Hacker Timesnew | past | comments | ask | show | jobs | submitlogin

(disclaimer: I work at Materialize and I work with Differential regularly)

Differential dataflow lets you write code such that the resulting programs are incremental e.g. if you were computing the most retweeted tweet in all of twitter or something like that and 5 minutes later 1000 new tweets showed up it would only take work proportional to the 1000 new tweets to update the results. It wouldn't need to redo the computation across all tweets.

Unlike every other similar framework I know of, Differential can also do this for programs with loops / recursion which makes it more possible to write algorithms.

Beyond that, as you've noted it parallelizes work nicely.

I wrote a blog post that was meant to explain "what does Differential do" and "when it is or isn't useful" and give some concrete examples that might be helpful. https://materialize.com/life-in-differential-dataflow/



> if you were computing the most retweeted tweet in all of twitter

are all the retweets counts of every tweet stored in the memory ?

Where are previous counts stored that are merged with new stream of tweets.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: