Ben, Note that Snappydata's primary objective is to be a distributed in-memory DB for mixed workloads (i.e. streaming with transactions and analytic queries). On the other hand, Spark, till date, is primarily designed as a processing engine over myriad storage engines (SnappyData being one). So, the marriage is quite complementary. The difference compared to other stores is that SnappyData realizes its solution by deeply integrating and collocating with Spark (i.e. share spark executor memory/resources with the store) avoiding serializations and shuffle in many situations.
On your specific thought about being similar to Structured streaming, a better discussion could be a comparison to the recently introduced State store <https://docs.google.com/document/d/1-ncawFx8JS5Zyfq1HAEGBx56RDet9wfVp_hDM8ZL254/edit#heading=h.2h7zw4ru3nw7> (perhaps this is what you meant). It proposes a KV store for streaming aggregations with support for updates. The proposed API will, at some point, be pluggable so vendors can easily support alternate implementations to storage, not just HDFS(default store in proposed State store). ----- Jags SnappyData blog <http://www.snappydata.io/blog> Download binary, source <https://github.com/SnappyDataInc/snappydata> On Wed, Jul 6, 2016 at 12:49 AM, Benjamin Kim <bbuil...@gmail.com> wrote: > I recently got a sales email from SnappyData, and after reading the > documentation about what they offer, it sounds very similar to what > Structured Streaming will offer w/o the underlying in-memory, > spill-to-disk, CRUD compliant data storage in SnappyData. I was wondering > if Structured Streaming is trying to achieve the same on its own or is > SnappyData contributing Streaming extensions that they built to the Spark > project. Lastly, what does the Spark community think of this so-called > “Spark Data Store”? > > Thanks, > Ben > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
