Brandon, Thanks for sharing this. Looks very promising.
The project mentions - "process petabytes of data in real-time". I'm curious to know if the architecture implemented in the Github repo was used to process petabytes? If yes, how many nodes did you use for this and did you use Spark standalone cluster or with YARN/Mesos ? I'm also interested to know what issues you had with Spray and Akka working at this scale. Thanks -Soumya On Fri, Aug 15, 2014 at 5:06 PM, Brandon Amos <[email protected]> wrote: > Hi Spark community, > > At Adobe Research, we're happy to open source a prototype > technology called Spindle we've been developing over > the past few months for processing analytics queries with Spark. > Please take a look at the repository on GitHub at > https://github.com/adobe-research/spindle, > and we welcome any feedback. Thanks! > > Regards, > Brandon. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Open-sourcing-Spindle-by-Adobe-Research-a-web-analytics-processing-engine-in-Scala-Spark-and-Parquet-tp12203.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
