@Jey, I'm willing to help, but I need some initial guidance. Please, check out issues tab in Spock.jl.
On Monday, May 25, 2015 at 7:05:01 AM UTC+3, Jey Kottalam wrote: > > Viral -- Given the clear interest in it, I'll work toward fleshing it > out more fully. However, if anyone is interested in contributing I'd > gladly take the help. > > On Fri, May 22, 2015 at 9:02 PM, Viral Shah <vi...@mayin.org <javascript:>> > wrote: > > Jey - just checking if you are listing what the project needs - or if > you would be able to participate to build these? > > > > -viral > > > > > > > >> On 23-May-2015, at 7:54 am, Jey Kottalam <j...@cs.berkeley.edu > <javascript:>> wrote: > >> > >> The core functionality is there, but there's also more to be > >> implemented to have a complete interface to spark-core: > >> > >> - distributed I/O > >> - broadcast vars > >> - accumulator vars > >> - custom partitioners > >> - persistence (caching) > >> - missing transforms and actions > >> > >> -Jey > >> > >> On Fri, May 22, 2015 at 12:46 AM, Jeff Waller <trut...@gmail.com > <javascript:>> wrote: > >>> > >>> > >>> On Thursday, May 21, 2015 at 4:55:16 PM UTC-4, Jey Kottalam wrote: > >>>> > >>>> Hi Jeff, > >>>> > >>>>> they relied on a 3rd party to containerize a Pythonprogram for > >>>>> transmission > >>>> > >>>> That is due to the pecularities of Python's serialization module than > >>>> anything intrinsic to creating a Spark binding. (E.g. Python's pickle > >>>> format doesn't have support for serializing code and closures, so > some > >>>> extra code was required.) This isn't an issue in Julia since > >>>> Base.serialize() already has the needed functionality. An initial > >>>> implementation of a Spark binding done in the same style as PySpark > is > >>>> available at http://github.com/jey/Spock.jl > >>>> > >>>> -Jey > >>> > >>> > >>> Hey awesome. Initial you say, what's missing? > > >
