@Mayur : I won't fight with the semantic of a fork but at the moment, no Spork does take the standard Pig as dependency. On that, we should agree.
As for my use of Pig, I have no limitation. I am however interested to see the rise of a 'no-sql high level non programming language' for Spark. @Zhang : Could you elaborate your reference about Twitter? Bertrand Dechoux On Tue, Jul 8, 2014 at 4:04 AM, 张包峰 <pelickzh...@qq.com> wrote: > Hi guys, previously I checked out the old "spork" and updated it to Hadoop > 2.0, Scala 2.10.3 and Spark 0.9.1, see github project of mine > https://github.com/pelick/flare-spork > > It it also highly experimental, and just directly mapping pig physical > operations to spark RDD transformations/actions. It works for simple > requests. :) > > I am also interested on the progress of spork, is it undergoing in Twitter > in an un open-source way? > > ------------------ > Thanks > Zhang Baofeng > Blog <http://blog.csdn.net/pelick> | Github <https://github.com/pelick> | > Weibo <http://weibo.com/pelickzhang> | LinkedIn > <http://www.linkedin.com/pub/zhang-baofeng/70/609/84> > > > > > ------------------ 原始邮件 ------------------ > *发件人:* "Mayur Rustagi";<mayur.rust...@gmail.com>; > *发送时间:* 2014年7月7日(星期一) 晚上11:55 > *收件人:* "user@spark.apache.org"<user@spark.apache.org>; > *主题:* Re: Pig 0.13, Spark, Spork > > That version is old :). > We are not forking pig but cleanly separating out pig execution engine. > Let me know if you are willing to give it a go. > > Also would love to know what features of pig you are using ? > > Regards > Mayur > > Mayur Rustagi > Ph: +1 (760) 203 3257 > http://www.sigmoidanalytics.com > @mayur_rustagi <https://twitter.com/mayur_rustagi> > > > > On Mon, Jul 7, 2014 at 8:46 PM, Bertrand Dechoux <decho...@gmail.com> > wrote: > >> I saw a wiki page from your company but with an old version of Spark. >> >> http://docs.sigmoidanalytics.com/index.php/Setting_up_spork_with_spark_0.8.1 >> >> I have no reason to use it yet but I am interested in the state of the >> initiative. >> What's your point of view (personal and/or professional) about the Pig >> 0.13 release? >> Is the pluggable execution engine flexible enough in order to avoid >> having Spork as a fork of Pig? Pig + Spark + Fork = Spork :D >> >> As a (for now) external observer, I am glad to see competition in that >> space. It can only be good for the community in the end. >> >> Bertrand Dechoux >> >> >> On Mon, Jul 7, 2014 at 5:00 PM, Mayur Rustagi <mayur.rust...@gmail.com> >> wrote: >> >>> Hi, >>> We have fixed many major issues around Spork & deploying it with some >>> customers. Would be happy to provide a working version to you to try out. >>> We are looking for more folks to try it out & submit bugs. >>> >>> Regards >>> Mayur >>> >>> Mayur Rustagi >>> Ph: +1 (760) 203 3257 >>> http://www.sigmoidanalytics.com >>> @mayur_rustagi <https://twitter.com/mayur_rustagi> >>> >>> >>> >>> On Mon, Jul 7, 2014 at 8:21 PM, Bertrand Dechoux <decho...@gmail.com> >>> wrote: >>> >>>> Hi, >>>> >>>> I was wondering what was the state of the Pig+Spark initiative now that >>>> the execution engine of Pig is pluggable? Granted, it was done in order to >>>> use Tez but could it be used by Spark? I know about a 'theoretical' project >>>> called Spork but I don't know any stable and maintained version of it. >>>> >>>> Regards >>>> >>>> Bertrand Dechoux >>>> >>> >>> >> >
