Terasort example

2014-11-11 Thread Ewan Higgs
Hi all, I saw that Reynold Xin had a Terasort example PR on Github[1]. It didn't appear to be similar to the Hadoop Terasort example, so I've tried to brush it into shape so it can generate Terasort files (teragen), sort the files (terasort) and validate the files (teravalidate). My

Re: Terasort example

2014-11-11 Thread Reynold Xin
This is great. I think the consensus from last time was that we would put performance stuff into spark-perf, so it is easy to test different Spark versions. On Tue, Nov 11, 2014 at 5:03 AM, Ewan Higgs wrote: > Hi all, > I saw that Reynold Xin had a Terasort example PR on Github[1]. It

Re: Terasort example

2014-11-11 Thread Ewan Higgs
great. I think the consensus from last time was that we would put performance stuff into spark-perf, so it is easy to test different Spark versions. On Tue, Nov 11, 2014 at 5:03 AM, Ewan Higgs <mailto:ewan.hi...@ugent.be>> wrote: Hi all, I saw that Reynold Xin had a Terasort e

Re: Terasort example

2014-11-11 Thread Josh Rosen
versions. > > > On Tue, Nov 11, 2014 at 5:03 AM, Ewan Higgs <mailto:ewan.hi...@ugent.be>> wrote: > > Hi all, > I saw that Reynold Xin had a Terasort example PR on Github[1]. It > didn't appear to be similar to the Hadoop Terasort example, so > I

running the Terasort example

2014-12-11 Thread Tim Harsch
Hi all, I just joined the list, so I don¹t have a message history that would allow me to reply to this post: http://apache-spark-developers-list.1001551.n3.nabble.com/Terasort-example- td9284.html I am interested in running the terasort example. I cloned the repo https://github.com/ehiggs/spark

Re: running the Terasort example

2014-12-16 Thread Ewan Higgs
: Hi all, I just joined the list, so I don¹t have a message history that would allow me to reply to this post: http://apache-spark-developers-list.1001551.n3.nabble.com/Terasort-example- td9284.html I am interested in running the terasort example. I cloned the repo https://github.com/ehiggs/spark and

Re: running the Terasort example

2014-12-16 Thread Tim Harsch
gt; >On 12/12/14 01:06, Tim Harsch wrote: >> Hi all, >> I just joined the list, so I don¹t have a message history that would >>allow >> me to reply to this post: >> >>http://apache-spark-developers-list.1001551.n3.nabble.com/Terasort-exampl >>e- >> td9

Re: running the Terasort example

2014-12-16 Thread Ewan Higgs
h wrote: >>> Hi all, >>> I just joined the list, so I don¹t have a message history that would >>> allow >>> me to reply to this post: >>> >>> http://apache-spark-developers-list.1001551.n3.nabble.com/Terasort-exampl >>> e- >>>

Re: running the Terasort example

2014-12-17 Thread Tim Harsch
im Harsch wrote: >>>> Hi all, >>>> I just joined the list, so I don¹t have a message history that would >>>> allow >>>> me to reply to this post: >>>> >>>> >>>>http://apache-spark-developers-list.1001551.n3.nabble