Re: Spark vs Tez

2014-10-21 Thread Tim Randles
...@gmail.com *Sent:* Saturday, October 18, 2014 7:38 AM *To:* user@hadoop.apache.org mailto:user@hadoop.apache.org *Subject:* Re: Spark vs Tez Check out PySpark. No Scala required. On Friday, October 17, 2014, Adaryl Bob Wakefield, MBA adaryl.wakefi...@hotmail.com mailto:adaryl.wakefi...@hotmail.com

Re: Spark vs Tez

2014-10-21 Thread Edward Capriolo
... B. *From:* Russell Jurney mailto:russell.jur...@gmail.com *Sent:* Saturday, October 18, 2014 7:38 AM *To:* user@hadoop.apache.org mailto:user@hadoop.apache.org *Subject:* Re: Spark vs Tez Check out PySpark. No Scala required. On Friday, October 17, 2014, Adaryl Bob Wakefield, MBA

Re: Spark vs Tez

2014-10-21 Thread Brian O'Neill
@hadoop.apache.org Subject: Re: Spark vs Tez scala is not an interpreted language, from my non authoritative view it seems to have 2-3 (thousand) more compile phases than java and as a result some of the things you are doing that look like they are interpreted are actually macro's that get converted

Re: Spark vs Tez

2014-10-20 Thread Adaryl Bob Wakefield, MBA
Using an interpreted scripting language with something that is billing itself as being fast doesn’t sound like the best idea... B. From: Russell Jurney Sent: Saturday, October 18, 2014 7:38 AM To: user@hadoop.apache.org Subject: Re: Spark vs Tez Check out PySpark. No Scala required

Re: Spark vs Tez

2014-10-19 Thread Niels Basjes
: Spark vs Tez What aspects of Tez and Spark are you comparing? They have different purposes and thus not directly comparable, as far as I understand. Regards, Shahab On Fri, Oct 17, 2014 at 2:06 PM, Adaryl Bob Wakefield, MBA adaryl.wakefi...@hotmail.com wrote: Does anybody have any

Re: Spark vs Tez

2014-10-19 Thread Mohan Radhakrishnan
:12 PM *To:* user@hadoop.apache.org *Subject:* Re: Spark vs Tez What aspects of Tez and Spark are you comparing? They have different purposes and thus not directly comparable, as far as I understand. Regards, Shahab On Fri, Oct 17, 2014 at 2:06 PM, Adaryl Bob Wakefield, MBA adaryl.wakefi

Re: Spark vs Tez

2014-10-18 Thread Niels Basjes
:* Shahab Yunus shahab.yu...@gmail.com *Sent:* Friday, October 17, 2014 1:12 PM *To:* user@hadoop.apache.org *Subject:* Re: Spark vs Tez What aspects of Tez and Spark are you comparing? They have different purposes and thus not directly comparable, as far as I understand. Regards, Shahab

Re: Spark vs Tez

2014-10-18 Thread Mohan Radhakrishnan
@hadoop.apache.org *Subject:* Re: Spark vs Tez What aspects of Tez and Spark are you comparing? They have different purposes and thus not directly comparable, as far as I understand. Regards, Shahab On Fri, Oct 17, 2014 at 2:06 PM, Adaryl Bob Wakefield, MBA adaryl.wakefi...@hotmail.com wrote

Re: Spark vs Tez

2014-10-18 Thread Jeff Zhang
Yunus shahab.yu...@gmail.com *Sent:* Friday, October 17, 2014 1:12 PM *To:* user@hadoop.apache.org *Subject:* Re: Spark vs Tez What aspects of Tez and Spark are you comparing? They have different purposes and thus not directly comparable, as far as I understand. Regards, Shahab On Fri

Spark vs Tez

2014-10-17 Thread Adaryl Bob Wakefield, MBA
Does anybody have any performance figures on how Spark stacks up against Tez? If you don’t have figures, does anybody have an opinion? Spark seems so popular but I’m not really seeing why. B.

Re: Spark vs Tez

2014-10-17 Thread Shahab Yunus
What aspects of Tez and Spark are you comparing? They have different purposes and thus not directly comparable, as far as I understand. Regards, Shahab On Fri, Oct 17, 2014 at 2:06 PM, Adaryl Bob Wakefield, MBA adaryl.wakefi...@hotmail.com wrote: Does anybody have any performance figures on

Re: Spark vs Tez

2014-10-17 Thread Alexander Pivovarov
Spark creator Amplab did some benchmarks. https://amplab.cs.berkeley.edu/benchmark/ On Fri, Oct 17, 2014 at 11:06 AM, Adaryl Bob Wakefield, MBA adaryl.wakefi...@hotmail.com wrote: Does anybody have any performance figures on how Spark stacks up against Tez? If you don’t have figures, does

Re: Spark vs Tez

2014-10-17 Thread kartik saxena
I did a performance benchmark during my summer internship . I am currently a grad student. Can't reveal much about the specific project but Spark is still faster than around 4-5th iteration of Tez of the same query/dataset. By Iteration I mean utilizing the hot-container property of Apache Tez .

Re: Spark vs Tez

2014-10-17 Thread Adaryl Bob Wakefield, MBA
: Spark vs Tez What aspects of Tez and Spark are you comparing? They have different purposes and thus not directly comparable, as far as I understand. Regards, Shahab On Fri, Oct 17, 2014 at 2:06 PM, Adaryl Bob Wakefield, MBA adaryl.wakefi...@hotmail.com wrote: Does anybody have any

Re: Spark vs Tez

2014-10-17 Thread Alexander Pivovarov
execution engine that replaces MapReduce and is also supposed to speed up batch processing. Is that not correct? B. *From:* Shahab Yunus shahab.yu...@gmail.com *Sent:* Friday, October 17, 2014 1:12 PM *To:* user@hadoop.apache.org *Subject:* Re: Spark vs Tez What aspects of Tez and Spark

Re: Spark vs Tez

2014-10-17 Thread Adaryl Bob Wakefield, MBA
, October 17, 2014 1:12 PM To: user@hadoop.apache.org Subject: Re: Spark vs Tez I did a performance benchmark during my summer internship . I am currently a grad student. Can't reveal much about the specific project but Spark is still faster than around 4-5th iteration of Tez of the same query/dataset

Re: Spark vs Tez

2014-10-17 Thread Gavin Yue
:* Re: Spark vs Tez What aspects of Tez and Spark are you comparing? They have different purposes and thus not directly comparable, as far as I understand. Regards, Shahab On Fri, Oct 17, 2014 at 2:06 PM, Adaryl Bob Wakefield, MBA adaryl.wakefi...@hotmail.com wrote: Does anybody have any