Re: Spark build time

2015-04-22 Thread Nicholas Chammas
I suggest searching the archives for this list as there were several previous discussions about this problem. JIRA also has several issues related to this. Some pointers: - SPARK-3431 https://issues.apache.org/jira/browse/SPARK-3431: Parallelize Scala/Java test execution -

Re: Spark build time

2015-04-22 Thread Olivier Girardot
I agree, it's what I did :) I was just wondering if it was considered a problem or something to work on, I personally think so because the feedback loop should be as quick as possible, and therefore if there was someone I could help. Le mar. 21 avr. 2015 à 22:20, Reynold Xin r...@databricks.com a

RE: Spark build time

2015-04-21 Thread Alex
: ‎4/‎21/‎2015 4:21 PM To: Olivier Girardot ssab...@gmail.com Cc: dev@spark.apache.org dev@spark.apache.org Subject: Re: Spark build time It runs tons of integration tests. I think most developers just let Jenkins run the full suite of them. On Tue, Apr 21, 2015 at 12:54 PM, Olivier Girardot ssab

Re: Spark build time

2015-04-21 Thread Reynold Xin
It runs tons of integration tests. I think most developers just let Jenkins run the full suite of them. On Tue, Apr 21, 2015 at 12:54 PM, Olivier Girardot ssab...@gmail.com wrote: Hi everyone, I was just wandering about the Spark full build time (including tests), 1h48 seems to me quite...

Spark build time

2015-04-21 Thread Olivier Girardot
Hi everyone, I was just wandering about the Spark full build time (including tests), 1h48 seems to me quite... spacious. What's taking most of the time ? Is the build mainly integration tests ? Is there any roadmap or jiras dedicated to that we can chip in ? Regards, Olivier.