Re: Speeding up Spark build during development

2015-05-06 Thread Pramod Biligiri
that you change and then run this main class. Best regards, Alexander -Original Message- From: Pramod Biligiri [mailto:pramodbilig...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during

Re: Speeding up Spark build during development

2015-05-05 Thread Iulian DragoČ™
, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I was wondering if there's a faster way to build than running the package target

Re: Speeding up Spark build during development

2015-05-04 Thread Pramod Biligiri
Subject: Speeding up Spark build during development Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I was wondering if there's a faster way to build than running the package target each time. Currently I'm using: mvn -DskipTests package

Re: Speeding up Spark build during development

2015-05-04 Thread Emre Sevinc
and then run this main class. Best regards, Alexander -Original Message- From: Pramod Biligiri [mailto:pramodbilig...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi

Re: Speeding up Spark build during development

2015-05-04 Thread Pramod Biligiri
[mailto:pramodbilig...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I was wondering if there's a faster way

Re: Speeding up Spark build during development

2015-05-04 Thread Emre Sevinc
that you change and then run this main class. Best regards, Alexander -Original Message- From: Pramod Biligiri [mailto:pramodbilig...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during

Re: Speeding up Spark build during development

2015-05-04 Thread Meethu Mathew
-Original Message- From: Pramod Biligiri [mailto:pramodbilig...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I was wondering

Re: Speeding up Spark build during development

2015-05-04 Thread Tathagata Das
...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I was wondering if there's a faster way to build than running

Re: Speeding up Spark build during development

2015-05-04 Thread Michael Armbrust
@spark.apache.org Subject: Speeding up Spark build during development Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I was wondering if there's a faster way to build than running the package target each time. Currently I'm using: mvn -DskipTests package All

Re: Speeding up Spark build during development

2015-05-03 Thread Mark Hamstra
change and then run this main class. Best regards, Alexander -Original Message- From: Pramod Biligiri [mailto:pramodbilig...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi

Re: Speeding up Spark build during development

2015-05-03 Thread Pramod Biligiri
:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I was wondering if there's a faster way to build than running the package target each time. Currently I'm using

Re: Speeding up Spark build during development

2015-05-01 Thread York, Brennon
-Original Message- From: Pramod Biligiri [mailto:pramodbilig...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I

RE: Speeding up Spark build during development

2015-05-01 Thread Ulanov, Alexander
...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I was wondering if there's a faster way to build than running the package target each time

Re: Speeding up Spark build during development

2015-05-01 Thread Ted Yu
the package that you change and then run this main class. Best regards, Alexander -Original Message- From: Pramod Biligiri [mailto:pramodbilig...@gmail.com] Sent: Friday, May 01, 2015 1:46 AM To: dev@spark.apache.org Subject: Speeding up Spark build during development Hi, I'm making

Speeding up Spark build during development

2015-05-01 Thread Pramod Biligiri
Hi, I'm making some small changes to the Spark codebase and trying it out on a cluster. I was wondering if there's a faster way to build than running the package target each time. Currently I'm using: mvn -DskipTests package All the nodes have the same filesystem mounted at the same mount point.

Re: Speeding up Spark build during development

2015-05-01 Thread Prashant Sharma
Hi Pramod, If you are using sbt as your build, then you need to do sbt assembly once and use sbt ~compile. Also export SPARK_PREPEND_CLASSES=1 this in your shell and all nodes. You can may be try this out ? Thanks, Prashant Sharma On Fri, May 1, 2015 at 2:16 PM, Pramod Biligiri