Re: how to run a dev spark project without fully rebuilding the fat jar ?

2014-10-24 Thread Akhil Das
You can use the --jars option to submit multiple jars using the spark-submit, so you can simply build the jar that you have modified. Thanks Best Regards On Thu, Oct 23, 2014 at 11:16 AM, Mohit Jaggi mohitja...@gmail.com wrote: i think you can give a list of jars - not just one - to

how to run a dev spark project without fully rebuilding the fat jar ?

2014-10-22 Thread Yang
during tests, I often modify my code a little bit and want to see the result. but spark-submit requires the full fat-jar, which takes quite a lot of time to build. I just need to run in --master local mode. is there a way to run it without rebuilding the fat jar? thanks Yang

Re: how to run a dev spark project without fully rebuilding the fat jar ?

2014-10-22 Thread Mohit Jaggi
i think you can give a list of jars - not just one - to spark-submit, so build only the one that has changed source code. On Wed, Oct 22, 2014 at 10:29 PM, Yang tedd...@gmail.com wrote: during tests, I often modify my code a little bit and want to see the result. but spark-submit