RE: Build with Hive 0.13.1 doesn't have datanucleus and parquet dependencies.

2014-10-27 Thread Cheng, Hao
Hive-thriftserver module is not included while specifying the profile hive-0.13.1. -Original Message- From: Jianshi Huang [mailto:jianshi.hu...@gmail.com] Sent: Monday, October 27, 2014 4:48 PM To: dev@spark.apache.org Subject: Build with Hive 0.13.1 doesn't have datanucleus and

Re: Build with Hive 0.13.1 doesn't have datanucleus and parquet dependencies.

2014-10-27 Thread Jianshi Huang
Ah I see. Thanks Hao! I'll wait for the fix. Jianshi On Mon, Oct 27, 2014 at 4:57 PM, Cheng, Hao hao.ch...@intel.com wrote: Hive-thriftserver module is not included while specifying the profile hive-0.13.1. -Original Message- From: Jianshi Huang [mailto:jianshi.hu...@gmail.com]

Re: best IDE for scala + spark development?

2014-10-27 Thread Dean Wampler
For what it's worth, I use Sublime Text + the SBT console for everything. I can live without the extra IDE features. However, if you like an IDE, the Eclipse Scala IDE 4.0 RC1 is a big improvement over previous releases. For one thing, it can now supports projects using different versions of

Re: best IDE for scala + spark development?

2014-10-27 Thread andy petrella
I second the S[B]T combo! I tried ATOM → lack of features and stability (atm) aℕdy ℙetrella about.me/noootsab [image: aℕdy ℙetrella on about.me] http://about.me/noootsab On Mon, Oct 27, 2014 at 2:15 PM, Dean Wampler deanwamp...@gmail.com wrote: For what it's worth, I use Sublime Text + the

Re: best IDE for scala + spark development?

2014-10-27 Thread Koert Kuipers
editor of your choice + sbt console works + grep great. if only folks stopped using wildcard imports (it has little benefits in terms of coding yet requires an IDE with 1G+ of ram to track em down). On Mon, Oct 27, 2014 at 9:17 AM, andy petrella andy.petre...@gmail.com wrote: I second the

Re: best IDE for scala + spark development?

2014-10-27 Thread Shivaram Venkataraman
Also ctags works fine with vim for browsing scala classes Shivaram On Mon, Oct 27, 2014 at 8:49 AM, Koert Kuipers ko...@tresata.com wrote: editor of your choice + sbt console works + grep great. if only folks stopped using wildcard imports (it has little benefits in terms of coding yet

jenkins downtime tomorrow morning ~6am-8am PDT

2014-10-27 Thread shane knapp
i'll be bringing jenkins down tomorrow morning for some system maintenance and to get our backups kicked off. i do expect to have the system back up and running before 8am. please let me know ASAP if i need to reschedule this. thanks, shane

jenkins emergency restart now, was Re: jenkins downtime tomorrow morning ~6am-8am PDT

2014-10-27 Thread shane knapp
so, i'm having a race condition between a plugin i installed putting jenkins in to quiet mode and it failing to perform a backup from this past weekend. i'll need to restart the process and get it out of the constantly-in-to-quiet-mode cycle it's in now. this will be quick, and i'll restart the

Re: jenkins emergency restart now, was Re: jenkins downtime tomorrow morning ~6am-8am PDT

2014-10-27 Thread shane knapp
ok we're back up and building. i've retriggered the jobs i killed. On Mon, Oct 27, 2014 at 1:24 PM, shane knapp skn...@berkeley.edu wrote: so, i'm having a race condition between a plugin i installed putting jenkins in to quiet mode and it failing to perform a backup from this past weekend.

Re: best IDE for scala + spark development?

2014-10-27 Thread Will Benton
I'll chime in as yet another user who is extremely happy with sbt and a text editor. (In my experience, running ack from the command line is usually just as easy and fast as using an IDE's find-in-project facility.) You can, of course, extend editors with Scala-specific IDE-like functionality

Re: HiveContext bug?

2014-10-27 Thread Marcelo Vanzin
Well, looks like a huge coincidence, but this was just sent to github: https://github.com/apache/spark/pull/2967 On Mon, Oct 27, 2014 at 3:25 PM, Marcelo Vanzin van...@cloudera.com wrote: Hey guys, I've been using the HiveFromSpark example to test some changes and I ran into an issue that

Workaround for python's inability to unzip zip64 spark assembly jar

2014-10-27 Thread Rahul Singhal
Hi All, We recently faced the known issue where pyspark does not work when the assembly jar contains more than 65K files. Our build and run time environment are both Java 7 but python fails to unzip the assembly jar as expected (https://issues.apache.org/jira/browse/SPARK-1911). All nodes in