Re: failed to build spark with maven for both 1.0.1 and latest master branch

2014-07-31 Thread yao
Great, thanks Ted. I just did a maven build on CentOS 6, everything looks good. So this is more like a Mac specific issue. On Thu, Jul 31, 2014 at 1:43 PM, Ted Yu wrote: > The following command succeeded (on Linux) on Spark master checked out this > morning: > > mvn -Pyarn -Phive -Phadoop-2.4 -

Re: failed to build spark with maven for both 1.0.1 and latest master branch

2014-07-31 Thread Ted Yu
The following command succeeded (on Linux) on Spark master checked out this morning: mvn -Pyarn -Phive -Phadoop-2.4 -DskipTests install FYI On Thu, Jul 31, 2014 at 1:36 PM, yao wrote: > Hi TD, > > I've asked my colleagues to do the same thing but compile still fails. > However, maven build su

Re: failed to build spark with maven for both 1.0.1 and latest master branch

2014-07-31 Thread yao
Hi TD, I've asked my colleagues to do the same thing but compile still fails. However, maven build succeeded once I built it on my personal macbook (with the latest MacOS Yosemite). So I guess there might be something wrong in my build environment. Wonder if anyone tried to compile spark using mav

Exception in Spark 1.0.1: com.esotericsoftware.kryo.KryoException: Buffer underflow

2014-07-31 Thread Andrew Ash
Hi everyone, I'm seeing the below exception coming out of Spark 1.0.1 when I call it from my application. I can't share the source to that application, but the quick gist is that it uses Spark's Java APIs to read from Avro files in HDFS, do processing, and write back to Avro files. It does this

Re: replacement for SPARK_JAVA_OPTS

2014-07-31 Thread Cody Koeninger
1. I've tried with and without escaping equals sign, it doesn't affect the results. 2. Yeah, exporting SPARK_SUBMIT_OPTS from spark-env.sh works for getting system properties set in the local shell (although not for executors). 3. We're using the default fine-grained mesos mode, not setting spark

Re: failed to build spark with maven for both 1.0.1 and latest master branch

2014-07-31 Thread Tathagata Das
Does a "mvn clean" or "sbt/sbt clean" help? TD On Wed, Jul 30, 2014 at 9:25 PM, yao wrote: > Hi Folks, > > Today I am trying to build spark using maven; however, the following > command failed consistently for both 1.0.1 and the latest master. (BTW, it > seems sbt works fine: *sbt/sbt -Dhadoop.

Re: Can I translate the documentations of Spark in Japanese?

2014-07-31 Thread Cheng Lian
Transifex (https://www.transifex.com/) may be helpful if you're considering online crowdsourcing. At least you may easily maintain a unified glossary with it among all translators. On Thu, Jul 31, 2014 at 3:40 PM, Yu Ishikawa wrote: > Hi Nick, > > > I know some projects get translations crowdso

Re: Can I translate the documentations of Spark in Japanese?

2014-07-31 Thread Yu Ishikawa
Hi Nick, > I know some projects get translations crowdsourced via one website or > other. Thank you for your comments. I think crowdsourced translation is fit for the translation project on github. Best, Yu -- View this message in context: http://apache-spark-developers-list.1001551.n3.nab

Re: Can I translate the documentations of Spark in Japanese?

2014-07-31 Thread Yu Ishikawa
Hi Kenichi Takagiwa, Thank you for commenting. I am going to proceed with the translation, will you please help me. Further details will be sent later. Best, Yu -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Can-I-translate-the-documentations-of-Sp