build/sbt -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver -Dhadoop.version=2.6.0 -DskipTests assembly
The above command fails on Mac. build/sbt -Pyarn -Phadoop-2.2 -Phive -Phive-thriftserver -Pkinesis-asl -DskipTests assembly The above command, used by Jenkins, passes. That's why the build error wasn't caught. FYI On Thu, Nov 5, 2015 at 11:07 AM, Dilip Biswal <dbis...@us.ibm.com> wrote: > Hello Ted, > > Thanks for your response. > > Here is the command i used : > > build/sbt clean > build/sbt -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver > -Dhadoop.version=2.6.0 -DskipTests assembly > > I am building on CentOS and on master branch. > > One other thing, i was able to build fine with the above command up until > recently. I think i have stared > to have problem after SPARK-11073 where the HashCodes import was added. > > Regards, > Dilip Biswal > Tel: 408-463-4980 > dbis...@us.ibm.com > > > > From: Ted Yu <yuzhih...@gmail.com> > To: Dilip Biswal/Oakland/IBM@IBMUS > Cc: Jean-Baptiste Onofré <j...@nanthrax.net>, "dev@spark.apache.org" > <dev@spark.apache.org> > Date: 11/05/2015 10:46 AM > Subject: Re: Master build fails ? > ------------------------------ > > > > Dilip: > Can you give the command you used ? > > Which release were you building ? > What OS did you build on ? > > Cheers > > On Thu, Nov 5, 2015 at 10:21 AM, Dilip Biswal <*dbis...@us.ibm.com* > <dbis...@us.ibm.com>> wrote: > Hello, > > I am getting the same build error about not being able tofind > com.google.common.hash.HashCodes. > > > Is there a solution to this ? > > Regards, > Dilip Biswal > Tel: *408-463-4980* <408-463-4980> > *dbis...@us.ibm.com* <dbis...@us.ibm.com> > > > > From: Jean-Baptiste Onofré <*j...@nanthrax.net* <j...@nanthrax.net>> > To: Ted Yu <*yuzhih...@gmail.com* <yuzhih...@gmail.com>> > Cc: "*dev@spark.apache.org* <dev@spark.apache.org>" < > *dev@spark.apache.org* <dev@spark.apache.org>> > Date: 11/03/2015 07:20 AM > Subject: Re: Master build fails ? > ------------------------------ > > > > Hi Ted, > > thanks for the update. The build with sbt is in progress on my box. > > Regards > JB > > On 11/03/2015 03:31 PM, Ted Yu wrote: > > Interesting, Sbt builds were not all failing: > > > > *https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/* > <https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/> > > > > FYI > > > > On Tue, Nov 3, 2015 at 5:58 AM, Jean-Baptiste Onofré <*j...@nanthrax.net* > <j...@nanthrax.net> > > <*mailto:j...@nanthrax.net* <j...@nanthrax.net>>> wrote: > > > > > Hi Jacek, > > > > it works fine with mvn: the problem is with sbt. > > > > I suspect a different reactor order in sbt compare to mvn. > > > > Regards > > JB > > > > On 11/03/2015 02:44 PM, Jacek Laskowski wrote: > > > > Hi, > > > > Just built the sources using the following command and it worked > > fine. > > > > ➜ spark git:(master) ✗ ./build/mvn -Pyarn -Phadoop-2.6 > > -Dhadoop.version=2.7.1 -Dscala-2.11 -Phive -Phive-thriftserver > > -DskipTests clean install > > ... > > [INFO] > > > ------------------------------------------------------------------------ > > [INFO] BUILD SUCCESS > > [INFO] > > > ------------------------------------------------------------------------ > > [INFO] Total time: 14:15 min > > [INFO] Finished at: 2015-11-03T14:40:40+01:00 > > [INFO] Final Memory: 438M/1972M > > [INFO] > > > ------------------------------------------------------------------------ > > > > ➜ spark git:(master) ✗ java -version > > java version "1.8.0_66" > > Java(TM) SE Runtime Environment (build 1.8.0_66-b17) > > Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) > > > > I'm on Mac OS. > > > > Pozdrawiam, > > Jacek > > > > -- > > Jacek Laskowski | > *http://blog.japila.pl* <http://blog.japila.pl/>| > > *http://blog.jaceklaskowski.pl* <http://blog.jaceklaskowski.pl/> > > > Follow me at *https://twitter.com/jaceklaskowski* > <https://twitter.com/jaceklaskowski> > > Upvote at > *http://stackoverflow.com/users/1305344/jacek-laskowski* > <http://stackoverflow.com/users/1305344/jacek-laskowski> > > > > > > > On Tue, Nov 3, 2015 at 1:37 PM, Jean-Baptiste Onofré > > <*j...@nanthrax.net* <j...@nanthrax.net><*mailto:j...@nanthrax.net* > <j...@nanthrax.net>>> wrote: > > > > Thanks for the update, I used mvn to build but without hive > > profile. > > > > Let me try with mvn with the same options as you and sbt > also. > > > > I keep you posted. > > > > Regards > > JB > > > > On 11/03/2015 12:55 PM, Jeff Zhang wrote: > > > > > > I found it is due to SPARK-11073. > > > > Here's the command I used to build > > > > build/sbt clean compile -Pyarn -Phadoop-2.6 -Phive > > -Phive-thriftserver > > -Psparkr > > > > On Tue, Nov 3, 2015 at 7:52 PM, Jean-Baptiste Onofré > > <*j...@nanthrax.net* <j...@nanthrax.net>< > *mailto:j...@nanthrax.net* <j...@nanthrax.net>> > > <*mailto:j...@nanthrax.net* <j...@nanthrax.net>< > *mailto:j...@nanthrax.net* <j...@nanthrax.net>>>> wrote: > > > > > Hi Jeff, > > > > it works for me (with skipping the tests). > > > > Let me try again, just to be sure. > > > > Regards > > JB > > > > > > On 11/03/2015 11:50 AM, Jeff Zhang wrote: > > > > Looks like it's due to guava version > > conflicts, I see both guava > > 14.0.1 > > and 16.0.1 under lib_managed/bundles. Anyone > > meet this issue too ? > > > > [error] > > > > > /Users/jzhang/github/spark_apache/core/src/main/scala/org/apache/spark/SecurityManager.scala:26: > > object HashCodes is not a member of package > > com.google.common.hash > > [error] import com.google.common.hash.HashCodes > > [error] ^ > > [info] Resolving > > org.apache.commons#commons-math;2.2 ... > > [error] > > > > > /Users/jzhang/github/spark_apache/core/src/main/scala/org/apache/spark/SecurityManager.scala:384: > > not found: value HashCodes > > [error] val cookie = > > HashCodes.fromBytes(secret).toString() > > [error] ^ > > > > > > > > > > -- > > Best Regards > > > > Jeff Zhang > > > > > > -- > > Jean-Baptiste Onofré > > *jbono...@apache.org* <jbono...@apache.org>< > *mailto:jbono...@apache.org* <jbono...@apache.org>> > > <*mailto:jbono...@apache.org* <jbono...@apache.org>< > *mailto:jbono...@apache.org* <jbono...@apache.org>>> > > *http://blog.nanthrax.net* <http://blog.nanthrax.net/> > > Talend - *http://www.talend.com* > <http://www.talend.com/> > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > > *dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org> > > <*mailto:dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org>> > > <*mailto:dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org> > > <*mailto:dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org>>> > > For additional commands, e-mail: > > *dev-h...@spark.apache.org* <dev-h...@spark.apache.org>< > *mailto:dev-h...@spark.apache.org* <dev-h...@spark.apache.org>> > > <*mailto:dev-h...@spark.apache.org* > <dev-h...@spark.apache.org> > > > <*mailto:dev-h...@spark.apache.org* > <dev-h...@spark.apache.org>>> > > > > > > > > > > -- > > Best Regards > > > > Jeff Zhang > > > > > > > > -- > > Jean-Baptiste Onofré > > *jbono...@apache.org* <jbono...@apache.org>< > *mailto:jbono...@apache.org* <jbono...@apache.org>> > > *http://blog.nanthrax.net* <http://blog.nanthrax.net/> > > Talend - *http://www.talend.com* <http://www.talend.com/> > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: *dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org> > > <*mailto:dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org>> > > For additional commands, e-mail: *dev-h...@spark.apache.org* > <dev-h...@spark.apache.org> > > <*mailto:dev-h...@spark.apache.org* > <dev-h...@spark.apache.org>> > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: *dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org> > > <*mailto:dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org>> > > For additional commands, e-mail: *dev-h...@spark.apache.org* > <dev-h...@spark.apache.org> > > <*mailto:dev-h...@spark.apache.org* <dev-h...@spark.apache.org>> > > > > > > -- > > Jean-Baptiste Onofré > > *jbono...@apache.org* <jbono...@apache.org>< > *mailto:jbono...@apache.org* <jbono...@apache.org>> > > *http://blog.nanthrax.net* <http://blog.nanthrax.net/> > > Talend - *http://www.talend.com* <http://www.talend.com/> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: *dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org> > > <*mailto:dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org>> > > For additional commands, e-mail: *dev-h...@spark.apache.org* > <dev-h...@spark.apache.org> > > <*mailto:dev-h...@spark.apache.org* <dev-h...@spark.apache.org>> > > > > > > -- > Jean-Baptiste Onofré > *jbono...@apache.org* <jbono...@apache.org> > *http://blog.nanthrax.net* <http://blog.nanthrax.net/> > Talend - *http://www.talend.com* <http://www.talend.com/> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: *dev-unsubscr...@spark.apache.org* > <dev-unsubscr...@spark.apache.org> > For additional commands, e-mail: *dev-h...@spark.apache.org* > <dev-h...@spark.apache.org> > > > > > >
