Hi Devaraj,
At the moment hadoop.version is set to 0.20.0 in Hive's build.properties file -- we're not setting it explicitly in the Hudson job. The build.properties file also defines the hadoop.security.version property, and Hive's build files use the values of these two properties to construct classpaths appropriate for secured and non-secured tests. I think the solution to this test failure involves manually setting the test classpath in build/shims.xml to use the secure version of hadoop indicated by hadoop.secure.version. I left some more details in HIVE-1927. Please let me know if they don't make sense. Thanks. Carl On Thu, Jan 20, 2011 at 8:32 AM, Devaraj Das <d...@yahoo-inc.com> wrote: > I checked the console output on the failed build. The command that's > used to run the tests is 'ant -Dhadoop.version=0.20.0 .....'. This > probably is the cause of the problem (though I haven't verified > locally). > Devaraj. > > On Jan 20, 2011, at 5:26 AM, "Carl Steinbach" <c...@cloudera.com> wrote: > > > HIVE-1696 introduced a build failure: > > > > Running org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge > > java.lang.NoSuchMethodError: > > org.apache.hadoop.security.UserGroupInformation.getCurrentUser()Lorg/ > > apache/hadoop/security/UserGroupInformation; > > > > This looks like a straightforward classpath problem (picking up 0.20.0 > > instead of 0.20.3-CDH3-SNAPSHOT), but when I run this locally it > > works, and > > dumping the classpath from the toplevel "test" target shows that > > 0.20.3 is > > getting pulled into the classpath. > > > > Anyone else have any ideas? > > > > Thanks. > > > > Carl > > > > > > On Thu, Jan 20, 2011 at 4:54 AM, Apache Hudson Server < > > hud...@hudson.apache.org> wrote: > > > >> See <https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/500/> > >> > >> ------------------------------------------ > >> [...truncated 7224 lines...] > >> create-dirs: > >> > >> init: > >> > >> compile: > >> [echo] Compiling: anttasks > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ant/build.xml > >> >:40: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> > >> deploy-ant-tasks: > >> > >> create-dirs: > >> > >> init: > >> > >> compile: > >> [echo] Compiling: anttasks > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ant/build.xml > >> >:40: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> > >> jar: > >> > >> init: > >> > >> install-hadoopcore: > >> > >> install-hadoopcore-default: > >> > >> ivy-init-dirs: > >> > >> ivy-download: > >> [get] Getting: > >> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar > >> [get] To: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/ivy/lib/ivy-2.1.0.jar > >>> > >> [get] Not modified - so not downloaded > >> > >> ivy-probe-antlib: > >> > >> ivy-init-antlib: > >> > >> ivy-init: > >> > >> ivy-retrieve-hadoop-source: > >> :: loading settings :: file = < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ivy/ivysettings.xml > >>> > >> [ivy:retrieve] :: resolving dependencies :: org.apache.hadoop.hive# > >> contrib;work...@vesta.apache.org <contrib%3bwork...@vesta.apache.org> < > contrib%3bwork...@vesta.apache.org <contrib%253bwork...@vesta.apache.org>> > >> [ivy:retrieve] confs: [default] > >> [ivy:retrieve] found hadoop#core;0.20.0 in hadoop-source > >> [ivy:retrieve] :: resolution report :: resolve 1086ms :: artifacts > >> dl 1ms > >> > >> --------------------------------------------------------------------- > >> | | modules || > >> artifacts > >> | > >> | conf | number| search|dwnlded|evicted|| > >> number|dwnlded| > >> > >> --------------------------------------------------------------------- > >> | default | 1 | 0 | 0 | 0 || 1 > >> | 0 > >> | > >> > >> --------------------------------------------------------------------- > >> [ivy:retrieve] :: retrieving :: org.apache.hadoop.hive#contrib > >> [ivy:retrieve] confs: [default] > >> [ivy:retrieve] 0 artifacts copied, 1 already retrieved (0kB/1ms) > >> > >> install-hadoopcore-internal: > >> > >> setup: > >> > >> compile: > >> [echo] Compiling: hbase-handler > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build-common.xml > >> >:283: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> > >> jar: > >> [echo] Jar: hbase-handler > >> > >> test: > >> > >> test-shims: > >> > >> test-conditions: > >> > >> gen-test: > >> > >> create-dirs: > >> > >> compile-ant-tasks: > >> > >> create-dirs: > >> > >> init: > >> > >> compile: > >> [echo] Compiling: anttasks > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ant/build.xml > >> >:40: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> > >> deploy-ant-tasks: > >> > >> create-dirs: > >> > >> init: > >> > >> compile: > >> [echo] Compiling: anttasks > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ant/build.xml > >> >:40: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> > >> jar: > >> > >> init: > >> > >> compile: > >> > >> ivy-init-dirs: > >> > >> ivy-download: > >> [get] Getting: > >> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar > >> [get] To: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/ivy/lib/ivy-2.1.0.jar > >>> > >> [get] Not modified - so not downloaded > >> > >> ivy-probe-antlib: > >> > >> ivy-init-antlib: > >> > >> ivy-init: > >> > >> ivy-retrieve-hadoop-source: > >> :: loading settings :: file = < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ivy/ivysettings.xml > >>> > >> [ivy:retrieve] :: resolving dependencies :: org.apache.hadoop.hive# > >> shims;work...@vesta.apache.org <shims%3bwork...@vesta.apache.org> < > shims%3bwork...@vesta.apache.org <shims%253bwork...@vesta.apache.org>> > >> [ivy:retrieve] confs: [default] > >> [ivy:retrieve] found hadoop#core;0.20.0 in hadoop-source > >> [ivy:retrieve] found hadoop#core;0.20.3-CDH3-SNAPSHOT in hadoop- > >> source > >> [ivy:retrieve] :: resolution report :: resolve 1738ms :: artifacts > >> dl 1ms > >> > >> --------------------------------------------------------------------- > >> | | modules || > >> artifacts > >> | > >> | conf | number| search|dwnlded|evicted|| > >> number|dwnlded| > >> > >> --------------------------------------------------------------------- > >> | default | 2 | 0 | 0 | 0 || 2 > >> | 0 > >> | > >> > >> --------------------------------------------------------------------- > >> [ivy:retrieve] :: retrieving :: org.apache.hadoop.hive#shims > >> [ivy:retrieve] confs: [default] > >> [ivy:retrieve] 0 artifacts copied, 2 already retrieved (0kB/2ms) > >> > >> install-hadoopcore-internal: > >> > >> build_shims: > >> [echo] Compiling shims against hadoop 0.20.0 (< > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/hadoopcore/hadoop-0.20.0 > >> )> > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/shims/build.xml > >> >:53: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> > >> ivy-init-dirs: > >> > >> ivy-download: > >> [get] Getting: > >> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar > >> [get] To: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/ivy/lib/ivy-2.1.0.jar > >>> > >> [get] Not modified - so not downloaded > >> > >> ivy-probe-antlib: > >> > >> ivy-init-antlib: > >> > >> ivy-init: > >> > >> ivy-retrieve-hadoop-source: > >> :: loading settings :: file = < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ivy/ivysettings.xml > >>> > >> [ivy:retrieve] :: resolving dependencies :: org.apache.hadoop.hive# > >> shims;work...@vesta.apache.org <shims%3bwork...@vesta.apache.org> < > shims%3bwork...@vesta.apache.org <shims%253bwork...@vesta.apache.org>> > >> [ivy:retrieve] confs: [default] > >> [ivy:retrieve] found hadoop#core;0.20.0 in hadoop-source > >> [ivy:retrieve] found hadoop#core;0.20.3-CDH3-SNAPSHOT in hadoop- > >> source > >> [ivy:retrieve] :: resolution report :: resolve 1986ms :: artifacts > >> dl 1ms > >> > >> --------------------------------------------------------------------- > >> | | modules || > >> artifacts > >> | > >> | conf | number| search|dwnlded|evicted|| > >> number|dwnlded| > >> > >> --------------------------------------------------------------------- > >> | default | 2 | 0 | 0 | 0 || 2 > >> | 0 > >> | > >> > >> --------------------------------------------------------------------- > >> [ivy:retrieve] :: retrieving :: org.apache.hadoop.hive#shims > >> [ivy:retrieve] confs: [default] > >> [ivy:retrieve] 0 artifacts copied, 2 already retrieved (0kB/2ms) > >> > >> install-hadoopcore-internal: > >> > >> build_shims: > >> [echo] Compiling shims against hadoop 0.20.3-CDH3-SNAPSHOT (< > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/hadoopcore/hadoop-0.20.3-CDH3-SNAPSHOT > >> )> > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/shims/build.xml > >> >:53: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/shims/build.xml > >> >:75: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> > >> compile-test: > >> > >> ivy-init-dirs: > >> > >> ivy-download: > >> [get] Getting: > >> http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar > >> [get] To: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/ivy/lib/ivy-2.1.0.jar > >>> > >> [get] Not modified - so not downloaded > >> > >> ivy-probe-antlib: > >> > >> ivy-init-antlib: > >> > >> ivy-init: > >> > >> ivy-retrieve-hadoop-source: > >> :: loading settings :: file = < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/ivy/ivysettings.xml > >>> > >> [ivy:retrieve] :: resolving dependencies :: org.apache.hadoop.hive# > >> shims;work...@vesta.apache.org <shims%3bwork...@vesta.apache.org> < > shims%3bwork...@vesta.apache.org <shims%253bwork...@vesta.apache.org>> > >> [ivy:retrieve] confs: [default] > >> [ivy:retrieve] found hadoop#core;0.20.0 in hadoop-source > >> [ivy:retrieve] found hadoop#core;0.20.3-CDH3-SNAPSHOT in hadoop- > >> source > >> [ivy:retrieve] :: resolution report :: resolve 1993ms :: artifacts > >> dl 1ms > >> > >> --------------------------------------------------------------------- > >> | | modules || > >> artifacts > >> | > >> | conf | number| search|dwnlded|evicted|| > >> number|dwnlded| > >> > >> --------------------------------------------------------------------- > >> | default | 2 | 0 | 0 | 0 || 2 > >> | 0 > >> | > >> > >> --------------------------------------------------------------------- > >> [ivy:retrieve] :: retrieving :: org.apache.hadoop.hive#shims > >> [ivy:retrieve] confs: [default] > >> [ivy:retrieve] 0 artifacts copied, 2 already retrieved (0kB/1ms) > >> > >> install-hadoopcore-internal: > >> > >> compile_secure_test: > >> [echo] Compiling shim tests against hadoop 0.20.3-CDH3-SNAPSHOT (< > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/hadoopcore/hadoop-0.20.3-CDH3-SNAPSHOT > >> )> > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/shims/build.xml > >> >:96: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> [javac] Compiling 1 source file to < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/shims/test/classes > >>> > >> [javac] < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/shims/build.xml > >> >:109: > >> warning: 'includeantruntime' was not set, defaulting to > >> build.sysclasspath=last; set to false for repeatable builds > >> > >> test-jar: > >> [jar] Building MANIFEST-only jar: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/shims/test/test-udfs.jar > >>> > >> > >> test-init: > >> [mkdir] Created dir: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/shims/test/data > >>> > >> [mkdir] Created dir: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/shims/test/logs/clientpositive > >>> > >> [mkdir] Created dir: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/shims/test/logs/clientnegative > >>> > >> [mkdir] Created dir: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/shims/test/logs/positive > >>> > >> [mkdir] Created dir: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/shims/test/logs/negative > >>> > >> [mkdir] Created dir: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/shims/test/data/warehouse > >>> > >> [mkdir] Created dir: < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build/shims/test/data/metadb > >>> > >> > >> test: > >> [junit] Running > >> org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge > >> [junit] java.lang.NoSuchMethodError: > >> org.apache.hadoop.security.UserGroupInformation.getCurrentUser() > >> Lorg/apache/hadoop/security/UserGroupInformation; > >> [junit] at > >> org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S > >> $Server.<init>(HadoopThriftAuthBridge20S.java:330) > >> [junit] at > >> org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge > >> $MyHadoopThriftAuthBridge20S$Server.<init> > >> (TestHadoop20SAuthBridge.java:53) > >> [junit] at > >> org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge > >> $MyHadoopThriftAuthBridge20S.createServer > >> (TestHadoop20SAuthBridge.java:48) > >> [junit] at > >> org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge > >> $MyHadoopThriftAuthBridge20S.createServer > >> (TestHadoop20SAuthBridge.java:43) > >> [junit] at > >> org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore > >> (HiveMetaStore.java:2973) > >> [junit] at > >> org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge$1.run > >> (TestHadoop20SAuthBridge.java:90) > >> [junit] at java.lang.Thread.run(Thread.java:619) > >> [junit] Running > >> org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge > >> [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec > >> [junit] Test org.apache.hadoop.hive.thrift.TestHadoop20SAuthBridge > >> FAILED (crashed) > >> > >> BUILD FAILED > >> < > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build.xml > >> >:220: > >> The following error occurred while executing this line: > >> < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build-common.xml > >> >:457: > >> The following error occurred while executing this line: > >> < > >> > https://hudson.apache.org/hudson/job/Hive-trunk-h0.20/ws/hive/build-common.xml > >> >:454: > >> Tests failed! > >> > >> Total time: 3 minutes 12 seconds > >> Archiving artifacts > >> Recording test results > >> > >> >
