- I could not reproduce the NoClassDefFoundError Artem reported with hadoop 2.6.0. I changed bin/oozie-setup.sh so that it prints loaded classes (passed -verbose:class to the JVM). Interestingly, org/apache/commons/io/ Charsets was not even loaded.
However, when I tried again with hadoop.version *2.7.5* , it resulted in the error mentioned: java.lang.NoClassDefFoundError: org/apache/commons/io/Charsets at org.apache.hadoop.security.Credentials.<clinit>(Credentials.java:222) at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:334) at org.apache.oozie.service.HadoopAccessorService.createJobConf( HadoopAccessorService.java:349) at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:171) at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:67) Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.Charsets It looks like https://issues.apache.org/jira/browse/OOZIE-2396 Some more info: - ./lib/commons-io-2.1.jar does not contain org/apache/commons/io/Charsets. - related hadoop 2.7.0 jira: https://issues.apache.org/jira/browse/HADOOP-11389 - related hadoop code: https://github.com/apache/hadoop/blame/branch-2.7.0/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Credentials.java#L36 - HADOOP-13444 replaced org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets so it shall not be a problem on hadoop 2.8+ Note: Oozie 5.0.0-beta1 includes OOZIE-2888 that add updated commons-io to 2.4. That is why it is not a problem there. On Wed, Jan 31, 2018 at 1:18 AM, Satish Saley <satishsa...@apache.org> wrote: > 1. However, json2.js is referenced by webapp/index.html and it > is not found (not a blocker in my opinion). > > -> It will show only in firebug/developer tools. Not on UI. > > 2. [WARNING] Unable to process class com/sun/tools/jdi/ClassTypeImpl.class > in > JarAnalyzer File > > -> We can ignore the warning as build is going through > > 3. /user/oozie/share/lib/lib_20180130164518 > Exception in thread "main" java.lang.NoClassDefFoundError: > > I am not able to reproduce this. > > saley@easyfamily-lm on Tue Jan 30 at 11:44 AM > ~/src/oozie-4.3.1/distro/target/oozie-4.3.1-distro/oozie-4.3.1 > $ bin/oozie-setup.sh sharelib create -fs hdfs://localhost -concurrency 8 > -locallib oozie-sharelib-4.3.1.tar.gz > setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m" > log4j:WARN No appenders could be found for logger > (org.apache.hadoop.util.Shell). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for > more info. > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/Users/saley/src/oozie-4.3.1/distro/target/ > oozie-4.3.1-distro/oozie-4.3.1/lib/slf4j-log4j12-1.6.6.jar! > /org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/Users/saley/src/oozie-4.3.1/distro/target/ > oozie-4.3.1-distro/oozie-4.3.1/lib/slf4j-simple-1.6.6.jar!/org/slf4j/impl/ > StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > the destination path for sharelib is: > /user/saley/share/lib/lib_20180130114509 > Running 410 copy tasks on 8 threads > > > saley@easyfamily-lm on Tue Jan 30 at 11:45 AM > ~/src/oozie-4.3.1/distro/target/oozie-4.3.1-distro/oozie-4.3.1 > $ hdfs dfs -ls > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/Users/saley/hadoop-2.6.3/share/hadoop/common/lib/ > slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/Users/saley/src/tez/tez-dist/target/tez-0.8.5- > minimal/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/ > StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > 18/01/30 11:45:27 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > Found 1 items > drwxr-xr-x - saley supergroup 0 2018-01-30 11:45 share > > > > > On Tue, Jan 30, 2018 at 8:52 AM, Artem Ervits <artemerv...@gmail.com> > wrote: > > > yes I realized that after hitting send, I encountered the following > issues > > but it may just be me > > > > hitting https://issues.apache.org/jira/browse/BCEL-173 when compiling > with > > > > bin/mkdistro.sh -Phadoop-2 -DskipTests -Puber -DtargetJavaVersion=1.8 > > -DjavaVersion=1.8 -Dhadoop.version=2.7.5 > > > > [WARNING] Unable to process class com/sun/tools/jdi/ClassTypeImpl.class > in > > JarAnalyzer File > > /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-0.b14.el7_4. > > x86_64/jre/../lib/tools.jar > > org.apache.bcel.classfile.ClassFormatException: Invalid byte tag in > > constant pool: 18 > > at org.apache.bcel.classfile.Constant.readConstant(Constant.java:146) > > at org.apache.bcel.classfile.ConstantPool.<init>(ConstantPool.java:67) > > at > > org.apache.bcel.classfile.ClassParser.readConstantPool( > > ClassParser.java:222) > > at org.apache.bcel.classfile.ClassParser.parse(ClassParser.java:136) > > at > > org.apache.maven.shared.jar.classes.JarClassesAnalysis. > > analyze(JarClassesAnalysis.java:92) > > at > > org.apache.maven.report.projectinfo.dependencies.Dependencies. > > getJarDependencyDetails(Dependencies.java:276) > > at > > org.apache.maven.report.projectinfo.dependencies. > > renderer.DependenciesRenderer.hasSealed(DependenciesRenderer.java:1374) > > at > > org.apache.maven.report.projectinfo.dependencies. > > renderer.DependenciesRenderer.renderSectionDependencyFileDet > > ails(DependenciesRenderer.java:544) > > at > > org.apache.maven.report.projectinfo.dependencies. > > renderer.DependenciesRenderer.renderBody(DependenciesRenderer.java:271) > > at > > org.apache.maven.reporting.AbstractMavenReportRenderer.render( > > AbstractMavenReportRenderer.java:79) > > at > > org.apache.maven.report.projectinfo.DependenciesReport.executeReport( > > DependenciesReport.java:206) > > at > > org.apache.maven.reporting.AbstractMavenReport.generate( > > AbstractMavenReport.java:190) > > at > > org.apache.maven.report.projectinfo.AbstractProjectInfoReport.execute( > > AbstractProjectInfoReport.java:202) > > at > > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo( > > DefaultBuildPluginManager.java:101) > > at > > org.apache.maven.lifecycle.internal.MojoExecutor.execute( > > MojoExecutor.java:209) > > at > > org.apache.maven.lifecycle.internal.MojoExecutor.execute( > > MojoExecutor.java:153) > > at > > org.apache.maven.lifecycle.internal.MojoExecutor.execute( > > MojoExecutor.java:145) > > at > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject( > > LifecycleModuleBuilder.java:84) > > at > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject( > > LifecycleModuleBuilder.java:59) > > at > > org.apache.maven.lifecycle.internal.LifecycleStarter. > singleThreadedBuild( > > LifecycleStarter.java:183) > > at > > org.apache.maven.lifecycle.internal.LifecycleStarter. > > execute(LifecycleStarter.java:161) > > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320) > > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) > > at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537) > > at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196) > > at org.apache.maven.cli.MavenCli.main(MavenCli.java:141) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java: > > 62) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke( > > DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > org.codehaus.plexus.classworlds.launcher.Launcher. > > launchEnhanced(Launcher.java:290) > > at > > org.codehaus.plexus.classworlds.launcher.Launcher. > > launch(Launcher.java:230) > > at > > org.codehaus.plexus.classworlds.launcher.Launcher. > > mainWithExitCode(Launcher.java:414) > > at org.codehaus.plexus.classworlds.launcher.Launcher. > > main(Launcher.java:357) > > > > Then after a successful build getting the following version string > > > > Oozie distro created, DATE[2018.01.30-16:22:57GMT] VC-REV[unavailable], > > available at [/tmp/oozie-4.3.1/distro/target] > > > > we can now pull VC-REV with https://issues.apache.org/ > > jira/browse/OOZIE-3152 > > > > finally saw the following > > > > [oozie@hadoop oozie-4.3.1]$ bin/oozie-setup.sh sharelib create -fs > > hdfs://localhost:9000 -concurrency 8 -locallib > oozie-sharelib-4.3.1.tar.gz > > setting CATALINA_OPTS="$CATALINA_OPTS -Xmx1024m" > > log4j:WARN No appenders could be found for logger > > (org.apache.hadoop.util.Shell). > > log4j:WARN Please initialize the log4j system properly. > > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for > > more info. > > SLF4J: Class path contains multiple SLF4J bindings. > > SLF4J: Found binding in > > [jar:file:/opt/oozie/oozie-4.3.1/distro/target/oozie-4.3.1- > > distro/oozie-4.3.1/lib/slf4j-log4j12-1.6.6.jar!/org/slf4j/ > > impl/StaticLoggerBinder.class] > > SLF4J: Found binding in > > [jar:file:/opt/oozie/oozie-4.3.1/distro/target/oozie-4.3.1- > > distro/oozie-4.3.1/lib/slf4j-simple-1.6.6.jar!/org/slf4j/ > > impl/StaticLoggerBinder.class] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > > explanation. > > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > > the destination path for sharelib is: > > /user/oozie/share/lib/lib_20180130164518 > > Exception in thread "main" java.lang.NoClassDefFoundError: > > org/apache/commons/io/Charsets > > at org.apache.hadoop.security.Credentials.<clinit>(Credentials.java:222) > > at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:334) > > at > > org.apache.oozie.service.HadoopAccessorService.createJobConf( > > HadoopAccessorService.java:349) > > at org.apache.oozie.tools.OozieSharelibCLI.run( > OozieSharelibCLI.java:171) > > at org.apache.oozie.tools.OozieSharelibCLI.main( > OozieSharelibCLI.java:67) > > Caused by: java.lang.ClassNotFoundException: > > org.apache.commons.io.Charsets > > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > > ... 5 more > > > > none were blockers as I was able to get the following output > > > > $ curl http://localhost:11000/oozie/v2/admin/status > > {"systemMode":"NORMAL"} > > > > > > On Tue, Jan 30, 2018 at 10:45 AM, Rohini Palaniswamy <roh...@apache.org> > > wrote: > > > > > > the server directory is missing altogether now. > > > That is as expected. It is supposed to be only in 5.0 > > > > > > On Mon, Jan 29, 2018 at 12:47 PM, Artem Ervits <artemerv...@gmail.com> > > > wrote: > > > > > > > the server directory is missing altogether now. > > > > > > > > On Mon, Jan 29, 2018 at 6:59 AM, Attila Sasvari < > asasv...@cloudera.com > > > > > > > wrote: > > > > > > > > > +1 (non-binding) > > > > > > > > > > - Verified hash, signature files, rat report > > > > > - Built Oozie distro with hadoop.version set to 2.6.0 using the > uber, > > > > > hadoop-2, spark-1 profiles > > > > > - Submitted example workflows/coordinators/bundles (except hive > > related > > > > > ones) on a hadoop 2.6.0 in pseudo distributed mode > > > > > - Verified Tomcat version was 6.0.53 > > > > > - Clicked through the web UI, log retrieval worked, job DAG-s were > > > > > rendered, version was correct. > > > > > - Verified instrumentation, SLA pages; they are now HTML based as > > > > expected > > > > > (OOZIE-2533). However, json2.js is referenced by webapp/index.html > > and > > > it > > > > > is not found (not a blocker in my opinion). > > > > > - Enabled SLA, re-submitted sla coordinator example, and retrieved > > SLA > > > > > details for the "one-op-wf" application, graph was displayed > > properly. > > > > > > > > > > I used JDK 1.8.0_121 > > > > > > > > > > On Sun, Jan 28, 2018 at 7:09 AM, Satish Saley < > > satishsa...@apache.org> > > > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I have created a build for Oozie 4.3.1, candidate 3. > > > > > > > > > > > > It includes all the changes decided earlier except OOZIE-3018, > > > > OOZIE-2771 > > > > > > and > > > > > > OOZIE-3072. OOZIE-3018 makes changes to SchemaCheckerService > which > > we > > > > are > > > > > > not picking. > > > > > > OOZIE-2771 makes changes to file created in OOZIE-2666. > > > > > > OOZIE-3072 makes changes to files created in OOZIE-1770. > > > > > > > > > > > > > > > > > > Keys to verify the signature of the release artifact are > available > > at > > > > > > http://www.apache.org/dist/oozie/KEYS > > > > > > > > > > > > Please download, test, and try it out: > > > > > > > > > > > > http://people.apache.org/~satishsaley/oozie-4.3.1-rc3/ > > > > > > > > > > > > > > > > > > The release, md5 signature, gpg signature, and rat report can be > > > found > > > > at > > > > > > the above address. > > > > > > > > > > > > Vote closes on 01/31/2018 11:59 PM PT. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > -- > > > > > Attila Sasvari > > > > > Software Engineer > > > > > <http://www.cloudera.com/> > > > > > > > > > > > > > > > -- -- Attila Sasvari Software Engineer <http://www.cloudera.com/>
