Hi, Kenn, Thanks for the quick response! Just FYI, I downgrade to 2.8.0 and the same project runs fine now. Will update the ticket accordingly.
-Yi On Wed, Jan 30, 2019 at 9:11 PM Kenneth Knowles <[email protected]> wrote: > Wow, thanks for the great report. Your configuration looks good to me. I > filed https://issues.apache.org/jira/browse/BEAM-6558 to figure this out. > > Kenn > > On Wed, Jan 30, 2019 at 7:01 PM Yi Pan <[email protected]> wrote: > >> Hi, all, >> >> Newbie here trying to figure out how to use published >> beam-sdks-java-extensions-sql-2.9.0 in my own project. >> >> I tried to create a gradle project to use BeamSQL sdk libraries. Here is >> the build.gradle I have: >> {code} >> plugins { >> id 'java' >> } >> >> group 'com.mycompany.myproject' >> version '1.0-SNAPSHOT' >> >> sourceCompatibility = 1.8 >> >> repositories { >> mavenCentral() >> } >> >> apply plugin: 'java' >> >> sourceSets.main.java.srcDirs = [ >> 'src/main/java' >> ] >> >> dependencies { >> compile 'org.apache.beam:beam-sdks-java-core:2.9.0' >> compile 'org.apache.beam:beam-sdks-java-extensions-sql:2.9.0' >> compile 'com.google.code.findbugs:jsr305:3.0.2' >> runtime 'org.apache.beam:beam-runners-direct-java:2.9.0' >> testCompile group: 'junit', name: 'junit', version: '4.12' >> } >> >> // Run basic SQL example >> task runBasicExample(type: JavaExec) { >> description = "Run basic SQL example" >> main = "com.mycompany.myproject.streamsql.examples.BeamSQLExample" >> classpath = sourceSets.main.runtimeClasspath >> args = ["--runner=DirectRunner"] >> println classpath.getAsPath() >> println args >> } >> {code} >> >> The example BeamSQLExample is just copied from >> https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/example/BeamSqlExample.java >> . >> >> I was able to compile the example in JDK8. However, when I tried to run >> it, I hit the following exception: >> {code} >> Exception in thread "main" java.util.ServiceConfigurationError: >> org.apache.beam.sdk.extensions.sql.impl.udf.BeamBuiltinFunctionProvider: >> Provider org.apache.beam.sdk.extensions.sql.impl.udf.BuiltinStringFunctions >> could not be instantiated >> at java.util.ServiceLoader.fail(ServiceLoader.java:232) >> at java.util.ServiceLoader.access$100(ServiceLoader.java:185) >> at >> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) >> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) >> at java.util.ServiceLoader$1.next(ServiceLoader.java:480) >> at >> org.apache.beam.sdk.extensions.sql.impl.BeamSqlEnv.loadBeamBuiltinFunctions(BeamSqlEnv.java:128) >> at >> org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:94) >> at >> org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:76) >> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537) >> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471) >> at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357) >> at >> com.mycompany.myprject.streamsql.examples.BeamSQLExample.main(BeamSQLExample.java:72) >> Caused by: java.lang.NoClassDefFoundError: >> org/apache/commons/codec/DecoderException >> at java.lang.Class.getDeclaredConstructors0(Native Method) >> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) >> at java.lang.Class.getConstructor0(Class.java:3075) >> at java.lang.Class.newInstance(Class.java:412) >> at >> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) >> ... 9 more >> Caused by: java.lang.ClassNotFoundException: >> org.apache.commons.codec.DecoderException >> at java.net.URLClassLoader.findClass(URLClassLoader.java:382) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> ... 14 more >> {code} >> >> When I traced into the code, it turns out that the failure occurs when >> the classloader tries to get the default constructor w/o any parameters for >> BuiltinStringFunctions.class. I double checked my local gradle cache and >> confirmed that the jar is there: >> {code} >> SJCMAC91THJHD4:beamsql-demo ypan$ ls -l >> ~/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-sdks-java-extensions-sql/2.9.0/67e7675519859ff332619c4c6ea5d26a505dbd50/beam-sdks-java-extensions-sql-2.9.0.jar >> -rw-r--r-- 1 ypan 192360288 12761025 Jan 29 18:02 >> /Users/ypan/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-sdks-java-extensions-sql/2.9.0/67e7675519859ff332619c4c6ea5d26a505dbd50/beam-sdks-java-extensions-sql-2.9.0.jar >> {code} >> >> I also have tried to compile the sql sdk libraries in Beam's source repo >> and just copying over the generated class files over to my runtime >> classpath. Apparently, that did not work well since there are many shadowed >> libraries now need explicit declaring dependencies. I tried to search for >> an example of maven or gradle dependency declaration that users will use to >> create their own application using the SQL SDKs but was not lucky enough to >> find one. >> >> Given the amount of debugging and searching effort I have put it there, I >> think that it is reasonable to ask the question to the community. Can >> someone shed some light on what went wrong in my build project? >> >> Thanks! >> >> -Yi >> >>
