Interesting. So two things: 1) As you correctly deduced, we switched to producing snapshots from the Gradle build, which doesn't include parent pom's. As a result, the parent pom snapshot you're referencing is stale. We should probably remove it from the repository to prevent accidental usage until we get this sorted out.
2) Depending on the parent pom directly isn't an intentionally supported scenario, but your use-case seems valid: successful execution on Dataflow requires specific version sets for Google Cloud dependencies. And those dependency versions are conveniently defined as properties in the beam-parent pom <https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/beam-parent-2.5.0-20180408.070218-37.pom> . Note that simply generating parent pom's from Gradle wouldn't fix this; we need to explicitly generate these version properties in the pom file. We could publish a special GCP parent pom just for this purpose. Although forcing a parent relationship seems unnecessary; is there a better convention within Maven for importing a set property values? On Wed, Apr 25, 2018 at 4:37 PM Eric Beach <[email protected]> wrote: > If I run $ mvn clean compile exec:java ... on my project, I get the > following error stack trace: > > java.lang.reflect.InvocationTargetException > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294) > > at java.lang.Thread.run(Thread.java:748) > > Caused by: java.lang.NoClassDefFoundError: > com/google/api/services/clouddebugger/v2/CloudDebugger > > at java.lang.Class.getDeclaredMethods0(Native Method) > > at java.lang.Class.privateGetDeclaredMethods(Class.java:2703) > > at java.lang.Class.getDeclaredMethod(Class.java:2130) > > at > org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:206) > > at > org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162) > > at > org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55) > > at org.apache.beam.sdk.Pipeline.create(Pipeline.java:150) > > at > com.google.cloud.pontem.CloudSpannerDatabaseBackup.main(CloudSpannerDatabaseBackup.java:359) > > ... 6 more > > Caused by: java.lang.ClassNotFoundException: > com.google.api.services.clouddebugger.v2.CloudDebugger > > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > > > When I dig deeper, the root problem seems to be that my project is pulling > in v2-rev8-1.22.0 instead of v2-rev233-1.23.0 (changing commit here > <https://github.com/apache/beam/commit/66fcf4bff19cda5831465ccedbbaa6fbaa648234#diff-600376dffeb79835ede4a0b285078036> > ). > > In fact, all the version changes in this commit > <https://github.com/apache/beam/commit/66fcf4bff19cda5831465ccedbbaa6fbaa648234#diff-600376dffeb79835ede4a0b285078036> > are missing. > > My project's pom.xml contains the following: > > <parent> > > <artifactId>beam-examples-parent</artifactId> > > <groupId>org.apache.beam</groupId> > > <version>2.5.0-SNAPSHOT</version> > > </parent> > > <modelVersion>4.0.0</modelVersion> > > > <groupId>com.google.cloud</groupId> > > <artifactId>xyz</artifactId> > > > <repositories> > > <repository> > > <id>apache.snapshots</id> > > <name>Apache Development Snapshot Repository</name> > > <url> > https://repository.apache.org/content/repositories/snapshots/</url> > > <releases> > > <enabled>false</enabled> > > </releases> > > <snapshots> > > <enabled>true</enabled> > > </snapshots> > > </repository> > > </repositories> > > My first round of attempts to debug included removing ~/.m2/ and a number > of other techniques. > > When I look at > https://repository.apache.org/content/repositories/snapshots, I notice > that beam-parent with 2.5.0-SNAPSHOT > <https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/> > is > very outdated and so all the version numbers inherited from the POM are > wrong (see > https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/). > If you look at > https://repository.apache.org/content/repositories/snapshots/org/apache/beam/ > you will see that the beam-parent is significantly old. > > So, it seems imperative that the snapshot be rebuilt before the finalized > 2.5.0 version is set or any projects depending on the Beam parent will > break. (I depend upon the Beam parent because trying to keep the versions > of all the different Google projects in sync is a total nightmare). > > > On Wed, Apr 25, 2018 at 6:58 PM Chamikara Jayalath <[email protected]> > wrote: > >> Ccing Eric for providing more context. >> >> Thanks, >> Cham >> >> On Wed, Apr 25, 2018 at 3:38 PM Scott Wegner <[email protected]> wrote: >> >>> Do you have any more context on how they were using the parent pom? In >>> the Gradle build, instead of using a parent pom hierarchy we are embedding >>> the complete set of dependencies and metadata in each generated pom. They >>> should be functionally equivalent. >>> >>> On Wed, Apr 25, 2018 at 3:09 PM Chamikara Jayalath <[email protected]> >>> wrote: >>> >>>> At least one user was depending on 'beam-parent' and ran into issues >>>> due to dependency update https://github.com/apache/beam/pull/5046 not >>>> being reflected there. I'm not sure if this is a usage pattern that we >>>> should discourage. >>>> >>>> Thanks, >>>> Cham >>>> >>>> >>>> On Wed, Apr 25, 2018 at 2:36 PM Alan Myrvold <[email protected]> >>>> wrote: >>>> >>>>> The gradle build is currently only generating pom files when there is >>>>> a jar file. >>>>> What are these parent poms used for? Do all the parent poms need to be >>>>> generated or just the top level beam-parent? >>>>> >>>>> On Wed, Apr 25, 2018 at 2:07 PM Chamikara Jayalath < >>>>> [email protected]> wrote: >>>>> >>>>>> Thanks. Should this be a 2.5.0 blocker ? >>>>>> >>>>>> On Wed, Apr 25, 2018 at 2:05 PM Alan Myrvold <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> I think this corresponds with the move from maven to gradle snapshot >>>>>>> releases. >>>>>>> Issue logged as https://issues.apache.org/jira/browse/BEAM-4170 >>>>>>> >>>>>>> On Wed, Apr 25, 2018 at 1:57 PM Chamikara Jayalath < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Seems like we are not building some of the SNAPSHOT artifacts since >>>>>>>> 4/9/2018. >>>>>>>> For example: >>>>>>>> >>>>>>>> >>>>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/ >>>>>>>> >>>>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-examples-parent/2.5.0-SNAPSHOT/ >>>>>>>> >>>>>>>> Users who depend on these artifacts are getting outdated >>>>>>>> dependencies. >>>>>>>> >>>>>>>> Any idea why ? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Cham >>>>>>>> >>>>>>> -- >>> >>> >>> Got feedback? http://go/swegner-feedback >>> >> -- Got feedback? http://go/swegner-feedback
