Thanks Scott and Ben. This makes sense, and of course the same error happens when I tried beam-parent instead of beam-examples-parent.
There is a big benefit for supporting explicit dependence on parent (either beam-examples-parent or beam-parent). In the project I'm involved in, I have Dataflow API (to query metrics for completed jobs), Google Cloud Storage, and Cloud Spanner (to execute queries outside of Beam). On Wed, Apr 25, 2018 at 7:55 PM Scott Wegner <sweg...@google.com> wrote: > Interesting. So two things: > > 1) As you correctly deduced, we switched to producing snapshots from the > Gradle build, which doesn't include parent pom's. As a result, the parent > pom snapshot you're referencing is stale. We should probably remove it from > the repository to prevent accidental usage until we get this sorted out. > > 2) Depending on the parent pom directly isn't an intentionally supported > scenario, but your use-case seems valid: successful execution on Dataflow > requires specific version sets for Google Cloud dependencies. And those > dependency versions are conveniently defined as properties in the beam-parent > pom > <https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/beam-parent-2.5.0-20180408.070218-37.pom> > . > > Note that simply generating parent pom's from Gradle wouldn't fix this; we > need to explicitly generate these version properties in the pom file. > > We could publish a special GCP parent pom just for this purpose. Although > forcing a parent relationship seems unnecessary; is there a better > convention within Maven for importing a set property values? > > On Wed, Apr 25, 2018 at 4:37 PM Eric Beach <ebe...@google.com> wrote: > >> If I run $ mvn clean compile exec:java ... on my project, I get the >> following error stack trace: >> >> java.lang.reflect.InvocationTargetException >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:498) >> >> at >> org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294) >> >> at java.lang.Thread.run(Thread.java:748) >> >> Caused by: java.lang.NoClassDefFoundError: >> com/google/api/services/clouddebugger/v2/CloudDebugger >> >> at java.lang.Class.getDeclaredMethods0(Native Method) >> >> at java.lang.Class.privateGetDeclaredMethods(Class.java:2703) >> >> at java.lang.Class.getDeclaredMethod(Class.java:2130) >> >> at >> org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:206) >> >> at >> org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162) >> >> at >> org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55) >> >> at org.apache.beam.sdk.Pipeline.create(Pipeline.java:150) >> >> at >> com.google.cloud.pontem.CloudSpannerDatabaseBackup.main(CloudSpannerDatabaseBackup.java:359) >> >> ... 6 more >> >> Caused by: java.lang.ClassNotFoundException: >> com.google.api.services.clouddebugger.v2.CloudDebugger >> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> >> >> When I dig deeper, the root problem seems to be that my project is >> pulling in v2-rev8-1.22.0 instead of v2-rev233-1.23.0 (changing commit >> here >> <https://github.com/apache/beam/commit/66fcf4bff19cda5831465ccedbbaa6fbaa648234#diff-600376dffeb79835ede4a0b285078036> >> ). >> >> In fact, all the version changes in this commit >> <https://github.com/apache/beam/commit/66fcf4bff19cda5831465ccedbbaa6fbaa648234#diff-600376dffeb79835ede4a0b285078036> >> are missing. >> >> My project's pom.xml contains the following: >> >> <parent> >> >> <artifactId>beam-examples-parent</artifactId> >> >> <groupId>org.apache.beam</groupId> >> >> <version>2.5.0-SNAPSHOT</version> >> >> </parent> >> >> <modelVersion>4.0.0</modelVersion> >> >> >> <groupId>com.google.cloud</groupId> >> >> <artifactId>xyz</artifactId> >> >> >> <repositories> >> >> <repository> >> >> <id>apache.snapshots</id> >> >> <name>Apache Development Snapshot Repository</name> >> >> <url> >> https://repository.apache.org/content/repositories/snapshots/</url> >> >> <releases> >> >> <enabled>false</enabled> >> >> </releases> >> >> <snapshots> >> >> <enabled>true</enabled> >> >> </snapshots> >> >> </repository> >> >> </repositories> >> >> My first round of attempts to debug included removing ~/.m2/ and a number >> of other techniques. >> >> When I look at >> https://repository.apache.org/content/repositories/snapshots, I notice >> that beam-parent with 2.5.0-SNAPSHOT >> <https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/> >> is >> very outdated and so all the version numbers inherited from the POM are >> wrong (see >> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/). >> If you look at >> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/ >> you will see that the beam-parent is significantly old. >> >> So, it seems imperative that the snapshot be rebuilt before the finalized >> 2.5.0 version is set or any projects depending on the Beam parent will >> break. (I depend upon the Beam parent because trying to keep the versions >> of all the different Google projects in sync is a total nightmare). >> >> >> On Wed, Apr 25, 2018 at 6:58 PM Chamikara Jayalath <chamik...@google.com> >> wrote: >> >>> Ccing Eric for providing more context. >>> >>> Thanks, >>> Cham >>> >>> On Wed, Apr 25, 2018 at 3:38 PM Scott Wegner <sweg...@google.com> wrote: >>> >>>> Do you have any more context on how they were using the parent pom? In >>>> the Gradle build, instead of using a parent pom hierarchy we are embedding >>>> the complete set of dependencies and metadata in each generated pom. They >>>> should be functionally equivalent. >>>> >>>> On Wed, Apr 25, 2018 at 3:09 PM Chamikara Jayalath < >>>> chamik...@google.com> wrote: >>>> >>>>> At least one user was depending on 'beam-parent' and ran into issues >>>>> due to dependency update https://github.com/apache/beam/pull/5046 not >>>>> being reflected there. I'm not sure if this is a usage pattern that we >>>>> should discourage. >>>>> >>>>> Thanks, >>>>> Cham >>>>> >>>>> >>>>> On Wed, Apr 25, 2018 at 2:36 PM Alan Myrvold <amyrv...@google.com> >>>>> wrote: >>>>> >>>>>> The gradle build is currently only generating pom files when there is >>>>>> a jar file. >>>>>> What are these parent poms used for? Do all the parent poms need to >>>>>> be generated or just the top level beam-parent? >>>>>> >>>>>> On Wed, Apr 25, 2018 at 2:07 PM Chamikara Jayalath < >>>>>> chamik...@google.com> wrote: >>>>>> >>>>>>> Thanks. Should this be a 2.5.0 blocker ? >>>>>>> >>>>>>> On Wed, Apr 25, 2018 at 2:05 PM Alan Myrvold <amyrv...@google.com> >>>>>>> wrote: >>>>>>> >>>>>>>> I think this corresponds with the move from maven to gradle >>>>>>>> snapshot releases. >>>>>>>> Issue logged as https://issues.apache.org/jira/browse/BEAM-4170 >>>>>>>> >>>>>>>> On Wed, Apr 25, 2018 at 1:57 PM Chamikara Jayalath < >>>>>>>> chamik...@google.com> wrote: >>>>>>>> >>>>>>>>> Seems like we are not building some of the SNAPSHOT artifacts >>>>>>>>> since 4/9/2018. >>>>>>>>> For example: >>>>>>>>> >>>>>>>>> >>>>>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/ >>>>>>>>> >>>>>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-examples-parent/2.5.0-SNAPSHOT/ >>>>>>>>> >>>>>>>>> Users who depend on these artifacts are getting outdated >>>>>>>>> dependencies. >>>>>>>>> >>>>>>>>> Any idea why ? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Cham >>>>>>>>> >>>>>>>> -- >>>> >>>> >>>> Got feedback? http://go/swegner-feedback >>>> >>> -- > > > Got feedback? http://go/swegner-feedback >
