Re: Apache Beam 2.50.0 - org.apache.beam.sdk.options.MemoryMonitorOptions ClassDef Not found

2023-10-14 Thread Bruno Volpato via dev 
Hi Jaymik,

This class (MemoryMonitorOptions) was indeed introduced in Beam 2.50.0.

My guess is that somehow dependencies were upgraded only partially (for
example, just the Runner module, but not Core).
A good way to make sure that all dependencies are upgraded correctly is to
use a BOM (Bill of Materials) and omit pinning a specific  for
every Beam artifact.

Please take a look at
https://beam.apache.org/blog/managing-beam-dependencies-in-java/ and see if
it helps.

In case it still doesn't work, running "mvn dependency:tree" might help
understanding which dependencies are being pulled.

Best,
Bruno



On Thu, Oct 12, 2023 at 5:50 PM Deliwala, Jaymik H. via dev <
dev@beam.apache.org> wrote:

> Hello Team
>
>
>
> Greetings!!
>
>
>
> As part of upgrading our Dataflow - Apache beam version from 2.46.0 to
> 2.49.0/2.50.0, we are able to compile the mvn package successfully.
>
> However, while running the compile exec command, we are getting an error
> as below - *org.apache.beam.sdk.options.MemoryMonitorOptions -
> NoClassefFoundError*
>
>
>
>
> 2023-10-12T10:39:17.9662373Z java.lang.NoClassDefFoundError: 
> org/apache/beam/sdk/options/MemoryMonitorOptions
>
>
> 2023-10-12T10:39:17.9665631Z at java.lang.ClassLoader.defineClass1 
> (Native Method)
>
>
> 2023-10-12T10:39:17.9668538Z at java.lang.ClassLoader.defineClass 
> (ClassLoader.java:1022)
>
>
> 2023-10-12T10:39:17.9670605Z at 
> java.security.SecureClassLoader.defineClass (SecureClassLoader.java:174)
>
>
> 2023-10-12T10:39:17.9672586Z at java.net.URLClassLoader.defineClass 
> (URLClassLoader.java:555)
>
>
> 2023-10-12T10:39:17.9674516Z at java.net.URLClassLoader$1.run 
> (URLClassLoader.java:458)
>
>
> 2023-10-12T10:39:17.9676939Z at java.net.URLClassLoader$1.run 
> (URLClassLoader.java:452)
>
>
> 2023-10-12T10:39:17.9678816Z at 
> java.security.AccessController.doPrivileged (Native Method)
>
>
> 2023-10-12T10:39:17.9690248Z at java.net.URLClassLoader.findClass 
> (URLClassLoader.java:451)
>
>
> 2023-10-12T10:39:17.9692252Z at java.lang.ClassLoader.loadClass 
> (ClassLoader.java:594)
>
>
> 2023-10-12T10:39:17.9694175Z at java.lang.ClassLoader.loadClass 
> (ClassLoader.java:527)
>
>
> 2023-10-12T10:39:17.9696026Z at java.lang.ClassLoader.defineClass1 
> (Native Method)
>
>
> 2023-10-12T10:39:17.961Z at java.lang.ClassLoader.defineClass 
> (ClassLoader.java:1022)
>
>
> 2023-10-12T10:39:17.9704965Z at 
> java.security.SecureClassLoader.defineClass (SecureClassLoader.java:174)
>
>
> 2023-10-12T10:39:17.9706908Z at java.net.URLClassLoader.defineClass 
> (URLClassLoader.java:555)
>
>
> 2023-10-12T10:39:17.9708814Z at java.net.URLClassLoader$1.run 
> (URLClassLoader.java:458)
>
>
> 2023-10-12T10:39:17.9710696Z at java.net.URLClassLoader$1.run 
> (URLClassLoader.java:452)
>
>
> 2023-10-12T10:39:17.9712552Z at 
> java.security.AccessController.doPrivileged (Native Method)
>
>
> 2023-10-12T10:39:17.9714469Z at java.net.URLClassLoader.findClass 
> (URLClassLoader.java:451)
>
>
> 2023-10-12T10:39:17.9716698Z at java.lang.ClassLoader.loadClass 
> (ClassLoader.java:594)
>
>
> 2023-10-12T10:39:17.9718622Z at java.lang.ClassLoader.loadClass 
> (ClassLoader.java:527)
>
>
> 2023-10-12T10:39:17.9720635Z at 
> org.apache.beam.runners.dataflow.DataflowPipelineRegistrar$Options.getPipelineOptions
>  (DataflowPipelineRegistrar.java:40)
>
>
> 2023-10-12T10:39:17.9722611Z at 
> org.apache.beam.sdk.options.PipelineOptionsFactory$Cache.initializeRegistry 
> (PipelineOptionsFactory.java:2090)
>
>
> 2023-10-12T10:39:17.9724487Z at 
> org.apache.beam.sdk.options.PipelineOptionsFactory$Cache. 
> (PipelineOptionsFactory.java:2083)
>
>
> 2023-10-12T10:39:17.9726421Z at 
> org.apache.beam.sdk.options.PipelineOptionsFactory$Cache. 
> (PipelineOptionsFactory.java:2047)
>
>
> 2023-10-12T10:39:17.9728327Z at 
> org.apache.beam.sdk.options.PipelineOptionsFactory.resetCache 
> (PipelineOptionsFactory.java:581)
>
>
> 2023-10-12T10:39:17.9730230Z at 
> org.apache.beam.sdk.options.PipelineOptionsFactory. 
> (PipelineOptionsFactory.java:547)
>
>
> 2023-10-12T10:39:17.9732113Z at cio.mmt.PubSubTopicsToBigQuery.main 
> (PubSubTopicsToBigQuery.java:55)
>
>
> 2023-10-12T10:39:17.9734009Z at org.codehaus.mojo.exec.ExecJavaMojo$1.run 
> (ExecJavaMojo.java:254)
>
> 2023-10-12T10:39:17.9743608Z at java.lang.Thread.run (Thread.java:829)
>
>
> 2023-10-12T10:39:17.9745720Z Caused by: java.lang.ClassNotFoundException: 
> org.apache.beam.sdk.options.MemoryMonitorOptions
>
>
> 2023-10-12T10:39:17.9748376Z at java.net.URLClassLoader.findClass 
> (URLClassLoader.java:476)
>
>
> 2023-10-12T10:39:17.9750298Z at java.lang.ClassLoader.loadClass 
> (ClassLoader.java:594)
>
>
> 2023-10-12T10:39:17.9752205Z at java.lang.ClassLoader.loadClass 
> (ClassLoader.java:527)
>
>
> 2023-10-12T10:39:17.9754041Z at java.lang.ClassLoader.defineClass1 
> (Native Method)
>
>
> 2023-10-12T10:39:17.9

Apache Beam 2.50.0 - org.apache.beam.sdk.options.MemoryMonitorOptions ClassDef Not found

2023-10-12 Thread Deliwala, Jaymik H. via dev 
Hello Team

Greetings!!

As part of upgrading our Dataflow - Apache beam version from 2.46.0 to 
2.49.0/2.50.0, we are able to compile the mvn package successfully.
However, while running the compile exec command, we are getting an error as 
below - org.apache.beam.sdk.options.MemoryMonitorOptions - NoClassefFoundError

2023-10-12T10:39:17.9662373Z java.lang.NoClassDefFoundError: 
org/apache/beam/sdk/options/MemoryMonitorOptions
2023-10-12T10:39:17.9665631Z at java.lang.ClassLoader.defineClass1 (Native 
Method)
2023-10-12T10:39:17.9668538Z at java.lang.ClassLoader.defineClass 
(ClassLoader.java:1022)
2023-10-12T10:39:17.9670605Z at java.security.SecureClassLoader.defineClass 
(SecureClassLoader.java:174)
2023-10-12T10:39:17.9672586Z at java.net.URLClassLoader.defineClass 
(URLClassLoader.java:555)
2023-10-12T10:39:17.9674516Z at java.net.URLClassLoader$1.run 
(URLClassLoader.java:458)
2023-10-12T10:39:17.9676939Z at java.net.URLClassLoader$1.run 
(URLClassLoader.java:452)
2023-10-12T10:39:17.9678816Z at java.security.AccessController.doPrivileged 
(Native Method)
2023-10-12T10:39:17.9690248Z at java.net.URLClassLoader.findClass 
(URLClassLoader.java:451)
2023-10-12T10:39:17.9692252Z at java.lang.ClassLoader.loadClass 
(ClassLoader.java:594)
2023-10-12T10:39:17.9694175Z at java.lang.ClassLoader.loadClass 
(ClassLoader.java:527)
2023-10-12T10:39:17.9696026Z at java.lang.ClassLoader.defineClass1 (Native 
Method)
2023-10-12T10:39:17.961Z at java.lang.ClassLoader.defineClass 
(ClassLoader.java:1022)
2023-10-12T10:39:17.9704965Z at java.security.SecureClassLoader.defineClass 
(SecureClassLoader.java:174)
2023-10-12T10:39:17.9706908Z at java.net.URLClassLoader.defineClass 
(URLClassLoader.java:555)
2023-10-12T10:39:17.9708814Z at java.net.URLClassLoader$1.run 
(URLClassLoader.java:458)
2023-10-12T10:39:17.9710696Z at java.net.URLClassLoader$1.run 
(URLClassLoader.java:452)
2023-10-12T10:39:17.9712552Z at java.security.AccessController.doPrivileged 
(Native Method)
2023-10-12T10:39:17.9714469Z at java.net.URLClassLoader.findClass 
(URLClassLoader.java:451)
2023-10-12T10:39:17.9716698Z at java.lang.ClassLoader.loadClass 
(ClassLoader.java:594)
2023-10-12T10:39:17.9718622Z at java.lang.ClassLoader.loadClass 
(ClassLoader.java:527)
2023-10-12T10:39:17.9720635Z at 
org.apache.beam.runners.dataflow.DataflowPipelineRegistrar$Options.getPipelineOptions
 (DataflowPipelineRegistrar.java:40)
2023-10-12T10:39:17.9722611Z at 
org.apache.beam.sdk.options.PipelineOptionsFactory$Cache.initializeRegistry 
(PipelineOptionsFactory.java:2090)
2023-10-12T10:39:17.9724487Z at 
org.apache.beam.sdk.options.PipelineOptionsFactory$Cache. 
(PipelineOptionsFactory.java:2083)
2023-10-12T10:39:17.9726421Z at 
org.apache.beam.sdk.options.PipelineOptionsFactory$Cache. 
(PipelineOptionsFactory.java:2047)
2023-10-12T10:39:17.9728327Z at 
org.apache.beam.sdk.options.PipelineOptionsFactory.resetCache 
(PipelineOptionsFactory.java:581)
2023-10-12T10:39:17.9730230Z at 
org.apache.beam.sdk.options.PipelineOptionsFactory. 
(PipelineOptionsFactory.java:547)
2023-10-12T10:39:17.9732113Z at cio.mmt.PubSubTopicsToBigQuery.main 
(PubSubTopicsToBigQuery.java:55)
2023-10-12T10:39:17.9734009Z at org.codehaus.mojo.exec.ExecJavaMojo$1.run 
(ExecJavaMojo.java:254)
2023-10-12T10:39:17.9743608Z at java.lang.Thread.run (Thread.java:829)
2023-10-12T10:39:17.9745720Z Caused by: java.lang.ClassNotFoundException: 
org.apache.beam.sdk.options.MemoryMonitorOptions
2023-10-12T10:39:17.9748376Z at java.net.URLClassLoader.findClass 
(URLClassLoader.java:476)
2023-10-12T10:39:17.9750298Z at java.lang.ClassLoader.loadClass 
(ClassLoader.java:594)
2023-10-12T10:39:17.9752205Z at java.lang.ClassLoader.loadClass 
(ClassLoader.java:527)
2023-10-12T10:39:17.9754041Z at java.lang.ClassLoader.defineClass1 (Native 
Method)
2023-10-12T10:39:17.9756293Z at java.lang.ClassLoader.defineClass 
(ClassLoader.java:1022)
2023-10-12T10:39:17.9758225Z at java.security.SecureClassLoader.defineClass 
(SecureClassLoader.java:174)
2023-10-12T10:39:17.9760115Z at java.net.URLClassLoader.defineClass 
(URLClassLoader.java:555)
2023-10-12T10:39:17.9761982Z at java.net.URLClassLoader$1.run 
(URLClassLoader.java:458)
2023-10-12T10:39:17.9763915Z at java.net.URLClassLoader$1.run 
(URLClassLoader.java:452)
2023-10-12T10:39:17.9765721Z at java.security.AccessController.doPrivileged 
(Native Method)
2023-10-12T10:39:17.9767732Z at java.net.URLClassLoader.findClass 
(URLClassLoader.java:451)
2023-10-12T10:39:17.9769643Z at java.lang.ClassLoader.loadClass 
(ClassLoader.java:594)
2023-10-12T10:39:17.9771552Z at java.lang.ClassLoader.loadClass 
(ClassLoader.java:527)
2023-10-12T10:39:17.9821607Z at java.lang.ClassLoader.defineClass1 (Native 
Method)
2023-10-12T10:39:17.9824027Z at java.lang.ClassLoader.defineClass 
(ClassLoader.java:102