Step 1: Try to identify which artifact is supposed to contain nonapi.io.github.classgraph.utils.URLPathEncoder Your IDE helps this.
Step 2: Get dependency graph of your project (mvn dependency:tree or gradlew dependencies). On Thu, Jan 26, 2023 at 19:30 Lydian <lydia...@gmail.com> wrote: > Hi, > > I found that my beam app run is not very stable. In the same pod, it > sometimes failed with > ``` > java.lang.ClassNotFoundException: > nonapi.io.github.classgraph.utils.URLPathEncoder > ``` > During the same pod, if I retried the submission again, sometimes it would > just succeed. (Namely, no change on the dependencies, but the first couple > submit usually would just fail) > Google search shows me it could potentially related to `classgraph` [1] > But I'm still not sure how I can fix this issue. > > I'm using beam 2.41.0 with flink 1.14.5 and here's the full error: > ``` > 2023-01-27 00:23:58,808 ERROR > org.apache.beam.runners.jobsubmission.InMemoryJobService [] - > Encountered Unexpected Exception for Preparation > job_1c650c6a-aee3-44eb-9ab8-a2b18cbda6a5 > io.github.classgraph.ClassGraphException: Uncaught exception during scan > at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1570) > ~[beam-runner.jar:?] > at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1587) > ~[beam-runner.jar:?] > at > org.apache.beam.runners.core.construction.resources.ClasspathScanningResourcesDetector.detect(ClasspathScanningResourcesDetector.java:51) > ~[beam-runner.jar:?] > at > org.apache.beam.runners.core.construction.resources.PipelineResources.detectClassPathResourcesToStage(PipelineResources.java:57) > ~[beam-runner.jar:?] > at > org.apache.beam.runners.flink.FlinkJobInvoker.invokeWithExecutor(FlinkJobInvoker.java:83) > ~[beam-runner.jar:?] > at > org.apache.beam.runners.jobsubmission.JobInvoker.invoke(JobInvoker.java:48) > ~[beam-runner.jar:?] > at > org.apache.beam.runners.jobsubmission.InMemoryJobService.run(InMemoryJobService.java:246) > [beam-runner.jar:?] > at > org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:949) > [beam-runner.jar:?] > at > org.apache.beam.vendor.grpc.v1p43p2.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182) > [beam-runner.jar:?] > at > org.apache.beam.vendor.grpc.v1p43p2.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35) > [beam-runner.jar:?] > at > org.apache.beam.vendor.grpc.v1p43p2.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23) > [beam-runner.jar:?] > at > org.apache.beam.vendor.grpc.v1p43p2.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40) > [beam-runner.jar:?] > at > org.apache.beam.vendor.grpc.v1p43p2.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86) > [beam-runner.jar:?] > at > org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:340) > [beam-runner.jar:?] > at > org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:866) > [beam-runner.jar:?] > at > org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) > [beam-runner.jar:?] > at > org.apache.beam.vendor.grpc.v1p43p2.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) > [beam-runner.jar:?] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [?:1.8.0_342] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [?:1.8.0_342] > at java.lang.Thread.run(Thread.java:750) [?:1.8.0_342] > Caused by: java.lang.NoClassDefFoundError: > nonapi/io/github/classgraph/utils/URLPathEncoder > at > io.github.classgraph.ClasspathElementZip.getURI(ClasspathElementZip.java:682) > ~[?:?] > at > io.github.classgraph.ClasspathElementZip.getAllURIs(ClasspathElementZip.java:695) > ~[?:?] > at > io.github.classgraph.ScanResult.getClasspathURIs(ScanResult.java:384) ~[?:?] > at > io.github.classgraph.ScanResult.getClasspathURLs(ScanResult.java:408) ~[?:?] > at > io.github.classgraph.ClassGraphClassLoader.<init>(ClassGraphClassLoader.java:108) > ~[?:?] > at io.github.classgraph.ScanResult.<init>(ScanResult.java:295) ~[?:?] > at io.github.classgraph.Scanner.performScan(Scanner.java:1013) ~[?:?] > at > io.github.classgraph.Scanner.openClasspathElementsThenScan(Scanner.java:1112) > ~[?:?] > at io.github.classgraph.Scanner.call(Scanner.java:1146) ~[?:?] > at io.github.classgraph.Scanner.call(Scanner.java:83) ~[?:?] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ~[?:1.8.0_342] > ... 3 more > Caused by: java.lang.ClassNotFoundException: > nonapi.io.github.classgraph.utils.URLPathEncoder > at java.net.URLClassLoader.findClass(URLClassLoader.java:387) > ~[?:1.8.0_342] > at java.lang.ClassLoader.loadClass(ClassLoader.java:418) > ~[?:1.8.0_342] > at > org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) > ~[flink-dist_2.12-1.14.5.jar:1.14.5] > at > org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) > ~[flink-dist_2.12-1.14.5.jar:1.14.5] > at > org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) > ~[flink-dist_2.12-1.14.5.jar:1.14.5] > at java.lang.ClassLoader.loadClass(ClassLoader.java:351) > ~[?:1.8.0_342] > at > io.github.classgraph.ClasspathElementZip.getURI(ClasspathElementZip.java:682) > ~[?:?] > at > io.github.classgraph.ClasspathElementZip.getAllURIs(ClasspathElementZip.java:695) > ~[?:?] > at > io.github.classgraph.ScanResult.getClasspathURIs(ScanResult.java:384) ~[?:?] > at > io.github.classgraph.ScanResult.getClasspathURLs(ScanResult.java:408) ~[?:?] > at > io.github.classgraph.ClassGraphClassLoader.<init>(ClassGraphClassLoader.java:108) > ~[?:?] > at io.github.classgraph.ScanResult.<init>(ScanResult.java:295) ~[?:?] > at io.github.classgraph.Scanner.performScan(Scanner.java:1013) ~[?:?] > at > io.github.classgraph.Scanner.openClasspathElementsThenScan(Scanner.java:1112) > ~[?:?] > at io.github.classgraph.Scanner.call(Scanner.java:1146) ~[?:?] > at io.github.classgraph.Scanner.call(Scanner.java:83) ~[?:?] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > ~[?:1.8.0_342] > ... 3 more > ``` > > Thanks for helping! > > Sincerely, > Lydian Lee > > -- Regards, Tomo
