Hi Zheng, Could you share a minimal example that reproduces the issue? Also, have you tried using Beam >2.35.0? I'm curious if this happens in the 2.41.0 release as well.
Brian On Tue, Aug 30, 2022 at 11:10 AM Zheng Ni <nizheng1...@gmail.com> wrote: > Hi There, > > I am using beam 2.35.0 to build a simple sql based pipeline but got below > error. I am trying to debug into the code and could not found this class: > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.metadata.BuiltInMetadata.ExplainVisibility.Handler. > Not sure if any lib missing in my project. > > Thanks, > Zheng > > > POM: > <dependency> > <groupId>org.apache.beam</groupId> > <artifactId>beam-sdks-java-extensions-sql</artifactId> > <version>2.35.0</version> > </dependency> > > > Stack trace: > Caused by: > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.commons.compiler.InternalCompilerException: > Compiling "GeneratedMetadata_ExplainVisibilityHandler" in Line 1, Column > 14: null > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:369) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.access$000(UnitCompiler.java:231) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler.java:333) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler$1.visitCompilationUnit(UnitCompiler.java:330) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.Java$CompilationUnit.accept(Java.java:367) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.compileUnit(UnitCompiler.java:330) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:245) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.SimpleCompiler.compileToClassLoader(SimpleCompiler.java:473) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:223) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.SimpleCompiler.cook(SimpleCompiler.java:209) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.commons.compiler.Cookable.cook(Cookable.java:82) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.commons.compiler.Cookable.cook(Cookable.java:77) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.compile(JaninoRelMetadataProvider.java:361) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.generateCompileAndInstantiate(JaninoRelMetadataProvider.java:182) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.lambda$static$0(JaninoRelMetadataProvider.java:75) > at > org.apache.beam.vendor.calcite.v1_28_0.com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165) > at > org.apache.beam.vendor.calcite.v1_28_0.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) > at > org.apache.beam.vendor.calcite.v1_28_0.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) > at > org.apache.beam.vendor.calcite.v1_28_0.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) > at > org.apache.beam.vendor.calcite.v1_28_0.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) > at > org.apache.beam.vendor.calcite.v1_28_0.com.google.common.cache.LocalCache.get(LocalCache.java:3951) > at > org.apache.beam.vendor.calcite.v1_28_0.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974) > at > org.apache.beam.vendor.calcite.v1_28_0.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4958) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.metadata.JaninoRelMetadataProvider.revise(JaninoRelMetadataProvider.java:381) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.metadata.RelMetadataQueryBase.revise(RelMetadataQueryBase.java:108) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.metadata.RelMetadataQuery.isVisibleInExplain(RelMetadataQuery.java:857) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.externalize.RelWriterImpl.explain_(RelWriterImpl.java:67) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.externalize.RelWriterImpl.done(RelWriterImpl.java:151) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.AbstractRelNode.explain(AbstractRelNode.java:252) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptUtil.toString(RelOptUtil.java:2409) > at > org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptUtil.toString(RelOptUtil.java:2392) > at > org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.convertToBeamRel(CalciteQueryPlanner.java:189) > at > org.apache.beam.sdk.extensions.sql.impl.BeamSqlEnv.parseQuery(BeamSqlEnv.java:112) > at > org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:171) > at > org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:110) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:548) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:482) > at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:363) > ... 117 common frames omitted > Caused by: java.lang.NullPointerException: null > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.ReflectionIClass.getDescriptor2(ReflectionIClass.java:125) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.IClass.getDescriptor(IClass.java:529) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.IClassLoader.defineIClass(IClassLoader.java:396) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.ClassLoaderIClassLoader.findIClass(ClassLoaderIClassLoader.java:93) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.IClassLoader.loadIClass(IClassLoader.java:319) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.findTypeByName(UnitCompiler.java:8673) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:6689) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6661) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.access$14400(UnitCompiler.java:231) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler$22.visitReferenceType(UnitCompiler.java:6556) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler$22.visitReferenceType(UnitCompiler.java:6551) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.Java$ReferenceType.accept(Java.java:4271) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6551) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.access$1300(UnitCompiler.java:231) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler$36.getInterfaces2(UnitCompiler.java:10134) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.IClass.getInterfaces(IClass.java:498) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.IClass.getIMethods(IClass.java:264) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.IClass.getIMethods(IClass.java:238) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:472) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:412) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.access$400(UnitCompiler.java:231) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:391) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler$2.visitPackageMemberClassDeclaration(UnitCompiler.java:386) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.Java$PackageMemberClassDeclaration.accept(Java.java:1692) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.compile(UnitCompiler.java:386) > at > org.apache.beam.vendor.calcite.v1_28_0.org.codehaus.janino.UnitCompiler.compile2(UnitCompiler.java:359) >
