Add another test case: - compiled tag calcite-1.35.0 with JDK8(1.8.0_391) on MacOS with M2 chip: cannot reproduce
Put it altogether, I agree with Julian that it might be something changed from 1.35.0 to 1.36.0 (such as the Guava version we're compiling with) which causes the problem (I'm not sure if it's a problem of Calcite). Benchao Li <libenc...@apache.org> 于2023年12月20日周三 11:28写道： > > I tried to reproduce this using Guillaume's > project(https://github.com/MasseGuillaume/asm-remapper-bug), and the > findings are below: > > - compiled latest main branch > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK8(1.8.0_391) on > MacOS with M2 chip: reproduced > - compiled latest main branch > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK8(1.8.0_391) on > Linux with x64 architecture: reproduced > - compiled latest main branch > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK11(1.11.0_21) on > MacOS with M2 chip: cannot reproduce > - compiled latest main branch > (50a20824c4536450dcae963b5e757cf4bbc7e406) with JDK17(1.17.0_7) on > MacOS with M2 chip: cannot reproduce > > It seems unrelated to platforms, and I assume it should not either, > since Java's bytecode should be platform independent. > > @Guillaume You said above that with 1.8.0_371-b11 on macos x64, you > cannot reproduce the problem, can you confirm that? Per my testing, > JDK8's output could all reproduce the problem > > Guillaume Masse <masse.guilla...@narrative.io.invalid> 于2023年12月18日周一 23:13写道： > > > > If I build 1.36.0 and 1.35.0 with the same java version/build javap can > > read the classfiles. Since the release process is manual it's really hard > > to know what causes the problem. My solution is to build and release each > > Calcite version myself and host it on our own S3 repository. > > > > On Mon, Dec 18, 2023, 8:56 AM Ruben Q L <rube...@gmail.com> wrote: > > > > > Hello, > > > > > > My project also recently upgraded to Calcite 1.36, and we are facing the > > > exact same issue when trying to shade with ASM. > > > > > > @Guillame , I can see that https://gitlab.ow2.org/asm/asm/-/issues/318008 > > > has > > > been closed, but I can't really understand why, since the explanation > > > mentions SqlFunctions-1.35.0, but the issue is with 1.36, do you have more > > > info on that? > > > > > > Best, > > > Ruben > > > > > > > > > On Fri, Nov 24, 2023 at 10:09 PM Guillaume Masse > > > <masse.guilla...@narrative.io.invalid> wrote: > > > > > > > We run spark in AWS EMR and it overrides the classpath, so we don't have > > > > control over it: > > > > > > > > Sparks will pull guava 14.0.1: > > > > > > > > > > > https://github.com/apache/spark/blob/master/dev/deps/spark-deps-hadoop-3-hive-2.3#L68 > > > > > > > > That's why we need to shade guava. > > > > > > > > > > > > On Fri, Nov 24, 2023 at 4:31 PM Julian Hyde <jh...@apache.org> wrote: > > > > > > > > > The version of Guava we compile against is relevant. In 1.35 we > > > > > compiled against Guava 19.0; in 1.36 we compiled against Guava > > > > > 32.1.3-jre. The effect would be the same if we compiled against any > > > > > version of Guava 20.0 or higher, due to the addition of > > > > > Preconditions.checkArgument methods that in earlier Guava versions > > > > > would be handled by varargs method. > > > > > > > > > > See https://issues.apache.org/jira/browse/CALCITE-5477 (which was > > > > > fixed in 1.35) and https://issues.apache.org/jira/browse/CALCITE-5763 > > > > > (which was fixed in 1.36 and essentially reverted 5477). > > > > > > > > > > Julian > > > > > > > > > > On Fri, Nov 24, 2023 at 10:20 AM Guillaume Masse > > > > > <masse.guilla...@narrative.io.invalid> wrote: > > > > > > > > > > > > Hi Benchao Li, > > > > > > > > > > > > thanks for giving me more details like the java version, > > > > > > > > > > > > I compiled with 1.8.0_371-b11 on macos x64 (emulated) and I was > > > > > > still > > > > > able > > > > > > to transform the classfile. The bug must come from something else. > > > Was > > > > > this > > > > > > compiled on your personal machine? Was it on windows/linux/mac? > > > What's > > > > > the > > > > > > processor architecture? > > > > > > > > > > > > 1.35.0 also pases the transformation and the same piece of code is > > > > there. > > > > > > How was 1.35.0 released? > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > On Thu, Nov 23, 2023 at 9:59 PM Benchao Li <libenc...@apache.org> > > > > wrote: > > > > > > > > > > > > > Guillaume, > > > > > > > > > > > > > > The binary release are always compiled with JDK8, this is a > > > required > > > > > > > procedure[1]. > > > > > > > > > > > > > > For 1.36.0, the JDK version I'm using is: > > > > > > > $ java -version > > > > > > > java version "1.8.0_371" > > > > > > > Java(TM) SE Runtime Environment (build 1.8.0_371-b11) > > > > > > > Java HotSpot(TM) 64-Bit Server VM (build 25.371-b11, mixed mode) > > > > > > > > > > > > > > [1] > > > > > https://calcite.apache.org/docs/howto.html#making-a-release-candidate > > > > > > > > > > > > > > Guillaume Masse <masse.guilla...@narrative.io.invalid> > > > > 于2023年11月24日周五 > > > > > > > 07:22写道： > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > We recently upgraded to calcite 1.36.0 and it broke our > > > deployment > > > > > > > process. > > > > > > > > We are using java asm <https://asm.ow2.io/> to shade (rename > > > > > packages > > > > > > > from > > > > > > > > the bytecode) since there are multiple dependencies clash when > > > > using > > > > > with > > > > > > > > Apache Spark. For example, Apache Spark is using a much older > > > > > version of > > > > > > > > guava. If I build calcite locally with Java Correto 17.0.7-amzn > > > > > > > > I > > > > can > > > > > > > > transform calcite's bytecode without any problem. Where can I > > > find > > > > > the > > > > > > > java > > > > > > > > version used to compiled calcite? Is there a process in place to > > > > > have a > > > > > > > > consistent java version? > > > > > > > > > > > > > > > > https://gitlab.ow2.org/asm/asm/-/issues/318008 > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Guillaume Massé > > > > > > > > [Gee-OHM] > > > > > > > > (马赛卫) > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Best, > > > > > > > Benchao Li > > > > > > > > > > > > > > > > > > > > > > > -- > > Best, > Benchao Li -- Best, Benchao Li
