Never tried it myself, but I notice that the RexBuilder has a withSimplifier method. I wonder whether you can create a noop simplifier.
Otherwise, this seems to be pretty deep inside Calcite - the builder is a core class, and it always seems to use the simplifier. Mihai ________________________________ From: Gonzalo Ortiz Jaureguizar <[email protected]> Sent: Thursday, July 3, 2025 4:42 AM To: [email protected] <[email protected]> Subject: Reduce class loading during query optimizing Hi there, Here at Apache Pinot, we utilize Apache Calcite for query optimization. Once the query is optimized at logical level (pushing predicates, simplifying expressions, etc), we transform the query tree to our own nodes and distribute the execution between different nodes. But we are having some issues in cases where we have 1000 queries per second. Specifically, we found that in these cases, we may end up with >80000 instances of org.codehaus.commons.compiler.util.reflect.ByteArrayClassLoader instances and, even worse, several query threads have to block for the lock of jdk.internal.loader.BuiltinClassLoader.loadClassOrNull. In these two cases, the stacktrace points to Janino. Specifically, Janino is being used to compile trivial expressions when queries are simplified. Is that necessary or even desirable? We may be doing something wrong (our Calcite knowledge is not as good as we would like), but it sounds expensive to generate bytecode to run code that will be executed only once. Is there a way to disable Janino for these cases? For the context, Stack trace of the thread allocating a new classloader: ``` <init>:57, ByteArrayClassLoader (org.codehaus.commons.compiler.util.reflect) run:357, SimpleCompiler$2 (org.codehaus.janino) run:351, SimpleCompiler$2 (org.codehaus.janino) doPrivileged:74, AccessController (java.security) getClassLoader2:351, SimpleCompiler (org.codehaus.janino) getClassLoader:341, SimpleCompiler (org.codehaus.janino) cook:308, ClassBodyEvaluator (org.codehaus.janino) cook2:297, ClassBodyEvaluator (org.codehaus.janino) cook:273, ClassBodyEvaluator (org.codehaus.janino) compile:64, RexExecutable (org.apache.calcite.rex) <init>:53, RexExecutable (org.apache.calcite.rex) reduce:144, RexExecutorImpl (org.apache.calcite.rex) simplifyCast:2304, RexSimplify (org.apache.calcite.rex) simplify:293, RexSimplify (org.apache.calcite.rex) lambda$simplifyList$2:682, RexSimplify (org.apache.calcite.rex) apply:-1, RexSimplify$$Lambda/0x00007faed7bfa218 (org.apache.calcite.rex) replaceAllRange:1803, ArrayList (java.util) replaceAll:1793, ArrayList (java.util) simplifyList:682, RexSimplify (org.apache.calcite.rex) simplifyComparison:523, RexSimplify (org.apache.calcite.rex) simplifyComparison:515, RexSimplify (org.apache.calcite.rex) simplify:311, RexSimplify (org.apache.calcite.rex) lambda$simplifyList$2:682, RexSimplify (org.apache.calcite.rex) apply:-1, RexSimplify$$Lambda/0x00007faed7bfa218 (org.apache.calcite.rex) replaceAllRange:1803, ArrayList (java.util) replaceAll:1793, ArrayList (java.util) simplifyList:682, RexSimplify (org.apache.calcite.rex) simplifyAnd:1529, RexSimplify (org.apache.calcite.rex) simplify:282, RexSimplify (org.apache.calcite.rex) simplifyUnknownAs:251, RexSimplify (org.apache.calcite.rex) simplifyUnknownAsFalse:240, RexSimplify (org.apache.calcite.rex) simplifyFilterPredicates:2898, RexSimplify (org.apache.calcite.rex) filter:1921, RelBuilder (org.apache.calcite.tools) filter:1886, RelBuilder (org.apache.calcite.tools) convertProject:393, PushProjector (org.apache.calcite.rel.rules) onMatch:179, ProjectFilterTransposeRule (org.apache.calcite.rel.rules) fireRule:350, AbstractRelOptPlanner (org.apache.calcite.plan) applyRule:541, HepPlanner (org.apache.calcite.plan.hep) depthFirstApply:370, HepPlanner (org.apache.calcite.plan.hep) depthFirstApply:384, HepPlanner (org.apache.calcite.plan.hep) depthFirstApply:384, HepPlanner (org.apache.calcite.plan.hep) depthFirstApply:384, HepPlanner (org.apache.calcite.plan.hep) applyRules:436, HepPlanner (org.apache.calcite.plan.hep) executeRuleCollection:285, HepPlanner (org.apache.calcite.plan.hep) execute:105, HepInstruction$RuleCollection$State (org.apache.calcite.plan.hep) lambda$executeProgram$0:210, HepPlanner (org.apache.calcite.plan.hep) accept:-1, HepPlanner$$Lambda/0x00007faed7c20f90 (org.apache.calcite.plan.hep) forEach:423, ImmutableList (com.google.common.collect) executeProgram:209, HepPlanner (org.apache.calcite.plan.hep) execute:118, HepProgram$State (org.apache.calcite.plan.hep) executeProgram:204, HepPlanner (org.apache.calcite.plan.hep) findBestExp:190, HepPlanner (org.apache.calcite.plan.hep) optimize:463, QueryEnvironment (org.apache.pinot.query) compileQuery:356, QueryEnvironment (org.apache.pinot.query) compile:283, QueryEnvironment (org.apache.pinot.query) compile:261, QueryEnvironment (org.apache.pinot.query) getTableNames:225, PinotQueryResource (org.apache.pinot.controller.api.resources) ... ``` Stack trace of the threads being blocked: ``` ## Thread has lock "multi-stage-query-compile-executor-2-thread-1" #374 [369] prio=5 os_prio=0 cpu=4267041.58ms elapsed=114126.21s tid=0x00007f7a3d1f5810 nid=369 runnable [0x00007f79c517b000] java.lang.Thread.State: RUNNABLE at java.lang.ClassLoader.findBootstrapClass([email protected]/Native Method) at java.lang.ClassLoader.findBootstrapClassOrNull([email protected] /ClassLoader.java:1277) at java.lang.System$2.findBootstrapClassOrNull([email protected] /System.java:2397) at jdk.internal.loader.ClassLoaders$BootClassLoader.loadClassOrNull([email protected] /ClassLoaders.java:140) at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull([email protected] /BuiltinClassLoader.java:700) at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull([email protected] /BuiltinClassLoader.java:676) - locked <0x000000030a1d87f8> (a java.lang.Object) at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull([email protected] /BuiltinClassLoader.java:700) at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull([email protected] /BuiltinClassLoader.java:676) - locked <0x000000030ab04140> (a java.lang.Object) at jdk.internal.loader.BuiltinClassLoader.loadClass([email protected] /BuiltinClassLoader.java:639) at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass([email protected] /ClassLoaders.java:188) at java.lang.ClassLoader.loadClass([email protected] /ClassLoader.java:526) at org.codehaus.janino.ClassLoaderIClassLoader.findIClass(ClassLoaderIClassLoader.java:75) at org.codehaus.janino.IClassLoader.loadIClass(IClassLoader.java:317) - locked <0x00000007f780cc68> (a org.codehaus.janino.ClassLoaderIClassLoader) at org.codehaus.janino.UnitCompiler.findTypeByName(UnitCompiler.java:9074) at org.codehaus.janino.UnitCompiler.getRawReferenceType(UnitCompiler.java:7242) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:7142) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:7023) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6994) at org.codehaus.janino.UnitCompiler.access$14900(UnitCompiler.java:240) at org.codehaus.janino.UnitCompiler$24.visitReferenceType(UnitCompiler.java:6891) at org.codehaus.janino.UnitCompiler$24.visitReferenceType(UnitCompiler.java:6888) ...... at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:281) at org.apache.calcite.rex.RexSimplify.simplifyUnknownAs(RexSimplify.java:250) at org.apache.calcite.rex.RexSimplify.simplifyUnknownAsFalse(RexSimplify.java:239) at org.apache.calcite.rex.RexSimplify.simplifyFilterPredicates(RexSimplify.java:2810) at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1809) at org.apache.calcite.tools.RelBuilder.filter(RelBuilder.java:1774) at org.apache.calcite.rel.rules.PushProjector.convertProject(PushProjector.java:394) at org.apache.calcite.rel.rules.ProjectFilterTransposeRule.onMatch(ProjectFilterTransposeRule.java:179) at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:337) at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:556) at org.apache.calcite.plan.hep.HepPlanner.depthFirstApply(HepPlanner.java:371) at org.apache.calcite.plan.hep.HepPlanner.depthFirstApply(HepPlanner.java:385) at org.apache.calcite.plan.hep.HepPlanner.depthFirstApply(HepPlanner.java:385) at org.apache.calcite.plan.hep.HepPlanner.depthFirstApply(HepPlanner.java:385) at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:437) at org.apache.calcite.plan.hep.HepPlanner.executeRuleCollection(HepPlanner.java:286) at org.apache.calcite.plan.hep.HepInstruction$RuleCollection$State.execute(HepInstruction.java:105) at org.apache.calcite.plan.hep.HepPlanner.lambda$executeProgram$0(HepPlanner.java:211) at org.apache.calcite.plan.hep.HepPlanner$$Lambda/0x00007f79fbf0a360.accept(Unknown Source) at org.apache.pinot.shaded.com.google.common.collect.ImmutableList.forEach(ImmutableList.java:423) at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:210) at org.apache.calcite.plan.hep.HepProgram$State.execute(HepProgram.java:118) at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:205) at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:191) at org.apache.pinot.query.QueryEnvironment.optimize(QueryEnvironment.java:394) at org.apache.pinot.query.QueryEnvironment.compileQuery(QueryEnvironment.java:347) at org.apache.pinot.query.QueryEnvironment.planQuery(QueryEnvironment.java:187) at org.apache.pinot.broker.requesthandler.MultiStageBrokerRequestHandler.lambda$handleRequest$2(MultiStageBrokerRequestHandler.java:222) Waiting threads: "multi-stage-query-compile-executor-2-thread-3" #376 [371] prio=5 os_prio=0 cpu=4261107.12ms elapsed=114126.12s tid=0x00007f79c7d8b010 nid=371 waiting for monitor entry [0x00007f79c48fb000] java.lang.Thread.State: BLOCKED (on object monitor) at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull([email protected] /BuiltinClassLoader.java:651) - waiting to lock <0x000000030ab04140> (a java.lang.Object) at jdk.internal.loader.BuiltinClassLoader.loadClass([email protected] /BuiltinClassLoader.java:639) at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass([email protected] /ClassLoaders.java:188) at java.lang.ClassLoader.loadClass([email protected] /ClassLoader.java:526) at org.codehaus.janino.ClassLoaderIClassLoader.findIClass(ClassLoaderIClassLoader.java:75) at org.codehaus.janino.IClassLoader.loadIClass(IClassLoader.java:317) - locked <0x00000007f70a2398> (a org.codehaus.janino.ClassLoaderIClassLoader) at org.codehaus.janino.UnitCompiler.findTypeByName(UnitCompiler.java:9074) at org.codehaus.janino.UnitCompiler.getRawReferenceType(UnitCompiler.java:7242) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:7142) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:7023) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6994) at org.codehaus.janino.UnitCompiler.access$14900(UnitCompiler.java:240) at org.codehaus.janino.UnitCompiler$24.visitReferenceType(UnitCompiler.java:6891) at org.codehaus.janino.UnitCompiler$24.visitReferenceType(UnitCompiler.java:6888) at org.codehaus.janino.Java$ReferenceType.accept(Java.java:4289) at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6888) at org.codehaus.janino.UnitCompiler.getRawType(UnitCompiler.java:6884) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7366) "multi-stage-query-compile-executor-2-thread-4" #396 [391] prio=5 os_prio=0 cpu=4264527.65ms elapsed=114125.23s tid=0x00007f79cdbc2010 nid=391 waiting for monitor entry [0x00007f79c12fb000] java.lang.Thread.State: BLOCKED (on object monitor) at jdk.internal.loader.BuiltinClassLoader.loadClassOrNull([email protected] /BuiltinClassLoader.java:651) - waiting to lock <0x000000030ab04140> (a java.lang.Object) at jdk.internal.loader.BuiltinClassLoader.loadClass([email protected] /BuiltinClassLoader.java:639) at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass([email protected] /ClassLoaders.java:188) at java.lang.ClassLoader.loadClass([email protected] /ClassLoader.java:526) at org.codehaus.janino.ClassLoaderIClassLoader.findIClass(ClassLoaderIClassLoader.java:75) at org.codehaus.janino.IClassLoader.loadIClass(IClassLoader.java:317) - locked <0x00000007f880ca18> (a org.codehaus.janino.ClassLoaderIClassLoader) at org.codehaus.janino.UnitCompiler.findTypeByName(UnitCompiler.java:9074) at org.codehaus.janino.UnitCompiler.getRawReferenceType(UnitCompiler.java:7242) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:7142) at org.codehaus.janino.UnitCompiler.getReferenceType(UnitCompiler.java:7023) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:6994) at org.codehaus.janino.UnitCompiler.access$14900(UnitCompiler.java:240) at org.codehaus.janino.UnitCompiler$24.visitReferenceType(UnitCompiler.java:6891) at org.codehaus.janino.UnitCompiler$24.visitReferenceType(UnitCompiler.java:6888) at org.codehaus.janino.Java$ReferenceType.accept(Java.java:4289) at org.codehaus.janino.UnitCompiler.getType(UnitCompiler.java:6888) at org.codehaus.janino.UnitCompiler.getRawType(UnitCompiler.java:6884) at org.codehaus.janino.UnitCompiler.getType2(UnitCompiler.java:7366) ```
