linghengqian commented on code in PR #5375: URL: https://github.com/apache/hive/pull/5375#discussion_r1804166021
########## data/conf/hive-log4j2.properties: ########## @@ -14,9 +14,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -status = INFO name = HiveLog4j2Test -packages = org.apache.hadoop.hive.ql.log Review Comment: > How we replace `packages` with the new way? I think removing `package` will cause some unkown issue. - I would say that even without merging this PR, `packages` never worked in Hive. Unless Hive has always used log4j1 and never used log4j2. Because the class `org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor` exists in the JAR of the final binary of Hive, `org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor` generates `Log4j2Plugins.dat`, which directly covers the operation of `packages`. It is located at `/hive/ql/target/hive-exec-4.1.0-SNAPSHOT.jar!/META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat`. `packages` is actually a `log4j1` thing. -  - You can determine from a comment by a log4j2 committer that this property does not actually take effect. See https://github.com/apache/logging-log4j2/issues/2791#issuecomment-2267168356 . > There is no problem in Apache Hive: I checked that the appropriate annotation processor is configured in the `hive-exec` project and there is a `Log4j2Plugins.dat` file in the JAR. You can safely remove the packages configuration attribute. - `org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor` is available since log4j2 2.0. The processor will generate the necessary `META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat` descriptor that replaces classpath scanning. See https://github.com/apache/logging-log4j2/issues/2434#issuecomment-2037064399 . There is also an intuitive PR at https://github.com/apache/jmeter/pull/6326 . - Since Hive has neither CI for JDK23 and JDK24 nor CI for GraalVM Native Image, we actually don't need to register a `org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor` in `pom.xml` as https://logging.apache.org/log4j/2.x/manual/plugins.html#plugin-registry . ```xml <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>${maven-compiler-plugin.version}</version> <executions> <execution> <id>generate-log4j-plugin-descriptor</id> <goals> <goal>compile</goal> </goals> <phase>process-classes</phase> <configuration> <proc>only</proc> <annotationProcessorPaths> <path> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.24.1</version> </path> </annotationProcessorPaths> <annotationProcessors> <processor>org.apache.logging.log4j.core.config.plugins.processor.PluginProcessor</processor> </annotationProcessors> </configuration> </execution> </executions> </plugin> ``` - You may ask why. The reason is simple, because for JDK8-JDK22, the default value of the JVM parameter `-proc` is `full`. The JVM parameter `-proc` did not become `only` until JDK23. `-proc` of `full` means that all processors are included by default. See https://github.com/apache/shardingsphere/pull/33025 and OpenJDK Committer’s Blog https://inside.java/2024/06/18/quality-heads-up/ . I also suspect that the ASF Jenkins node has an outdated JDK8 version and does not actually recognize the `-proc` JVM parameter, just like the Temurin JDK 8 on the Github Actions Runner also does not recognize the `-proc` JVM parameter. - Are there unit tests for Hive Exec under GraalVM Native Image compiled by GraalVM CE For JDK22 and GraalVM CE For JDK23? Yes, they are at https://github.com/linghengqian/hive-server2-jdbc-driver and https://github.com/apache/shardingsphere . Due to the ongoing dispute of https://issues.apache.org/jira/browse/HIVE-28445, I have to maintain Hive's Thin JAR outside of Hive to provide it for unit testing of ShardingSphere under GraalVM Native Image. - Does `packages = org.apache.hadoop.hive.ql.log` affect things like `logger.Operator.name = org.apache.hadoop.hive.ql.exec.Operator`? Apparently not. Most people use `packages` because they are actually using log4j1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
