----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66557/ -----------------------------------------------------------
Review request for hive and Ashutosh Chauhan. Bugs: HIVE-17477 https://issues.apache.org/jira/browse/HIVE-17477 Repository: hive-git Description ------- PROBLEM: We can use pid in th filePattern: appender.DRFA.filePattern = ${sys:hive.log.dir}/${sys:hive.log.file}.%d {yyyy-MM-dd} .%pid But when its time roll the logs at the end of the RollingAppender just renames hive.log by applying the pattern described above and keeps logs that are not related to the process this log was named by. The issue is that all processes are writing to the same log and we cannot separate them by pid. SOLUTION: I have discovered apache log4j2 project and documentation and found that existed PidFilePatternConverter allowed appending PID only when TimeBasedTriggeringPolicy was occured. I have created ported https://github.com/apache/logging-log4j2/blob/master/log4j-core/src/main/java/org/apache/logging/log4j/core/pattern/ProcessIdPatternConverter.java and configured new RoutingAppender in hive-log4j2.properties. I have removed old PidFilePatternConverter.java file and create new ProcessIdPatternConverter.java because first class contain in name "FilePattern" and I think it could be confused. PidFilePatternConverter's implementation required adding %pid to appender.XXX.filePattern in hive-log4j2.properties. But ProcessIdPatternConverter depends not on filePattern, it depends on LogEvents, and %pid should be added to rolling.layout.pattern field. Diffs ----- common/src/main/resources/hive-log4j2.properties b1719907571ead14686e7fcc14a0687158dec941 llap-server/src/main/resources/llap-cli-log4j2.properties 687c97399a6632205318c24665ed9a97dc30c552 ql/src/java/org/apache/hadoop/hive/ql/log/PidFilePatternConverter.java c49f53fd3cdcf71ebca36dec185021e18d0b1ef9 ql/src/java/org/apache/hadoop/hive/ql/log/ProcessIdPatternConverter.java PRE-CREATION ql/src/test/org/apache/hadoop/hive/ql/log/ProcessIdPatternConverterTest.java PRE-CREATION standalone-metastore/src/main/resources/metastore-log4j2.properties ec5039b4fbdf880fee2a3f86ec3402ba02f3d21d Diff: https://reviews.apache.org/r/66557/diff/1/ Testing ------- Thanks, Bohdan Chupika