[ https://issues.apache.org/jira/browse/HIVE-25970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Stamatis Zampetakis resolved HIVE-25970. ---------------------------------------- Fix Version/s: 4.0.0 Resolution: Fixed Fixed in https://github.com/apache/hive/commit/d3cd596aa15ebedd58f99628d43a03eb2f5f3909. Thanks for the review [~kgyrtkirk]! > Missing messages in HS2 operation logs > -------------------------------------- > > Key: HIVE-25970 > URL: https://issues.apache.org/jira/browse/HIVE-25970 > Project: Hive > Issue Type: Bug > Components: HiveServer2 > Reporter: Stamatis Zampetakis > Assignee: Stamatis Zampetakis > Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > > After HIVE-22753 & HIVE-24590, with some unlucky timing of events, operation > log messages can get lost and never appear in the appropriate files. > The changes in HIVE-22753 will prevent a {{HushableRandomAccessFileAppender}} > from being created if the latter refers to a file that has been closed in the > last second. Preventing the creation of the appender also means that the > message which triggered the creation will be lost forever. In fact any > message (for the same query) that comes in the interval of 1 second will be > lost forever. > Before HIVE-24590 the appender/file was closed only once (explicitly by HS2) > and thus the problem may be very hard to notice in practice. However, with > the arrival of HIVE-24590 appenders may close much more frequently (and not > via HS2) making the issue reproducible rather easily. It suffices to set > _hive.server2.operation.log.purgePolicy.timeToLive_ property very low and > check the operation logs. > The problem was discovered by investigating some intermittent failures in > operation logging tests (e.g., TestOperationLoggingAPIWithTez). -- This message was sent by Atlassian Jira (v8.20.1#820001)