Stamatis Zampetakis created HIVE-25970:
------------------------------------------
Summary: Missing messages in HS2 operation logs
Key: HIVE-25970
URL: https://issues.apache.org/jira/browse/HIVE-25970
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
After HIVE-22753 & HIVE-24590, with some unlucky timing of events, operation
log messages can get lost and never appear in the appropriate files.
The changes in HIVE-22753 will prevent a {{HushableRandomAccessFileAppender}}
from being created if the latter refers to a file that has been closed in the
last second. Preventing the creation of the appender also means that the
message which triggered the creation will be lost forever. In fact any message
(for the same query) that comes in the interval of 1 second will be lost
forever.
Before HIVE-24590 the appender/file was closed only once (explicitly by HS2)
and thus the problem may be very hard to notice in practice. However, with the
arrival of HIVE-24590 appenders may close much more frequently (and not via
HS2) making the issue reproducible rather easily. It suffices to set
_hive.server2.operation.log.purgePolicy.timeToLive_ property very low and check
the operation logs.
The problem was discovered by investigating some intermittent failures in
operation logging tests (e.g., TestOperationLoggingAPIWithTez).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
