[ 
https://issues.apache.org/jira/browse/HIVE-17128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Sherman updated HIVE-17128:
----------------------------------
    Description: 
[HIVE-16061] and [HIVE-16400] changed operation logging to use the Log4j2 
RoutingAppender to automatically output the log for each query into each 
individual operation log file. As log4j does not know when a query is finished 
it keeps the OutputStream in the Appender open even when the query completes. 
The stream holds a file descriptor and so we leak file descriptors. Note that 
we are already careful to close any streams reading from the operation log file.

h2. Fix

To fix this we use a technique described in the comments of [LOG4J2-510] which 
uses reflection to close the appender. The test in TestOperationLoggingLayout 
will be extended to check that the Appender is closed.

  was:
[HIVE-16061] and [HIVE-16400] changed operation logging to use the Log4j2 
RoutingAppender to automatically output the log for each query into each 
individual operation log file. As log4j does not know when a query is finished 
it keeps the OutputStream in the Appender open even when the query completes. 
The stream holds a file descriptor and so we leak file descriptors. Note that 
we are already careful to close any streams reading from the operation log file.

h2. Fix

To fix this we use a technique described in [LOG4J2-510] which uses reflection 
to close the appender. The test in TestOperationLoggingLayout will be extended 
to check that the Appender is closed.


> Operation Logging leaks file descriptors as the log4j Appender is never closed
> ------------------------------------------------------------------------------
>
>                 Key: HIVE-17128
>                 URL: https://issues.apache.org/jira/browse/HIVE-17128
>             Project: Hive
>          Issue Type: Bug
>          Components: Logging
>            Reporter: Andrew Sherman
>            Assignee: Andrew Sherman
>
> [HIVE-16061] and [HIVE-16400] changed operation logging to use the Log4j2 
> RoutingAppender to automatically output the log for each query into each 
> individual operation log file. As log4j does not know when a query is 
> finished it keeps the OutputStream in the Appender open even when the query 
> completes. The stream holds a file descriptor and so we leak file 
> descriptors. Note that we are already careful to close any streams reading 
> from the operation log file.
> h2. Fix
> To fix this we use a technique described in the comments of [LOG4J2-510] 
> which uses reflection to close the appender. The test in 
> TestOperationLoggingLayout will be extended to check that the Appender is 
> closed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to