[
https://issues.apache.org/jira/browse/IMPALA-12595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17795197#comment-17795197
]
ASF subversion and git services commented on IMPALA-12595:
----------------------------------------------------------
Commit 3381fbf761989ce46da4f83a952bd247dc652ff8 in impala's branch
refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3381fbf76 ]
IMPALA-12595: Allow automatic removal of old logs from previous PID
IMPALA-11184 add code to target specific PID for log rotation. This
align with glog behavior and grant safety. That is, it is strictly limit
log rotation to only consider log files made by the currently running
Impalad and exclude logs made by previous PID or other living-colocated
Impalads. The downside of this limit is that logs can start accumulate
in a node when impalad is frequently restarted and is only resolvable by
admin doing manual log removal.
To help avoid this manual removal, this patch adds a backend flag
'log_rotation_match_pid' that relax the limit by dropping the PID in
glob pattern. Default value for this new flag is False. However, for
testing purpose, start-impala-cluster.py will override it to True since
test minicluster logs to a common log directory. Setting
'log_rotation_match_pid' to True will prevent one impalad from
interfering with log rotation of other impalad in minicluster.
As a minimum exercise for this new log rotation behavior,
test_breakpad.py::TestLogging is modified to invoke
start-impala-cluster.py with 'log_rotation_match_pid' set to False.
Testing:
- Add test_excessive_cerr_ignore_pid and test_excessive_cerr_match_pid.
- Split TestLogging into two. One run test_excessive_cerr_ignore_pid in
core exploration, while the other run the rest of logging tests in
exhaustive exploration.
- Pass exhaustive tests.
Change-Id: I599799e73f27f941a1d7f3dec0f40b4f05ea5ceb
Reviewed-on: http://gerrit.cloudera.org:8080/20754
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Allow automatic removal of old logs from previous PID
> -----------------------------------------------------
>
> Key: IMPALA-12595
> URL: https://issues.apache.org/jira/browse/IMPALA-12595
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 4.3.0
> Reporter: Riza Suminto
> Assignee: Riza Suminto
> Priority: Major
>
> IMPALA-11184 add code to target specific file name pattern for log rotation
> {code:java}
> // We specifically target the base file name created by glog.
> // Glog's default base file name follow this pattern:
> // "<program name>.<hostname>.<user name>.log.<severity
> level>.<date>-<time>.<pid>"
> inline string GlobPatternForLog(google::LogSeverity severity) {
> return strings::Substitute("$0/$1*.log.$2.*.$3", FLAGS_log_dir,
> google::ProgramInvocationShortName(),
> google::GetLogSeverityName(severity),
> getpid());
> }{code}
> The <pid> component in the pattern grant safety. That is, it is strictly
> limit log rotation to only consider log files made by the currently running
> Impalad and exclude logs made by previous PID or other living-colocated
> Impalads.
> The downside of this limit is that logs can start accumulate in a node when
> impalad is frequently restarted and is only resolvable by admin doing manual
> log removal. To help avoid this manual removal, Impala should consider
> relaxing the limit a bit by dropping the <pid> pattern, at least once on
> Impalad startup or at very first log rotation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
