[
https://issues.apache.org/jira/browse/YARN-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15404788#comment-15404788
]
Vrushali C edited comment on YARN-5382 at 8/2/16 9:08 PM:
----------------------------------------------------------
So with the last uploaded patch v9 on branch-2.7
(https://issues.apache.org/jira/secure/attachment/12821498/YARN-5382-branch-2.7.09.patch)
. The 2.7 patch does not have logging in ClientRMService.
I see only one audit log message when I ran a sleep job and killed it on
pseudo-distributed setup on my laptop.
{code}
[machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -i
Rmauditlogg logs/yarn-vchannapattan-resourcemanager-machine13-channapattan.log
| grep -i Kill
2016-08-02 14:00:19,186 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1470171585834_0001
2016-08-02 14:00:19,195 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan
OPERATION=Application Finished - Killed TARGET=RMAppManager
RESULT=SUCCES APPID=application_1470171585834_0001
[machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
{code}
On another window:
{code}
[machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ bin/hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.4-SNAPSHOT.jar
sleep -m 100 -r 1000 -mt 300 -rt 300
16/08/02 14:00:03 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
16/08/02 14:00:03 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
16/08/02 14:00:05 INFO mapreduce.JobSubmitter: number of splits:100
16/08/02 14:00:05 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1470171585834_0001
16/08/02 14:00:05 INFO impl.YarnClientImpl: Submitted application
application_1470171585834_0001
16/08/02 14:00:05 INFO mapreduce.Job: The url to track the job:
http://localhost:8088/proxy/application_1470171585834_0001/
16/08/02 14:00:05 INFO mapreduce.Job: Running job: job_1470171585834_0001
^C
[machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ bin/yarn
application -kill application_1470171585834_0001
16/08/02 14:00:16 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
16/08/02 14:00:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Killing application application_1470171585834_0001
16/08/02 14:00:19 INFO impl.YarnClientImpl: Killed application
application_1470171585834_0001
[t-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
{code}
I need to update the patch for trunk to include removal of the audit logging
upon isAppFinalStateStored check.
was (Author: vrushalic):
So with the last uploaded patch v9 on branch-2.7
(https://issues.apache.org/jira/secure/attachment/12821498/YARN-5382-branch-2.7.09.patch)
. The 2.7 patch does not have logging in ClientRMService.
I see only one audit log message when I ran a sleep job and killed it on
pseudo-distributed setup on my laptop.
{code}
[machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -i
Rmauditlogg logs/yarn-vchannapattan-resourcemanager-machine13-channapattan.log
| grep -i Kill
2016-08-02 14:00:19,186 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan
IP=127.0.0.1 OPERATION=Kill Application Request TARGET=RMAppImpl
RESULT=SUCCESS APPID=application_1470171585834_0001
2016-08-02 14:00:19,195 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vchannapattan
OPERATION=Application Finished - Killed TARGET=RMAppManager
RESULT=SUCCES APPID=application_1470171585834_0001
[machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
{code}
On another window:
[machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ bin/hadoop jar
share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.4-SNAPSHOT.jar
sleep -m 100 -r 1000 -mt 300 -rt 300
16/08/02 14:00:03 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
16/08/02 14:00:03 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
16/08/02 14:00:05 INFO mapreduce.JobSubmitter: number of splits:100
16/08/02 14:00:05 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1470171585834_0001
16/08/02 14:00:05 INFO impl.YarnClientImpl: Submitted application
application_1470171585834_0001
16/08/02 14:00:05 INFO mapreduce.Job: The url to track the job:
http://localhost:8088/proxy/application_1470171585834_0001/
16/08/02 14:00:05 INFO mapreduce.Job: Running job: job_1470171585834_0001
^C
[machine13-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ bin/yarn
application -kill application_1470171585834_0001
16/08/02 14:00:16 INFO client.RMProxy: Connecting to ResourceManager at
/0.0.0.0:8032
16/08/02 14:00:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Killing application application_1470171585834_0001
16/08/02 14:00:19 INFO impl.YarnClientImpl: Killed application
application_1470171585834_0001
[t-channapattan hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
{code}
I need to update the patch for trunk to include removal of the audit logging
upon isAppFinalStateStored check.
> RM does not audit log kill request for active applications
> ----------------------------------------------------------
>
> Key: YARN-5382
> URL: https://issues.apache.org/jira/browse/YARN-5382
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.7.2
> Reporter: Jason Lowe
> Assignee: Vrushali C
> Attachments: YARN-5382-branch-2.7.01.patch,
> YARN-5382-branch-2.7.02.patch, YARN-5382-branch-2.7.03.patch,
> YARN-5382-branch-2.7.04.patch, YARN-5382-branch-2.7.05.patch,
> YARN-5382-branch-2.7.09.patch, YARN-5382.06.patch, YARN-5382.07.patch,
> YARN-5382.08.patch, YARN-5382.09.patch
>
>
> ClientRMService will audit a kill request but only if it either fails to
> issue the kill or if the kill is sent to an already finished application. It
> does not create a log entry when the application is active which is arguably
> the most important case to audit.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
