[
https://issues.apache.org/jira/browse/PIG-4921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-4921:
------------------------------------
Attachment: PIG-4921-2.patch
bq. TezLaucher.kill still kill individual job, is that redundant as well?
TezLaucher.kill is now not called as part of HangingJobKiller shutdown hook.
Only called on interrupted exception. TezSessionManager shutdown hook will
shutdown all sessions.
bq. For MR, kill will be invoked in both exception handling and shutdown hook,
will this an issue?
No. If already killed in exception handling then shutdown hook will be a
no-op as it kills jc.getRunningJobs. Actually found that PIG-3524 broke
HangingJobKiller for mapreduce as it was setting jc = null and it skipped
killing jobs because of that. That was a unnecessary change. Reverted that to
fix the issue and also added System.err logging statement to mapreduce as well.
> Kill running jobs on InterruptedException
> -----------------------------------------
>
> Key: PIG-4921
> URL: https://issues.apache.org/jira/browse/PIG-4921
> Project: Pig
> Issue Type: Bug
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Fix For: 0.17.0
>
> Attachments: PIG-4921-1.patch, PIG-4921-2.patch
>
>
> Shutdown hook kills running jobs, but when running in Oozie launcher
> sometimes NodeManager can issue a SIGKILL after AM unregisters and before
> shutdown hook gets to execute causing orphaned jobs that continue to run. So
> it is better to kill when we see the InterruptedException.
> Added additional System.err logging to code in shutdown hooks as
> LogManager.shutdown() is called by mapreduce AM (Oozie launcher job is
> mapreduce) and log4j logging does not appear anymore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
