[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash updated MAPREDUCE-4135:
Status: Open (was: Patch Available)
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Ravi Prakash
Attachments: MAPREDUCE-4135.branch-0.23.patch, MAPREDUCE-4135.patch,
MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash updated MAPREDUCE-4135:
Attachment: MAPREDUCE-4135.branch-0.23.patch
Wait! I thought of another way.
3. We can @Override stop() in MRAppMaster to call super.stop() and then do the
FileSystem.closeAll(). Set a flag to denote that stop() has already been
called. In the MRAppMasterShutdownHook, if the flag is set, NOT call the
FileSystem.closeAll(). Sheesh! Now I feel stupid for not thinking about this
earlier :(
What do all y'all think of this way?
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.branch-0.23.patch, MAPREDUCE-4135.patch,
MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Attachment: MAPREDUCE-4135.patch
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Affects Version/s: 2.0.0
Status: Patch Available (was: Open)
As Alejandro mentioned in MAPREDUCE-4137, Calling FileSystem.closeAll() from
MRAppMaster is not required because the FileSystem has its own shutdown hook
ClientFinalizer which does closeAll().
I have removed it and attached the patch. I have not added any tests because it
is just code removal and verified it manually.
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Status: Open (was: Patch Available)
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch, MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Attachment: MAPREDUCE-4135.patch
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch, MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Status: Patch Available (was: Open)
Re-attaching the same patch to run Hadoop QA.
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch, MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
