[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13816210#comment-13816210
]
Xi Fang commented on MAPREDUCE-5508:
One way to confirm that is to set to
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13800914#comment-13800914
]
Xi Fang commented on MAPREDUCE-5508:
Thanks Chris and viswanathan. And I think
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13776023#comment-13776023
]
Xi Fang commented on MAPREDUCE-5508:
Thanks Chris and Sandy!
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5508:
---
Attachment: MAPREDUCE-5508.2.patch
JobTracker memory leak caused by unreleased FileSystem
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13774322#comment-13774322
]
Xi Fang commented on MAPREDUCE-5508:
Thanks Chris. I attached a new patch and will
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5508:
---
Attachment: MAPREDUCE-5508.3.patch
JobTracker memory leak caused by unreleased FileSystem
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13775904#comment-13775904
]
Xi Fang commented on MAPREDUCE-5508:
Thanks Chris and Sandy. I just finished the
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13774280#comment-13774280
]
Xi Fang commented on MAPREDUCE-5508:
[~cnauroth], thanks for your comments.
bq.
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13772073#comment-13772073
]
Xi Fang commented on MAPREDUCE-5508:
I set both staging and system dirs to hdfs on
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5508:
---
Attachment: MAPREDUCE-5508.1.patch
JobTracker memory leak caused by unreleased FileSystem
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13771276#comment-13771276
]
Xi Fang commented on MAPREDUCE-5508:
Thanks Chris and Sandy. I made a draft patch
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767702#comment-13767702
]
Xi Fang commented on MAPREDUCE-5508:
Thanks [~sandyr] and [~cnauroth]. Actually,
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767847#comment-13767847
]
Xi Fang commented on MAPREDUCE-5508:
Thanks Chris for filing HDFS-5211. That
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767401#comment-13767401
]
Xi Fang commented on MAPREDUCE-5508:
[~sandyr] Thanks for your comments.
bq. Have
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767402#comment-13767402
]
Xi Fang commented on MAPREDUCE-5508:
Just found Chris was also working on this
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767525#comment-13767525
]
Xi Fang commented on MAPREDUCE-5508:
Thanks Sandy for the information on
Xi Fang created MAPREDUCE-5508:
--
Summary: Memory Leak caused by unreleased FileSystem objects in
JobInProgress#cleanupJob
Key: MAPREDUCE-5508
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5508
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5508:
---
Summary: Memory leak caused by unreleased FileSystem objects in
JobInProgress#cleanupJob (was:
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13767210#comment-13767210
]
Xi Fang commented on MAPREDUCE-5508:
This bug was found in Microsoft's large scale
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5508:
---
Description:
MAPREDUCE-5351 fixed a memory leak problem but introducing another filesystem
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5508:
---
Description:
MAPREDUCE-5351 fixed a memory leak problem but introducing another filesystem
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5508:
---
Attachment: MAPREDUCE-5508.patch
Memory leak caused by unreleased FileSystem objects in
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on MAPREDUCE-5508 started by Xi Fang.
Memory leak caused by unreleased FileSystem objects in
JobInProgress#cleanupJob
[
https://issues.apache.org/jira/browse/MAPREDUCE-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5508:
---
Summary: JobTracker memory leak caused by unreleased FileSystem objects in
[
https://issues.apache.org/jira/browse/MAPREDUCE-5405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714194#comment-13714194
]
Xi Fang commented on MAPREDUCE-5405:
Sounds good to me! I also did some tests on
[
https://issues.apache.org/jira/browse/MAPREDUCE-5391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708988#comment-13708988
]
Xi Fang commented on MAPREDUCE-5391:
Thanks Chris. The patch looks good to me!
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Attachment: MAPREDUCE-5278.5.patch
Distributed cache is broken when JT staging dir is not
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13703619#comment-13703619
]
Xi Fang commented on MAPREDUCE-5278:
Thanks, Chris. A new patch has been attached.
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13703695#comment-13703695
]
Xi Fang commented on MAPREDUCE-5278:
Thanks, Chris
Distributed
[
https://issues.apache.org/jira/browse/MAPREDUCE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13702176#comment-13702176
]
Xi Fang commented on MAPREDUCE-5371:
Thanks Chris!
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Attachment: MAPREDUCE-5278.4.patch
Distributed cache is broken when JT staging dir is not
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13699544#comment-13699544
]
Xi Fang commented on MAPREDUCE-5278:
Thanks Bikas. A new patch was attached.
[
https://issues.apache.org/jira/browse/MAPREDUCE-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13698205#comment-13698205
]
Xi Fang commented on MAPREDUCE-5330:
Thanks Ivan and Chris!
JVM
Xi Fang created MAPREDUCE-5371:
--
Summary: TestProxyUserFromEnv#testProxyUserFromEnvironment failed
caused by domains of windows users
Key: MAPREDUCE-5371
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5371
[
https://issues.apache.org/jira/browse/MAPREDUCE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5371:
---
Attachment: MAPREDUCE-5371.patch
TestProxyUserFromEnv#testProxyUserFromEnvironment failed
[
https://issues.apache.org/jira/browse/MAPREDUCE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on MAPREDUCE-5371 started by Xi Fang.
TestProxyUserFromEnv#testProxyUserFromEnvironment failed caused by domains of
windows users
[
https://issues.apache.org/jira/browse/MAPREDUCE-5371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13698240#comment-13698240
]
Xi Fang commented on MAPREDUCE-5371:
The attached patch removed the domains from
[
https://issues.apache.org/jira/browse/MAPREDUCE-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695116#comment-13695116
]
Xi Fang commented on MAPREDUCE-5109:
Hi Vinod, thanks for your patch. If Hadoop
Xi Fang created MAPREDUCE-5330:
--
Summary: Killing M/R JVM's leads to metrics not being uploaded
Key: MAPREDUCE-5330
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5330
Project: Hadoop Map/Reduce
[
https://issues.apache.org/jira/browse/MAPREDUCE-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5330:
---
Attachment: MAPREDUCE-5330.patch
Killing M/R JVM's leads to metrics not being uploaded
[
https://issues.apache.org/jira/browse/MAPREDUCE-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687346#comment-13687346
]
Xi Fang commented on MAPREDUCE-5330:
If Signal.TERM is sent to a process, then we
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13686060#comment-13686060
]
Xi Fang commented on MAPREDUCE-5278:
Thanks Bikas for your comments. For your
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Attachment: MAPREDUCE-5278.3.patch
Distributed cache is broken when JT staging dir is not
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13686233#comment-13686233
]
Xi Fang commented on MAPREDUCE-5278:
Thanks Bikas. A config name was added in
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Attachment: MAPREDUCE-5278.2.patch
Distributed cache is broken when JT staging dir is not
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13680075#comment-13680075
]
Xi Fang commented on MAPREDUCE-5278:
Thanks Ivan. I have added a classpath check
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on MAPREDUCE-5278 started by Xi Fang.
Perf: Distributed cache is broken when JT staging dir is not on the default FS
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Attachment: MAPREDUCE-5278.patch
A patch is attached.
In this patch, we added a property
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Fix Version/s: 1-win
Target Version/s: 1-win
Status: Patch Available (was:
Xi Fang created MAPREDUCE-5278:
--
Summary: Perf: Distributed cache is broken when JT staging dir is
not on the default FS
Key: MAPREDUCE-5278
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5278
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Assignee: Xi Fang
Perf: Distributed cache is broken when JT staging dir is not on the
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Description:
Today, we set the JobTracker staging dir
(mapreduce.jobtracker.staging.root.dir)
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668119#comment-13668119
]
Xi Fang commented on MAPREDUCE-5224:
Thanks Ivan!
JobTracker
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13668117#comment-13668117
]
Xi Fang commented on MAPREDUCE-5278:
Basically, if a remote file system is
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Description:
Today, the JobTracker staging dir (mapreduce.jobtracker.staging.root.dir) is
set
[
https://issues.apache.org/jira/browse/MAPREDUCE-5278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5278:
---
Description:
Today, the JobTracker staging dir (mapreduce.jobtracker.staging.root.dir) is
set
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Attachment: MAPREDUCE-5224.5.patch
JobTracker should allow the system directory to be in
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13667437#comment-13667437
]
Xi Fang commented on MAPREDUCE-5224:
Thank Ivan for MAPREDUCE-5224.5.patch.
Here
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665679#comment-13665679
]
Xi Fang commented on MAPREDUCE-5224:
Thanks Ivan for your detailed comments. These
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Attachment: MAPREDUCE-5224.4.patch
Above comments have been addressed. Thanks.
BTW, I changed
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Attachment: MAPREDUCE-5224.3.patch
JobTracker should allow the system directory to be in
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13663457#comment-13663457
]
Xi Fang commented on MAPREDUCE-5224:
Thanks Chuan and Ivan.
1. For Chuan's
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13662398#comment-13662398
]
Xi Fang commented on MAPREDUCE-5224:
Hi Ivan, I was addressing your fourth
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13662400#comment-13662400
]
Xi Fang commented on MAPREDUCE-5224:
Sorry for the format! The system changed my
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661415#comment-13661415
]
Xi Fang commented on MAPREDUCE-5224:
Thanks Ivan for the comments. That is of
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661206#comment-13661206
]
Xi Fang commented on MAPREDUCE-5224:
Thanks Chuan.
JobTracker
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Attachment: MAPREDUCE-5224.2.patch
JobTracker should allow the system directory to be in
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13656644#comment-13656644
]
Xi Fang commented on MAPREDUCE-5224:
I updated the patch. It includes the unit
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Attachment: MAPREDUCE-5224.patch
JobTracker should allow the system directory to be in
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Attachment: (was: MAPREDUCE-5224.patch)
JobTracker should allow the system directory to
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Attachment: MAPREDUCE-5224.1.patch
JobTracker should allow the system directory to be in
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Attachment: (was: MAPREDUCE-5224.1.patch)
JobTracker should allow the system directory
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on MAPREDUCE-5224 started by Xi Fang.
JobTracker should allow the system directory to be in non-default FS
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Attachment: MAPREDUCE-5224.patch
JobTracker should allow the system directory to be in
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13653473#comment-13653473
]
Xi Fang commented on MAPREDUCE-5224:
The original motivation of this JIRA is
Xi Fang created MAPREDUCE-5224:
--
Summary: JobTracker should allow the system directory to be in
non-default FS
Key: MAPREDUCE-5224
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5224
Project:
[
https://issues.apache.org/jira/browse/MAPREDUCE-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5224:
---
Description:
JobTracker today expects the system directory to be in the default file system
77 matches
Mail list logo