[jira] [Created] (MAPREDUCE-3881) building fail under Windows
building fail under Windows
---
Key: MAPREDUCE-3881
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3881
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: build
Environment: D:\os\hadoopcommonmvn --version
Apache Maven 3.0.4 (r1232337; 2012-01-17 16:44:56+0800)
Maven home: C:\portable\maven\bin\..
Java version: 1.7.0_02, vendor: Oracle Corporation
Java home: C:\Program Files (x86)\Java\jdk1.7.0_02\jre
Default locale: zh_CN, platform encoding: GBK
OS name: windows 7, version: 6.1, arch: x86, family: windows
Reporter: Changming Sun
Priority: Minor
hadoop-mapreduce-project\hadoop-yarn\hadoop-yarn-common\pom.xml is not portable.
execution
idgenerate-version/id
phasegenerate-sources/phase
configuration
executablescripts/saveVersion.sh/executable
arguments
argument${project.version}/argument
argument${project.build.directory}/argument
/arguments
/configuration
goals
goalexec/goal
/goals
/execution
when I built it under windows , I got a such error:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec (gen
erate-version) on project hadoop-yarn-common: Command execution failed. Cannot r
un program scripts\saveVersion.sh (in directory D:\os\hadoopcommon\hadoop-map
reduce-project\hadoop-yarn\hadoop-yarn-common): CreateProcess error=2,
? - [Help 1]
we can modify it like this: (copied from
hadoop-common-project\hadoop-common\pom.xml)
configuration
target
mkdir dir=${project.build.directory}/generated-sources/java/
exec executable=sh
arg
line=${basedir}/dev-support/saveVersion.sh
${project.version} ${project.build.directory}/generated-sources/java/
/exec
/target
/configuration
/execution
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3881) building fail under Windows
[
https://issues.apache.org/jira/browse/MAPREDUCE-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Changming Sun updated MAPREDUCE-3881:
-
Description:
hadoop-mapreduce-project\hadoop-yarn\hadoop-yarn-common\pom.xml is not portable.
execution
idgenerate-version/id
phasegenerate-sources/phase
configuration
executablescripts/saveVersion.sh/executable
arguments
argument${project.version}/argument
argument${project.build.directory}/argument
/arguments
/configuration
goals
goalexec/goal
/goals
/execution
when I built it under windows , I got a such error:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec (gen
erate-version) on project hadoop-yarn-common: Command execution failed. Cannot r
un program scripts\saveVersion.sh (in directory D:\os\hadoopcommon\hadoop-map
reduce-project\hadoop-yarn\hadoop-yarn-common): CreateProcess error=2,
? - [Help 1]
we should modify it like this: (copied from
hadoop-common-project\hadoop-common\pom.xml)
configuration
target
mkdir dir=${project.build.directory}/generated-sources/java/
exec executable=sh
arg
line=${basedir}/dev-support/saveVersion.sh
${project.version} ${project.build.directory}/generated-sources/java/
/exec
/target
/configuration
/execution
was:
hadoop-mapreduce-project\hadoop-yarn\hadoop-yarn-common\pom.xml is not portable.
execution
idgenerate-version/id
phasegenerate-sources/phase
configuration
executablescripts/saveVersion.sh/executable
arguments
argument${project.version}/argument
argument${project.build.directory}/argument
/arguments
/configuration
goals
goalexec/goal
/goals
/execution
when I built it under windows , I got a such error:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec (gen
erate-version) on project hadoop-yarn-common: Command execution failed. Cannot r
un program scripts\saveVersion.sh (in directory D:\os\hadoopcommon\hadoop-map
reduce-project\hadoop-yarn\hadoop-yarn-common): CreateProcess error=2,
? - [Help 1]
we can modify it like this: (copied from
hadoop-common-project\hadoop-common\pom.xml)
configuration
target
mkdir dir=${project.build.directory}/generated-sources/java/
exec executable=sh
arg
line=${basedir}/dev-support/saveVersion.sh
${project.version} ${project.build.directory}/generated-sources/java/
/exec
/target
/configuration
/execution
building fail under Windows
---
Key: MAPREDUCE-3881
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3881
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: build
Environment: D:\os\hadoopcommonmvn --version
Apache Maven 3.0.4 (r1232337; 2012-01-17 16:44:56+0800)
Maven home: C:\portable\maven\bin\..
Java version: 1.7.0_02, vendor: Oracle Corporation
Java home: C:\Program Files (x86)\Java\jdk1.7.0_02\jre
Default locale: zh_CN, platform encoding: GBK
OS name: windows 7, version: 6.1, arch: x86, family: windows
Reporter: Changming Sun
Priority: Minor
Original Estimate: 1h
Remaining Estimate: 1h
hadoop-mapreduce-project\hadoop-yarn\hadoop-yarn-common\pom.xml is not
portable.
execution
idgenerate-version/id
phasegenerate-sources/phase
configuration
executablescripts/saveVersion.sh/executable
arguments
argument${project.version}/argument
argument${project.build.directory}/argument
/arguments
/configuration
goals
goalexec/goal
/goals
/execution
when I built it under windows , I got a such error:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec
(gen
erate-version) on project hadoop-yarn-common: Command execution failed.
Cannot r
un program scripts\saveVersion.sh (in directory
D:\os\hadoopcommon\hadoop-map
reduce-project\hadoop-yarn\hadoop-yarn-common): CreateProcess error=2,
? - [Help 1]
we should modify it like this: (copied from
hadoop-common-project\hadoop-common\pom.xml)
configuration
target
mkdir
dir=${project.build.directory}/generated-sources/java/
[jira] [Commented] (MAPREDUCE-3881) building fail under Windows
[
https://issues.apache.org/jira/browse/MAPREDUCE-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210883#comment-13210883
]
Arun C Murthy commented on MAPREDUCE-3881:
--
Changmin, could you please provide a patch? Thanks!
building fail under Windows
---
Key: MAPREDUCE-3881
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3881
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: build
Environment: D:\os\hadoopcommonmvn --version
Apache Maven 3.0.4 (r1232337; 2012-01-17 16:44:56+0800)
Maven home: C:\portable\maven\bin\..
Java version: 1.7.0_02, vendor: Oracle Corporation
Java home: C:\Program Files (x86)\Java\jdk1.7.0_02\jre
Default locale: zh_CN, platform encoding: GBK
OS name: windows 7, version: 6.1, arch: x86, family: windows
Reporter: Changming Sun
Priority: Minor
Original Estimate: 1h
Remaining Estimate: 1h
hadoop-mapreduce-project\hadoop-yarn\hadoop-yarn-common\pom.xml is not
portable.
execution
idgenerate-version/id
phasegenerate-sources/phase
configuration
executablescripts/saveVersion.sh/executable
arguments
argument${project.version}/argument
argument${project.build.directory}/argument
/arguments
/configuration
goals
goalexec/goal
/goals
/execution
when I built it under windows , I got a such error:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec
(gen
erate-version) on project hadoop-yarn-common: Command execution failed.
Cannot r
un program scripts\saveVersion.sh (in directory
D:\os\hadoopcommon\hadoop-map
reduce-project\hadoop-yarn\hadoop-yarn-common): CreateProcess error=2,
? - [Help 1]
we should modify it like this: (copied from
hadoop-common-project\hadoop-common\pom.xml)
configuration
target
mkdir
dir=${project.build.directory}/generated-sources/java/
exec executable=sh
arg
line=${basedir}/dev-support/saveVersion.sh
${project.version} ${project.build.directory}/generated-sources/java/
/exec
/target
/configuration
/execution
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3881) building fail under Windows
[
https://issues.apache.org/jira/browse/MAPREDUCE-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Changming Sun updated MAPREDUCE-3881:
-
Status: Patch Available (was: Open)
Index: hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
===
--- hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
(revision 1245861)
+++ hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/pom.xml
(working copy)
@@ -108,8 +108,9 @@
idgenerate-version/id
phasegenerate-sources/phase
configuration
- executablescripts/saveVersion.sh/executable
+ executablesh/executable
arguments
+ argumentscripts/saveVersion.sh/argument
argument${project.version}/argument
argument${project.build.directory}/argument
/arguments
building fail under Windows
---
Key: MAPREDUCE-3881
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3881
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: build
Environment: D:\os\hadoopcommonmvn --version
Apache Maven 3.0.4 (r1232337; 2012-01-17 16:44:56+0800)
Maven home: C:\portable\maven\bin\..
Java version: 1.7.0_02, vendor: Oracle Corporation
Java home: C:\Program Files (x86)\Java\jdk1.7.0_02\jre
Default locale: zh_CN, platform encoding: GBK
OS name: windows 7, version: 6.1, arch: x86, family: windows
Reporter: Changming Sun
Priority: Minor
Original Estimate: 1h
Remaining Estimate: 1h
hadoop-mapreduce-project\hadoop-yarn\hadoop-yarn-common\pom.xml is not
portable.
execution
idgenerate-version/id
phasegenerate-sources/phase
configuration
executablescripts/saveVersion.sh/executable
arguments
argument${project.version}/argument
argument${project.build.directory}/argument
/arguments
/configuration
goals
goalexec/goal
/goals
/execution
when I built it under windows , I got a such error:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec
(gen
erate-version) on project hadoop-yarn-common: Command execution failed.
Cannot r
un program scripts\saveVersion.sh (in directory
D:\os\hadoopcommon\hadoop-map
reduce-project\hadoop-yarn\hadoop-yarn-common): CreateProcess error=2,
? - [Help 1]
we should modify it like this: (copied from
hadoop-common-project\hadoop-common\pom.xml)
configuration
target
mkdir
dir=${project.build.directory}/generated-sources/java/
exec executable=sh
arg
line=${basedir}/dev-support/saveVersion.sh
${project.version} ${project.build.directory}/generated-sources/java/
/exec
/target
/configuration
/execution
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3881) building fail under Windows
[
https://issues.apache.org/jira/browse/MAPREDUCE-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210892#comment-13210892
]
Hadoop QA commented on MAPREDUCE-3881:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515081/pom.xml.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1894//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1894//console
This message is automatically generated.
building fail under Windows
---
Key: MAPREDUCE-3881
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3881
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: build
Environment: D:\os\hadoopcommonmvn --version
Apache Maven 3.0.4 (r1232337; 2012-01-17 16:44:56+0800)
Maven home: C:\portable\maven\bin\..
Java version: 1.7.0_02, vendor: Oracle Corporation
Java home: C:\Program Files (x86)\Java\jdk1.7.0_02\jre
Default locale: zh_CN, platform encoding: GBK
OS name: windows 7, version: 6.1, arch: x86, family: windows
Reporter: Changming Sun
Priority: Minor
Attachments: pom.xml.patch
Original Estimate: 1h
Remaining Estimate: 1h
hadoop-mapreduce-project\hadoop-yarn\hadoop-yarn-common\pom.xml is not
portable.
execution
idgenerate-version/id
phasegenerate-sources/phase
configuration
executablescripts/saveVersion.sh/executable
arguments
argument${project.version}/argument
argument${project.build.directory}/argument
/arguments
/configuration
goals
goalexec/goal
/goals
/execution
when I built it under windows , I got a such error:
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec
(gen
erate-version) on project hadoop-yarn-common: Command execution failed.
Cannot r
un program scripts\saveVersion.sh (in directory
D:\os\hadoopcommon\hadoop-map
reduce-project\hadoop-yarn\hadoop-yarn-common): CreateProcess error=2,
? - [Help 1]
we should modify it like this: (copied from
hadoop-common-project\hadoop-common\pom.xml)
configuration
target
mkdir
dir=${project.build.directory}/generated-sources/java/
exec executable=sh
arg
line=${basedir}/dev-support/saveVersion.sh
${project.version} ${project.build.directory}/generated-sources/java/
/exec
/target
/configuration
/execution
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3882) fix some compile warnings of hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Changming Sun updated MAPREDUCE-3882: - Status: Patch Available (was: Open) fix some compile warnings of hadoop-mapreduce-examples -- Key: MAPREDUCE-3882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3882 Project: Hadoop Map/Reduce Issue Type: Improvement Environment: Windows 7 Reporter: Changming Sun Priority: Minor Original Estimate: 2m Remaining Estimate: 2m fix some compile warnings of hadoop-mapreduce-examples -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3882) fix some compile warnings of hadoop-mapreduce-examples
fix some compile warnings of hadoop-mapreduce-examples -- Key: MAPREDUCE-3882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3882 Project: Hadoop Map/Reduce Issue Type: Improvement Environment: Windows 7 Reporter: Changming Sun Priority: Minor fix some compile warnings of hadoop-mapreduce-examples -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3882) fix some compile warnings of hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Changming Sun updated MAPREDUCE-3882: - Attachment: examples.patch fix some compile warnings of hadoop-mapreduce-examples -- Key: MAPREDUCE-3882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3882 Project: Hadoop Map/Reduce Issue Type: Improvement Environment: Windows 7 Reporter: Changming Sun Priority: Minor Attachments: examples.patch Original Estimate: 2m Remaining Estimate: 2m fix some compile warnings of hadoop-mapreduce-examples -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3880) Allow for 32-bit container-executor
[ https://issues.apache.org/jira/browse/MAPREDUCE-3880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210908#comment-13210908 ] Hudson commented on MAPREDUCE-3880: --- Integrated in Hadoop-Hdfs-trunk #959 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/959/]) MAPREDUCE-3880. Changed LCE binary to be 32-bit. (Revision 1245867) Result = SUCCESS acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245867 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml Allow for 32-bit container-executor --- Key: MAPREDUCE-3880 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3880 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, mrv2 Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3880.patch Currently we can't pass in -m32 to LCE build. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3862) Nodemanager can appear to hang on shutdown due to lingering DeletionService threads
[
https://issues.apache.org/jira/browse/MAPREDUCE-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210907#comment-13210907
]
Hudson commented on MAPREDUCE-3862:
---
Integrated in Hadoop-Hdfs-trunk #959 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/959/])
MAPREDUCE-3862 Nodemanager can appear to hang on shutdown due to lingering
DeletionService threads (Jason Lowe via bobby) (Revision 1245781)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245781
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDeletionService.java
Nodemanager can appear to hang on shutdown due to lingering DeletionService
threads
---
Key: MAPREDUCE-3862
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3862
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, nodemanager
Affects Versions: 0.23.1
Reporter: Jason Lowe
Assignee: Jason Lowe
Fix For: 0.23.2
Attachments: MAPREDUCE-3862.patch, MAPREDUCE-3862.patch
When a nodemanager attempts to shutdown cleanly, it's possible for it to
appear to hang due to lingering DeletionService threads. This can occur when
yarn.nodemanager.delete.debug-delay-sec is set to a relatively large value
and one or more containers executes on the node shortly before the shutdown.
The DeletionService is never calling
{{setExecuteExistingDelayedTasksAfterShutdownPolicy()}} on the
ScheduledThreadPoolExecutor, and it defaults to waiting for all scheduled
tasks to complete before exiting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3856) Instances of RunningJob class givs incorrect job tracking urls when mutiple jobs are submitted from same client jvm.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210913#comment-13210913 ] Hudson commented on MAPREDUCE-3856: --- Integrated in Hadoop-Hdfs-trunk #959 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/959/]) MAPREDUCE-3856. Committing to 0.23.1. (Revision 1245840) Result = SUCCESS acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245840 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt Instances of RunningJob class givs incorrect job tracking urls when mutiple jobs are submitted from same client jvm. Key: MAPREDUCE-3856 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3856 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Eric Payne Assignee: Eric Payne Priority: Critical Fix For: 0.23.1 Attachments: MAPREDUCE-3856-1.txt When multiple jobs are submitted from the same client JVM, each call to RunningJob.getTrackingURL() always returns the tracking URL from the first job. This happens even if the jobs are submitted and the client waits for the job to complete before submitting the subsequent job. Each job runs fine and is definitely a new, unique job, but the call to getTrackingURL() still returns the URL for the first job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3882) fix some compile warnings of hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Changming Sun updated MAPREDUCE-3882: - Attachment: (was: examples.patch) fix some compile warnings of hadoop-mapreduce-examples -- Key: MAPREDUCE-3882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3882 Project: Hadoop Map/Reduce Issue Type: Improvement Environment: Windows 7 Reporter: Changming Sun Priority: Minor Original Estimate: 2m Remaining Estimate: 2m fix some compile warnings of hadoop-mapreduce-examples -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3882) fix some compile warnings of hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Changming Sun updated MAPREDUCE-3882: - Status: Patch Available (was: Open) fix some compile warnings of hadoop-mapreduce-examples -- Key: MAPREDUCE-3882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3882 Project: Hadoop Map/Reduce Issue Type: Improvement Environment: Windows 7 Reporter: Changming Sun Priority: Minor Attachments: mapreduce-3882.patch Original Estimate: 2m Remaining Estimate: 2m fix some compile warnings of hadoop-mapreduce-examples -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3882) fix some compile warnings of hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Changming Sun updated MAPREDUCE-3882: - Attachment: mapreduce-3882.patch fix some compile warnings of hadoop-mapreduce-examples -- Key: MAPREDUCE-3882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3882 Project: Hadoop Map/Reduce Issue Type: Improvement Environment: Windows 7 Reporter: Changming Sun Priority: Minor Attachments: mapreduce-3882.patch Original Estimate: 2m Remaining Estimate: 2m fix some compile warnings of hadoop-mapreduce-examples -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3877) Add a test to formalise the current state transitions of the yarn lifecycle
[ https://issues.apache.org/jira/browse/MAPREDUCE-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210929#comment-13210929 ] Hudson commented on MAPREDUCE-3877: --- Integrated in Hadoop-Hdfs-0.23-Commit #560 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/560/]) MAPREDUCE-3877 Add a test to formalise the current state transitions of the yarn lifecycle (Revision 1245914) MAPREDUCE-3877 Add a test to formalise the current state transitions of the yarn lifecycle (Revision 1245913) Result = SUCCESS stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245914 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/BreakableService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/ServiceAssert.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/TestServiceLifecycle.java stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245913 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt Add a test to formalise the current state transitions of the yarn lifecycle --- Key: MAPREDUCE-3877 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3877 Project: Hadoop Map/Reduce Issue Type: Test Components: mrv2 Affects Versions: 0.24.0 Reporter: Steve Loughran Assignee: Steve Loughran Priority: Minor Attachments: MAPREDUCE-3877.patch Original Estimate: 1h Remaining Estimate: 1h Add a test service that counts the number of times it's state methods are called; and can be set to raise an exception on any such entry. Use it to show what the current lifecycle state model is, so that unintentional regressions can be detected. It will also act as a foundation for intentional changes to that lifecycle -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3877) Add a test to formalise the current state transitions of the yarn lifecycle
[ https://issues.apache.org/jira/browse/MAPREDUCE-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210932#comment-13210932 ] Hudson commented on MAPREDUCE-3877: --- Integrated in Hadoop-Common-0.23-Commit #574 (See [https://builds.apache.org/job/Hadoop-Common-0.23-Commit/574/]) MAPREDUCE-3877 Add a test to formalise the current state transitions of the yarn lifecycle (Revision 1245914) MAPREDUCE-3877 Add a test to formalise the current state transitions of the yarn lifecycle (Revision 1245913) Result = SUCCESS stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245914 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/BreakableService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/ServiceAssert.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/TestServiceLifecycle.java stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245913 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt Add a test to formalise the current state transitions of the yarn lifecycle --- Key: MAPREDUCE-3877 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3877 Project: Hadoop Map/Reduce Issue Type: Test Components: mrv2 Affects Versions: 0.24.0 Reporter: Steve Loughran Assignee: Steve Loughran Priority: Minor Attachments: MAPREDUCE-3877.patch Original Estimate: 1h Remaining Estimate: 1h Add a test service that counts the number of times it's state methods are called; and can be set to raise an exception on any such entry. Use it to show what the current lifecycle state model is, so that unintentional regressions can be detected. It will also act as a foundation for intentional changes to that lifecycle -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3877) Add a test to formalise the current state transitions of the yarn lifecycle
[ https://issues.apache.org/jira/browse/MAPREDUCE-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-3877: -- Resolution: Fixed Fix Version/s: 0.23.2 0.24.0 Status: Resolved (was: Patch Available) Add a test to formalise the current state transitions of the yarn lifecycle --- Key: MAPREDUCE-3877 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3877 Project: Hadoop Map/Reduce Issue Type: Test Components: mrv2 Affects Versions: 0.24.0 Reporter: Steve Loughran Assignee: Steve Loughran Priority: Minor Fix For: 0.24.0, 0.23.2 Attachments: MAPREDUCE-3877.patch Original Estimate: 1h Remaining Estimate: 1h Add a test service that counts the number of times it's state methods are called; and can be set to raise an exception on any such entry. Use it to show what the current lifecycle state model is, so that unintentional regressions can be detected. It will also act as a foundation for intentional changes to that lifecycle -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3877) Add a test to formalise the current state transitions of the yarn lifecycle
[ https://issues.apache.org/jira/browse/MAPREDUCE-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210934#comment-13210934 ] Steve Loughran commented on MAPREDUCE-3877: --- I'm going to note that these tests show flaws in the current lifecycle that need to be addressed. Specifically # as the ensureState checks are performed at the end of all subclasses start/stop/init operations, they always perform their operations regardless of the current state. This allows you to enter these states more than once # if anything goes wrong during init and start the system is left in an indeterminate state # you can't stop without starting, yet init can acquire resources that need to be explicitly released -and which you may want to release without starting the service. Add a test to formalise the current state transitions of the yarn lifecycle --- Key: MAPREDUCE-3877 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3877 Project: Hadoop Map/Reduce Issue Type: Test Components: mrv2 Affects Versions: 0.24.0 Reporter: Steve Loughran Assignee: Steve Loughran Priority: Minor Fix For: 0.24.0, 0.23.2 Attachments: MAPREDUCE-3877.patch Original Estimate: 1h Remaining Estimate: 1h Add a test service that counts the number of times it's state methods are called; and can be set to raise an exception on any such entry. Use it to show what the current lifecycle state model is, so that unintentional regressions can be detected. It will also act as a foundation for intentional changes to that lifecycle -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3882) fix some compile warnings of hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210936#comment-13210936 ] Hadoop QA commented on MAPREDUCE-3882: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12515083/examples.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1895//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1895//console This message is automatically generated. fix some compile warnings of hadoop-mapreduce-examples -- Key: MAPREDUCE-3882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3882 Project: Hadoop Map/Reduce Issue Type: Improvement Environment: Windows 7 Reporter: Changming Sun Priority: Minor Attachments: mapreduce-3882.patch Original Estimate: 2m Remaining Estimate: 2m fix some compile warnings of hadoop-mapreduce-examples -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3877) Add a test to formalise the current state transitions of the yarn lifecycle
[ https://issues.apache.org/jira/browse/MAPREDUCE-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210938#comment-13210938 ] Hudson commented on MAPREDUCE-3877: --- Integrated in Hadoop-Hdfs-trunk-Commit #1827 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1827/]) MAPREDUCE-3877 Add a test to formalise the current state transitions of the yarn lifecycle (Revision 1245918) Result = FAILURE stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245918 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/BreakableService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/ServiceAssert.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/TestServiceLifecycle.java Add a test to formalise the current state transitions of the yarn lifecycle --- Key: MAPREDUCE-3877 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3877 Project: Hadoop Map/Reduce Issue Type: Test Components: mrv2 Affects Versions: 0.24.0 Reporter: Steve Loughran Assignee: Steve Loughran Priority: Minor Fix For: 0.24.0, 0.23.2 Attachments: MAPREDUCE-3877.patch Original Estimate: 1h Remaining Estimate: 1h Add a test service that counts the number of times it's state methods are called; and can be set to raise an exception on any such entry. Use it to show what the current lifecycle state model is, so that unintentional regressions can be detected. It will also act as a foundation for intentional changes to that lifecycle -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3882) fix some compile warnings of hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210944#comment-13210944 ] Hadoop QA commented on MAPREDUCE-3882: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12515084/mapreduce-3882.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1896//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1896//console This message is automatically generated. fix some compile warnings of hadoop-mapreduce-examples -- Key: MAPREDUCE-3882 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3882 Project: Hadoop Map/Reduce Issue Type: Improvement Environment: Windows 7 Reporter: Changming Sun Priority: Minor Attachments: mapreduce-3882.patch Original Estimate: 2m Remaining Estimate: 2m fix some compile warnings of hadoop-mapreduce-examples -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3862) Nodemanager can appear to hang on shutdown due to lingering DeletionService threads
[
https://issues.apache.org/jira/browse/MAPREDUCE-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210946#comment-13210946
]
Hudson commented on MAPREDUCE-3862:
---
Integrated in Hadoop-Mapreduce-0.23-Build #200 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/200/])
svn merge -c 1245781 from trunk to branch 0.23 FIXES MAPREDUCE-3862.
Nodemanager can appear to hang on shutdown due to lingering DeletionService
threads (Jason Lowe via bobby) (Revision 1245794)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245794
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDeletionService.java
Nodemanager can appear to hang on shutdown due to lingering DeletionService
threads
---
Key: MAPREDUCE-3862
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3862
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, nodemanager
Affects Versions: 0.23.1
Reporter: Jason Lowe
Assignee: Jason Lowe
Fix For: 0.23.2
Attachments: MAPREDUCE-3862.patch, MAPREDUCE-3862.patch
When a nodemanager attempts to shutdown cleanly, it's possible for it to
appear to hang due to lingering DeletionService threads. This can occur when
yarn.nodemanager.delete.debug-delay-sec is set to a relatively large value
and one or more containers executes on the node shortly before the shutdown.
The DeletionService is never calling
{{setExecuteExistingDelayedTasksAfterShutdownPolicy()}} on the
ScheduledThreadPoolExecutor, and it defaults to waiting for all scheduled
tasks to complete before exiting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3880) Allow for 32-bit container-executor
[ https://issues.apache.org/jira/browse/MAPREDUCE-3880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210947#comment-13210947 ] Hudson commented on MAPREDUCE-3880: --- Integrated in Hadoop-Mapreduce-0.23-Build #200 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/200/]) Merge -c 1245867 from trunk to branch-0.23 to fix MAPREDUCE-3880. Changed LCE binary to be 32-bit. (Revision 1245868) Result = FAILURE acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245868 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml Allow for 32-bit container-executor --- Key: MAPREDUCE-3880 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3880 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, mrv2 Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Blocker Fix For: 0.23.1 Attachments: MAPREDUCE-3880.patch Currently we can't pass in -m32 to LCE build. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3862) Nodemanager can appear to hang on shutdown due to lingering DeletionService threads
[
https://issues.apache.org/jira/browse/MAPREDUCE-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210954#comment-13210954
]
Hudson commented on MAPREDUCE-3862:
---
Integrated in Hadoop-Mapreduce-trunk #994 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/994/])
MAPREDUCE-3862 Nodemanager can appear to hang on shutdown due to lingering
DeletionService threads (Jason Lowe via bobby) (Revision 1245781)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245781
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDeletionService.java
Nodemanager can appear to hang on shutdown due to lingering DeletionService
threads
---
Key: MAPREDUCE-3862
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3862
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, nodemanager
Affects Versions: 0.23.1
Reporter: Jason Lowe
Assignee: Jason Lowe
Fix For: 0.23.2
Attachments: MAPREDUCE-3862.patch, MAPREDUCE-3862.patch
When a nodemanager attempts to shutdown cleanly, it's possible for it to
appear to hang due to lingering DeletionService threads. This can occur when
yarn.nodemanager.delete.debug-delay-sec is set to a relatively large value
and one or more containers executes on the node shortly before the shutdown.
The DeletionService is never calling
{{setExecuteExistingDelayedTasksAfterShutdownPolicy()}} on the
ScheduledThreadPoolExecutor, and it defaults to waiting for all scheduled
tasks to complete before exiting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3856) Instances of RunningJob class givs incorrect job tracking urls when mutiple jobs are submitted from same client jvm.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210960#comment-13210960 ] Hudson commented on MAPREDUCE-3856: --- Integrated in Hadoop-Mapreduce-trunk #994 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/994/]) MAPREDUCE-3856. Committing to 0.23.1. (Revision 1245840) Result = SUCCESS acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245840 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt Instances of RunningJob class givs incorrect job tracking urls when mutiple jobs are submitted from same client jvm. Key: MAPREDUCE-3856 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3856 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Eric Payne Assignee: Eric Payne Priority: Critical Fix For: 0.23.1 Attachments: MAPREDUCE-3856-1.txt When multiple jobs are submitted from the same client JVM, each call to RunningJob.getTrackingURL() always returns the tracking URL from the first job. This happens even if the jobs are submitted and the client waits for the job to complete before submitting the subsequent job. Each job runs fine and is definitely a new, unique job, but the call to getTrackingURL() still returns the URL for the first job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3583) ProcfsBasedProcessTree#constructProcessInfo() may throw NumberFormatException
[
https://issues.apache.org/jira/browse/MAPREDUCE-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210953#comment-13210953
]
Hudson commented on MAPREDUCE-3583:
---
Integrated in Hadoop-Mapreduce-0.23-Build #200 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/200/])
svn merge -c 1245828 from trunk for MAPREDUCE-3583. (Revision 1245831)
Result = FAILURE
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245831
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ProcfsBasedProcessTree.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java
*
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java
ProcfsBasedProcessTree#constructProcessInfo() may throw NumberFormatException
-
Key: MAPREDUCE-3583
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3583
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.20.205.0
Environment: 64-bit Linux:
asf011.sp2.ygridcore.net
Linux asf011.sp2.ygridcore.net 2.6.32-33-server #71-Ubuntu SMP Wed Jul 20
17:42:25 UTC 2011 x86_64 GNU/Linux
Reporter: Zhihong Yu
Assignee: Zhihong Yu
Priority: Critical
Fix For: 0.24.0, 0.23.2
Attachments: mapreduce-3583-trunk-v2.txt,
mapreduce-3583-trunk-v2.txt, mapreduce-3583-trunk-v3.txt,
mapreduce-3583-trunk-v4.txt, mapreduce-3583-trunk-v5.txt,
mapreduce-3583-trunk-v6.txt, mapreduce-3583-trunk-v7.txt,
mapreduce-3583-trunk.txt, mapreduce-3583-v2.txt, mapreduce-3583-v3.txt,
mapreduce-3583-v4.txt, mapreduce-3583-v5.txt, mapreduce-3583.txt
HBase PreCommit builds frequently gave us NumberFormatException.
From
https://builds.apache.org/job/PreCommit-HBASE-Build/553//testReport/org.apache.hadoop.hbase.mapreduce/TestHFileOutputFormat/testMRIncrementalLoad/:
{code}
2011-12-20 01:44:01,180 WARN [main] mapred.JobClient(784): No job jar file
set. User classes may not be found. See JobConf(Class) or
JobConf#setJar(String).
java.lang.NumberFormatException: For input string: 18446743988060683582
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:422)
at java.lang.Long.parseLong(Long.java:468)
at
org.apache.hadoop.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:413)
at
org.apache.hadoop.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:148)
at
org.apache.hadoop.util.LinuxResourceCalculatorPlugin.getProcResourceValues(LinuxResourceCalculatorPlugin.java:401)
at org.apache.hadoop.mapred.Task.initialize(Task.java:536)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{code}
From hadoop 0.20.205 source code, looks like ppid was 18446743988060683582,
causing NFE:
{code}
// Set (name) (ppid) (pgrpId) (session) (utime) (stime) (vsize) (rss)
pinfo.updateProcessInfo(m.group(2), Integer.parseInt(m.group(3)),
{code}
You can find information on the OS at the beginning of
https://builds.apache.org/job/PreCommit-HBASE-Build/553/console:
{code}
asf011.sp2.ygridcore.net
Linux asf011.sp2.ygridcore.net 2.6.32-33-server #71-Ubuntu SMP Wed Jul 20
17:42:25 UTC 2011 x86_64 GNU/Linux
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 20
file size (blocks, -f) unlimited
pending signals (-i) 16382
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 6
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 2048
virtual memory (kbytes, -v) unlimited
file locks
[jira] [Commented] (MAPREDUCE-3856) Instances of RunningJob class givs incorrect job tracking urls when mutiple jobs are submitted from same client jvm.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210951#comment-13210951 ] Hudson commented on MAPREDUCE-3856: --- Integrated in Hadoop-Mapreduce-0.23-Build #200 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/200/]) MAPREDUCE-3856. Committing to 0.23.1. (Revision 1245842) Result = FAILURE acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245842 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt Instances of RunningJob class givs incorrect job tracking urls when mutiple jobs are submitted from same client jvm. Key: MAPREDUCE-3856 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3856 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Eric Payne Assignee: Eric Payne Priority: Critical Fix For: 0.23.1 Attachments: MAPREDUCE-3856-1.txt When multiple jobs are submitted from the same client JVM, each call to RunningJob.getTrackingURL() always returns the tracking URL from the first job. This happens even if the jobs are submitted and the client waits for the job to complete before submitting the subsequent job. Each job runs fine and is definitely a new, unique job, but the call to getTrackingURL() still returns the URL for the first job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3583) ProcfsBasedProcessTree#constructProcessInfo() may throw NumberFormatException
[
https://issues.apache.org/jira/browse/MAPREDUCE-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210962#comment-13210962
]
Hudson commented on MAPREDUCE-3583:
---
Integrated in Hadoop-Mapreduce-trunk #994 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/994/])
MAPREDUCE-3583. Change pid to String and stime to BigInteger in order to
handle integers larger than Long.MAX_VALUE. Contributed by Zhihong Yu
(Revision 1245828)
Result = SUCCESS
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245828
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ProcfsBasedProcessTree.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java
ProcfsBasedProcessTree#constructProcessInfo() may throw NumberFormatException
-
Key: MAPREDUCE-3583
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3583
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.20.205.0
Environment: 64-bit Linux:
asf011.sp2.ygridcore.net
Linux asf011.sp2.ygridcore.net 2.6.32-33-server #71-Ubuntu SMP Wed Jul 20
17:42:25 UTC 2011 x86_64 GNU/Linux
Reporter: Zhihong Yu
Assignee: Zhihong Yu
Priority: Critical
Fix For: 0.24.0, 0.23.2
Attachments: mapreduce-3583-trunk-v2.txt,
mapreduce-3583-trunk-v2.txt, mapreduce-3583-trunk-v3.txt,
mapreduce-3583-trunk-v4.txt, mapreduce-3583-trunk-v5.txt,
mapreduce-3583-trunk-v6.txt, mapreduce-3583-trunk-v7.txt,
mapreduce-3583-trunk.txt, mapreduce-3583-v2.txt, mapreduce-3583-v3.txt,
mapreduce-3583-v4.txt, mapreduce-3583-v5.txt, mapreduce-3583.txt
HBase PreCommit builds frequently gave us NumberFormatException.
From
https://builds.apache.org/job/PreCommit-HBASE-Build/553//testReport/org.apache.hadoop.hbase.mapreduce/TestHFileOutputFormat/testMRIncrementalLoad/:
{code}
2011-12-20 01:44:01,180 WARN [main] mapred.JobClient(784): No job jar file
set. User classes may not be found. See JobConf(Class) or
JobConf#setJar(String).
java.lang.NumberFormatException: For input string: 18446743988060683582
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:422)
at java.lang.Long.parseLong(Long.java:468)
at
org.apache.hadoop.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:413)
at
org.apache.hadoop.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:148)
at
org.apache.hadoop.util.LinuxResourceCalculatorPlugin.getProcResourceValues(LinuxResourceCalculatorPlugin.java:401)
at org.apache.hadoop.mapred.Task.initialize(Task.java:536)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{code}
From hadoop 0.20.205 source code, looks like ppid was 18446743988060683582,
causing NFE:
{code}
// Set (name) (ppid) (pgrpId) (session) (utime) (stime) (vsize) (rss)
pinfo.updateProcessInfo(m.group(2), Integer.parseInt(m.group(3)),
{code}
You can find information on the OS at the beginning of
https://builds.apache.org/job/PreCommit-HBASE-Build/553/console:
{code}
asf011.sp2.ygridcore.net
Linux asf011.sp2.ygridcore.net 2.6.32-33-server #71-Ubuntu SMP Wed Jul 20
17:42:25 UTC 2011 x86_64 GNU/Linux
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 20
file size (blocks, -f) unlimited
pending signals (-i) 16382
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 6
pipe size(512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 2048
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
6
[jira] [Commented] (MAPREDUCE-3877) Add a test to formalise the current state transitions of the yarn lifecycle
[ https://issues.apache.org/jira/browse/MAPREDUCE-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210965#comment-13210965 ] Hudson commented on MAPREDUCE-3877: --- Integrated in Hadoop-Mapreduce-0.23-Commit #576 (See [https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/576/]) MAPREDUCE-3877 Add a test to formalise the current state transitions of the yarn lifecycle (Revision 1245914) MAPREDUCE-3877 Add a test to formalise the current state transitions of the yarn lifecycle (Revision 1245913) Result = ABORTED stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245914 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/BreakableService.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/ServiceAssert.java * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/TestServiceLifecycle.java stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245913 Files : * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt Add a test to formalise the current state transitions of the yarn lifecycle --- Key: MAPREDUCE-3877 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3877 Project: Hadoop Map/Reduce Issue Type: Test Components: mrv2 Affects Versions: 0.24.0 Reporter: Steve Loughran Assignee: Steve Loughran Priority: Minor Fix For: 0.24.0, 0.23.2 Attachments: MAPREDUCE-3877.patch Original Estimate: 1h Remaining Estimate: 1h Add a test service that counts the number of times it's state methods are called; and can be set to raise an exception on any such entry. Use it to show what the current lifecycle state model is, so that unintentional regressions can be detected. It will also act as a foundation for intentional changes to that lifecycle -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3851) Allow more aggressive action on detection of the jetty issue
[ https://issues.apache.org/jira/browse/MAPREDUCE-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210966#comment-13210966 ] Thomas Graves commented on MAPREDUCE-3851: -- Without this change there are 10 findbugs warnings. This one introduces: org.apache.hadoop.mapred.ShuffleExceptionTracker.doAbort() invokes System.exit(...), which shuts down the entire virtual machine This is expected, I will update patch to exclude this. [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. [exec] Allow more aggressive action on detection of the jetty issue Key: MAPREDUCE-3851 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3851 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 1.0.0 Reporter: Kihwal Lee Assignee: Thomas Graves Fix For: 1.1.0, 1.0.1 Attachments: MAPREDUCE-3851.patch, MAPREDUCE-3851.patch MAPREDUCE-2529 added the useful failure detection mechanism. In this jira, I propose we add a periodic check inside TT and configurable action to self-destruct. Blacklisting helps but is not enough. Hung jetty still accepts connection and it takes very long time for clients to fail out. Short jobs are delayed for hours because of this. This feature will be a nice companion to MAPREDUCE-3184. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3851) Allow more aggressive action on detection of the jetty issue
[ https://issues.apache.org/jira/browse/MAPREDUCE-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated MAPREDUCE-3851: - Status: Open (was: Patch Available) Allow more aggressive action on detection of the jetty issue Key: MAPREDUCE-3851 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3851 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 1.0.0 Reporter: Kihwal Lee Assignee: Thomas Graves Fix For: 1.1.0, 1.0.1 Attachments: MAPREDUCE-3851.patch, MAPREDUCE-3851.patch MAPREDUCE-2529 added the useful failure detection mechanism. In this jira, I propose we add a periodic check inside TT and configurable action to self-destruct. Blacklisting helps but is not enough. Hung jetty still accepts connection and it takes very long time for clients to fail out. Short jobs are delayed for hours because of this. This feature will be a nice companion to MAPREDUCE-3184. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3877) Add a test to formalise the current state transitions of the yarn lifecycle
[ https://issues.apache.org/jira/browse/MAPREDUCE-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13210970#comment-13210970 ] Hudson commented on MAPREDUCE-3877: --- Integrated in Hadoop-Mapreduce-trunk-Commit #1764 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1764/]) MAPREDUCE-3877 Add a test to formalise the current state transitions of the yarn lifecycle (Revision 1245918) Result = ABORTED stevel : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1245918 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/BreakableService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/ServiceAssert.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/service/TestServiceLifecycle.java Add a test to formalise the current state transitions of the yarn lifecycle --- Key: MAPREDUCE-3877 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3877 Project: Hadoop Map/Reduce Issue Type: Test Components: mrv2 Affects Versions: 0.24.0 Reporter: Steve Loughran Assignee: Steve Loughran Priority: Minor Fix For: 0.24.0, 0.23.2 Attachments: MAPREDUCE-3877.patch Original Estimate: 1h Remaining Estimate: 1h Add a test service that counts the number of times it's state methods are called; and can be set to raise an exception on any such entry. Use it to show what the current lifecycle state model is, so that unintentional regressions can be detected. It will also act as a foundation for intentional changes to that lifecycle -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3851) Allow more aggressive action on detection of the jetty issue
[ https://issues.apache.org/jira/browse/MAPREDUCE-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated MAPREDUCE-3851: - Attachment: MAPREDUCE-3851.patch exclude findbugs warning. test-patch output now matches branch-1.0 without this patch. [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 6 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] -1 findbugs. The patch appears to introduce 10 new Findbugs (version 1.3.9) warnings. Allow more aggressive action on detection of the jetty issue Key: MAPREDUCE-3851 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3851 Project: Hadoop Map/Reduce Issue Type: Bug Components: tasktracker Affects Versions: 1.0.0 Reporter: Kihwal Lee Assignee: Thomas Graves Fix For: 1.1.0, 1.0.1 Attachments: MAPREDUCE-3851.patch, MAPREDUCE-3851.patch, MAPREDUCE-3851.patch MAPREDUCE-2529 added the useful failure detection mechanism. In this jira, I propose we add a periodic check inside TT and configurable action to self-destruct. Blacklisting helps but is not enough. Hung jetty still accepts connection and it takes very long time for clients to fail out. Short jobs are delayed for hours because of this. This feature will be a nice companion to MAPREDUCE-3184. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-1416) New JIRA components for Map/Reduce project
[ https://issues.apache.org/jira/browse/MAPREDUCE-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-1416. --- Resolution: Fixed New JIRA components for Map/Reduce project -- Key: MAPREDUCE-1416 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1416 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Vinod Kumar Vavilapalli Assignee: Steve Loughran We need more JIRA components for the Map/Reduce project for better tracking. Some missing ones: DistributedCache, TaskController, contrib/vaidya, contrib/mruit, contrib/dynamic-scheduler, contrib/data_join. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3685) There are some bugs in implementation of MergeManager
[ https://issues.apache.org/jira/browse/MAPREDUCE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated MAPREDUCE-3685: -- Assignee: Anthony Urso Summary: There are some bugs in implementation of MergeManager (was: There are some bugs in implementation of MergeMnager :) There are some bugs in implementation of MergeManager - Key: MAPREDUCE-3685 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3685 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: anty.rao Assignee: Anthony Urso Priority: Minor Fix For: 0.23.1 Attachments: MAPREDUCE-3685-branch-0.23.1.patch, MAPREDUCE-3685.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3685) There are some bugs in implementation of MergeManager
[ https://issues.apache.org/jira/browse/MAPREDUCE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211080#comment-13211080 ] Zhihong Yu commented on MAPREDUCE-3685: --- @Anty: When you reference code, please use curly braces around the word 'code' instead of square brackets. There are some bugs in implementation of MergeManager - Key: MAPREDUCE-3685 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3685 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: anty.rao Assignee: Anthony Urso Priority: Minor Fix For: 0.23.1 Attachments: MAPREDUCE-3685-branch-0.23.1.patch, MAPREDUCE-3685.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3807) JobTracker needs fix similar to HDFS-94
[
https://issues.apache.org/jira/browse/MAPREDUCE-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Forza Mehlano updated MAPREDUCE-3807:
-
Release Note: The Heap Size printed in the JobTracker WebUI is accurate
Status: Patch Available (was: Open)
No unit test created as similar patch for HDFS didn't require one.
JobTracker needs fix similar to HDFS-94
---
Key: MAPREDUCE-3807
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3807
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Harsh J
Labels: newbie
1.0 JobTracker's jobtracker.jsp page currently shows:
{code}
h2Cluster Summary (Heap Size is %=
StringUtils.byteDesc(Runtime.getRuntime().totalMemory()) %/%=
StringUtils.byteDesc(Runtime.getRuntime().maxMemory()) %)/h2
{code}
It could use an improvement same as HDFS-94 to reflect live heap usage more
accurately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3807) JobTracker needs fix similar to HDFS-94
[
https://issues.apache.org/jira/browse/MAPREDUCE-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1324#comment-1324
]
Hadoop QA commented on MAPREDUCE-3807:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515105/MAPREDUCE-3807.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
-1 patch. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1898//console
This message is automatically generated.
JobTracker needs fix similar to HDFS-94
---
Key: MAPREDUCE-3807
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3807
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Harsh J
Labels: newbie
Attachments: MAPREDUCE-3807.patch
1.0 JobTracker's jobtracker.jsp page currently shows:
{code}
h2Cluster Summary (Heap Size is %=
StringUtils.byteDesc(Runtime.getRuntime().totalMemory()) %/%=
StringUtils.byteDesc(Runtime.getRuntime().maxMemory()) %)/h2
{code}
It could use an improvement same as HDFS-94 to reflect live heap usage more
accurately.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3685) There are some bugs in implementation of MergeManager
[ https://issues.apache.org/jira/browse/MAPREDUCE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Urso reassigned MAPREDUCE-3685: --- Assignee: Zhihong Yu (was: Anthony Urso) I am not Anty Rao, but I can't find him on the list of asignees, so I will assign this back to you. There are some bugs in implementation of MergeManager - Key: MAPREDUCE-3685 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3685 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: anty.rao Assignee: Zhihong Yu Priority: Minor Fix For: 0.23.1 Attachments: MAPREDUCE-3685-branch-0.23.1.patch, MAPREDUCE-3685.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3685) There are some bugs in implementation of MergeManager
[ https://issues.apache.org/jira/browse/MAPREDUCE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211164#comment-13211164 ] anty.rao commented on MAPREDUCE-3685: - @Zhihong Thanks,Sorry for the messed up reference code. Next time i will keep that in mind. There are some bugs in implementation of MergeManager - Key: MAPREDUCE-3685 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3685 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: anty.rao Assignee: Zhihong Yu Priority: Minor Fix For: 0.23.1 Attachments: MAPREDUCE-3685-branch-0.23.1.patch, MAPREDUCE-3685-branch-0.23.1.patch, MAPREDUCE-3685.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (MAPREDUCE-3746) Nodemanagers are not automatically shut down after decommissioning
[ https://issues.apache.org/jira/browse/MAPREDUCE-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli reopened MAPREDUCE-3746: Nodemanagers are not automatically shut down after decommissioning -- Key: MAPREDUCE-3746 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3746 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Jason Lowe Priority: Critical Fix For: 0.23.2 Attachments: jstack Nodemanagers are not automatically shutdown after decommissioning. MAPREDUCE-2775 does not seem to fix the issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3862) Nodemanager can appear to hang on shutdown due to lingering DeletionService threads
[
https://issues.apache.org/jira/browse/MAPREDUCE-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13211209#comment-13211209
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3862:
bq. we have to be able to handle starting up from an unclean shutdown, so files
need to be deleted on startup as well. +1
Yes, that is what happens in 1.0.* too. You know of an open ticket for this?
Nodemanager can appear to hang on shutdown due to lingering DeletionService
threads
---
Key: MAPREDUCE-3862
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3862
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, nodemanager
Affects Versions: 0.23.1
Reporter: Jason Lowe
Assignee: Jason Lowe
Fix For: 0.23.2
Attachments: MAPREDUCE-3862.patch, MAPREDUCE-3862.patch
When a nodemanager attempts to shutdown cleanly, it's possible for it to
appear to hang due to lingering DeletionService threads. This can occur when
yarn.nodemanager.delete.debug-delay-sec is set to a relatively large value
and one or more containers executes on the node shortly before the shutdown.
The DeletionService is never calling
{{setExecuteExistingDelayedTasksAfterShutdownPolicy()}} on the
ScheduledThreadPoolExecutor, and it defaults to waiting for all scheduled
tasks to complete before exiting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
