[jira] [Created] (MAPREDUCE-7232) Reduce excessive getFileLinkInfo calls in MR
Jonathan Eagles created MAPREDUCE-7232:
--
Summary: Reduce excessive getFileLinkInfo calls in MR
Key: MAPREDUCE-7232
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7232
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Jonathan Eagles
Assignee: Jonathan Eagles
The MR Client redundantly calls getFileLinkInfo for the same file sometimes
over 10 times. The files under question originate from
mapreduce.job.cache.files and mapreduce.job.classpath.files and their archive
equivalences.
Client Observation
Case 1 & 2
{code}
19/08/15 10:57:55 INFO hdfs.DFSClient:
getFileLinkInfo:/shared/tez-exploded/tez-0.9.2-SNAPSHOT-minimal/tez-runtime-internals-0.9.2-SNAPSHOT.jar
java.lang.Exception
at org.apache.hadoop.hdfs.DFSClient.getFileLinkInfo(DFSClient.java:1683)
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1486)
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1483)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileLinkStatus(DistributedFileSystem.java:1498)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:772)
at
org.apache.hadoop.mapreduce.v2.util.MRApps.addToClasspathIfNotJar(MRApps.java:341)
at
org.apache.hadoop.mapreduce.v2.util.MRApps.addClasspathToEnv(MRApps.java:301)
at
org.apache.hadoop.mapreduce.v2.util.MRApps.setClasspath(MRApps.java:262)
at
org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:467)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:296)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:244)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1341)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1338)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1953)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1338)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1359)
at org.apache.hadoop.mapreduce.SleepJob.run(SleepJob.java:273)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.mapreduce.SleepJob.main(SleepJob.java:194)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at
org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:136)
at
org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:144)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
{code}
{code}
19/08/15 10:57:55 INFO hdfs.DFSClient:
getFileLinkInfo:/shared/tez-exploded/tez-0.9.2-SNAPSHOT-minimal/tez-runtime-internals-0.9.2-SNAPSHOT.jar
java.lang.Exception
at org.apache.hadoop.hdfs.DFSClient.getFileLinkInfo(DFSClient.java:1683)
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1486)
at
org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1483)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileLinkStatus(DistributedFileSystem.java:1498)
at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:772)
at
org.apache.hadoop.mapreduce.v2.util.MRApps.addToClasspathIfNotJar(MRApps.java:341)
at
org.apache.hadoop.mapreduce.v2.util.MRApps.addClasspathToEnv(MRApps.java:301)
at
org.apache.hadoop.mapreduce.v2.util.MRApps.setClasspath(MRApps.java:263)
at
org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:467)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:296)
at
[jira] [Created] (MAPREDUCE-6850) Shuffle Handler keep-alive connections are closed from the server side
Jonathan Eagles created MAPREDUCE-6850: -- Summary: Shuffle Handler keep-alive connections are closed from the server side Key: MAPREDUCE-6850 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6850 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles Assignee: Jonathan Eagles When performance testing tez shuffle handler (TEZ-3334), it was noticed the keep-alive connections are closed from the server-side. The client silently recovers and logs the connection as keep-alive, despite reestablishing a connection. This jira aims to remove the close from the server side, fixing the bug preventing keep-alive connections. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6808) Log map attempts as part of shuffle handler audit log
Jonathan Eagles created MAPREDUCE-6808: -- Summary: Log map attempts as part of shuffle handler audit log Key: MAPREDUCE-6808 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6808 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles With the introduction tez, multiple unrelated reducers in the same job will have the same id. The audit log won't be able to distinguish which map attempt the reduce was fetching. This jira is to discuss the possibility to add map attempts logging to distinguish between the two. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6385) mapreduce.map.env and mapreduce.reduce.env need to fall back to mapred.child.env for compatibility
Jonathan Eagles created MAPREDUCE-6385: -- Summary: mapreduce.map.env and mapreduce.reduce.env need to fall back to mapred.child.env for compatibility Key: MAPREDUCE-6385 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6385 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6285) ClientServiceDelegate should not retry upon AuthenticationException
Jonathan Eagles created MAPREDUCE-6285: -- Summary: ClientServiceDelegate should not retry upon AuthenticationException Key: MAPREDUCE-6285 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6285 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles Assignee: Jonathan Eagles ClientServiceDelegate tries really hard to getJobStatus from a job through retries. However, if an AuthorizationException occurs, it will always continue to fail no matter how many retries it makes. AuthorizationException is also thrown when mapred job -status is call on unsupported services like tez jobs. This jira will limit the negative effect on AMs in both scenarios. Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Protocol interface org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB is not known. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6018) Create a framework specific config to enable timeline server
Jonathan Eagles created MAPREDUCE-6018: -- Summary: Create a framework specific config to enable timeline server Key: MAPREDUCE-6018 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6018 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Jonathan Eagles -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (MAPREDUCE-5116) PUMA Benchmark Suite
[ https://issues.apache.org/jira/browse/MAPREDUCE-5116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles resolved MAPREDUCE-5116. Resolution: Later Target Version/s: 1.1.2, 0.23.5, 2.0.3-alpha, 0.20.205.0 (was: 0.20.205.0, 2.0.3-alpha, 0.23.5, 1.1.2) PUMA Benchmark Suite Key: MAPREDUCE-5116 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5116 Project: Hadoop Map/Reduce Issue Type: New Feature Components: benchmarks Reporter: Faraz Ahmad Assignee: Faraz Ahmad Original Estimate: 48h Time Spent: 48h Remaining Estimate: 0h A benchmark suite which represents a broad range of real-world MapReduce applications exhibiting application characteristics with high/low computation and high/low shuffle volumes. These benchmarks have been published as part of MaRCO (http://dx.doi.org/10.1016/j.jpdc.2012.12.012) project in JPDC '12. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (MAPREDUCE-3551) web proxy returns internal server error when application invalid
[ https://issues.apache.org/jira/browse/MAPREDUCE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles resolved MAPREDUCE-3551. Resolution: Fixed This is fix in some previous version of hadoop. web proxy returns internal server error when application invalid Key: MAPREDUCE-3551 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3551 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Assignee: Jonathan Eagles querying for an absent app i.e. say http://RM:8088/proxy/application_1323054018624_0004/ws/v1/mapreduce/info - returns a 500 with a stack trace instead of a 404. From an api spec point of view, this is not really an error but just a not found. This occurs with web services as well as any web ui page. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (MAPREDUCE-3120) JobHistory is not providing correct count failed,killed task
[ https://issues.apache.org/jira/browse/MAPREDUCE-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles resolved MAPREDUCE-3120. Resolution: Fixed Target Version/s: 2.0.0-alpha, 0.23.3, 3.0.0 (was: 0.23.3, 2.0.0-alpha, 3.0.0) Duping to MAPREDUCE-3032 as we haven't heard from the reporter on this issue in some time and believe this issue is already fixed. Feel free to reopen this ticket if this is still and issue on the latest 2.4.0 build. JobHistory is not providing correct count failed,killed task Key: MAPREDUCE-3120 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3120 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Subroto Sanyal Assignee: Subroto Sanyal Fix For: 0.24.0 Attachments: JobFail.PNG Please refer the attachment JobFail.PNG. Here the Job (WordCount) Failed as all Map Attempts were killed(intensionally) but, still the Table in UI shows 0 Killed Attempts and no reason for Failure is also available. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (MAPREDUCE-5765) Update hadoop-pipes examples README
Jonathan Eagles created MAPREDUCE-5765: -- Summary: Update hadoop-pipes examples README Key: MAPREDUCE-5765 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5765 Project: Hadoop Map/Reduce Issue Type: Bug Components: pipes Affects Versions: 3.0.0, 2.5.0 Reporter: Jonathan Eagles wordcount-simple is in the native/examples directory -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (MAPREDUCE-5645) TestFixedLengthInputFormat fails with native libs
Jonathan Eagles created MAPREDUCE-5645: -- Summary: TestFixedLengthInputFormat fails with native libs Key: MAPREDUCE-5645 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5645 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 2.3.0 Reporter: Jonathan Eagles Assignee: Mit Desai mvn clean install -Pnative -DskipTests hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient mvn clean test -Dtest=TestFixedLengthInputFormat Running org.apache.hadoop.mapreduce.lib.input.TestFixedLengthInputFormat Tests run: 8, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 39.957 sec FAILURE! - in org.apache.hadoop.mapreduce.lib.input.TestFixedLengthInputFormat testGzipWithTwoInputs(org.apache.hadoop.mapreduce.lib.input.TestFixedLengthInputFormat) Time elapsed: 0.029 sec ERROR! java.lang.NullPointerException: null at org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:65) at org.apache.hadoop.io.compress.GzipCodec.createOutputStream(GzipCodec.java:162) at org.apache.hadoop.mapreduce.lib.input.TestFixedLengthInputFormat.writeFile(TestFixedLengthInputFormat.java:397) at org.apache.hadoop.mapreduce.lib.input.TestFixedLengthInputFormat.testGzipWithTwoInputs(TestFixedLengthInputFormat.java:229) testFormatCompressedIn(org.apache.hadoop.mapreduce.lib.input.TestFixedLengthInputFormat) Time elapsed: 0.01 sec ERROR! java.lang.NullPointerException: null at org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:65) at org.apache.hadoop.io.compress.GzipCodec.createOutputStream(GzipCodec.java:162) at org.apache.hadoop.mapreduce.lib.input.TestFixedLengthInputFormat.createFile(TestFixedLengthInputFormat.java:261) at org.apache.hadoop.mapreduce.lib.input.TestFixedLengthInputFormat.runRandomTests(TestFixedLengthInputFormat.java:314) at org.apache.hadoop.mapreduce.lib.input.TestFixedLengthInputFormat.testFormatCompressedIn(TestFixedLengthInputFormat.java:96) Running org.apache.hadoop.mapred.TestFixedLengthInputFormat Tests run: 8, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 46.151 sec FAILURE! - in org.apache.hadoop.mapred.TestFixedLengthInputFormat testPartialRecordCompressedIn(org.apache.hadoop.mapred.TestFixedLengthInputFormat) Time elapsed: 0.031 sec ERROR! java.lang.NullPointerException: null at org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:65) at org.apache.hadoop.io.compress.GzipCodec.createOutputStream(GzipCodec.java:162) at org.apache.hadoop.mapred.TestFixedLengthInputFormat.writeFile(TestFixedLengthInputFormat.java:357) at org.apache.hadoop.mapred.TestFixedLengthInputFormat.runPartialRecordTest(TestFixedLengthInputFormat.java:386) at org.apache.hadoop.mapred.TestFixedLengthInputFormat.testPartialRecordCompressedIn(TestFixedLengthInputFormat.java:182) testGzipWithTwoInputs(org.apache.hadoop.mapred.TestFixedLengthInputFormat) Time elapsed: 0.009 sec ERROR! java.lang.NullPointerException: null at org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:65) at org.apache.hadoop.io.compress.GzipCodec.createOutputStream(GzipCodec.java:162) at org.apache.hadoop.mapred.TestFixedLengthInputFormat.writeFile(TestFixedLengthInputFormat.java:357) at org.apache.hadoop.mapred.TestFixedLengthInputFormat.testGzipWithTwoInputs(TestFixedLengthInputFormat.java:201) testFormatCompressedIn(org.apache.hadoop.mapred.TestFixedLengthInputFormat) Time elapsed: 0.017 sec ERROR! java.lang.NullPointerException: null at org.apache.hadoop.io.compress.zlib.ZlibFactory.isNativeZlibLoaded(ZlibFactory.java:65) at org.apache.hadoop.io.compress.GzipCodec.createOutputStream(GzipCodec.java:162) at org.apache.hadoop.mapred.TestFixedLengthInputFormat.createFile(TestFixedLengthInputFormat.java:234) at org.apache.hadoop.mapred.TestFixedLengthInputFormat.runRandomTests(TestFixedLengthInputFormat.java:287) at org.apache.hadoop.mapred.TestFixedLengthInputFormat.testFormatCompressedIn(TestFixedLengthInputFormat.java:90) Results : Tests in error: TestFixedLengthInputFormat.testGzipWithTwoInputs:229-writeFile:397 » NullPointer TestFixedLengthInputFormat.testFormatCompressedIn:96-runRandomTests:314-createFile:261 » NullPointer TestFixedLengthInputFormat.testPartialRecordCompressedIn:182-runPartialRecordTest:386-writeFile:357 » NullPointer TestFixedLengthInputFormat.testGzipWithTwoInputs:201-writeFile:357 » NullPointer TestFixedLengthInputFormat.testFormatCompressedIn:90-runRandomTests:287-createFile:234 » NullPointer Tests run: 16, Failures: 0, Errors: 5, Skipped: 0 -- This message was sent by Atlassian JIRA
[jira] [Created] (MAPREDUCE-5631) TestJobEndNotifier.testNotifyRetries fails with Should have taken more than 5 seconds in jdk7
Jonathan Eagles created MAPREDUCE-5631: -- Summary: TestJobEndNotifier.testNotifyRetries fails with Should have taken more than 5 seconds in jdk7 Key: MAPREDUCE-5631 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5631 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 2.3.0, 0.23.10 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Configuration settings are bleeding over from test to test in jdk7 environment since tests are run in random order. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (MAPREDUCE-5625) TestFixedLengthInputFormat fails in jdk7 environment
Jonathan Eagles created MAPREDUCE-5625: -- Summary: TestFixedLengthInputFormat fails in jdk7 environment Key: MAPREDUCE-5625 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5625 Project: Hadoop Map/Reduce Issue Type: Test Affects Versions: 3.0.0, 2.3.0 Reporter: Jonathan Eagles -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (MAPREDUCE-5610) TestSleepJob fails in jdk7
Jonathan Eagles created MAPREDUCE-5610: -- Summary: TestSleepJob fails in jdk7 Key: MAPREDUCE-5610 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5610 Project: Hadoop Map/Reduce Issue Type: Test Reporter: Jonathan Eagles Assignee: Jonathan Eagles In jdk7 tests methods in a class do not run in file order, but rather in random order. TestSleepJob hosts are not initialized and a NullPointerException is thrown unless testRandomLocation was run first. This can be easily seen by running tests individually. org.apache.hadoop.mapred.gridmix.TestSleepJob#testStressSubmit org.apache.hadoop.mapred.gridmix.TestSleepJob#testReplaySubmit org.apache.hadoop.mapred.gridmix.TestSleepJob#testSerialSubmit org.apache.hadoop.mapred.gridmix.TestSleepJob#testMapTasksOnlySleepJobs org.apache.hadoop.mapred.gridmix.TestSleepJob#testRandomLocation -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (MAPREDUCE-5607) Backport MAPREDUCE-5086 - MR app master deletes staging dir when sent a reboot command from the RM
Jonathan Eagles created MAPREDUCE-5607: -- Summary: Backport MAPREDUCE-5086 - MR app master deletes staging dir when sent a reboot command from the RM Key: MAPREDUCE-5607 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5607 Project: Hadoop Map/Reduce Issue Type: Sub-task Reporter: Jonathan Eagles Assignee: Jian He Fix For: 2.1.0-beta If the RM is restarted when the MR job is running, then it sends a reboot command to the job. The job ends up deleting the staging dir and that causes the next attempt to fail. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (MAPREDUCE-5587) TestTextOutputFormat fails on JDK7
Jonathan Eagles created MAPREDUCE-5587: -- Summary: TestTextOutputFormat fails on JDK7 Key: MAPREDUCE-5587 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5587 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles Assignee: Jonathan Eagles -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (MAPREDUCE-5585) TestCopyCommitter#testNoCommitAction Fails on JDK7
[ https://issues.apache.org/jira/browse/MAPREDUCE-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles resolved MAPREDUCE-5585. Resolution: Fixed Fix Version/s: 0.23.10 2.3.0 3.0.0 TestCopyCommitter#testNoCommitAction Fails on JDK7 -- Key: MAPREDUCE-5585 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5585 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 0.23.9, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 3.0.0, 2.3.0, 0.23.10 Attachments: MAPREDUCE-5585.patch, MAPREDUCE-5585.patch TestCopyCommitter#testNoCommitAction fails on JDK7 when run after testAtomicCommitMissingFinal or testAtomicCommitExistingFinal. Config settings are from atomic tests are being accidentally used for testNoCommitAction. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (MAPREDUCE-5585) TestCopyCommitter#testNoCommitAction Fails on JDK7
Jonathan Eagles created MAPREDUCE-5585: -- Summary: TestCopyCommitter#testNoCommitAction Fails on JDK7 Key: MAPREDUCE-5585 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5585 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.9, 3.0.0, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles TestCopyCommitter#testNoCommitAction fails on JDK7 when run after testAtomicCommitMissingFinal or testAtomicCommitExistingFinal. Config settings are from atomic tests are being accidentally used for testNoCommitAction. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (MAPREDUCE-5586) TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory
Jonathan Eagles created MAPREDUCE-5586: -- Summary: TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory Key: MAPREDUCE-5586 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5586 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles Assignee: Jonathan Eagles -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (MAPREDUCE-3835) RM capacity scheduler web UI doesn't show active users
[ https://issues.apache.org/jira/browse/MAPREDUCE-3835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles resolved MAPREDUCE-3835. Resolution: Duplicate Target Version/s: 2.0.0-alpha, 0.23.3, 3.0.0 (was: 0.23.3, 2.0.0-alpha, 3.0.0) RM capacity scheduler web UI doesn't show active users -- Key: MAPREDUCE-3835 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3835 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Priority: Minor On the jobtracker, the web ui showed the active users for each queue and how much resources each of those users were using. That currently isn't being displayed on the RM capacity scheduler web ui. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4269) Gridmix has javadoc warnings in StressJobFactory
Jonathan Eagles created MAPREDUCE-4269: -- Summary: Gridmix has javadoc warnings in StressJobFactory Key: MAPREDUCE-4269 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4269 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.3, 2.0.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4220) RM apps page starttime/endtime sorts are incorrect
Jonathan Eagles created MAPREDUCE-4220: -- Summary: RM apps page starttime/endtime sorts are incorrect Key: MAPREDUCE-4220 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4220 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.3, 2.0.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Minor Attachments: MAPREDUCE-4220.patch Sorting by start time and end time sort lexically instead of temporally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4215) RM app page shows 500 error on appid parse error
Jonathan Eagles created MAPREDUCE-4215: -- Summary: RM app page shows 500 error on appid parse error Key: MAPREDUCE-4215 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4215 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.3 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 2.0.0 For example, cluster/app/application_1335823499485_000a has displays this error -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4211) Error conditions (missing appid, appid not found) are masked in the RM app page
Jonathan Eagles created MAPREDUCE-4211: -- Summary: Error conditions (missing appid, appid not found) are masked in the RM app page Key: MAPREDUCE-4211 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4211 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.3, 2.0.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4206) Sorting by Last Health-Update on the RM nodes page sorts does not work correctly
Jonathan Eagles created MAPREDUCE-4206: -- Summary: Sorting by Last Health-Update on the RM nodes page sorts does not work correctly Key: MAPREDUCE-4206 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4206 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.3, 2.0.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Minor column is mistakenly sorted lexically -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2777) Backport MAPREDUCE-220 to Hadoop 20 security branch
Backport MAPREDUCE-220 to Hadoop 20 security branch --- Key: MAPREDUCE-2777 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2777 Project: Hadoop Map/Reduce Issue Type: New Feature Affects Versions: 0.20.205.0 Reporter: Jonathan Eagles -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2665) MR-279: Add hostname:port to NM and RM info/about page
MR-279: Add hostname:port to NM and RM info/about page -- Key: MAPREDUCE-2665 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2665 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Minor Adding host port information to the RM NM info/about pages -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2628) MR-279: Add compiled on date to NM and RM info/about page
MR-279: Add compiled on date to NM and RM info/about page - Key: MAPREDUCE-2628 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2628 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Minor Compiled on dates were present on the JobTracker UI. Bring compiled on dates to resource manager and node manager UI. NM and RM retrieves build version for hadoop and yarn version via the getBuildVersion util api. This function used to contain the compiled on date, but since has been removed since that function is used to determine hadoop compatible versions, but was too restrictive with build date being present. Instead, a getDate call should be used to retrieve the compiled on date. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2569) MR-279: Restarting resource manager with root capacity not equal to 100 percent should result in error
MR-279: Restarting resource manager with root capacity not equal to 100 percent should result in error -- Key: MAPREDUCE-2569 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2569 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Minor -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2526) Forward port MAPREDUCE-1966 Add task tracker graylisting
Forward port MAPREDUCE-1966 Add task tracker graylisting Key: MAPREDUCE-2526 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2526 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Reporter: Jonathan Eagles Assignee: Greg Roelofs The current heuristic of rolling up fixed number of job failures per tracker isn't working well, we need better design/heuristics. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2490) Log blacklist debug count
Log blacklist debug count - Key: MAPREDUCE-2490 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2490 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker Affects Versions: 0.20.204.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Trivial Gain some insight into blacklist increments/decrements by enhancing the debug logging -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
