[jira] [Resolved] (MAPREDUCE-252) Create an InputFormat for reading lines of text as Java Strings

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-252. Resolution: Incomplete Closing this as stale. > Create an InputFormat for readi

[jira] [Resolved] (MAPREDUCE-2827) Test code can create Integer.MIN_INT when trying to create a random non-negative integer

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-2827. - Resolution: Incomplete Closing this as stale. > Test code can create Integer

[jira] [Resolved] (MAPREDUCE-589) StreamXMLRecordReader does not support gzipped files

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-589. Resolution: Incomplete Closing this as stale, given the number of changes to str

[jira] [Resolved] (MAPREDUCE-149) Class JobControl needs to be rewritten for safe concurrency

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-149. Resolution: Incomplete Closing this as stale. > Class JobControl needs to be re

[jira] [Resolved] (MAPREDUCE-60) Nested class TaskTracker.TaskInProgress needs additional synchronization

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-60. --- Resolution: Incomplete Closing this as stale. > Nested class TaskTracker.TaskInPro

[jira] [Resolved] (MAPREDUCE-497) TaskTracker.addDiagnostics(String file, int num, String tag) could exit early if num==0

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-497. Resolution: Incomplete Closing this as stale. > TaskTracker.addDiagnostics(Stri

[jira] [Updated] (MAPREDUCE-505) StatusHttpServer should include hostname/port when bind fails

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-505: --- Labels: newbie (was: ) > StatusHttpServer should include hostname/port when bind

[jira] [Resolved] (MAPREDUCE-235) Custom Splitter for handling many small files

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-235. Resolution: Not a Problem Closing this with Not A Problem. > Custom Splitter fo

[jira] [Resolved] (MAPREDUCE-621) hadoop streaming to support shell pipes: "tr ' ' '\t' | cut -f 3,4"

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-621. Resolution: Not a Problem Resolving as Not a Problem. > hadoop streaming to sup

[jira] [Resolved] (MAPREDUCE-436) NPE in TaskRunner.run if hadoop.log.dir not set

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-436. Resolution: Duplicate Closing as a dupe. > NPE in TaskRunner.run if hadoop.log.

[jira] [Resolved] (MAPREDUCE-606) Implement a binary input/output format for Streaming

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-606. Resolution: Fixed I'm closing this as fixed. The streaming input/output system

[jira] [Resolved] (MAPREDUCE-212) want InputFormat for task logs

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-212. Resolution: Won't Fix I'm going to close this as won't fix. In some cases, Hado

[jira] [Commented] (MAPREDUCE-460) Should be able to re-run jobs, collecting only missing output

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066019#comment-14066019 ] Allen Wittenauer commented on MAPREDUCE-460: So is there actually another J

[jira] [Commented] (MAPREDUCE-5966) MR1 FairScheduler use of custom weight adjuster is not thread safe for comparisons

2014-07-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065847#comment-14065847 ] Hadoop QA commented on MAPREDUCE-5966: -- {color:red}-1 overall{color}. Here are t

[jira] [Updated] (MAPREDUCE-5966) MR1 FairScheduler use of custom weight adjuster is not thread safe for comparisons

2014-07-17 Thread Anubhav Dhoot (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Dhoot updated MAPREDUCE-5966: - Attachment: MAPREDUCE-5966.001.patch Testing: Verified running ant test under src/con

[jira] [Updated] (MAPREDUCE-5966) MR1 FairScheduler use of custom weight adjuster is not thread safe for comparisons

2014-07-17 Thread Anubhav Dhoot (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Dhoot updated MAPREDUCE-5966: - Status: Patch Available (was: Open) Submitting patch that does not allow changes in

[jira] [Commented] (MAPREDUCE-5962) Support CRC32C in IFile

2014-07-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065829#comment-14065829 ] Hadoop QA commented on MAPREDUCE-5962: -- {color:red}-1 overall{color}. Here are t

[jira] [Commented] (MAPREDUCE-5979) MR1 FairScheduler zero weight can cause sort failures

2014-07-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065796#comment-14065796 ] Hadoop QA commented on MAPREDUCE-5979: -- {color:red}-1 overall{color}. Here are t

[jira] [Updated] (MAPREDUCE-5979) MR1 FairScheduler zero weight can cause sort failures

2014-07-17 Thread Anubhav Dhoot (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Dhoot updated MAPREDUCE-5979: - Attachment: MAPREDUCE-5979.001.patch > MR1 FairScheduler zero weight can cause sort f

[jira] [Updated] (MAPREDUCE-5979) MR1 FairScheduler zero weight can cause sort failures

2014-07-17 Thread Anubhav Dhoot (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Dhoot updated MAPREDUCE-5979: - Status: Patch Available (was: Open) > MR1 FairScheduler zero weight can cause sort f

[jira] [Commented] (MAPREDUCE-5979) MR1 FairScheduler zero weight can cause sort failures

2014-07-17 Thread Anubhav Dhoot (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065766#comment-14065766 ] Anubhav Dhoot commented on MAPREDUCE-5979: -- Uploading a patch which prevents

[jira] [Commented] (MAPREDUCE-5974) Allow map output collector fallback

2014-07-17 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065752#comment-14065752 ] Chris Douglas commented on MAPREDUCE-5974: -- Could this be equivalently implem

[jira] [Assigned] (MAPREDUCE-5979) MR1 FairScheduler zero weight can cause sort failures

2014-07-17 Thread Anubhav Dhoot (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anubhav Dhoot reassigned MAPREDUCE-5979: Assignee: Anubhav Dhoot > MR1 FairScheduler zero weight can cause sort failures

[jira] [Created] (MAPREDUCE-5979) MR1 FairScheduler zero weight can cause sort failures

2014-07-17 Thread Anubhav Dhoot (JIRA)
Anubhav Dhoot created MAPREDUCE-5979: Summary: MR1 FairScheduler zero weight can cause sort failures Key: MAPREDUCE-5979 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5979 Project: Hadoop Ma

[jira] [Updated] (MAPREDUCE-5962) Support CRC32C in IFile

2014-07-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-5962: --- Attachment: mapreduce-5962.txt The RawKVIteratorReader used in the shuffle wasn't properl

[jira] [Resolved] (MAPREDUCE-152) getMapOutput() keeps failing too many times before the tasktracker fails

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-152. Resolution: Incomplete At this point, lots of have changed have happened. So clo

[jira] [Updated] (MAPREDUCE-573) reduce scans/copies while reading data in hadoop streaming

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-573: --- Labels: newbie (was: ) > reduce scans/copies while reading data in hadoop streami

[jira] [Updated] (MAPREDUCE-464) Command options should be provided in examples

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-464: --- Labels: newbie (was: ) > Command options should be provided in examples > ---

[jira] [Resolved] (MAPREDUCE-283) We should reuse key and value objects in the MultithreadedMapRunner.

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-283. Resolution: Incomplete Closing this as stale. > We should reuse key and value o

[jira] [Resolved] (MAPREDUCE-345) Investigate whether the array in the JobInProgress that holds TIP references can be removed

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-345. Resolution: Incomplete Closing this as stale. > Investigate whether the array i

[jira] [Resolved] (MAPREDUCE-448) Facility to connect back to a job from the command line.

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-448. Resolution: Won't Fix Closing this as won't fix. > Facility to connect back to

[jira] [Resolved] (MAPREDUCE-300) Ability to thread task execution

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-300. Resolution: Not a Problem Stale! > Ability to thread task execution > -

[jira] [Updated] (MAPREDUCE-340) TextInputFormat should not create input splits for 0 byte files

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-340: --- Labels: newbie (was: ) > TextInputFormat should not create input splits for 0 byt

[jira] [Commented] (MAPREDUCE-340) TextInputFormat should not create input splits for 0 byte files

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065582#comment-14065582 ] Allen Wittenauer commented on MAPREDUCE-340: Marking this as a newbie bug s

[jira] [Commented] (MAPREDUCE-5978) native-task CompressTest failure on Ubuntu

2014-07-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065570#comment-14065570 ] Todd Lipcon commented on MAPREDUCE-5978: I see the following in the logs: loa

[jira] [Resolved] (MAPREDUCE-286) Optimize the last merge of the map output files

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-286. Resolution: Fixed Stale. > Optimize the last merge of the map output files > --

[jira] [Created] (MAPREDUCE-5978) native-task CompressTest failure on Ubuntu

2014-07-17 Thread Todd Lipcon (JIRA)
Todd Lipcon created MAPREDUCE-5978: -- Summary: native-task CompressTest failure on Ubuntu Key: MAPREDUCE-5978 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5978 Project: Hadoop Map/Reduce

[jira] [Resolved] (MAPREDUCE-313) Add additional jobs of new types to gridmix

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-313. Resolution: Fixed gridmix has been rewritten a few times now. Closing. > Add ad

[jira] [Resolved] (MAPREDUCE-556) Refactor Hadoop package structure and source tree.

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-556. Resolution: Fixed Closing this as fixed, since this is like four refactors ago.

[jira] [Resolved] (MAPREDUCE-262) Optimize finding of speculative tasks

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-262. Resolution: Fixed Spec exec got optimized somewhere along the way. Closing. > O

[jira] [Resolved] (MAPREDUCE-475) JobConf should validate key names in well-defined namespaces and warn on misspelling

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-475. Resolution: Fixed I'm going to closed this as fixed since Hadoop is much better

[jira] [Resolved] (MAPREDUCE-274) Hadoop JobClient can return specific exit codes for specific classes of exceptions

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-274. Resolution: Won't Fix I'm closing this as won't fix. Using the exit code like t

[jira] [Resolved] (MAPREDUCE-427) Earlier key-value buffer from MapTask.java is still referenced even though its not required anymore.

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-427. Resolution: Fixed Resolved! > Earlier key-value buffer from MapTask.java is sti

[jira] [Updated] (MAPREDUCE-2593) Random read benchmark for DFS

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-2593: Status: Open (was: Patch Available) Ha. Cancelling the patch so jenkins isn't

[jira] [Commented] (MAPREDUCE-2593) Random read benchmark for DFS

2014-07-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065496#comment-14065496 ] Hadoop QA commented on MAPREDUCE-2593: -- {color:red}-1 overall{color}. Here are t

[jira] [Commented] (MAPREDUCE-5962) Support CRC32C in IFile

2014-07-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065488#comment-14065488 ] Hadoop QA commented on MAPREDUCE-5962: -- {color:red}-1 overall{color}. Here are t

[jira] [Resolved] (MAPREDUCE-78) tasks should not run on nodes where they were previously lost

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-78. --- Resolution: Fixed Fairly confident this has been fixed. > tasks should not run on

[jira] [Commented] (MAPREDUCE-2593) Random read benchmark for DFS

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065470#comment-14065470 ] Allen Wittenauer commented on MAPREDUCE-2593: - Ping! > Random read benchm

[jira] [Resolved] (MAPREDUCE-390) Corner case exists in detecting Java process deaths that might lead to orphan pipes processes lying around in memory

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-390. Resolution: Fixed I believe this one has actually been resolved with the rewrite

[jira] [Resolved] (MAPREDUCE-344) Remove dead code block in JobInProgress.completedTask

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-344. Resolution: Fixed Stale. > Remove dead code block in JobInProgress.completedTa

[jira] [Commented] (MAPREDUCE-489) Force the task tracker to exit when the task is complete, prevents nodes from dying due to resource starvation from impropertly written map/reduce tasks

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065462#comment-14065462 ] Allen Wittenauer commented on MAPREDUCE-489: Doing a quick pass through the

[jira] [Updated] (MAPREDUCE-489) Force the task tracker to exit when the task is complete, prevents nodes from dying due to resource starvation from impropertly written map/reduce tasks

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-489: --- Labels: newbie (was: ) > Force the task tracker to exit when the task is complete

[jira] [Updated] (MAPREDUCE-165) the map task output servlet doesn't protect against ".." attacks

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-165: --- Labels: newbie security (was: security) > the map task output servlet doesn't pro

[jira] [Commented] (MAPREDUCE-302) Maintaining cluster information across multiple job submissions

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065445#comment-14065445 ] Allen Wittenauer commented on MAPREDUCE-302: I think YARN sort of makes thi

[jira] [Updated] (MAPREDUCE-165) the map task output servlet doesn't protect against ".." attacks

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-165: --- Labels: security (was: ) > the map task output servlet doesn't protect against ".

[jira] [Commented] (MAPREDUCE-165) the map task output servlet doesn't protect against ".." attacks

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065441#comment-14065441 ] Allen Wittenauer commented on MAPREDUCE-165: I suspect this issue can be cl

[jira] [Resolved] (MAPREDUCE-574) Fix -file option in Streaming to use Distributed Cache

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-574. Resolution: Fixed Stale. > Fix -file option in Streaming to use Distributed Cac

[jira] [Resolved] (MAPREDUCE-224) limit running tasks per job

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-224. Resolution: Duplicate I'm going to close this out as a duplicate of MAPREDEUCE-5

[jira] [Resolved] (MAPREDUCE-186) TaskLogServlet returns 410 when trying to access log early in task life

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-186. Resolution: Incomplete I'm going to close this out as stale. > TaskLogServlet r

[jira] [Resolved] (MAPREDUCE-265) check permissions for job inputs and outputs

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-265. Resolution: Fixed I believe this has been fixed. > check permissions for job in

[jira] [Resolved] (MAPREDUCE-215) Improve facilities for job-control, job-queues etc.

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-215. Resolution: Fixed I'm going to close as fixed for a variety of reasons: - some

[jira] [Commented] (MAPREDUCE-455) Hadoop needs a better XML Input

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065392#comment-14065392 ] Allen Wittenauer commented on MAPREDUCE-455: Is it time to revisit this? >

[jira] [Resolved] (MAPREDUCE-594) Streaming: org.apache.hadoop.mapred.lib.IdentityMapper should not inserted unnecessary keys

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-594. Resolution: Fixed Closing this as fixed for a variety of reasons (alternative pr

[jira] [Resolved] (MAPREDUCE-386) The combiner in pipes is closed before the last values are passed in.

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-386. Resolution: Fixed Stale issue. I'm sure this has been fixed by now! > The combi

[jira] [Updated] (MAPREDUCE-5910) MRAppMaster should handle Resync from RM instead of shutting down.

2014-07-17 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He updated MAPREDUCE-5910: --- Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed Status: Resol

[jira] [Commented] (MAPREDUCE-618) Need to be able to re-run specific map tasks (when -reducer NONE)

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065373#comment-14065373 ] Allen Wittenauer commented on MAPREDUCE-618: I'm tempted to close this beca

[jira] [Commented] (MAPREDUCE-5910) MRAppMaster should handle Resync from RM instead of shutting down.

2014-07-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065361#comment-14065361 ] Hudson commented on MAPREDUCE-5910: --- FAILURE: Integrated in Hadoop-trunk-Commit #590

[jira] [Resolved] (MAPREDUCE-598) Streaming: better conrol over input splits

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-598. Resolution: Fixed This has been fixed in a variety of use cases. Closing. > Str

[jira] [Resolved] (MAPREDUCE-620) Streaming: support local execution

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-620. Resolution: Won't Fix Unix already provides this functionality. > Streaming: su

[jira] [Resolved] (MAPREDUCE-229) Provide a command line option to check if a Hadoop jobtracker is idle

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-229. Resolution: Fixed I'm going to close this as fixed. Clients can ask the JT if t

[jira] [Resolved] (MAPREDUCE-2829) TestMiniMRMapRedDebugScript times out

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-2829. - Resolution: Fixed Yup. Definitely. Closing. > TestMiniMRMapRedDebugScript ti

[jira] [Resolved] (MAPREDUCE-1767) Steaming infrastructures should provide statisics about job

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-1767. - Resolution: Fixed All of this information is available via other means. > St

[jira] [Resolved] (MAPREDUCE-611) Streaming infrastructure should report information about runtime errors

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-611. Resolution: Won't Fix The stderr log was built to handle this type of thing. Clo

[jira] [Commented] (MAPREDUCE-5910) MRAppMaster should handle Resync from RM instead of shutting down.

2014-07-17 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065309#comment-14065309 ] Jian He commented on MAPREDUCE-5910: committing this. > MRAppMaster should handle

[jira] [Resolved] (MAPREDUCE-1769) Streaming command should be able to take its output to a "file", rather then to stdout

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-1769. - Resolution: Won't Fix Same reasoning as its buddy jira. > Streaming command

[jira] [Resolved] (MAPREDUCE-600) Streaming command should be able to take its input from a "file", rather then from stdin

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-600. Resolution: Won't Fix I'm going to close this as won't fix. For applications tha

[jira] [Updated] (MAPREDUCE-636) Inconsistency in config parameter for RandomWriter

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-636: --- Labels: newbie (was: ) > Inconsistency in config parameter for RandomWriter > ---

[jira] [Resolved] (MAPREDUCE-155) Each task tracker should not execute more than one speculative task

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-155. Resolution: Incomplete Closing this as stale since spec exec has undergone quite

[jira] [Resolved] (MAPREDUCE-108) Blacklisted hosts may not be able to serve map outputs

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-108. Resolution: Incomplete I'm going to close this out as stale. Blacklisting has u

[jira] [Commented] (MAPREDUCE-5962) Support CRC32C in IFile

2014-07-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065291#comment-14065291 ] Todd Lipcon commented on MAPREDUCE-5962: (fwiw this depends on [~james.thomas]

[jira] [Updated] (MAPREDUCE-5962) Support CRC32C in IFile

2014-07-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-5962: --- Status: Patch Available (was: Open) > Support CRC32C in IFile > ---

[jira] [Commented] (MAPREDUCE-5910) MRAppMaster should handle Resync from RM instead of shutting down.

2014-07-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065293#comment-14065293 ] Hadoop QA commented on MAPREDUCE-5910: -- {color:green}+1 overall{color}. Here are

[jira] [Updated] (MAPREDUCE-5962) Support CRC32C in IFile

2014-07-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-5962: --- Attachment: mapreduce-5962.txt Attached patch adds a new configuration to set the IFile c

[jira] [Commented] (MAPREDUCE-2830) Document config parameters for each Map-Reduce class/interface

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065280#comment-14065280 ] Allen Wittenauer commented on MAPREDUCE-2830: - Ping! > Document config pa

[jira] [Assigned] (MAPREDUCE-5962) Support CRC32C in IFile

2014-07-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon reassigned MAPREDUCE-5962: -- Assignee: Todd Lipcon (was: James Thomas) > Support CRC32C in IFile >

[jira] [Resolved] (MAPREDUCE-492) Pending, running, completed tasks should also be shown as percentage

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-492. Resolution: Fixed A different percentage was committed eons ago. > Pending, run

[jira] [Commented] (MAPREDUCE-5974) Allow map output collector fallback

2014-07-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065268#comment-14065268 ] Hadoop QA commented on MAPREDUCE-5974: -- {color:red}-1 overall{color}. Here are t

[jira] [Resolved] (MAPREDUCE-129) job_null_0001 in jobid

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-129. Resolution: Fixed > job_null_0001 in jobid > -- > >

[jira] [Commented] (MAPREDUCE-272) Job tracker should report the number of splits that are local to some task trackers

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065261#comment-14065261 ] Allen Wittenauer commented on MAPREDUCE-272: Don't we essentially have this

[jira] [Resolved] (MAPREDUCE-230) Need to document the controls for sorting and grouping into the reduce

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-230. Resolution: Won't Fix This ship sailed a looong time ago. > Need to documen

[jira] [Updated] (MAPREDUCE-5975) Fix native-task build on Ubuntu 13.10

2014-07-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated MAPREDUCE-5975: --- Attachment: mr-5975.txt Trivial patch to add some missing unistd.h includes which were ne

[jira] [Commented] (MAPREDUCE-5975) Fix native-task build on Ubuntu 13.10

2014-07-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065248#comment-14065248 ] Todd Lipcon commented on MAPREDUCE-5975: [~clockfly] and [~decster] -- since y

[jira] [Resolved] (MAPREDUCE-2833) Job Tracker needs to collect more job/task execution stats and save them to DFS file

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-2833. - Resolution: Fixed I'm closing this as fixed since the history files pretty mu

[jira] [Created] (MAPREDUCE-5977) Fix or suppress native-task gcc warnings

2014-07-17 Thread Todd Lipcon (JIRA)
Todd Lipcon created MAPREDUCE-5977: -- Summary: Fix or suppress native-task gcc warnings Key: MAPREDUCE-5977 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5977 Project: Hadoop Map/Reduce

[jira] [Resolved] (MAPREDUCE-209) Support for metrics aggregation module in JobTracker

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-209. Resolution: Fixed Closing this as stale. > Support for metrics aggregation modu

[jira] [Resolved] (MAPREDUCE-428) Reducers reported completion % is generally incorrect when consuming compressed map outputs

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-428. Resolution: Duplicate > Reducers reported completion % is generally incorrect wh

[jira] [Created] (MAPREDUCE-5976) native-task should not fail to build if snappy is missing

2014-07-17 Thread Todd Lipcon (JIRA)
Todd Lipcon created MAPREDUCE-5976: -- Summary: native-task should not fail to build if snappy is missing Key: MAPREDUCE-5976 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5976 Project: Hadoop Ma

[jira] [Created] (MAPREDUCE-5975) Fix native-task build on Ubuntu 13.10

2014-07-17 Thread Todd Lipcon (JIRA)
Todd Lipcon created MAPREDUCE-5975: -- Summary: Fix native-task build on Ubuntu 13.10 Key: MAPREDUCE-5975 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5975 Project: Hadoop Map/Reduce I

[jira] [Commented] (MAPREDUCE-210) want InputFormat for zip files

2014-07-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065216#comment-14065216 ] Allen Wittenauer commented on MAPREDUCE-210: Ping! > want InputFormat fo

[jira] [Commented] (MAPREDUCE-2841) Task level native optimization

2014-07-17 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14065207#comment-14065207 ] Todd Lipcon commented on MAPREDUCE-2841: Thanks Sean. This patch looks better.

  1   2   >