[jira] Created: (MAPREDUCE-832) Too man y WARN messages about deprecated memorty config variables in JobTacker log
Too man y WARN messages about deprecated memorty config variables in JobTacker log -- Key: MAPREDUCE-832 URL: https://issues.apache.org/jira/browse/MAPREDUCE-832 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.1 Reporter: Karam Singh When user submit a mapred job using old memory config vairiable (mapred.task.maxmem) followinig message too many times in JobTracker logs -: [ WARN org.apache.hadoop.mapred.JobConf: The variable mapred.task.maxvmem is no longer used instead use mapred.job.map.memory.mb and mapred.job.reduce.memory.mb ] -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-833) Jobclient does not print any warning message when old memory config variable used with -D option from command line
Jobclient does not print any warning message when old memory config variable used with -D option from command line -- Key: MAPREDUCE-833 URL: https://issues.apache.org/jira/browse/MAPREDUCE-833 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.1 Reporter: Karam Singh -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-834) When TaskTracker config use old memory management values its memory monitoring is diabled.
When TaskTracker config use old memory management values its memory monitoring is diabled. -- Key: MAPREDUCE-834 URL: https://issues.apache.org/jira/browse/MAPREDUCE-834 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Karam Singh TaskTracker memory config values -: mapred.tasktracker.vmem.reserved=8589934592 mapred.task.default.maxvmem=2147483648 mapred.task.limit.maxvmem=4294967296 mapred.tasktracker.pmem.reserved=2147483648 TaskTracker start as -: 2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.tasktracker.vmem.reserved is no longer used 2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.tasktracker.pmem.reserved is no longer used 2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.task.default.maxvmem is no longer used 2009-08-05 12:39:03,308 WARN org.apache.hadoop.mapred.TaskTracker: The variable mapred.task.limit.maxvmem is no longer used 2009-08-05 12:39:03,308 INFO org.apache.hadoop.mapred.TaskTracker: Starting thread: Map-events fetcher for all reduce tasks on 2009-08-05 12:39:03,309 INFO org.apache.hadoop.mapred.TaskTracker: Using MemoryCalculatorPlugin : org.apache.hadoop.util.linuxmemorycalculatorplu...@19be4777 2009-08-05 12:39:03,311 WARN org.apache.hadoop.mapred.TaskTracker: TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is disabled. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Reopened: (MAPREDUCE-796) Encountered "ClassCastException" on tasktracker while running wordcount with MultithreadedMapRunner
[ https://issues.apache.org/jira/browse/MAPREDUCE-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amar Kamat reopened MAPREDUCE-796: -- > Encountered "ClassCastException" on tasktracker while running wordcount with > MultithreadedMapRunner > --- > > Key: MAPREDUCE-796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: examples >Affects Versions: 0.20.1 >Reporter: Suman Sehgal >Assignee: Amar Kamat > > ClassCastException for OutOfMemoryError is encountered on tasktracker while > running wordcount example with MultithreadedMapRunner. > Stack trace : > = > java.lang.ClassCastException: java.lang.OutOfMemoryError cannot be cast to > java.lang.RuntimeException > at > org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper.run(MultithreadedMapper.java:149) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:581) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:303) > at org.apache.hadoop.mapred.Child.main(Child.java:170) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAPREDUCE-796) Encountered "ClassCastException" on tasktracker while running wordcount with MultithreadedMapRunner
[ https://issues.apache.org/jira/browse/MAPREDUCE-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved MAPREDUCE-796. --- Resolution: Fixed Fix Version/s: 0.20.1 Hadoop Flags: [Reviewed] I just committed this. Thanks, Amar! > Encountered "ClassCastException" on tasktracker while running wordcount with > MultithreadedMapRunner > --- > > Key: MAPREDUCE-796 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-796 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: examples >Affects Versions: 0.20.1 >Reporter: Suman Sehgal >Assignee: Amar Kamat > Fix For: 0.20.1 > > Attachments: MAPREDUCE-796-v1.0.patch > > > ClassCastException for OutOfMemoryError is encountered on tasktracker while > running wordcount example with MultithreadedMapRunner. > Stack trace : > = > java.lang.ClassCastException: java.lang.OutOfMemoryError cannot be cast to > java.lang.RuntimeException > at > org.apache.hadoop.mapreduce.lib.map.MultithreadedMapper.run(MultithreadedMapper.java:149) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:581) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:303) > at org.apache.hadoop.mapred.Child.main(Child.java:170) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Reopened: (MAPREDUCE-757) JobConf will not be deleted from the logs folder if job retires from finalizeJob()
[ https://issues.apache.org/jira/browse/MAPREDUCE-757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amar Kamat reopened MAPREDUCE-757: -- > JobConf will not be deleted from the logs folder if job retires from > finalizeJob() > -- > > Key: MAPREDUCE-757 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-757 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Reporter: Amar Kamat >Assignee: Amar Kamat > Attachments: MAPREDUCE-757-v1.0.patch, > MAPREDUCE-757-v2.0-branch-0.20.patch, MAPREDUCE-757-v2.0.patch > > > MAPREDUCE-130 fixed the case where the job is retired from the retire jobs > thread. But jobs can also retire when the num-job-per-user limit is exceeded. > In such cases the conf file will not be deleted. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-835) hadoop-mapred examples,test and tools jar iles are being packaged when ant binary or bin-package
hadoop-mapred examples,test and tools jar iles are being packaged when ant binary or bin-package Key: MAPREDUCE-835 URL: https://issues.apache.org/jira/browse/MAPREDUCE-835 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.21.0 Reporter: Karam Singh When checking mapreduce trunk. If run ant binary or ant bin-package commands-: hadoop-mapred-test-0.21.0-dev.jar, hadoop-mapred-examples-0.21.0-dev.jar, hadoop-mapred-tools-0.21.0-dev.jar are being in tar or build/hadoop-mapred-0.21.0-dev packe directory. But they present under build directory. For ant tar and ant package they are being packaged correclty. buid/hadoop-mapred-0.21.0-dev directory. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-836) Examples of hadoop pipes a even when -Dcompile.native=yes -Dcompile.c++=yes option are used while running ant package or tar or similar commands.
Examples of hadoop pipes a even when -Dcompile.native=yes -Dcompile.c++=yes option are used while running ant package or tar or similar commands. - Key: MAPREDUCE-836 URL: https://issues.apache.org/jira/browse/MAPREDUCE-836 Project: Hadoop Map/Reduce Issue Type: Bug Components: examples Affects Versions: 0.20.1, 0.21.0 Reporter: Karam Singh Examples of hadoop pies and python are not packed even when -Dcompile.native=yes -Dcompile.c++=yes option are used while running ant package or tar or similar commands. The pipes examples are compiled and copied under build/c++-examples but are not being packaged. Similar is case with python examples also. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-837) harchive fail when output directory has URI with default port of 8020
harchive fail when output directory has URI with default port of 8020 - Key: MAPREDUCE-837 URL: https://issues.apache.org/jira/browse/MAPREDUCE-837 Project: Hadoop Map/Reduce Issue Type: Bug Components: harchive Affects Versions: 0.20.1 Reporter: Koji Noguchi Priority: Minor % hadoop archive -archiveName abc.har /user/knoguchi/abc hdfs://mynamenode:8020/user/knoguchi doesn't work on 0.18 nor 0.20 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-838) Task succeeds even when committer.commitTask fails with IOException
Task succeeds even when committer.commitTask fails with IOException
---
Key: MAPREDUCE-838
URL: https://issues.apache.org/jira/browse/MAPREDUCE-838
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: task
Affects Versions: 0.20.1
Reporter: Koji Noguchi
In MAPREDUCE-837, job succeeded with empty output even though all the tasks
were throwing IOException at commiter.commitTask.
{noformat}
2009-08-07 17:51:47,458 INFO org.apache.hadoop.mapred.TaskRunner: Task
attempt_200907301448_8771_r_00_0 is allowed to commit now
2009-08-07 17:51:47,466 WARN org.apache.hadoop.mapred.TaskRunner: Failure
committing: java.io.IOException: Can not get the relative path: \
base =
hdfs://mynamenode:8020/user/knoguchi/test2.har/_temporary/_attempt_200907301448_8771_r_00_0
\
child =
hdfs://mynamenode/user/knoguchi/test2.har/_temporary/_attempt_200907301448_8771_r_00_0/_index
at
org.apache.hadoop.mapred.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:150)
at
org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:106)
at
org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:126)
at
org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:86)
at
org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:171)
at org.apache.hadoop.mapred.Task.commit(Task.java:768)
at org.apache.hadoop.mapred.Task.done(Task.java:692)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
2009-08-07 17:51:47,468 WARN org.apache.hadoop.mapred.TaskRunner: Failure
asking whether task can commit: java.io.IOException: \
Can not get the relative path: base =
hdfs://mynamenode:8020/user/knoguchi/test2.har/_temporary/_attempt_200907301448_8771_r_00_0
\
child =
hdfs://mynamenode/user/knoguchi/test2.har/_temporary/_attempt_200907301448_8771_r_00_0/_index
at
org.apache.hadoop.mapred.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:150)
at
org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:106)
at
org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:126)
at
org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:86)
at
org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:171)
at org.apache.hadoop.mapred.Task.commit(Task.java:768)
at org.apache.hadoop.mapred.Task.done(Task.java:692)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
2009-08-07 17:51:47,469 INFO org.apache.hadoop.mapred.TaskRunner: Task
attempt_200907301448_8771_r_00_0 is allowed to commit now
2009-08-07 17:51:47,472 INFO org.apache.hadoop.mapred.TaskRunner: Task
'attempt_200907301448_8771_r_00_0' done.
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-839) unit test TestMiniMRChildTask fails on mac os-x
unit test TestMiniMRChildTask fails on mac os-x --- Key: MAPREDUCE-839 URL: https://issues.apache.org/jira/browse/MAPREDUCE-839 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.21.0 Reporter: Hong Tang Priority: Minor The unit test TestMiniMRChildTask fails on Mac OS-X (10.5.8) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-840) DBInputFormat leaves open transaction
DBInputFormat leaves open transaction - Key: MAPREDUCE-840 URL: https://issues.apache.org/jira/browse/MAPREDUCE-840 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Aaron Kimball Assignee: Aaron Kimball Priority: Minor DBInputFormat.getSplits() does not connection.commit() after the COUNT query. This can leave an open transaction against the database which interferes with other connections to the same table. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (MAPREDUCE-841) Protect Job Tracker against memory exhaustion due to very large InputSplit or JobConf objects
Protect Job Tracker against memory exhaustion due to very large InputSplit or JobConf objects - Key: MAPREDUCE-841 URL: https://issues.apache.org/jira/browse/MAPREDUCE-841 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobtracker Affects Versions: 0.20.1 Reporter: Hong Tang Fix For: 0.21.0 JobTracker only needs to examine a subset of information contained by InputSplit or JobConf objects. But currently JobTracker loads the complete user-defined InputSplit and JobConf objects in memory. This design would leave JobTracker susceptible to memory exhaustion particularly in cases when some bugs in user code which could result in very large input splits or job conf objects (e.g. PIG-901). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
