[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Resolution: Fixed
Fix Version/s: 2.9.0
Target Version/s: (was: 3.0.0, 2.9.0)
Tags: (was: MRBench)
Status: Resolved (was: Patch Available)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.3.0
>Reporter: Michael Noll
>Assignee: Wilfred Spiegelenburg
>Priority: Minor
> Fix For: 2.9.0
>
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398.2.patch,
> MAPREDUCE-2398_0.20.2.patch, MAPREDUCE-2398_v2-0.20.203.0.patch,
> MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Hadoop Flags: Reviewed
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.3.0
>Reporter: Michael Noll
>Assignee: Wilfred Spiegelenburg
>Priority: Minor
> Fix For: 2.9.0
>
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398.2.patch,
> MAPREDUCE-2398_0.20.2.patch, MAPREDUCE-2398_v2-0.20.203.0.patch,
> MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Status: Patch Available (was: Reopened)
(Sorry for all the extra status noise; Marking as Patch-Available finally)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.3.0
>Reporter: Michael Noll
>Assignee: Wilfred Spiegelenburg
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398.2.patch,
> MAPREDUCE-2398_0.20.2.patch, MAPREDUCE-2398_v2-0.20.203.0.patch,
> MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Assignee: (was: Yang Hao)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.3.0
>Reporter: Michael Noll
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398.2.patch,
> MAPREDUCE-2398_0.20.2.patch, MAPREDUCE-2398_v2-0.20.203.0.patch,
> MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wilfred Spiegelenburg updated MAPREDUCE-2398:
-
Attachment: MAPREDUCE-2398.2.patch
This seems to have gone stale for a long time and just ran into this.
I have updated the patch and made sure it works passing in the value from the
command line. As part of the change I also cleaned up the directory creation
and cleanup.
The INPUT_DIR automatically gets created by the {{generateTextFile()}} call.
The clean up should not delete the BASE_DIR because it could exist and have
other data in it. Only the OUTPUT_DIR and INPUT_DIR that were created for the
run should be removed for cleanup.
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.3.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398.2.patch,
> MAPREDUCE-2398_0.20.2.patch, MAPREDUCE-2398_v2-0.20.203.0.patch,
> MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Release Note: (was: Fixes MRBench to properly handle the -baseDir
parameter.)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.3.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Target Version/s: 3.0.0, 2.9.0 (was: 2.8.0)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.3.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Affects Version/s: (was: 0.20.2)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.3.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated MAPREDUCE-2398:
Target Version/s: 2.6.0 (was: 1.3.0, 2.5.0)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 2.3.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Fix Version/s: (was: 2.5.0)
(was: 1.3.0)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 2.3.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Affects Version/s: (was: 2.5.0)
(was: 2.4.0)
(was: 0.22.0)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 2.3.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Target Version/s: 2.5.0 (was: 2.3.0, 2.4.0, 2.5.0)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 2.3.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated MAPREDUCE-2398:
---
Target Version/s: 1.3.0, 2.5.0 (was: 2.5.0)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 2.3.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Hao updated MAPREDUCE-2398:
Target Version/s: 2.3.0, 2.4.0, 2.5.0 (was: 2.3.0, 2.5.0)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 0.22.0, 2.3.0, 2.4.0, 2.5.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Fix For: 1.3.0, 2.5.0
>
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Hao updated MAPREDUCE-2398:
Tags: MRBench
Fix Version/s: (was: 0.22.1)
2.5.0
1.3.0
Assignee: Yang Hao
Target Version/s: 2.3.0, 2.5.0
Affects Version/s: 2.5.0
2.4.0
2.3.0
Status: In Progress (was: Patch Available)
This problem still exists in the version Hadoop-2.3.0, so please fix it in the
java "org.apache.hadoop.mapred.MRBench".Sometimes we don't have the permission
to operate all the HDFS file, so we have to change the BASE_DIR , and the
INPUT_DIR and OUTPUT_DIR should be updated when BASE_DIR is set. Only two extra
lines be put into the java source code can solve the bug
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.3.0, 0.22.0, 0.20.2, 2.4.0, 2.5.0
>Reporter: Michael Noll
>Assignee: Yang Hao
>Priority: Minor
> Fix For: 1.3.0, 2.5.0
>
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Noll updated MAPREDUCE-2398:
Attachment: MAPREDUCE-2398_v2-trunk.patch
New version of patch, integrating Harsh J's suggestions. This patch is based on
trunk (SVN rev 1166009 / git 2647680).
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 0.22.0
>Reporter: Michael Noll
>Priority: Minor
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch, MAPREDUCE-2398_v2-trunk.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Noll updated MAPREDUCE-2398:
Attachment: MAPREDUCE-2398_v2-0.20.203.0.patch
New version of patch, integrating Harsh J's suggestions. This patch is based
on Hadoop release 0.20.203.0.
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 0.22.0
>Reporter: Michael Noll
>Priority: Minor
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Noll updated MAPREDUCE-2398:
Fix Version/s: 0.22.0
Release Note: Fixes MRBench to properly handle the -baseDir parameter.
Status: Patch Available (was: Open)
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 0.22.0
>Reporter: Michael Noll
>Priority: Minor
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Noll updated MAPREDUCE-2398:
Attachment: MAPREDUCE-2398-trunk.patch
Patch in svn diff format for trunk.
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 0.22.0
>Reporter: Michael Noll
>Priority: Minor
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Noll updated MAPREDUCE-2398:
Description:
The optional {{-baseDir}} parameter lets user specify the base DFS path for
output/input of MRBench.
However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
(MRBench.java) are not updated in the case that the default value of
{{-baseDir}} is actually overwritten by the user. Hence any input and output is
always written to the default locations ({{/benchmarks/MRBench/...}}), even
though the user-supplied location for {{-baseDir}} is created (and eventually
deleted again) on HDFS.
The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
March 21, 2011.
was:
The optional {{-baseDir}} parameter lets user specify the base DFS path for
output/input of MRBench.
However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
(MRBench.java) are not updated in the case that the default value of
{{-baseDir}} is actually overwritten by the user. Hence any input and output is
always written to the default locations ({{/benchmarks/MRBench/...}}), even
though the user-supplied location for {{-baseDir}} is created (and eventually
deleted again) on HDFS.
Affects Version/s: 0.22.0
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 0.22.0
>Reporter: Michael Noll
>Priority: Minor
> Attachments: MAPREDUCE-2398_0.20.2.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Noll updated MAPREDUCE-2398:
Attachment: MAPREDUCE-2398_0.20.2.patch
Patch in unified diff format (old/src/... versus new/src/...) for Hadoop 0.20.2
attached.
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2
>Reporter: Michael Noll
>Priority: Minor
> Attachments: MAPREDUCE-2398_0.20.2.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
