[
https://issues.apache.org/jira/browse/HADOOP-3770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Douglas updated HADOOP-3770:
----------------------------------
Status: Open (was: Patch Available)
I haven't been over the details, but had a few general suggestions after a
first pass:
* Most of this doesn't conform to the coding guidelines. Converting tabs to
spaces, removing commented-out code, putting constants and defaults in a
reasonable place, etc. should be done before this can be committed.
* Temporary directories should be configurable and default off of a single,
configurable temp dir rather than being hard-coded off /tmp
* If independent from the original, the configuration and drivers of gridmix2
should not be in the same place. If this is intended as a replacement for
gridmix, it should modify the existing benchmark rather than creating _file_,
_file2_ pairs. If it's a new benchmark, it should be in src/benchmarks/gridmix2.
* Would it be possible to split the pig benchmarks into a separate JIRA? This
is simply too large to review well.
* GridMixRunner is unnecessarily enormous. Most of the methods are setting
defaults and performing work best encapsulated in the \*Creator classes that
currently do trivial work. This class would also benefit from utility methods
converting results from Configuration::getStrings to int[] (instead of
subclassing Configuration), abstracting out the creation of unique Strings for
runs (the use of Calendar/Date may not be the correct choice), javadoc, and
general cleanup
* Exceptions are almost always ignored; most probably should not be.
> improve composition, submission and result collection of gridmix
> ----------------------------------------------------------------
>
> Key: HADOOP-3770
> URL: https://issues.apache.org/jira/browse/HADOOP-3770
> Project: Hadoop Core
> Issue Type: Improvement
> Reporter: Lingyun Yang
> Assignee: Runping Qi
> Attachments: patch-3770.txt, patch-3770.txt, patch-3770.v2.txt
>
>
> Current gridmix submits jobs using a set of scripts, which is inconvenient
> and the results are difficult to collect. To improve the gridmix submission
> and results collection, we implemented a new program using JobControl to
> submit and collect the results of jobs
> Also the new gridmix allows to have more different types of jobs such as, pig
> jobs, jobs with combiners etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
