[
https://issues.apache.org/jira/browse/HADOOP-11984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549091#comment-14549091
]
Haohui Mai commented on HADOOP-11984:
-------------------------------------
bq. ~2 hours only for HDFS. The next closest (IIRC) is mapreduce-jobclient
which comes in at 20 minutes. Perhaps the HDFS folks should take a serious look
at re-arranging the universe, not running integration tests in unit tests,
start paying attention to the nightly build, etc.
Obviously we can do better here and there, but even optimizing for the
{{mapreduce-jobclient}} you're optimizing for 1% of the execution time. Is it
worthwhile to trade the additional complexity and possibly reduced robustness
for the optimized execution time?
bq. test-patch, ... MUST be able to run in a variety of hostile environments.
(No, Mac OS X and Linux are NOT good enough.)
Absolutely agree.
bq. FWIW, the only big portability problem with the current version of
test-patch.sh
The UNIX world supports bash pretty well. However, Hadoop officially supports
Windows as well. How does the compatibility story work for Windows folks?
Unfortunately things like path separators are quite annoying to handle.
bq. If I had my way, I'd have written this in perl 5. It's a significantly
better choice for the things we need to do here (text processing! OS
manipulation!)
To clarify I have no problems writing things in bash today. What I'm trying to
explore here from a maintainability / productivity prospective whether we
should allow writing supplement scripts in languages that have a wide range of
libraries to take care of things like text processing / OS manipulation,
regardless it's Python, Perl or other languages. From a longer term
prospective, however, having the option to leverage libraries can help improve
the productivity.
> Enable parallel JUnit tests in pre-commit.
> ------------------------------------------
>
> Key: HADOOP-11984
> URL: https://issues.apache.org/jira/browse/HADOOP-11984
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-11984.001.patch, HADOOP-11984.002.patch,
> HADOOP-11984.003.patch, HADOOP-11984.004.patch, HADOOP-11984.005.patch
>
>
> HADOOP-9287 and related issues implemented the parallel-tests Maven profile
> for running JUnit tests in multiple concurrent processes. This issue
> proposes to activate that profile during pre-commit to speed up execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
