[
https://issues.apache.org/jira/browse/PIG-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan A. Veselovsky updated PIG-2898:
------------------------------------
Attachment: PIG-2898-trunk-8.patch
Hi, Rohini,
in the attached "PIG-2898-trunk-8.patch" I renamed the methods
globalSetup2()/globalCleanup2() to
globalSetupConditional()/globalCleanupConditional() respectively.
This is different from the names onGroupRunXxx() suggested by you because of 2
reasons:
1) globalSetup2() executed once per test config file in sequential mode, and
once per test group in parallel mode, so, "onGroupRun" is not quite exact.
2) globalXxx2() methods complement the globalXxx() methods (are their
conditional parts), so, their names should be similar to each other.
> Parallel execution of e2e tests
> -------------------------------
>
> Key: PIG-2898
> URL: https://issues.apache.org/jira/browse/PIG-2898
> Project: Pig
> Issue Type: Improvement
> Components: e2e harness
> Affects Versions: 0.10.0
> Reporter: Andrey Klochkov
> Assignee: Ivan A. Veselovsky
> Labels: test
> Fix For: 0.11, 0.12
>
> Attachments: PIG-2898-branch-0.10-6-final.patch,
> PIG-2898-branch-0.10-7.patch, PIG-2898-trunk-3.patch,
> PIG-2898-trunk-6-final.patch, PIG-2898-trunk-7.patch, PIG-2898-trunk-8.patch
>
>
> Today it takes ~19 hours to run the full set of e2e tests in mapred mode. The
> bottleneck here is the client side, and per our observations it can help a
> lot if the e2e harness would be able to run tests in parallel threads.
> We prototyped changes in e2e harness allowing to run tests in a configurable
> number of threads. Preliminary results show more than 6x reduction in
> execution time when using a small 3-nodes M/R cluster with modest
> configuration. Going to share a patch shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
