[
https://issues.apache.org/jira/browse/PIG-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487221#comment-13487221
]
Rohini Palaniswamy commented on PIG-2898:
-----------------------------------------
Thanks Ivan. Will commit soon. Just FYI. It is not required to cleanup after by
deleting older patches. We usually leave them in the jira so that the history
of the progress is there.
> Parallel execution of e2e tests
> -------------------------------
>
> Key: PIG-2898
> URL: https://issues.apache.org/jira/browse/PIG-2898
> Project: Pig
> Issue Type: Improvement
> Components: e2e harness
> Affects Versions: 0.10.0
> Reporter: Andrey Klochkov
> Assignee: Ivan A. Veselovsky
> Labels: test
> Fix For: 0.11, 0.12
>
> Attachments: PIG-2898-trunk-3.patch, PIG-2898-trunk-8.patch
>
>
> Today it takes ~19 hours to run the full set of e2e tests in mapred mode. The
> bottleneck here is the client side, and per our observations it can help a
> lot if the e2e harness would be able to run tests in parallel threads.
> We prototyped changes in e2e harness allowing to run tests in a configurable
> number of threads. Preliminary results show more than 6x reduction in
> execution time when using a small 3-nodes M/R cluster with modest
> configuration. Going to share a patch shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
