> On Sept. 20, 2012, 9:06 p.m., Rohini Palaniswamy wrote:
> > http://svn.apache.org/repos/asf/pig/trunk/test/e2e/pig/drivers/TestDriverPig.pm,
> > lines 136-154
> > <https://reviews.apache.org/r/7053/diff/1/?file=152857#file152857line136>
> >
> > This was taking 40 secs for each group in parallel mode as opposed to 6
> > secs in sequential mode and I am suspecting it is due to lot of forking. So
> > running two tests with tests.to.run option took double the time of
> > sequential (5 min to 12 min). That cost is immaterial if we are running the
> > whole suite. But reducing it helps as most of the time you will be running
> > a subset for testing.
> >
> > Would be good to optimize by combining the two hdfs mkdirs into one
> > command and use perl mkdir instead of IPC::Run (which internally does fork
> > and 1 sec sleep).
The results of fully sequential run under the same conditions:
[exec] Final results , PASSED: 555 FAILED: 2 SKIPPED: 65 ABORTED:
15 FAILED DEPENDENCY: 0
That is, the results are the same that in the paralellized mode.)
The run took 250 minutes, that is, 2.7 times larger than the parallelized one.
- Ivan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7053/#review11749
-----------------------------------------------------------
On Sept. 28, 2012, 12:20 p.m., Ivan Veselovsky wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7053/
> -----------------------------------------------------------
>
> (Updated Sept. 28, 2012, 12:20 p.m.)
>
>
> Review request for pig and Rohini Palaniswamy.
>
>
> Description
> -------
>
> Please see https://issues.apache.org/jira/browse/PIG-2898 for details.
>
>
> This addresses bug https://issues.apache.org/jira/browse/PIG-2898.
>
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/PIG-2898
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/pig/trunk/test/e2e/harness/TestDriver.pm
> 1390615
> http://svn.apache.org/repos/asf/pig/trunk/test/e2e/harness/test_harness.pl
> 1390615
> http://svn.apache.org/repos/asf/pig/trunk/test/e2e/pig/build.xml 1390615
> http://svn.apache.org/repos/asf/pig/trunk/test/e2e/pig/conf/local.conf
> 1390615
>
> http://svn.apache.org/repos/asf/pig/trunk/test/e2e/pig/deployers/ExistingClusterDeployer.pm
> 1390615
>
> http://svn.apache.org/repos/asf/pig/trunk/test/e2e/pig/drivers/TestDriverPig.pm
> 1390615
>
> http://svn.apache.org/repos/asf/pig/trunk/test/e2e/pig/tests/streaming_local.conf
> 1390615
>
> Diff: https://reviews.apache.org/r/7053/diff/
>
>
> Testing
> -------
>
> Tested e2e tests execution in both sequential (default) and parallel modes.
> The test run duration measurement data (in dependency on the fork parameters)
> will be available soon.
>
>
> Thanks,
>
> Ivan Veselovsky
>
>
