FYI, Starting Jetty in Hadoop on my machine takes much longer than you. Normally, it takes at least 15 seconds each time here when it takes sub-second in your logs. I will have to dig more.
08/05/10 23:22:01 INFO fs.FSNamesystem: Registered FSNamesystemStatusMBean 08/05/10 23:22:16 INFO util.Credential: Checking Resource aliases 08/05/10 23:22:16 INFO http.HttpServer: Version Jetty/5.1.x 08/05/10 23:22:17 INFO dfs.DataNode: Balancing bandwith is 1048576 bytes/s 08/05/10 23:22:32 INFO http.HttpServer: Version Jetty/5.1.x 08/05/10 23:22:33 INFO util.Container: Started [EMAIL PROTECTED] 08/05/10 23:22:33 INFO dfs.DataNode: BlockReport of 0 blocks got processed in 5 msecs 08/05/10 23:22:48 INFO http.HttpServer: Version Jetty/5.1.x 08/05/10 23:22:48 INFO util.Container: Started [EMAIL PROTECTED] 08/05/10 23:22:49 INFO dfs.DataNode: BlockReport of 0 blocks got processed in 6 msecs 08/05/10 23:23:04 INFO http.HttpServer: Version Jetty/5.1.x 08/05/10 23:23:04 INFO util.Container: Started [EMAIL PROTECTED] 08/05/10 23:23:04 INFO dfs.DataNode: BlockReport of 0 blocks got processed in 8 msecs 08/05/10 23:23:19 INFO http.HttpServer: Version Jetty/5.1.x 08/05/10 23:23:20 INFO util.Container: Started [EMAIL PROTECTED] 08/05/10 23:23:36 INFO mapred.MiniMRCluster: mapred.local.dir is /tmp/hadoop-root/mapred/local/0_0 08/05/10 23:23:51 INFO http.HttpServer: Version Jetty/5.1.x 08/05/10 23:23:51 INFO util.Container: Started [EMAIL PROTECTED] On Tue, May 13, 2008 at 8:54 AM, pi song <[EMAIL PROTECTED]> wrote: > Thanks for your information. > I'll look at this tonight. > > > On 5/13/08, Olga Natkovich <[EMAIL PROTECTED]> wrote: > > > > We do see similar slowdown testing with hadoop 0.17 and still trying to > > figure out why this is happening. I wonder if this could be related. > > > > > -----Original Message----- > > > From: Alan Gates [mailto:[EMAIL PROTECTED] > > > Sent: Monday, May 12, 2008 9:41 AM > > > To: [email protected] > > > Subject: Re: Slow unit testing > > > > > > Sorry, I should have been more clear. Time results similar > > > to Olga's, not Pi's. > > > > > > Alan. > > > > > > Alan Gates wrote: > > > > I run the unit tests on a mac laptop and am seeing similar > > > time results. > > > > > > > > Alan. > > > > > > > > Olga Natkovich wrote: > > > >> Pi, > > > >> > > > >> Inline my stdout content. Attached test output files. I ran this > > > >> against latest trunk on a linux box. > > > >> > > > >> test-local: > > > >> > > > >> testOneMode: > > > >> [mkdir] Created dir: > > > >> /home/olgan/src/pig-apache3/trunk/build/test/logs-local > > > >> [junit] Running org.apache.pig.test.TestAlgebraicEval > > > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time > > > elapsed: 2.93 sec > > > >> [junit] Running org.apache.pig.test.TestBinaryStorage > > > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time > > > elapsed: 0.662 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestBuiltin > > > >> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: > > > >> 3.849 sec > > > >> [junit] Running org.apache.pig.test.TestCmdLineParser > > > >> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: > > > >> 0.011 sec > > > >> [junit] Running org.apache.pig.test.TestCombiner > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 0.421 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestCompressedFiles > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 0.337 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestCustomSlicer > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 0.214 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestDataBag > > > >> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: > > > >> 1.461 sec > > > >> [junit] Running org.apache.pig.test.TestDataModel > > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time > > > elapsed: 0.126 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestEvalPipeline > > > >> [junit] Tests run: 7, Failures: 0, Errors: 0, Time > > > elapsed: 5.035 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestExGenCogroup > > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time > > > elapsed: 1.246 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestExGenEval > > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time > > > elapsed: 1.172 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestFilterOpNumeric > > > >> [junit] Tests run: 8, Failures: 0, Errors: 0, Time > > > elapsed: 1.76 sec > > > >> [junit] Running org.apache.pig.test.TestFilterOpString > > > >> [junit] Tests run: 6, Failures: 0, Errors: 0, Time > > > elapsed: 1.385 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestGrunt > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 0.238 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestInfixArithmetic > > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: > > > >> 57.608 sec > > > >> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder > > > >> [junit] Tests run: 49, Failures: 0, Errors: 0, Time > > > elapsed: 1.04 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestMapReduce > > > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: > > > >> 73.303 sec > > > >> [junit] Running > > > org.apache.pig.test.TestMapReduceResultRecycling > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 1.542 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestParamSubPreproc > > > >> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: > > > >> 0.633 sec > > > >> [junit] Running org.apache.pig.test.TestParser > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 0.206 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestPigContext > > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time > > > elapsed: 0.806 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestPigFile > > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time > > > elapsed: 3.344 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestPigScriptParser > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 0.441 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestPigServer > > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time > > > elapsed: 5.76 sec > > > >> [junit] Running org.apache.pig.test.TestPigSplit > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 0.546 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestReversibleLoadStore > > > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time > > > elapsed: 0.622 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestStore > > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time > > > elapsed: 2.139 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestStreaming > > > >> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: > > > >> 1.074 sec > > > >> > > > >> test-mapreduce: > > > >> > > > >> testOneMode: > > > >> [mkdir] Created dir: > > > >> /home/olgan/src/pig-apache3/trunk/build/test/logs-mapreduce > > > >> [junit] Running org.apache.pig.test.TestAlgebraicEval > > > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: > > > >> 74.738 sec > > > >> [junit] Running org.apache.pig.test.TestBinaryStorage > > > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time > > > elapsed: 36.99 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestBuiltin > > > >> [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: > > > >> 4.513 sec > > > >> [junit] Running org.apache.pig.test.TestCmdLineParser > > > >> [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: > > > >> 0.034 sec > > > >> [junit] Running org.apache.pig.test.TestCombiner > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 24.9 sec > > > >> [junit] Running org.apache.pig.test.TestCompressedFiles > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: > > > >> 29.658 sec > > > >> [junit] Running org.apache.pig.test.TestCustomSlicer > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: > > > >> 90.949 sec > > > >> [junit] Running org.apache.pig.test.TestDataBag > > > >> [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: > > > >> 1.605 sec > > > >> [junit] Running org.apache.pig.test.TestDataModel > > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time > > > elapsed: 0.194 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestEvalPipeline > > > >> [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: > > > >> 203.826 sec > > > >> [junit] Running org.apache.pig.test.TestExGenCogroup > > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: > > > >> 15.116 sec > > > >> [junit] Running org.apache.pig.test.TestExGenEval > > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time > > > elapsed: 8.609 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestFilterOpNumeric > > > >> [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: > > > >> 104.898 sec > > > >> [junit] Running org.apache.pig.test.TestFilterOpString > > > >> [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: > > > >> 83.079 sec > > > >> [junit] Running org.apache.pig.test.TestGrunt > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 0.296 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestInfixArithmetic > > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: > > > >> 50.802 sec > > > >> [junit] Running org.apache.pig.test.TestLogicalPlanBuilder > > > >> [junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: > > > >> 1.328 sec > > > >> [junit] Running org.apache.pig.test.TestMapReduce > > > >> [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: > > > >> 52.169 sec > > > >> [junit] Running > > > org.apache.pig.test.TestMapReduceResultRecycling > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: > > > >> 43.827 sec > > > >> [junit] Running org.apache.pig.test.TestParamSubPreproc > > > >> [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: > > > >> 0.724 sec > > > >> [junit] Running org.apache.pig.test.TestParser > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 8.08 sec > > > >> [junit] Running org.apache.pig.test.TestPigContext > > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time > > > elapsed: 1.209 > > > >> sec > > > >> [junit] Running org.apache.pig.test.TestPigFile > > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time > > > elapsed: 0.95 sec > > > >> [junit] Running org.apache.pig.test.TestPigScriptParser > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time > > > elapsed: 0.26 sec > > > >> [junit] Running org.apache.pig.test.TestPigServer > > > >> [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: > > > >> 10.313 sec > > > >> [junit] Running org.apache.pig.test.TestPigSplit > > > >> [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: > > > >> 109.971 sec > > > >> [junit] Running org.apache.pig.test.TestReversibleLoadStore > > > >> [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: > > > >> 11.803 sec > > > >> [junit] Running org.apache.pig.test.TestStore > > > >> [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: > > > >> 125.27 sec > > > >> [junit] Running org.apache.pig.test.TestStreaming > > > >> [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: > > > >> 142.835 sec > > > >> > > > >> test: > > > >> > > > >> BUILD SUCCESSFUL > > > >> Total time: 26 minutes 4 seconds > > > >> ucdev3:~/src/pig-apache3/trunk> cd > > > >> ~/src/pig-yahoo/trunk/pig/conf/piglet/hadoop-site.xml > > > > > > >
