As a follow-up: I've been running the "ant test" suite since noon, trying ot validate PIG-1870. It's now 10 pm and I'm up to TestSkewedJoin. A few tests have failed, even though my patch didn't touch any of the related code (TestPigContext, TestGrunt, TestJobSubmission, TestSecondarySort, TestEvalPipeline2) -- I wonder if these pass normally, or if they are just ignored by virtue of not being in test-commit and taking far too long....
D On Fri, Apr 15, 2011 at 2:59 PM, Dmitriy Ryaboy <[email protected]> wrote: > There are quite a few tests in the suite that use ExecType.MAPREDUCE. It > seems to me that we can now get away with ExecType.LOCAL for many of these, > since local is now using Hadoop. The difference in test speed is fairly > dramatic -- when I switched HBaseStorage tests, they went from roughly 50 > seconds each to roughly 12 seconds each. The only tests for which (I think) > we need the full MAPREDUCE exec mode is when we are messing with classpaths, > need to ensure new jvms pick up the right classes, etc. Objections to > switching these tests over? Suggestions on what to watch out for? > > D > >
