Hey Jonathan, Try calling Pipeline.enableDebug before you kick off your job-- I think it does what you're looking for, at the expense of some additional overhead for extra logging and try-catch blocks.
J On Tue, Aug 7, 2012 at 9:02 AM, Jonathan Dinu <[email protected]> wrote: > Hello, > > I am curious as to how exceptions get handled inside the Crunch runtime. I > am trying to do some input/output error checking and throwing the appropriate > exceptions. The exceptions in my code are getting throw but seem to be eaten > by Crunch and instead of my Exception (throw in a MapFn) bubbling up to the > console, a Crunch runtime exception is throw instead: > > org.apache.crunch.impl.mr.run.CrunchRuntimeException: java.io.IOException: No > files found to materialize at: /tmp/crunch72870282/p3 > > Is there anyway to break out of a parallelDo or a way gracefully handle > things such as bad data formats encountered? Or do I need to just catch my > exceptions and give Crunch an appropriate response. > > I tried throwing a RuntimeException in my run() method as well as the MapFn's > map() but both of those do not seem to stop Crunch from trying to materialize > the collection and subsequently throw a Crunch RuntimeException. Any help is > greatly appreciated. > > thanks!
