Slow iteration is a problem with Pig. I still write MR jobs mainly in Java because (1) I control the execution plan, (2) can do things nearly zero-copy, and (3) I can get a quick iteration cycle by using JUnit to test mappers, reducers, and other components.
On Tue, May 20, 2014 at 3:02 PM, Kevin Burton <[email protected]> wrote: > I've noticed that while working with pig my stress level and frustration > with the system is higher than other systems I've worked with. > > I think it's because the iteration cycle is longer. > > Even pig -x local takes a while to execute. > > Is this just me? > > If you're trying to learn and debug python lists, dictionaries, etc. It's > almost instant response time. > > But with pig literally everything takes 30-60 seconds to play with. > > -- > > Founder/CEO Spinn3r.com > Location: *San Francisco, CA* > Skype: *burtonator* > blog: http://burtonator.wordpress.com > … or check out my Google+ > profile<https://plus.google.com/102718274791889610666/posts> > <http://spinn3r.com> > War is peace. Freedom is slavery. Ignorance is strength. Corporations are > people. -- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype [email protected]
