I thought it was too late in the workflow to do this, but it would be really cool if you could. I don't think to think about MapReduce much when I Pig, except to group my scripts by jobs... so this was a surprise for me. Made sense once i thought of it. But it was a surprise.
On Fri, Mar 2, 2012 at 1:49 PM, Bill Graham <billgra...@gmail.com> wrote: > I was also curious about this and will try it, but my initial thought was > that at that point it might be tool late in the workflow of the job. I'll > give it a shot and report back. > > > On Fri, Mar 2, 2012 at 1:45 PM, Dmitriy Ryaboy <dvrya...@gmail.com> wrote: > > > In a StoreFunc, you could do that when you get passed the jobconf, right? > > > > On Thu, Mar 1, 2012 at 9:37 PM, Bill Graham <billgra...@gmail.com> > wrote: > > > I don't think so. We just do it in the pig script before using the > store > > > func: > > > > > > SET mapred.map.tasks.speculative.execution false > > > > > > > > > On Thu, Mar 1, 2012 at 4:19 PM, Russell Jurney < > russell.jur...@gmail.com > > >wrote: > > > > > >> Can you turn off speculative execution in a StoreFunc? I believe it > is > > >> leading to duplicates in MongoStorage(); > > >> > > >> -- > > >> Russell Jurney twitter.com/rjurney russell.jur...@gmail.com > > >> datasyndrome.com > > >> > > > > > > > > > > > > -- > > > *Note that I'm no longer using my Yahoo! email address. Please email me > > at > > > billgra...@gmail.com going forward.* > > > > > > -- > *Note that I'm no longer using my Yahoo! email address. Please email me at > billgra...@gmail.com going forward.* > -- Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome.com
