Re: Is Intermediate data written to disk?

2010-02-03 Thread bharath v
count the regular IO that needs to happen for a standard Map-Reduce > join, and any possible spills to disk required when buffers run out of > memory and such. > > Hope this helps. > > Dmitriy > > On Wed, Feb 3, 2010 at 4:17 AM, bharath v > wrote: > > Dimitr

Re: Is Intermediate data written to disk?

2010-02-03 Thread bharath v
> the intermediate result of joining A to B. > > -Dmitriy > > On Tue, Feb 2, 2010 at 10:52 PM, bharath v > wrote: > > Hi , > > > > I have a small doubt in how pig handles queries containing join of more > than > > 2 tables . > > > > Suppose we hav

Is Intermediate data written to disk?

2010-02-02 Thread bharath v
Hi , I have a small doubt in how pig handles queries containing join of more than 2 tables . Suppose we have 3 tables A,B,C .. and the plan is "((AB)C)" .. We can join A,B in a map reduce job and join the resultant table with "C". I have a doubt whether the result of "AB" is stored to disk befor