Thanks Ashutosh! Your explanation and some debugging where quite enlightening!
Renato M. 2010/10/29 Ashutosh Chauhan <hashut...@apache.org> > Currently parsing and logical plan building is mixed. So, as script is > getting parsed, its corresponding logical plan > is built. So, at the end of parsing phase, a logical plan is emitted. > Inner plans are also built at the same time. The whole plan > construction is done in once shot. This plan is then run through > validators which do various kind of validations and checks, is then > optimized, then translated into physical plan, then translated into > Map reduce plan, then optimized again and is finally submitted via job > client for execution. > > Hope it helps, > Ashutosh > > On Thu, Oct 28, 2010 at 19:22, Renato Marroquín Mogrovejo > <renatoj.marroq...@gmail.com> wrote: > > Thanks Ashutosh! I read the wiki and it was really illustrating (: > > But there is one thing, are all plans (top level and inner plans) made at > > the same time? I mean after parsing the query, or is the top level plan > made > > first and then when the executing one of its nodes, then an inner plan is > > made for it? > > Is there anywhere else where I could find any extra information? > > > > Renato M. > > > > > > 2010/10/24 Ashutosh Chauhan <hashut...@apache.org> > > > >> http://wiki.apache.org/pig/NestedLogicalPlan > >> > >> This doesn't reflect the current implementation but fundamental idea > >> remains the same. You may want to start from there. > >> > >> Ashutosh > >> On Sun, Oct 24, 2010 at 12:59, Renato Marroquín Mogrovejo > >> <renatoj.marroq...@gmail.com> wrote: > >> > Hi everyone, > >> > > >> > I know Pig builds a logical plan for every bag that the user defines, > and > >> > that every new bag will have as its logical plan the combination of > the > >> > input bags' logical plans and the comands it uses. Is there any other > >> > command that creates a logical plan? I guess the one that creates the > >> first > >> > logical plan is the STORE comand. And how are the other commands' > logical > >> > plans combined? For example if I have a join between two relations, it > >> will > >> > create a single logical plan that contains the loading logical plans, > but > >> > how does Pig decide which relation goes where? Or is that a part that > >> > happens while compiling the logical plan into MapReduce and not in the > >> > logical plan? > >> > Thanks in advanced. > >> > > >> > Renato M. > >> > > >> > > >
