The optimization code is in the package org.apache.pig.experimental. The
code is not used by default unless you set system property
pig.usenewlogicalplan to true. A good starting point might be
"org.apache.pig.experimental.logical.optimizer.LogicalPlanOptimizer".
Daniel
Renato Marroquín Mogrovejo wrote:
Thanks Daniel and Dmitriy for your answers, now I have a much clear idea of
what type of optimization work is being done on PIG. And just another quick
question, do you guys know where in the code those optimizations are?
I just want to give it a closer look (:
Renato M.
2010/7/1 Dmitriy Ryaboy <dvrya...@gmail.com>
Renato,
I just want to make sure folks know -- Pig already has a number of such
optimizations. Daniel's work is aimed at making it (much) easier to write
such rules and to add a couple new ones. But some of the classic
optimizations like projection and filter push-down already exist in the
released versions of Pig.
D
On Thu, Jul 1, 2010 at 5:32 PM, Daniel Dai <jiany...@yahoo-inc.com> wrote:
Yes, they are classic logical optimization plus some Pig only
optimization.
All these are rule based.
https://issues.apache.org/jira/browse/PIG-1319is an umbrella Jira to track
all new optimization rules.
Daniel
Renato Marroquín Mogrovejo wrote:
Hi,
I am also interested in this logical plan optimization framework
functionality. You mentioned that rules are being developed, could you
explain a little bit more about them? Are they like the classic logical
optimizations (early projection, early filtering, among others)?
Thanks in advance.
Renato M.
logical plan optimization framework
2010/6/28 Alan Gates <ga...@yahoo-inc.com>
On Jun 28, 2010, at 12:36 AM, Swati Jain wrote:
Thanks for the prompt reply. As you mentioned optimization is in its
developing stage, does it mean optimization framework is not complete
or
only rules are in developing stage? In addition to that, I would
really
appreciate if you could give a rough idea when the patch will be
available
and what functionality will it contain?
At this point we believe the framework is complete and rules are
being
developed. But the framework has never been used in user testing
situations
(alpha or beta testing) so there will be a whole round of bugs to fix
once
that testing is done.
The current plan is to switch to this code as the actual optimizer with
0.8, which we hope to release late this year (no promises).
Alan.
