Yan, I got the same failure when I turned off the optimization, so I think something else was to blame. Still haven't quite figured out what is going wrong in the script I was debugging -- it was complex and had multiple problems. I will send something along if I am able to come up with a simpler script that reproduces this issue.
-D On Wed, Oct 13, 2010 at 2:04 PM, Yan Zhou <[email protected]> wrote: > Dimitry, > > Could you send me your filter expression on which the logical expression > simplifier failed upon to help me debug the problem as this sits in the > critical path of pig optimization? > > I do not need your full query script and not even the exact filter > expression but just the same expression structure with whatever field names > you may put there. For instance if you have a filter as follows: > > B = FILTER a BY (twitterField1 > 7 AND twitterField2 <3); > > It will suffice for me to debug if you send me the following > > B = FILTER a BY (foo1 > 7 AND foo2 <3); > > In other word, it is the expression tree structure and operators that > matters not the field names on the tree. > > Thanks for help! > > Yan > > -----Original Message----- > From: Yan Zhou > Sent: Monday, October 11, 2010 5:02 PM > To: '[email protected]' > Cc: '[email protected]' > Subject: RE: Turning off individual rules in new optimizer > > Please try "-t FilterLogicExpressionSimplifier" to turn it off. > > In the meanwhile, could you send me your filter expression that you suspect > the simplifier fails on? > > Thanks, > > Yan > > -----Original Message----- > From: Dmitriy Ryaboy [mailto:[email protected]] > Sent: Monday, October 11, 2010 4:51 PM > To: [email protected] > Subject: Turning off individual rules in new optimizer > > I am getting mysterious errors that I think are due to the new optimizer in > pig 8. > How can I turn off individual rules (LogicalExpressionSimplifier in this > case)? > > Thanks, > -Dmitriy >
