Re: Bug in new logical optimizer framework?

2010-07-06 Thread Daniel Dai
The optimization code is in the package org.apache.pig.experimental. The code is not used by default unless you set system property pig.usenewlogicalplan to true. A good starting point might be "org.apache.pig.experimental.logical.optimizer.LogicalPlanOptimizer". Daniel Renato MarroquĂ­n Mogro

[jira] Commented: (PIG-1478) Add progress notification listener to PigRunner API

2010-07-06 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885606#action_12885606 ] Richard Ding commented on PIG-1478: --- Run core tests manually and they passed. > Add progre

Re: dynamically change an operator's behavior

2010-07-06 Thread Daniel Dai
map() have access to the entire map plan. So you can traverse through the plan, find your node inside map(). Daniel Gang Luo wrote: Hi all, I am implementing an operator that needs runtime information and dynamically changes its behaviors. I want to tell that operator certain requirement is

[jira] Commented: (PIG-1478) Add progress notification listener to PigRunner API

2010-07-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885625#action_12885625 ] Hadoop QA commented on PIG-1478: -1 overall. Here are the results of testing the latest atta

[jira] Commented: (PIG-1295) Binary comparator for secondary sort

2010-07-06 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885626#action_12885626 ] Daniel Dai commented on PIG-1295: - Patch looks good, one comment: Where is the code "if the u

[jira] Commented: (PIG-928) UDFs in scripting languages

2010-07-06 Thread Julien Le Dem (JIRA)
[ https://issues.apache.org/jira/browse/PIG-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885633#action_12885633 ] Julien Le Dem commented on PIG-928: --- actually, I retract the init() method as it seems this

[jira] Commented: (PIG-928) UDFs in scripting languages

2010-07-06 Thread Julien Le Dem (JIRA)
[ https://issues.apache.org/jira/browse/PIG-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885635#action_12885635 ] Julien Le Dem commented on PIG-928: --- Argh... Sorry about that {code} /** *

[jira] Commented: (PIG-1481) PigServer throws exception if it cannot find hadoop-site.xml or core-site.xml

2010-07-06 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885650#action_12885650 ] Daniel Dai commented on PIG-1481: - You will need hadoop conf file even in MiniCluster. Pig wi

[jira] Commented: (PIG-1471) inline UDFs in scripting languages

2010-07-06 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885652#action_12885652 ] Daniel Dai commented on PIG-1471: - Yes, Jython works because of JythonFunction has the abilit

[jira] Updated: (PIG-1462) No informative error message on parse problem

2010-07-06 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1462: Fix Version/s: 0.8.0 > No informative error message on parse problem > --

RE: PIG Logical Optimization: Use CNF in SplitFilter

2010-07-06 Thread Yan Zhou
Swati, I happen to be working on the logical expression simplification effort (https://issues.apache.org/jira/browse/PIG-1399), but not on the filter split front. So I guess our interests will have some overlaps. I think the filter logic split problem can be divided into 2 parts: 1) the filtering

[jira] Resolved: (PIG-1476) Add trailing flag to commands to prevent retention of relation name in field names: STRIP ?

2010-07-06 Thread Russell Jurney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Russell Jurney resolved PIG-1476. - Resolution: Fixed This is actually ok. > Add trailing flag to commands to prevent retention of rel

[jira] Updated: (PIG-200) Pig Performance Benchmarks

2010-07-06 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-200: --- Attachment: pigmix2.patch > Pig Performance Benchmarks > -- > > Key: PIG

[jira] Updated: (PIG-200) Pig Performance Benchmarks

2010-07-06 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-200: --- Attachment: (was: pigmix2.patch) > Pig Performance Benchmarks > -- > >

[jira] Updated: (PIG-1389) Implement Pig counter to track number of rows for each input files

2010-07-06 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Ding updated PIG-1389: -- Attachment: PIG-1389_2.patch > Implement Pig counter to track number of rows for each input files >

[jira] Updated: (PIG-1389) Implement Pig counter to track number of rows for each input files

2010-07-06 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Ding updated PIG-1389: -- Status: Patch Available (was: Open) > Implement Pig counter to track number of rows for each input files

[jira] Commented: (PIG-1389) Implement Pig counter to track number of rows for each input files

2010-07-06 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885700#action_12885700 ] Richard Ding commented on PIG-1389: --- New patch to address above comments. bq. 2. In PigRec

[jira] Updated: (PIG-1389) Implement Pig counter to track number of rows for each input files

2010-07-06 Thread Richard Ding (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Ding updated PIG-1389: -- Status: Open (was: Patch Available) > Implement Pig counter to track number of rows for each input files

[jira] Updated: (PIG-928) UDFs in scripting languages

2010-07-06 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-928: --- Attachment: RegisterPythonUDFFinale3.patch Thanks Dmitriy and Julien for your help. Attached is the patc

[jira] Updated: (PIG-928) UDFs in scripting languages

2010-07-06 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-928: --- Status: Patch Available (was: Open) > UDFs in scripting languages > --- > >

[jira] Commented: (PIG-928) UDFs in scripting languages

2010-07-06 Thread Julien Le Dem (JIRA)
[ https://issues.apache.org/jira/browse/PIG-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885737#action_12885737 ] Julien Le Dem commented on PIG-928: --- ScriptEvalFunc does not do much anymore, I would sugges

[jira] Updated: (PIG-506) Does pig need a NATIVE keyword?

2010-07-06 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-506: --- Fix Version/s: 0.8.0 > Does pig need a NATIVE keyword? > --- > >

[jira] Commented: (PIG-1389) Implement Pig counter to track number of rows for each input files

2010-07-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885804#action_12885804 ] Hadoop QA commented on PIG-1389: -1 overall. Here are the results of testing the latest atta

[jira] Commented: (PIG-928) UDFs in scripting languages

2010-07-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/PIG-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885822#action_12885822 ] Hadoop QA commented on PIG-928: --- -1 overall. Here are the results of testing the latest attachm

[jira] Updated: (PIG-1472) Optimize serialization/deserialization between Map and Reduce and between MR jobs

2010-07-06 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated PIG-1472: --- Attachment: PIG-1472.patch Summary of changes in the patch - 1. The default TupleFactory is now BinSede

[jira] Commented: (PIG-1295) Binary comparator for secondary sort

2010-07-06 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885831#action_12885831 ] Thejas M Nair commented on PIG-1295: 1. If utf8 encoded ordering is same as java string

[jira] Commented: (PIG-1472) Optimize serialization/deserialization between Map and Reduce and between MR jobs

2010-07-06 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885835#action_12885835 ] Thejas M Nair commented on PIG-1472: I ran the pigmix1 queries with changes to specify ty

[jira] Commented: (PIG-1482) Pig gets confused when more than one loader is involved

2010-07-06 Thread Ankur (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885840#action_12885840 ] Ankur commented on PIG-1482: forgot to add Include this change as well for the above script to w

[jira] Commented: (PIG-1482) Pig gets confused when more than one loader is involved

2010-07-06 Thread Ankur (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885839#action_12885839 ] Ankur commented on PIG-1482: Casting early alleviates the problem. So this makes the above script

[jira] Commented: (PIG-1482) Pig gets confused when more than one loader is involved

2010-07-06 Thread Ankur (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12885838#action_12885838 ] Ankur commented on PIG-1482: ERROR 1065: Found more than one load function to use: [PigStorage, T

[jira] Created: (PIG-1482) Pig gets confused when more than one loader is involved

2010-07-06 Thread Ankur (JIRA)
Pig gets confused when more than one loader is involved --- Key: PIG-1482 URL: https://issues.apache.org/jira/browse/PIG-1482 Project: Pig Issue Type: Bug Affects Versions: 0.7.0