[jira] Commented: (PIG-1303) unable to set outgoing format for org.apache.pig.piggybank.evaluation.util.apachelogparser.DateExtractor

2010-04-22 Thread Dmitriy V. Ryaboy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860142#action_12860142 ] Dmitriy V. Ryaboy commented on PIG-1303: hc, I haven't gone through the code, but I t

[jira] Commented: (PIG-1385) UDF to create tuples and bags

2010-04-22 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860134#action_12860134 ] Hadoop QA commented on PIG-1385: -1 overall. Here are the results of testing the latest atta

[jira] Commented: (PIG-1386) UDF to extend functionalities of MaxTupleBy1stField

2010-04-22 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860121#action_12860121 ] Hadoop QA commented on PIG-1386: -1 overall. Here are the results of testing the latest atta

[jira] Assigned: (PIG-1390) Provide a target to generate eclipse-related classpath and files

2010-04-22 Thread Ashutosh Chauhan (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned PIG-1390: - Assignee: V.V.Chaitanya Krishna > Provide a target to generate eclipse-related classpath an

[jira] Updated: (PIG-1386) UDF to extend functionalities of MaxTupleBy1stField

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1386: - Attachment: (was: PIG-1386-trunk.patch) > UDF to extend functionalities of MaxTupleBy1stField > ---

[jira] Updated: (PIG-1386) UDF to extend functionalities of MaxTupleBy1stField

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1386: - Attachment: PIG-1386-trunk.patch a92218b0c641363439af8f2d9e5ecbc0 > UDF to extend functionalities of MaxTupleBy1st

[jira] Commented: (PIG-1386) UDF to extend functionalities of MaxTupleBy1stField

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860104#action_12860104 ] hc busy commented on PIG-1386: -- I'm having trouble writing this UDF because of the bug similar t

[jira] Commented: (PIG-1303) unable to set outgoing format for org.apache.pig.piggybank.evaluation.util.apachelogparser.DateExtractor

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860103#action_12860103 ] hc busy commented on PIG-1303: -- Okay, so, here's a thought: I'm kind of stuck writing the initi

Re: Consider cleaning up backend code

2010-04-22 Thread Jianyong Dai
+1 for removing. This interface does not bring us any value when we decide to move closer to hadoop. Writing a backend is almost writing half of Pig. I don't think this interface is attractive to most developers. Instead, I +1 for Milind's idea to make intermediate artifacts available, or provi

[jira] Updated: (PIG-1391) pig unit tests leave behind files in temp directory because MiniCluster files don't get deleted

2010-04-22 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated PIG-1391: --- Attachment: minicluster.patch This is a path for review before I go ahead and change around 70+ test fi

[jira] Assigned: (PIG-1391) pig unit tests leave behind files in temp directory because MiniCluster files don't get deleted

2010-04-22 Thread Thejas M Nair (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair reassigned PIG-1391: -- Assignee: Thejas M Nair > pig unit tests leave behind files in temp directory because MiniCluster

[jira] Created: (PIG-1391) pig unit tests leave behind files in temp directory because MiniCluster files don't get deleted

2010-04-22 Thread Thejas M Nair (JIRA)
pig unit tests leave behind files in temp directory because MiniCluster files don't get deleted --- Key: PIG-1391 URL: https://issues.apache.org/jira/browse/PIG-1391

Re: Consider cleaning up backend code

2010-04-22 Thread Alan Gates
A couple of years ago we had this concept that Pig as is should be able to run on other backends (like say Dryad if it were open source). So we built this whole backend interface and (mostly) kept Hadoop specific objects out of the front end. Recently we have modified that stand and said t

Re: Consider cleaning up backend code

2010-04-22 Thread Arun C Murthy
On Apr 22, 2010, at 4:38 PM, Richard Ding wrote: Yes. The abstraction layer I was referring to is src/org/apache/pig/backend/executionengine and src/org/apache/pig/backend/datastorage. Thanks for the clarification. +1 Arun Thanks, -Richard -Original Message- From: Arun C Murthy

RE: Consider cleaning up backend code

2010-04-22 Thread Richard Ding
Yes. The abstraction layer I was referring to is src/org/apache/pig/backend/executionengine and src/org/apache/pig/backend/datastorage. Thanks, -Richard -Original Message- From: Arun C Murthy [mailto:a...@yahoo-inc.com] Sent: Thursday, April 22, 2010 4:14 PM To: pig-dev@hadoop.apache.o

[jira] Updated: (PIG-1386) UDF to extend functionalities of MaxTupleBy1stField

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1386: - Status: Patch Available (was: Open) resubmitting patch for the build system to check. > UDF to extend functionali

[jira] Updated: (PIG-1386) UDF to extend functionalities of MaxTupleBy1stField

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1386: - Status: Open (was: Patch Available) > UDF to extend functionalities of MaxTupleBy1stField > --

[jira] Updated: (PIG-1385) UDF to create tuples and bags

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1385: - Attachment: PIG-1385-trunk.patch changed so that the unit test builds and runs. > UDF to create tuples and bags >

[jira] Updated: (PIG-1385) UDF to create tuples and bags

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1385: - Status: Patch Available (was: Open) resubmitting patch for the build system to check. > UDF to create tuples and

[jira] Updated: (PIG-1385) UDF to create tuples and bags

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1385: - Status: Open (was: Patch Available) > UDF to create tuples and bags > - > >

[jira] Updated: (PIG-1385) UDF to create tuples and bags

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1385: - Attachment: (was: PIG-1385-trunk.patch) > UDF to create tuples and bags > - > >

[jira] Updated: (PIG-1386) UDF to extend functionalities of MaxTupleBy1stField

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1386: - Attachment: PIG-1386-trunk.patch checked to be sure the unittest builds and runs. > UDF to extend functionalities

[jira] Updated: (PIG-1386) UDF to extend functionalities of MaxTupleBy1stField

2010-04-22 Thread hc busy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hc busy updated PIG-1386: - Attachment: (was: PIG-1386-trunk.patch) > UDF to extend functionalities of MaxTupleBy1stField > ---

Re: Consider cleaning up backend code

2010-04-22 Thread Arun C Murthy
I read it as getting rid of concepts parallel to hadoop in src/org/ apache/pig/backend/hadoop/datastorage. Is that true? thanks, Arun On Apr 22, 2010, at 1:34 PM, Dmitriy Ryaboy wrote: I kind of dig the concept of being able to plug in a different backend, though I definitely thing we shou

[jira] Commented: (PIG-1387) Syntactical Sugar for PIG-1385

2010-04-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860063#action_12860063 ] Alan Gates commented on PIG-1387: - Adding () for tuples, {} for bags, and [] for maps seems r

[jira] Updated: (PIG-1384) Adding contrib javadoc to main Pig javadoc

2010-04-22 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1384: Status: Resolved (was: Patch Available) Hadoop Flags: [Reviewed] Resolution: Fixed Patch com

[jira] Updated: (PIG-1342) [Zebra] Avoid making unnecessary name node calls for writes in Zebra

2010-04-22 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Zhou updated PIG-1342: -- Status: Resolved (was: Patch Available) Resolution: Fixed Committed to the trunk. > [Zebra] Avoid makin

[jira] Commented: (PIG-1342) [Zebra] Avoid making unnecessary name node calls for writes in Zebra

2010-04-22 Thread Yan Zhou (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860042#action_12860042 ] Yan Zhou commented on PIG-1342: --- +1 > [Zebra] Avoid making unnecessary name node calls for wri

[jira] Commented: (PIG-1342) [Zebra] Avoid making unnecessary name node calls for writes in Zebra

2010-04-22 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860035#action_12860035 ] Hadoop QA commented on PIG-1342: +1 overall. Here are the results of testing the latest atta

Re: Consider cleaning up backend code

2010-04-22 Thread Milind A Bhandarkar
I think it is a great idea to be able to plug-in a different back-ends. But the way to do that, IMHO, is to make the intermediate artifacts public (akin to making byte-code specs public). That way, independent projects can spring up that take the translated pig script, and provide a new interpret

[jira] Commented: (PIG-1384) Adding contrib javadoc to main Pig javadoc

2010-04-22 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12860012#action_12860012 ] Olga Natkovich commented on PIG-1384: - +1 on the latest patch > Adding contrib javadoc t

[jira] Updated: (PIG-1384) Adding contrib javadoc to main Pig javadoc

2010-04-22 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-1384: Attachment: PIG-1384.apirel.patch One additional patch to change the api doc site link from absolute path to

Re: Consider cleaning up backend code

2010-04-22 Thread Dmitriy Ryaboy
I kind of dig the concept of being able to plug in a different backend, though I definitely thing we should get rid of the dead localmode code. Can you give an example of how this will simplify the codebase? Is it more than just GenericClass foo = new SpecificClass(), and the associated extra files

Re: Consider cleaning up backend code

2010-04-22 Thread Arun C Murthy
+1 Arun On Apr 22, 2010, at 11:35 AM, Richard Ding wrote: Pig has an abstraction layer (interfaces and abstract classes) to support multiple execution engines. After PIG-1053, Hadoop is the only execution engine supported by Pig. I wonder if we should remove this layer of code, and make Hadoop

[jira] Commented: (PIG-1320) Pig/Zebra 0.7.0 Docs

2010-04-22 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859979#action_12859979 ] Olga Natkovich commented on PIG-1320: - Latest patch committed to both trunk and 0.7.0 bra

[jira] Updated: (PIG-1390) Provide a target to generate eclipse-related classpath and files

2010-04-22 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-1390: Fix Version/s: (was: 0.7.0) we already branched for 0.7.0. New functioanlity should go to 0.8.0

[jira] Resolved: (PIG-803) Pig Latin Reference Manual - discussion of Pig streaming is incomplete

2010-04-22 Thread Olga Natkovich (JIRA)
[ https://issues.apache.org/jira/browse/PIG-803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich resolved PIG-803. Resolution: Fixed Resolved with the latest path in PIG-1320 > Pig Latin Reference Manual - discussion

Re: Consider cleaning up backend code

2010-04-22 Thread Milind A Bhandarkar
+1. - milind On 4/22/10 11:35 AM, "Richard Ding" wrote: > Pig has an abstraction layer (interfaces and abstract classes) to > support multiple execution engines. After PIG-1053, Hadoop is the only > execution engine supported by Pig. I wonder if we should remove this > layer of code, and make

[jira] Commented: (PIG-1384) Adding contrib javadoc to main Pig javadoc

2010-04-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859965#action_12859965 ] Alan Gates commented on PIG-1384: - +1 Changes look good. > Adding contrib javadoc to main Pi

Consider cleaning up backend code

2010-04-22 Thread Richard Ding
Pig has an abstraction layer (interfaces and abstract classes) to support multiple execution engines. After PIG-1053, Hadoop is the only execution engine supported by Pig. I wonder if we should remove this layer of code, and make Hadoop THE execution engine for Pig. This will simplify a lot the bac

[jira] Updated: (PIG-1390) Provide a target to generate eclipse-related classpath and files

2010-04-22 Thread V.V.Chaitanya Krishna (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] V.V.Chaitanya Krishna updated PIG-1390: --- Attachment: PIG-eclipse_support.patch Attaching a patch with the new ant target "eclips

[jira] Created: (PIG-1390) Provide a target to generate eclipse-related classpath and files

2010-04-22 Thread V.V.Chaitanya Krishna (JIRA)
Provide a target to generate eclipse-related classpath and files Key: PIG-1390 URL: https://issues.apache.org/jira/browse/PIG-1390 Project: Pig Issue Type: Improvement

[jira] Created: (PIG-1389) Implement Pig counter to track number of rows for each input files

2010-04-22 Thread Richard Ding (JIRA)
Implement Pig counter to track number of rows for each input files --- Key: PIG-1389 URL: https://issues.apache.org/jira/browse/PIG-1389 Project: Pig Issue Type: Improvement

[jira] Updated: (PIG-1320) Pig/Zebra 0.7.0 Docs

2010-04-22 Thread Corinne Chandel (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Corinne Chandel updated PIG-1320: - Attachment: pig-zebra-070-GA.patch Apply patch to TRUNK and 0.7.0 branch (if created). Note: No ne

Re: Steps to get pig source code in Eclipse environment

2010-04-22 Thread Ashutosh Chauhan
Chaitanya, Awesome! I do these steps manually (atleast) 10 times a week which your patch will automate. Hugely useful. The way the patch process works for Apache projects is that you create a jira issue on https://issues.apache.org/jira/secure/CreateIssue!default.jspa and attach your patch on the

Re: Steps to get pig source code in Eclipse environment

2010-04-22 Thread chaitanya krishna
Attached here is the patch with the "eclipse-files" target. On Thu, Apr 22, 2010 at 1:54 PM, Dmitriy Ryaboy wrote: > At some point you need to run "ant" so that it pulls down various > dependencies and autogenerate some code -- this is probably the step that > was missing when you used the subcl

[jira] Created: (PIG-1388) hadoop18 jar should be removed from Pig trunk

2010-04-22 Thread Alan Gates (JIRA)
hadoop18 jar should be removed from Pig trunk - Key: PIG-1388 URL: https://issues.apache.org/jira/browse/PIG-1388 Project: Pig Issue Type: Bug Reporter: Alan Gates Priority:

Re: Shouldn't hadoop18.jar be removed from lib of trunk?

2010-04-22 Thread Alan Gates
It should be removed. I filed https://issues.apache.org/jira/browse/PIG-1388 so we'll remember to remove it in 0.8. Alan. On Apr 21, 2010, at 10:24 PM, chaitanya krishna wrote: Hi, Since pig-trunk now supports hadoop-0.20 and as it already has hadoop20.jar, shouldn't the hadoop18.jar be re

[jira] Updated: (PIG-1342) [Zebra] Avoid making unnecessary name node calls for writes in Zebra

2010-04-22 Thread Chao Wang (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Wang updated PIG-1342: --- Status: Patch Available (was: Open) >From the test result log, it looks like the testcase TestFinish failed.

[jira] Updated: (PIG-1342) [Zebra] Avoid making unnecessary name node calls for writes in Zebra

2010-04-22 Thread Chao Wang (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Wang updated PIG-1342: --- Status: Open (was: Patch Available) > [Zebra] Avoid making unnecessary name node calls for writes in Zebra > -

Re: Steps to get pig source code in Eclipse environment

2010-04-22 Thread Dmitriy Ryaboy
At some point you need to run "ant" so that it pulls down various dependencies and autogenerate some code -- this is probably the step that was missing when you used the subclipse plugin. I know people have used subclipse successfully before (me, I'm more of a command-line type). An ant target tha