Pig 0.10 on YARN

2013-03-29 Thread Konstantin Boudnik
Guys, I am trying to run TestPigTest against a fanctional Hadoop 2.0 YARN cluster - I can run MR jobs using Hive or plain MR examples. However, it is going somewhat unsuccessful. While using pig far jar file - e.g. with everything repacked inside of it - I seeminly can run TestPigTest against a

Re: [VOTE] Release Pig 0.11.1 (candidate 0)

2013-03-29 Thread Bill Graham
With 3 binding +1s (Daniel, Julien, BillG) this vote passes. I'll start the release process. On Wed, Mar 27, 2013 at 6:22 PM, Bill Graham billgra...@gmail.com wrote: +1 On Mon, Mar 25, 2013 at 3:42 PM, Daniel Dai da...@hortonworks.com wrote: Yes, it is Ok with me. Daniel On Mon, Mar

Re: [VOTE] Release Pig 0.11.1 (candidate 0)

2013-03-29 Thread Prashant Kommireddi
Great, thanks Bill. Sent from my iPhone On Mar 28, 2013, at 11:52 PM, Bill Graham billgra...@gmail.com wrote: With 3 binding +1s (Daniel, Julien, BillG) this vote passes. I'll start the release process. On Wed, Mar 27, 2013 at 6:22 PM, Bill Graham billgra...@gmail.com wrote: +1 On

[jira] [Created] (PIG-3263) Resolving UDFs fails while using pig embedded code in Python when job parallelism is used.

2013-03-29 Thread Jakub Glapa (JIRA)
Jakub Glapa created PIG-3263: Summary: Resolving UDFs fails while using pig embedded code in Python when job parallelism is used. Key: PIG-3263 URL: https://issues.apache.org/jira/browse/PIG-3263

Re: Pig 0.10 on YARN

2013-03-29 Thread Dmitriy Ryaboy
Can you run `pig -secretDebugCmd` and look at your paths? I suspect there's something going on there, like not having the right configs or hadoop jars. On Thu, Mar 28, 2013 at 11:32 PM, Konstantin Boudnik c...@apache.org wrote: Guys, I am trying to run TestPigTest against a fanctional Hadoop

Re: GSoC 2013

2013-03-29 Thread Dmitriy Ryaboy
Hi Burakk, The general idea of making graph processing easier is a good one. I'm not sure what exactly you are proposing to do, though. Could you be more detailed about what you are thinking? On Thu, Mar 28, 2013 at 1:28 PM, burakkk burak.isi...@gmail.com wrote: Hi, I might be a little bit

Why does Pig not use default resources from the Configuration object?

2013-03-29 Thread Bhooshan Mogal
Hi Folks, I had implemented the Hadoop FileSystem abstract class for a storage system at work. This implementation uses some config files that are similar in structure to hadoop config files. They have a *-default.xml and a *-site.xml for users to override default properties. In the class that

[jira] [Commented] (PIG-3262) Pig contrib 0.11 doesn't compile on certain rpm systems

2013-03-29 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617494#comment-13617494 ] Mark Grover commented on PIG-3262: -- [~cheolsoo] It's ok to commit to just trunk if the vote

[jira] [Commented] (PIG-3263) Resolving UDFs fails while using pig embedded code in Python when using parallel execution

2013-03-29 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617496#comment-13617496 ] Cheolsoo Park commented on PIG-3263: Thank you Jakub for reporting the issue. I am

Re: Why does Pig not use default resources from the Configuration object?

2013-03-29 Thread Prashant Kommireddi
Hi Bhooshan, There is a patch that addresses what you need, and is part of 0.12 (unreleased). Take a look and see if you can apply the patch to the version you are using. https://issues.apache.org/jira/browse/PIG-3135. With this patch, the following property will allow you to override the

Re: Pig 0.10 on YARN

2013-03-29 Thread Konstantin Boudnik
There's no problem with Pig itself. If I use Pig CLI I can run some PigLatin scripts and they go all the way to the completion. The problem arises while running TestPigTest _programmatically_ e.g. using JUnit runner. Is there any secret debug options in this case? I've looked through the code but

Re: Pig 0.10 on YARN

2013-03-29 Thread Dmitriy Ryaboy
I missed the part where you are running it from JUnit. Still suspect a classpath issue :). Can you dump your classpath as seen by JUnit, and what happens when you run -secretDebugCmd and see if something is missing? -D On Fri, Mar 29, 2013 at 10:38 AM, Konstantin Boudnik c...@apache.org wrote:

[jira] [Commented] (PIG-3186) tar/deb/pkg ant targets should depend on piggybank

2013-03-29 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617680#comment-13617680 ] Alan Gates commented on PIG-3186: - Is this ready for review? If so please click Submit

[jira] [Commented] (PIG-3262) Pig contrib 0.11 doesn't compile on certain rpm systems

2013-03-29 Thread Roman Shaposhnik (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617712#comment-13617712 ] Roman Shaposhnik commented on PIG-3262: --- Cheolsoo, it would be *extremely* appreciated

[jira] [Created] (PIG-3264) mvn signanddeploy target broken for pigunit and pigsmoke

2013-03-29 Thread Bill Graham (JIRA)
Bill Graham created PIG-3264: Summary: mvn signanddeploy target broken for pigunit and pigsmoke Key: PIG-3264 URL: https://issues.apache.org/jira/browse/PIG-3264 Project: Pig Issue Type: Bug

[jira] [Commented] (PIG-3262) Pig contrib 0.11 doesn't compile on certain rpm systems

2013-03-29 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617752#comment-13617752 ] Mark Grover commented on PIG-3262: -- Actually, that would be really nice. In general, Apache

[jira] [Updated] (PIG-3264) mvn signanddeploy target broken for pigunit and pigsmoke

2013-03-29 Thread Bill Graham (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Graham updated PIG-3264: - Attachment: PIG_3264.1.patch PIG_3264_branch11.1.patch Attaching trunk and branch 11

[jira] [Updated] (PIG-3264) mvn signanddeploy target broken for pigunit and pigsmoke

2013-03-29 Thread Bill Graham (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Graham updated PIG-3264: - Status: Patch Available (was: Open) mvn signanddeploy target broken for pigunit and pigsmoke

[jira] [Updated] (PIG-3264) mvn signanddeploy target broken for pigunit, pigsmoke and piggybank

2013-03-29 Thread Bill Graham (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Graham updated PIG-3264: - Summary: mvn signanddeploy target broken for pigunit, pigsmoke and piggybank (was: mvn signanddeploy

[jira] [Updated] (PIG-3264) mvn signanddeploy target broken for pigunit, pigsmoke and piggybank

2013-03-29 Thread Bill Graham (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Graham updated PIG-3264: - Resolution: Fixed Status: Resolved (was: Patch Available) mvn signanddeploy target broken

Re: Pig 0.10 on YARN

2013-03-29 Thread Konstantin Boudnik
I certainly agree with you. I have managed to narrow it down to the difference between pig-0.10.0-withouthadoop.jar and maven pig-0.10.0.jar. Basically, I can run: java -cp \ /etc/hadoop/conf:\ /etc/pig/conf:\ /root/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:\

Re: Pig 0.10 on YARN

2013-03-29 Thread Johnny Zhang
Hi, cos: I met the same problem before. Can you try below in your Bigtop pig maven execution .pom ? It will actually resolve other problem as well (for example, you may see Pig cannot get kerberos principle in secure YARN cluster) dependencies dependency

Re: Pig 0.10 on YARN

2013-03-29 Thread Konstantin Boudnik
Hey Johnny. Yup, I've been using pretty much this without -hs artifact. However, adding hadoop-mapreduce-client-hs didn't help ;( Cos On Fri, Mar 29, 2013 at 03:25PM, Johnny Zhang wrote: Hi, cos: I met the same problem before. Can you try below in your Bigtop pig maven execution .pom ? It

[jira] [Commented] (PIG-3262) Pig contrib 0.11 doesn't compile on certain rpm systems

2013-03-29 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617834#comment-13617834 ] Cheolsoo Park commented on PIG-3262: [~rvs], the vote is passed now.

[jira] [Commented] (PIG-3262) Pig contrib 0.11 doesn't compile on certain rpm systems

2013-03-29 Thread Mark Grover (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13617843#comment-13617843 ] Mark Grover commented on PIG-3262: -- OK, let me do that. Pig contrib 0.11

Apache Pig 0.11.1 release candidate

2013-03-29 Thread Mark Grover
Hi all, I am a contributor to Apache Bigtop http://bigtop.apache.org and have a question for you. Bigtop is a TLP responsible for performing packaging and interoperability testing of various projects in the Hadoop ecosystem, including Apache Pig. We are planning to include Pig 0.11 in our soon to

[jira] Subscription: PIG patch available

2013-03-29 Thread jira
Issue Subscription Filter: PIG patch available (36 issues) Subscriber: pigdaily Key Summary PIG-3261User set PIG_CLASSPATH entries must be prepended to the CLASSPATH, not appended https://issues.apache.org/jira/browse/PIG-3261 PIG-3257Add unique identifier