date:20110720

[jira] [Commented] (PIG-2178) Filtering a source and then merging the filtered rows only generates data from one half of the filtering

2011-07-20 Thread Thejas M Nair (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068806#comment-13068806
 ] 

Thejas M Nair commented on PIG-2178:


No problem, thanks for reporting issues as when you see them, and helping 
improve pig!

> Filtering a source and then merging the filtered rows only generates data 
> from one half of the filtering
> 
>
> Key: PIG-2178
> URL: https://issues.apache.org/jira/browse/PIG-2178
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.1
>Reporter: Derek Wollenstein
> Fix For: 0.8.1
>
>
> Pig is generating a plan that eliminates half of input data when using FILTER 
> BY
> To better illustrate, I created a small test case.
> 1. Create a file in HDFS called "/testinput"
>The contents of the file should be:
> "1\ta\taline\n1\tb\tbline"
> 2. Run the following pig script:
> ORIG = LOAD '/testinput' USING PigStorage() AS (parent_id: chararray, 
> child_id:chararray, value:chararray);
> -- Split into two inputs based on the value of child_id
> A = FILTER ORIG BY child_id =='a';
> B = FILTER ORIG BY child_id =='b';
> -- Project out the column which chooses the correct data set
> APROJ = FOREACH A GENERATE parent_id, value;
> BPROJ = FOREACH B GENERATE parent_id, value;
> -- Merge both datasets by parent id
> ABMERGE = JOIN APROJ by parent_id FULL OUTER, BPROJ by parent_id;
> -- Project the result
> ABPROJ = FOREACH ABMERGE GENERATE APROJ::parent_id AS parent_id, 
> APROJ::value,BPROJ::value;
> DUMP ABPROJ;
> 3. The resulting tuple will be
> (1,aline,aline)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (PIG-2178) Filtering a source and then merging the filtered rows only generates data from one half of the filtering

2011-07-20 Thread Derek Wollenstein (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Derek Wollenstein resolved PIG-2178.


   Resolution: Not A Problem
Fix Version/s: 0.8.1

I took a look, and you were right.  I was using 0.8.0, so this bug is 
incorrect.  And I'll take your word for the fact that this isn't an issue in 
0.8.1

Locally I was able to correct the problem by loading the file twice (ORIGA and 
ORIGB).

I just wanted to make sure this was noted for future fixes.  If 0.8.1 takes 
care of that then I'll go and upgrade on my end.  Sorry for the trouble.

> Filtering a source and then merging the filtered rows only generates data 
> from one half of the filtering
> 
>
> Key: PIG-2178
> URL: https://issues.apache.org/jira/browse/PIG-2178
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.1
>Reporter: Derek Wollenstein
> Fix For: 0.8.1
>
>
> Pig is generating a plan that eliminates half of input data when using FILTER 
> BY
> To better illustrate, I created a small test case.
> 1. Create a file in HDFS called "/testinput"
>The contents of the file should be:
> "1\ta\taline\n1\tb\tbline"
> 2. Run the following pig script:
> ORIG = LOAD '/testinput' USING PigStorage() AS (parent_id: chararray, 
> child_id:chararray, value:chararray);
> -- Split into two inputs based on the value of child_id
> A = FILTER ORIG BY child_id =='a';
> B = FILTER ORIG BY child_id =='b';
> -- Project out the column which chooses the correct data set
> APROJ = FOREACH A GENERATE parent_id, value;
> BPROJ = FOREACH B GENERATE parent_id, value;
> -- Merge both datasets by parent id
> ABMERGE = JOIN APROJ by parent_id FULL OUTER, BPROJ by parent_id;
> -- Project the result
> ABPROJ = FOREACH ABMERGE GENERATE APROJ::parent_id AS parent_id, 
> APROJ::value,BPROJ::value;
> DUMP ABPROJ;
> 3. The resulting tuple will be
> (1,aline,aline)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk-commit #868

2011-07-20 Thread Apache Jenkins Server

See 

Changes:

[thejas] fix import in testcase added as part of - PIG-2146: 
POStore.getSchema() returns null because of which PigOutputCommitter
 is not storing schema while cleanup

--
[...truncated 39692 lines...]
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] at org.apache.hadoop.ipc.Client.call(Client.java:740)
[junit] at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:3264)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3188)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$LeaseChecker.close(DFSClient.java:1043)
[junit] at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:237)
[junit] at 
org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:269)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsClusters(MiniGenericCluster.java:83)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsAndMrClusters(MiniGenericCluster.java:77)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutDown(MiniGenericCluster.java:68)
[junit] at 
org.apache.pig.test.TestStore.oneTimeTearDown(TestStore.java:127)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit] at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit] at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit] at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
[junit] at 
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] 11/07/21 02:23:43 WARN hdfs.StateChange: DIR* 
NameSystem.completeFile: failed to complete 
/tmp/TestStore-output-1881025561287049326.txt_cleanupOnFailure_succeeded 
because dir.getFileBlocks() is null  and pendingFile is null
[junit] 11/07/21 02:23:43 INFO ipc.Server: IPC Server handler 7 on 37270, 
call 
complete(/tmp/TestStore-output-1881025561287049326.txt_cleanupOnFailure_succeeded,
 DFSClient_492464520) from 127.0.0.1:34834: error: java.io.IOException: Could 
not complete write to file 
/tmp/TestStore-output-1881025561287049326.txt_cleanupOnFailure_succeeded by 
DFSClient_492464520
[junit] java.io.IOException: Could not complete write to file 
/tmp/TestStore-output-1881025561287049326.txt_cleanupOnFailure_succeeded by 
DFSClient_492464520
[junit] at 
org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:449)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at org.apache.hadoop.ipc.RPC$Ser

Build failed in Jenkins: Pig-trunk-commit #867

2011-07-20 Thread Apache Jenkins Server

See 

Changes:

[daijy] PIG-2125: Make Pig work with hadoop .NEXT (PIG-2125-5.patch)

--
[...truncated 39623 lines...]
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] org.apache.hadoop.ipc.RemoteException: java.io.IOException: Could 
not complete write to file 
/tmp/TestStore-output--809804008553611944.txt_cleanupOnFailure_succeeded2 by 
DFSClient_-700297148
[junit] at 
org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:449)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] at org.apache.hadoop.ipc.Client.call(Client.java:740)
[junit] at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:3264)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3188)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$LeaseChecker.close(DFSClient.java:1043)
[junit] at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:237)
[junit] at 
org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:269)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsClusters(MiniGenericCluster.java:83)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsAndMrClusters(MiniGenericCluster.java:77)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutDown(MiniGenericCluster.java:68)
[junit] at 
org.apache.pig.test.TestStore.oneTimeTearDown(TestStore.java:127)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit] at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit] at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit] at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
[junit] at 
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] 11/07/21 01:29:12 WARN hdfs.StateChange: DIR* 
NameSystem.completeFile: failed to complete 
/tmp/TestStore-output-4697537299435714937.txt_cleanupOnFailure_succeeded 
because dir.getFileBlocks() is null  and pendingFile is null
[junit

[jira] [Assigned] (PIG-2183) Pig 0.8.1 not working with Hadoop 0.20.203.0

2011-07-20 Thread Daniel Dai (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai reassigned PIG-2183:
---

Assignee: Daniel Dai

> Pig 0.8.1 not working with Hadoop 0.20.203.0
> 
>
> Key: PIG-2183
> URL: https://issues.apache.org/jira/browse/PIG-2183
> Project: Pig
>  Issue Type: Bug
>  Components: grunt, site
>Affects Versions: 0.8.1
> Environment: Gentoo Linux Kernel: 2.6.38-gentoo-r6
> java version "1.6.0_26"
> Ant version 1.8.1
>Reporter: John Lium
>Assignee: Daniel Dai
>
> When running pig, I get the following error.
> Error before Pig is launched
> 
> ERROR 2999: Unexpected internal error. Failed to create DataStorage
> java.lang.RuntimeException: Failed to create DataStorage
>   at 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
>   at 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.(HDataStorage.java:58)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:183)
>   at org.apache.pig.PigServer.(PigServer.java:226)
>   at org.apache.pig.PigServer.(PigServer.java:215)
>   at org.apache.pig.tools.grunt.Grunt.(Grunt.java:55)
>   at org.apache.pig.Main.run(Main.java:452)
>   at org.apache.pig.Main.main(Main.java:107)
> Caused by: java.io.IOException: Call to rasputin/192.168.1.3:9000 failed on 
> local exception: java.io.EOFException
>   at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
>   at org.apache.hadoop.ipc.Client.call(Client.java:743)
>   at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>   at $Proxy0.getProtocolVersion(Unknown Source)
>   at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>   at 
> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
>   at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:207)
>   at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:170)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
>   at 
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
>   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
>   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
>   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>   at 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
>   ... 9 more
> Caused by: java.io.EOFException
>   at java.io.DataInputStream.readInt(DataInputStream.java:375)
>   at 
> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
>   at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
> 
> My env vars are defined in bin/pig as the following
> export JAVA_HOME="/etc/java-config-2/current-system-vm"
> export 
> PIG_CLASSPATH="/var/hadoop/pig/pig-withouthadoop.jar:$HADOOP_HOME/hadoop-core-0.20.203.0.jar:$HADOOP_HOME/lib:$HADOOP_CONF_DIR"

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2125) Make Pig work with hadoop .NEXT

2011-07-20 Thread Daniel Dai (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068744#comment-13068744
 ] 

Daniel Dai commented on PIG-2125:
-

PIG-2125-5.patch committed to both trunk and 0.9 branch.

> Make Pig work with hadoop .NEXT
> ---
>
> Key: PIG-2125
> URL: https://issues.apache.org/jira/browse/PIG-2125
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Affects Versions: 0.10
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2125-1.patch, PIG-2125-2.patch, PIG-2125-3.patch, 
> PIG-2125-4.patch, PIG-2125-5.patch
>
>
> We need to make Pig work with hadoop .NEXT, the svn branch currently is: 
> https://svn.apache.org/repos/asf/hadoop/common/branches/MR-279

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2179) tests in TestLoad are failing

2011-07-20 Thread Daniel Dai (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068741#comment-13068741
 ] 

Daniel Dai commented on PIG-2179:
-

Patch looks good. Add the comments in findMetaFile that it deals with comma 
separated filenames before commit.

> tests in TestLoad are failing 
> --
>
> Key: PIG-2179
> URL: https://issues.apache.org/jira/browse/PIG-2179
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.10
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.10
>
> Attachments: PIG-2179.1.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2125) Make Pig work with hadoop .NEXT

2011-07-20 Thread Richard Ding (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068739#comment-13068739
 ] 

Richard Ding commented on PIG-2125:
---

+1

> Make Pig work with hadoop .NEXT
> ---
>
> Key: PIG-2125
> URL: https://issues.apache.org/jira/browse/PIG-2125
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Affects Versions: 0.10
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2125-1.patch, PIG-2125-2.patch, PIG-2125-3.patch, 
> PIG-2125-4.patch, PIG-2125-5.patch
>
>
> We need to make Pig work with hadoop .NEXT, the svn branch currently is: 
> https://svn.apache.org/repos/asf/hadoop/common/branches/MR-279

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2125) Make Pig work with hadoop .NEXT

2011-07-20 Thread Daniel Dai (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-2125:


Attachment: PIG-2125-5.patch

There are some piggybank and zebra test failures, attach PIG-2125-5.patch for 
the fix.

> Make Pig work with hadoop .NEXT
> ---
>
> Key: PIG-2125
> URL: https://issues.apache.org/jira/browse/PIG-2125
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Affects Versions: 0.10
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 0.10
>
> Attachments: PIG-2125-1.patch, PIG-2125-2.patch, PIG-2125-3.patch, 
> PIG-2125-4.patch, PIG-2125-5.patch
>
>
> We need to make Pig work with hadoop .NEXT, the svn branch currently is: 
> https://svn.apache.org/repos/asf/hadoop/common/branches/MR-279

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk-commit #866

2011-07-20 Thread Apache Jenkins Server

See 

Changes:

[thejas] PIG-2146: POStore.getSchema() returns null because of which 
PigOutputCommitter
 is not storing schema while cleanup

--
[...truncated 39831 lines...]
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] org.apache.hadoop.ipc.RemoteException: java.io.IOException: Could 
not complete write to file 
/tmp/TestStore-output--3939521172535858951.txt_cleanupOnFailure_succeeded2 by 
DFSClient_-634647566
[junit] at 
org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:449)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] at org.apache.hadoop.ipc.Client.call(Client.java:740)
[junit] at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:3264)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3188)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$LeaseChecker.close(DFSClient.java:1043)
[junit] at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:237)
[junit] at 
org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:269)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsClusters(MiniGenericCluster.java:83)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsAndMrClusters(MiniGenericCluster.java:77)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutDown(MiniGenericCluster.java:68)
[junit] at 
org.apache.pig.test.TestStore.oneTimeTearDown(TestStore.java:127)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit] at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit] at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit] at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
[junit] at 
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] 11/07/20 23:28:36 WARN hdfs.StateChange: DIR* 
NameSystem.completeFile: failed to complete 
/tmp/TestStore-output-5551418102219430049.txt_cleanupOnFailure_succeeded1 
because dir.getFileBlocks() is null  and pendingFile is null
[junit] 11/07/20 23:28:36 INFO ipc.Server: IPC Server handler 4 on 40868, 
call 
complete(/tmp/TestStore-output-5551418102219430049.txt_cleanupOnFailure_succeeded1,
 DFSClient_-634647566) from

[jira] [Updated] (PIG-2146) POStore.getSchema() returns null because of which PigOutputCommitter is not storing schema while cleanup

2011-07-20 Thread Thejas M Nair (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2146:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> POStore.getSchema() returns null because of which PigOutputCommitter is not 
> storing schema while cleanup
> 
>
> Key: PIG-2146
> URL: https://issues.apache.org/jira/browse/PIG-2146
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Vivek Padmanabhan
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2146.1.patch
>
>
> The below is my script;
> {code}
> register piggybank.jar;
> a = load 'myinput' using PigStorage(',') as 
> (f1:chararray,f2:chararray,f3:chararray);
> b = distinct a;
> c = limit b 2;
> store c into 'pss001' using 
> org.apache.pig.piggybank.storage.PigStorageSchema();
> {code}
> Input
> ---
> a,1,aa
> b,2,bb
> c,3,cc
> For this script , PigStorageSchema is not generating  .pig_headers and 
> .pig_schema files. While debugging I could see that storeSchema(..) method 
> itself is not invoked.The schema object for the store is returned as  null 
> (POStore.getSchema()) because of which PigOutputCommitter is not invoking the 
> storSchema.
> The same schema object is valid when I run it in local mode. This issue is 
> happening for Pig 0.9 also.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2146) POStore.getSchema() returns null because of which PigOutputCommitter is not storing schema while cleanup

2011-07-20 Thread Thejas M Nair (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068692#comment-13068692
 ] 

Thejas M Nair commented on PIG-2146:


Patch committed to 0.9 branch and trunk.


> POStore.getSchema() returns null because of which PigOutputCommitter is not 
> storing schema while cleanup
> 
>
> Key: PIG-2146
> URL: https://issues.apache.org/jira/browse/PIG-2146
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Vivek Padmanabhan
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2146.1.patch
>
>
> The below is my script;
> {code}
> register piggybank.jar;
> a = load 'myinput' using PigStorage(',') as 
> (f1:chararray,f2:chararray,f3:chararray);
> b = distinct a;
> c = limit b 2;
> store c into 'pss001' using 
> org.apache.pig.piggybank.storage.PigStorageSchema();
> {code}
> Input
> ---
> a,1,aa
> b,2,bb
> c,3,cc
> For this script , PigStorageSchema is not generating  .pig_headers and 
> .pig_schema files. While debugging I could see that storeSchema(..) method 
> itself is not invoked.The schema object for the store is returned as  null 
> (POStore.getSchema()) because of which PigOutputCommitter is not invoking the 
> storSchema.
> The same schema object is valid when I run it in local mode. This issue is 
> happening for Pig 0.9 also.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk #1050

2011-07-20 Thread Apache Jenkins Server

See 

Changes:

[daijy] PIG-2175: Switch Pig wiki to use confluence

[daijy] PIG-2027: NPE if Pig don't have permission for log file

--
[...truncated 39312 lines...]
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] org.apache.hadoop.ipc.RemoteException: java.io.IOException: Could 
not complete write to file 
/tmp/TestStore-output-6378947281920216709.txt_cleanupOnFailure_succeeded2 by 
DFSClient_1722705116
[junit] at 
org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:449)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] at org.apache.hadoop.ipc.Client.call(Client.java:740)
[junit] at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:3264)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3188)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$LeaseChecker.close(DFSClient.java:1043)
[junit] at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:237)
[junit] at 
org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:269)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsClusters(MiniGenericCluster.java:83)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsAndMrClusters(MiniGenericCluster.java:77)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutDown(MiniGenericCluster.java:68)
[junit] at 
org.apache.pig.test.TestStore.oneTimeTearDown(TestStore.java:127)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit] at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit] at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit] at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
[junit] at 
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] 11/07/20 22:33:58 WARN hdfs.StateChange: DIR* 
NameSystem.completeFile: failed to complete 
/tmp/TestStore-output-7340733076336181835.txt_cleanupOnFailure_succeeded 
because dir.getFileBlocks() i

[jira] [Resolved] (PIG-2175) Switch Pig wiki to use confluence

2011-07-20 Thread Daniel Dai (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai resolved PIG-2175.
-

  Resolution: Fixed
Hadoop Flags: [Reviewed]

Patch committed to site. Some links in existing Pig docs may still refer to old 
wiki. We will keep the old wiki site running, but new material should go to new 
wiki.

> Switch Pig wiki to use confluence
> -
>
> Key: PIG-2175
> URL: https://issues.apache.org/jira/browse/PIG-2175
> Project: Pig
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: PIG-2175-1.patch
>
>
> Confluence gives us more functionality and more permission control features. 
> We plan to migrate our wiki to confluence. I migrated part of our wiki to 
> https://cwiki.apache.org/confluence/display/PIG. I also put a link to the old 
> wiki on that site. Attached patch change links on Pig main site.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2183) Pig 0.8.1 not working with Hadoop 0.20.203.0

2011-07-20 Thread John Lium (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068623#comment-13068623
 ] 

John Lium commented on PIG-2183:


Google waited till after I posted this issue to show me this.
https://issues.apache.org/jira/browse/PIG-2148

> Pig 0.8.1 not working with Hadoop 0.20.203.0
> 
>
> Key: PIG-2183
> URL: https://issues.apache.org/jira/browse/PIG-2183
> Project: Pig
>  Issue Type: Bug
>  Components: grunt, site
>Affects Versions: 0.8.1
> Environment: Gentoo Linux Kernel: 2.6.38-gentoo-r6
> java version "1.6.0_26"
> Ant version 1.8.1
>Reporter: John Lium
>
> When running pig, I get the following error.
> Error before Pig is launched
> 
> ERROR 2999: Unexpected internal error. Failed to create DataStorage
> java.lang.RuntimeException: Failed to create DataStorage
>   at 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
>   at 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.(HDataStorage.java:58)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:183)
>   at org.apache.pig.PigServer.(PigServer.java:226)
>   at org.apache.pig.PigServer.(PigServer.java:215)
>   at org.apache.pig.tools.grunt.Grunt.(Grunt.java:55)
>   at org.apache.pig.Main.run(Main.java:452)
>   at org.apache.pig.Main.main(Main.java:107)
> Caused by: java.io.IOException: Call to rasputin/192.168.1.3:9000 failed on 
> local exception: java.io.EOFException
>   at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
>   at org.apache.hadoop.ipc.Client.call(Client.java:743)
>   at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>   at $Proxy0.getProtocolVersion(Unknown Source)
>   at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>   at 
> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
>   at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:207)
>   at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:170)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
>   at 
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
>   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
>   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
>   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>   at 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
>   ... 9 more
> Caused by: java.io.EOFException
>   at java.io.DataInputStream.readInt(DataInputStream.java:375)
>   at 
> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
>   at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
> 
> My env vars are defined in bin/pig as the following
> export JAVA_HOME="/etc/java-config-2/current-system-vm"
> export 
> PIG_CLASSPATH="/var/hadoop/pig/pig-withouthadoop.jar:$HADOOP_HOME/hadoop-core-0.20.203.0.jar:$HADOOP_HOME/lib:$HADOOP_CONF_DIR"

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2183) Pig 0.8.1 not working with Hadoop 0.20.203.0

2011-07-20 Thread John Lium (JIRA)

Pig 0.8.1 not working with Hadoop 0.20.203.0


 Key: PIG-2183
 URL: https://issues.apache.org/jira/browse/PIG-2183
 Project: Pig
  Issue Type: Bug
  Components: grunt, site
Affects Versions: 0.8.1
 Environment: Gentoo Linux Kernel: 2.6.38-gentoo-r6
java version "1.6.0_26"
Ant version 1.8.1
Reporter: John Lium


When running pig, I get the following error.
Error before Pig is launched

ERROR 2999: Unexpected internal error. Failed to create DataStorage

java.lang.RuntimeException: Failed to create DataStorage
at 
org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
at 
org.apache.pig.backend.hadoop.datastorage.HDataStorage.(HDataStorage.java:58)
at 
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214)
at 
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134)
at org.apache.pig.impl.PigContext.connect(PigContext.java:183)
at org.apache.pig.PigServer.(PigServer.java:226)
at org.apache.pig.PigServer.(PigServer.java:215)
at org.apache.pig.tools.grunt.Grunt.(Grunt.java:55)
at org.apache.pig.Main.run(Main.java:452)
at org.apache.pig.Main.main(Main.java:107)
Caused by: java.io.IOException: Call to rasputin/192.168.1.3:9000 failed on 
local exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
at org.apache.hadoop.ipc.Client.call(Client.java:743)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at $Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
at 
org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:207)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:170)
at 
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
at 
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
at 
org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
... 9 more
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at 
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)


My env vars are defined in bin/pig as the following
export JAVA_HOME="/etc/java-config-2/current-system-vm"
export 
PIG_CLASSPATH="/var/hadoop/pig/pig-withouthadoop.jar:$HADOOP_HOME/hadoop-core-0.20.203.0.jar:$HADOOP_HOME/lib:$HADOOP_CONF_DIR"



--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2183) Pig 0.8.1 not working with Hadoop 0.20.203.0

2011-07-20 Thread John Lium (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-2183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068616#comment-13068616
 ] 

John Lium commented on PIG-2183:


Also, forgot to mention that I built pig with "ant jar-withouthadoop"

> Pig 0.8.1 not working with Hadoop 0.20.203.0
> 
>
> Key: PIG-2183
> URL: https://issues.apache.org/jira/browse/PIG-2183
> Project: Pig
>  Issue Type: Bug
>  Components: grunt, site
>Affects Versions: 0.8.1
> Environment: Gentoo Linux Kernel: 2.6.38-gentoo-r6
> java version "1.6.0_26"
> Ant version 1.8.1
>Reporter: John Lium
>
> When running pig, I get the following error.
> Error before Pig is launched
> 
> ERROR 2999: Unexpected internal error. Failed to create DataStorage
> java.lang.RuntimeException: Failed to create DataStorage
>   at 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
>   at 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.(HDataStorage.java:58)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:214)
>   at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:134)
>   at org.apache.pig.impl.PigContext.connect(PigContext.java:183)
>   at org.apache.pig.PigServer.(PigServer.java:226)
>   at org.apache.pig.PigServer.(PigServer.java:215)
>   at org.apache.pig.tools.grunt.Grunt.(Grunt.java:55)
>   at org.apache.pig.Main.run(Main.java:452)
>   at org.apache.pig.Main.main(Main.java:107)
> Caused by: java.io.IOException: Call to rasputin/192.168.1.3:9000 failed on 
> local exception: java.io.EOFException
>   at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
>   at org.apache.hadoop.ipc.Client.call(Client.java:743)
>   at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>   at $Proxy0.getProtocolVersion(Unknown Source)
>   at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>   at 
> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
>   at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:207)
>   at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:170)
>   at 
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
>   at 
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
>   at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>   at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
>   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
>   at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
>   at 
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
>   ... 9 more
> Caused by: java.io.EOFException
>   at java.io.DataInputStream.readInt(DataInputStream.java:375)
>   at 
> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
>   at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
> 
> My env vars are defined in bin/pig as the following
> export JAVA_HOME="/etc/java-config-2/current-system-vm"
> export 
> PIG_CLASSPATH="/var/hadoop/pig/pig-withouthadoop.jar:$HADOOP_HOME/hadoop-core-0.20.203.0.jar:$HADOOP_HOME/lib:$HADOOP_CONF_DIR"

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk-commit #865

2011-07-20 Thread Apache Jenkins Server

See 

Changes:

[daijy] PIG-2175: Switch Pig wiki to use confluence

--
[...truncated 39819 lines...]
[junit] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] org.apache.hadoop.ipc.RemoteException: java.io.IOException: Could 
not complete write to file 
/tmp/TestStore-output--6173355577045242468.txt_cleanupOnFailure_succeeded2 by 
DFSClient_-402372977
[junit] at 
org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:449)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] at org.apache.hadoop.ipc.Client.call(Client.java:740)
[junit] at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:3264)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3188)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$LeaseChecker.close(DFSClient.java:1043)
[junit] at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:237)
[junit] at 
org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:269)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsClusters(MiniGenericCluster.java:83)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsAndMrClusters(MiniGenericCluster.java:77)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutDown(MiniGenericCluster.java:68)
[junit] at 
org.apache.pig.test.TestStore.oneTimeTearDown(TestStore.java:127)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit] at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit] at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit] at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
[junit] at 
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] 11/07/20 20:28:54 WARN hdfs.StateChange: DIR* 
NameSystem.completeFile: failed to complete 
/tmp/TestStore-output--8318435738590698187.txt_cleanupOnFailure_succeeded 
because dir.getFileBlocks() is null  and pendingFile is null
[junit] 11/07/20 20:28:54 INFO ipc.Server: IPC Server handler 8 on 54827, 
call 
complete(/tmp/TestStore-output--8318435738590698187.txt_cleanupOnFailure_succeeded,
 DFSClient_-402372977) from 12

[jira] [Updated] (PIG-2182) Add more append support to DataByteArray

2011-07-20 Thread Grant Ingersoll (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Grant Ingersoll updated PIG-2182:
-

Attachment: PIG-2182.patch

adds appends and has append return a DataByteArray.

> Add more append support to DataByteArray
> 
>
> Key: PIG-2182
> URL: https://issues.apache.org/jira/browse/PIG-2182
> Project: Pig
>  Issue Type: Improvement
>Reporter: Grant Ingersoll
>Priority: Trivial
> Attachments: PIG-2182.patch
>
>
> I was recently writing a UDF to use a DataByteArray similar to CONCAT and 
> thought it would be convenient if DBA supported more append options similar 
> to Appendable and also that you can string them together if you wish as in 
> dba.append("foo").append("bar").

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2182) Add more append support to DataByteArray

2011-07-20 Thread Grant Ingersoll (JIRA)

Add more append support to DataByteArray


 Key: PIG-2182
 URL: https://issues.apache.org/jira/browse/PIG-2182
 Project: Pig
  Issue Type: Improvement
Reporter: Grant Ingersoll
Priority: Trivial


I was recently writing a UDF to use a DataByteArray similar to CONCAT and 
thought it would be convenient if DBA supported more append options similar to 
Appendable and also that you can string them together if you wish as in 
dba.append("foo").append("bar").

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2175) Switch Pig wiki to use confluence

2011-07-20 Thread Alan Gates (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-2175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068543#comment-13068543
 ] 

Alan Gates commented on PIG-2175:
-

+1

> Switch Pig wiki to use confluence
> -
>
> Key: PIG-2175
> URL: https://issues.apache.org/jira/browse/PIG-2175
> Project: Pig
>  Issue Type: Improvement
>  Components: documentation
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: PIG-2175-1.patch
>
>
> Confluence gives us more functionality and more permission control features. 
> We plan to migrate our wiki to confluence. I migrated part of our wiki to 
> https://cwiki.apache.org/confluence/display/PIG. I also put a link to the old 
> wiki on that site. Attached patch change links on Pig main site.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Pig-trunk-commit #864

2011-07-20 Thread Apache Jenkins Server

See 

Changes:

[daijy] PIG-2027: NPE if Pig don't have permission for log file

--
[...truncated 39819 lines...]
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] org.apache.hadoop.ipc.RemoteException: java.io.IOException: Could 
not complete write to file 
/tmp/TestStore-output--4558572667977626699.txt_cleanupOnFailure_succeeded1 by 
DFSClient_-912609163
[junit] at 
org.apache.hadoop.hdfs.server.namenode.NameNode.complete(NameNode.java:449)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
[junit] at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
[junit] at java.security.AccessController.doPrivileged(Native Method)
[junit] at javax.security.auth.Subject.doAs(Subject.java:396)
[junit] at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
[junit] 
[junit] at org.apache.hadoop.ipc.Client.call(Client.java:740)
[junit] at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
[junit] at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
[junit] at $Proxy0.complete(Unknown Source)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:3264)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3188)
[junit] at 
org.apache.hadoop.hdfs.DFSClient$LeaseChecker.close(DFSClient.java:1043)
[junit] at org.apache.hadoop.hdfs.DFSClient.close(DFSClient.java:237)
[junit] at 
org.apache.hadoop.hdfs.DistributedFileSystem.close(DistributedFileSystem.java:269)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsClusters(MiniGenericCluster.java:83)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutdownMiniDfsAndMrClusters(MiniGenericCluster.java:77)
[junit] at 
org.apache.pig.test.MiniGenericCluster.shutDown(MiniGenericCluster.java:68)
[junit] at 
org.apache.pig.test.TestStore.oneTimeTearDown(TestStore.java:127)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit] at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit] at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit] at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
[junit] at 
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] 11/07/20 18:30:09 WARN hdfs.StateChange: DIR* 
NameSystem.completeFile: failed to complete 
/tmp/TestStore-output-2315760537085322334.txt_cleanupOnFailure_succeeded2 
because dir.getFileBlocks() is null  and pendingFile is null
[junit] 11/07/20 18:30:09 INFO ipc.Server: IPC Server handler 0 on 59045, 
call 
complete(/tmp/TestStore-output-2315760537085322334.txt_cleanupOnFailure_succeeded2,
 DFSClient_-912609163) from 127.0.0.1:52994: error: java.io.IOException: Could 
not complete write to file 
/tmp/TestStore-output-2315760537085322334.txt_cleanupOnFailu

[jira] [Commented] (PIG-2178) Filtering a source and then merging the filtered rows only generates data from one half of the filtering

2011-07-20 Thread Thejas M Nair (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068503#comment-13068503
 ] 

Thejas M Nair commented on PIG-2178:


I get the correct results - (1,aline,bline) with the following -
- pig 0.8.1 released version 
- latest jar from pig 0.8 svn branch
- latest jar from pig 0.9 svn branch

Are you the first release of pig 0.8 (ie not 0.8.1 ?) . 0.8.1 has a bunch of 
bug fixes, it is the stable release of 0.8, you should use that.



> Filtering a source and then merging the filtered rows only generates data 
> from one half of the filtering
> 
>
> Key: PIG-2178
> URL: https://issues.apache.org/jira/browse/PIG-2178
> Project: Pig
>  Issue Type: Bug
>  Components: impl
>Affects Versions: 0.8.1
>Reporter: Derek Wollenstein
>
> Pig is generating a plan that eliminates half of input data when using FILTER 
> BY
> To better illustrate, I created a small test case.
> 1. Create a file in HDFS called "/testinput"
>The contents of the file should be:
> "1\ta\taline\n1\tb\tbline"
> 2. Run the following pig script:
> ORIG = LOAD '/testinput' USING PigStorage() AS (parent_id: chararray, 
> child_id:chararray, value:chararray);
> -- Split into two inputs based on the value of child_id
> A = FILTER ORIG BY child_id =='a';
> B = FILTER ORIG BY child_id =='b';
> -- Project out the column which chooses the correct data set
> APROJ = FOREACH A GENERATE parent_id, value;
> BPROJ = FOREACH B GENERATE parent_id, value;
> -- Merge both datasets by parent id
> ABMERGE = JOIN APROJ by parent_id FULL OUTER, BPROJ by parent_id;
> -- Project the result
> ABPROJ = FOREACH ABMERGE GENERATE APROJ::parent_id AS parent_id, 
> APROJ::value,BPROJ::value;
> DUMP ABPROJ;
> 3. The resulting tuple will be
> (1,aline,aline)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2181) Improvement : for error message when describe misses alias

2011-07-20 Thread Daniel Dai (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-2181:


Labels: newbie  (was: )

> Improvement : for error message when describe misses alias
> --
>
> Key: PIG-2181
> URL: https://issues.apache.org/jira/browse/PIG-2181
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Vivek Padmanabhan
>Priority: Minor
>  Labels: newbie
>
> In Pig 0.9, if I have a describe without an alias, it throws a 
> NullPointerException like below.
> ERROR 2999: Unexpected internal error. null
> java.lang.NullPointerException
> at 
> org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:270)
> at 
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
> at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> at org.apache.pig.Main.run(Main.java:553)
> at org.apache.pig.Main.main(Main.java:108)
> For example;
> describe;
> This message is of no use from a users perspective. Especially when my script 
> becomes large and I have added couple of describe statements. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2027) NPE if Pig don't have permission for log file

2011-07-20 Thread Daniel Dai (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-2027:


  Resolution: Fixed
Hadoop Flags: [Reviewed]
  Status: Resolved  (was: Patch Available)

Patch committed to trunk

> NPE if Pig don't have permission for log file
> -
>
> Key: PIG-2027
> URL: https://issues.apache.org/jira/browse/PIG-2027
> Project: Pig
>  Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Trivial
> Fix For: 0.10
>
> Attachments: PIG-2027-1.patch
>
>
> If specify a log file to Pig, but Pig don't have write permission, if any 
> failure in Pig script, we will get a NPE in addition to Pig script failure:
> 2011-05-02 13:18:36,493 [main] ERROR org.apache.pig.tools.grunt.Grunt - 
> java.lang.NullPointerException
> at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:172)
> at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:79)
> at 
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:131)
> at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:180)
> at 
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:152)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:554)
> at org.apache.pig.Main.main(Main.java:109)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2179) tests in TestLoad are failing

2011-07-20 Thread Thejas M Nair (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2179:
---

Status: Patch Available  (was: Open)

test-patch and test-commit succeeded. 
Changes in the patch -
- JsonMetadata.java - If the input file has comma separated filenames, lookup 
schema file corresponding to each file separately.
- TestLoad.java - the har file being loaded in  test testNonDfsLocation does 
not exist, so use '-noschema' option .

> tests in TestLoad are failing 
> --
>
> Key: PIG-2179
> URL: https://issues.apache.org/jira/browse/PIG-2179
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.10
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.10
>
> Attachments: PIG-2179.1.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (PIG-2146) POStore.getSchema() returns null because of which PigOutputCommitter is not storing schema while cleanup

2011-07-20 Thread Thejas M Nair (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair reassigned PIG-2146:
--

Assignee: Thejas M Nair

> POStore.getSchema() returns null because of which PigOutputCommitter is not 
> storing schema while cleanup
> 
>
> Key: PIG-2146
> URL: https://issues.apache.org/jira/browse/PIG-2146
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.8.1, 0.9.0
>Reporter: Vivek Padmanabhan
>Assignee: Thejas M Nair
> Fix For: 0.9.0
>
> Attachments: PIG-2146.1.patch
>
>
> The below is my script;
> {code}
> register piggybank.jar;
> a = load 'myinput' using PigStorage(',') as 
> (f1:chararray,f2:chararray,f3:chararray);
> b = distinct a;
> c = limit b 2;
> store c into 'pss001' using 
> org.apache.pig.piggybank.storage.PigStorageSchema();
> {code}
> Input
> ---
> a,1,aa
> b,2,bb
> c,3,cc
> For this script , PigStorageSchema is not generating  .pig_headers and 
> .pig_schema files. While debugging I could see that storeSchema(..) method 
> itself is not invoked.The schema object for the store is returned as  null 
> (POStore.getSchema()) because of which PigOutputCommitter is not invoking the 
> storSchema.
> The same schema object is valid when I run it in local mode. This issue is 
> happening for Pig 0.9 also.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2179) tests in TestLoad are failing

2011-07-20 Thread Thejas M Nair (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated PIG-2179:
---

Attachment: PIG-2179.1.patch

> tests in TestLoad are failing 
> --
>
> Key: PIG-2179
> URL: https://issues.apache.org/jira/browse/PIG-2179
> Project: Pig
>  Issue Type: Bug
>Affects Versions: 0.10
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Fix For: 0.10
>
> Attachments: PIG-2179.1.patch
>
>


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (PIG-2181) Improvement : for error message when describe misses alias

2011-07-20 Thread Vivek Padmanabhan (JIRA)

Improvement : for error message when describe misses alias
--

 Key: PIG-2181
 URL: https://issues.apache.org/jira/browse/PIG-2181
 Project: Pig
  Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Vivek Padmanabhan
Priority: Minor


In Pig 0.9, if I have a describe without an alias, it throws a 
NullPointerException like below.

ERROR 2999: Unexpected internal error. null

java.lang.NullPointerException
at 
org.apache.pig.tools.grunt.GruntParser.processDescribe(GruntParser.java:270)
at 
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:317)
at 
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
at 
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
at org.apache.pig.Main.run(Main.java:553)
at org.apache.pig.Main.main(Main.java:108)


For example;
describe;

This message is of no use from a users perspective. Especially when my script 
becomes large and I have added couple of describe statements. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (PIG-2114) Enhancements to PIG HBaseStorage Load & Store Func with extra scan configurations

2011-07-20 Thread Hariprasad Kuppuswamy (JIRA)


 [ 
https://issues.apache.org/jira/browse/PIG-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hariprasad Kuppuswamy updated PIG-2114:
---

Status: Patch Available  (was: Open)

> Enhancements to PIG HBaseStorage Load & Store Func with extra scan 
> configurations
> -
>
> Key: PIG-2114
> URL: https://issues.apache.org/jira/browse/PIG-2114
> Project: Pig
>  Issue Type: New Feature
>  Components: impl
>Affects Versions: 0.9.0
>Reporter: Hariprasad Kuppuswamy
>Assignee: Hariprasad Kuppuswamy
>Priority: Minor
>  Labels: hbase, storage
> Fix For: 0.10
>
> Attachments: 
> Enhancments-to-enable-timestampversion-based-row-scan.patch
>
>
> - Added capability to specify scan based on timestamps (Hariprasad 
> Kuppuswwamy)
> - Ability to specify number of versions to be fetched with current scan 
> (Hariprasad Kuppuswwamy)
> - Configure the rowkey prefixes filter for the scan (Hariprasad Kuppuswwamy)
> - Added ability to omit nulls when dealing with hbase storage (Greg Bowyer)
> - Added ability to specify Put timestamps while insertion (Hariprasad 
> Kuppuswamy)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (PIG-2178) Filtering a source and then merging the filtered rows only generates data from one half of the filtering

[jira] [Resolved] (PIG-2178) Filtering a source and then merging the filtered rows only generates data from one half of the filtering

Build failed in Jenkins: Pig-trunk-commit #868

Build failed in Jenkins: Pig-trunk-commit #867

[jira] [Assigned] (PIG-2183) Pig 0.8.1 not working with Hadoop 0.20.203.0

[jira] [Commented] (PIG-2125) Make Pig work with hadoop .NEXT

[jira] [Commented] (PIG-2179) tests in TestLoad are failing

[jira] [Commented] (PIG-2125) Make Pig work with hadoop .NEXT

[jira] [Updated] (PIG-2125) Make Pig work with hadoop .NEXT

Build failed in Jenkins: Pig-trunk-commit #866

[jira] [Updated] (PIG-2146) POStore.getSchema() returns null because of which PigOutputCommitter is not storing schema while cleanup

[jira] [Commented] (PIG-2146) POStore.getSchema() returns null because of which PigOutputCommitter is not storing schema while cleanup

Build failed in Jenkins: Pig-trunk #1050

[jira] [Resolved] (PIG-2175) Switch Pig wiki to use confluence

[jira] [Commented] (PIG-2183) Pig 0.8.1 not working with Hadoop 0.20.203.0

[jira] [Created] (PIG-2183) Pig 0.8.1 not working with Hadoop 0.20.203.0

[jira] [Commented] (PIG-2183) Pig 0.8.1 not working with Hadoop 0.20.203.0

Build failed in Jenkins: Pig-trunk-commit #865

[jira] [Updated] (PIG-2182) Add more append support to DataByteArray

[jira] [Created] (PIG-2182) Add more append support to DataByteArray

[jira] [Commented] (PIG-2175) Switch Pig wiki to use confluence

Build failed in Jenkins: Pig-trunk-commit #864

[jira] [Commented] (PIG-2178) Filtering a source and then merging the filtered rows only generates data from one half of the filtering

[jira] [Updated] (PIG-2181) Improvement : for error message when describe misses alias

[jira] [Updated] (PIG-2027) NPE if Pig don't have permission for log file

[jira] [Updated] (PIG-2179) tests in TestLoad are failing

[jira] [Assigned] (PIG-2146) POStore.getSchema() returns null because of which PigOutputCommitter is not storing schema while cleanup

[jira] [Updated] (PIG-2179) tests in TestLoad are failing

[jira] [Created] (PIG-2181) Improvement : for error message when describe misses alias

[jira] [Updated] (PIG-2114) Enhancements to PIG HBaseStorage Load & Store Func with extra scan configurations

30 matches

Site Navigation

Mail list logo

Footer information