pig load function

2014-02-04 Thread Krishna Prasad Ambaripeta
Hi.I am new to pig. Have a basic doubt. when we write a = load 'a/y.txt' as(a,b) , which pig function will it call. is it LoadFunc? Thanks for the support. Thanks,Krishna Prasad

Re: pig load function

2014-02-04 Thread Mark Wagner
Hi Krishna, By default it uses the PigStorage LoadFunc. You can change that behavior though by setting pig.default.load.func to your LoadFunc. -Mark On Mon, Feb 3, 2014 at 8:37 PM, Krishna Prasad Ambaripeta krishna.ambarip...@hotmail.com wrote: Hi.I am new to pig. Have a basic doubt. when we

Review Request 17681: [PIG-3742] Set MR runtime settings on tez runtime

2014-02-04 Thread Rohini Palaniswamy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17681/ --- Review request for pig, Cheolsoo Park and Daniel Dai. Bugs: PIG-3742

[jira] [Commented] (PIG-3347) Store invocation brings side effect

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890903#comment-13890903 ] Daniel Dai commented on PIG-3347: - All unit tests pass with the patch. Store invocation

[jira] [Updated] (PIG-3347) Store invocation brings side effect

2014-02-04 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3347: -- Attachment: PIG-3347-4-testonly.patch Thanks [~daijy]. Adding one more testcase that I believe should

[jira] [Commented] (PIG-3347) Store invocation brings side effect

2014-02-04 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890942#comment-13890942 ] Koji Noguchi commented on PIG-3347: --- bq. UID is to track column lineage so in logical

[jira] [Updated] (PIG-3347) Store invocation brings side effect

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3347: Attachment: PIG-3347-5.patch Attach another patch which also address Koji's new case. Store invocation

[jira] [Commented] (PIG-3741) Utils.setTmpFileCompressionOnConf can cause side effect for SequenceFileInterStorage

2014-02-04 Thread Julien Le Dem (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891030#comment-13891030 ] Julien Le Dem commented on PIG-3741: Ideally each store would get its own config object,

[jira] [Commented] (PIG-3347) Store invocation brings side effect

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891032#comment-13891032 ] Daniel Dai commented on PIG-3347: - [~knoguchi], in the B = foreach A generate a as col1, a

[jira] [Updated] (PIG-259) allow store to overwrite existing directroy

2014-02-04 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nezih Yigitbasi updated PIG-259: Attachment: PIG-259.7.patch Daniel, Thanks for the comments. 1. Good catch. 2. Updated

[jira] [Updated] (PIG-3567) LogicalPlanPrinter throws OOM for large scripts

2014-02-04 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3567: Fix Version/s: 0.13.0 0.12.1 LogicalPlanPrinter throws OOM for large scripts

[jira] [Resolved] (PIG-259) allow store to overwrite existing directroy

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-259. Resolution: Fixed Fix Version/s: 0.13.0 Hadoop Flags: Reviewed Also add some comment to

Re: Review Request 17681: [PIG-3742] Set MR runtime settings on tez runtime

2014-02-04 Thread Daniel Dai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17681/#review33632 --- Seems the patch cannot apply cleanly. Can you rebase? - Daniel Dai

[jira] [Commented] (PIG-3347) Store invocation brings side effect

2014-02-04 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891127#comment-13891127 ] Koji Noguchi commented on PIG-3347: --- bq. we will need to generate a new uid for col2 to

[jira] [Commented] (PIG-259) allow store to overwrite existing directroy

2014-02-04 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891145#comment-13891145 ] Nezih Yigitbasi commented on PIG-259: - Daniel, one question. Do you think the

[jira] [Updated] (PIG-3744) SequenceFileLoader does not support BytesWritable

2014-02-04 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3744: Attachment: PIG-3744-1.patch SequenceFileLoader does not support BytesWritable

[jira] [Updated] (PIG-3744) SequenceFileLoader does not support BytesWritable

2014-02-04 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3744: Status: Patch Available (was: Open) SequenceFileLoader does not support BytesWritable

[jira] [Commented] (PIG-259) allow store to overwrite existing directroy

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891158#comment-13891158 ] Daniel Dai commented on PIG-259: Yes, good point. Let's remove the method to keep interface

[jira] [Commented] (PIG-3744) SequenceFileLoader does not support BytesWritable

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891160#comment-13891160 ] Daniel Dai commented on PIG-3744: - +1 SequenceFileLoader does not support BytesWritable

[jira] [Updated] (PIG-3744) SequenceFileLoader does not support BytesWritable

2014-02-04 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3744: Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk.

[jira] [Updated] (PIG-3744) SequenceFileLoader does not support BytesWritable

2014-02-04 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3744: Attachment: PIG-3744-2.patch Test failed while running before commit as a single quote in

[jira] [Commented] (PIG-3347) Store invocation brings side effect

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891178#comment-13891178 ] Daniel Dai commented on PIG-3347: - It could but introduce a lot of complications. Currently

[jira] [Commented] (PIG-3441) Allow Pig to use default resources from Configuration objects

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891228#comment-13891228 ] Daniel Dai commented on PIG-3441: - [~bdmogal] I see several more places we instantiate

[jira] [Updated] (PIG-259) allow store to overwrite existing directroy

2014-02-04 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nezih Yigitbasi updated PIG-259: Attachment: PIG-259.8.patch When I removed the isOverwritable method tests failed. Because, the user

[jira] [Commented] (PIG-3441) Allow Pig to use default resources from Configuration objects

2014-02-04 Thread Bhooshan Mogal (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891280#comment-13891280 ] Bhooshan Mogal commented on PIG-3441: - [~daijy], yes, I agree. There are a lot of places

Re: Review Request 17681: [PIG-3742] Set MR runtime settings on tez runtime

2014-02-04 Thread Rohini Palaniswamy
On Feb. 4, 2014, 8:05 p.m., Daniel Dai wrote: Seems the patch cannot apply cleanly. Can you rebase? This is because of the svn mv of the util classes. Uploaded a patch after deleting the moved files and creating them newly. Will do svn mv during the commit. - Rohini

Re: Review Request 17681: [PIG-3742] Set MR runtime settings on tez runtime

2014-02-04 Thread Rohini Palaniswamy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17681/ --- (Updated Feb. 4, 2014, 10:08 p.m.) Review request for pig, Cheolsoo Park and

[jira] [Commented] (PIG-259) allow store to overwrite existing directroy

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891303#comment-13891303 ] Daniel Dai commented on PIG-259: Sounds good. Since the patch is committed, can you upload

[jira] [Updated] (PIG-259) allow store to overwrite existing directroy

2014-02-04 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nezih Yigitbasi updated PIG-259: Attachment: PIG-259.9.patch Daniel, delta patch added. Thanks for the review. allow store to

[jira] [Commented] (PIG-259) allow store to overwrite existing directroy

2014-02-04 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891440#comment-13891440 ] Daniel Dai commented on PIG-259: PIG-259.9.patch committed. Thanks Nezih! allow store to

[jira] Subscription: PIG patch available

2014-02-04 Thread jira
Issue Subscription Filter: PIG patch available (17 issues) Subscriber: pigdaily Key Summary PIG-3741Utils.setTmpFileCompressionOnConf can cause side effect for SequenceFileInterStorage https://issues.apache.org/jira/browse/PIG-3741 PIG-3737Bundle dependent jars in

[jira] [Updated] (PIG-3623) HBaseStorage: setting loadKey and noWAL to false doesn't have any affect

2014-02-04 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3623: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch