Hadoop 2.3 and pig

I know many of you are trying out Hadoop 2.x. Just FYI for those to save time if they hit the following issue when they are building directly off the branch. pig joins (replication, skewed and merge joins) are broken with a recent checkin - HADOOP-9877. For details refer https://issues.apache.org

Re: Review Request 13950: Tez backend layout

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13950/ --- (Updated Sept. 4, 2013, 1:16 a.m.) Review request for pig. Changes --- D

[jira] [Updated] (PIG-3448) Tez backend layout

[ https://issues.apache.org/jira/browse/PIG-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3448: --- Attachment: PIG-3448-2.patch > Tez backend layout > -- > > Key: PIG

[jira] Subscription: PIG patch available

Issue Subscription Filter: PIG patch available (17 issues) Subscriber: pigdaily Key Summary PIG-3449Move JobCreationException to org.apache.pig.backend.hadoop.executionengine https://issues.apache.org/jira/browse/PIG-3449 PIG-3448Tez backend layout

Re: Review Request 13950: Tez backend layout

> On Sept. 3, 2013, 9:48 p.m., Mark Wagner wrote: > > src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java, line 29 > > > > > > Do we still need this when we have the DAG api from Tez? It seems > > strange

Re: Review Request 13950: Tez backend layout

> On Sept. 3, 2013, 9:48 p.m., Mark Wagner wrote: > > src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java, line 29 > > > > > > Do we still need this when we have the DAG api from Tez? It seems > > strange

[jira] [Updated] (PIG-3448) Tez backend layout

[ https://issues.apache.org/jira/browse/PIG-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3448: --- Status: Patch Available (was: Open) > Tez backend layout > -- > >

[jira] [Updated] (PIG-3449) Move JobCreationException to org.apache.pig.backend.hadoop.executionengine

[ https://issues.apache.org/jira/browse/PIG-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3449: --- Status: Patch Available (was: Open) > Move JobCreationException to org.apache.pig.backend.hadoop.e

[jira] [Updated] (PIG-3449) Move JobCreationException to org.apache.pig.backend.hadoop.executionengine

[ https://issues.apache.org/jira/browse/PIG-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3449: --- Attachment: PIG-3446-1.patch Uploading a patch that moves the JobCreationException to the proposed des

[jira] [Updated] (PIG-3059) Global configurable minimum 'bad record' thresholds

[ https://issues.apache.org/jira/browse/PIG-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3059: --- Fix Version/s: (was: 0.12) > Global configurable minimum 'bad record' thresholds >

[jira] [Updated] (PIG-3346) New property that controls the number of combined splits

[ https://issues.apache.org/jira/browse/PIG-3346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3346: --- Fix Version/s: (was: 0.12) Status: Open (was: Patch Available) > New property that

[jira] [Updated] (PIG-3288) Kill jobs if the number of output files is over a configurable limit

[ https://issues.apache.org/jira/browse/PIG-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3288: --- Fix Version/s: (was: 0.12) > Kill jobs if the number of output files is over a configurable lim

[jira] [Updated] (PIG-3400) FS commands do not work with S3 paths

[ https://issues.apache.org/jira/browse/PIG-3400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3400: --- Fix Version/s: (was: 0.12) > FS commands do not work with S3 paths > --

[jira] [Updated] (PIG-3370) Add New Reserved Keywords To The Pig Docs

[ https://issues.apache.org/jira/browse/PIG-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3370: --- Fix Version/s: 0.12 > Add New Reserved Keywords To The Pig Docs > -

Re: Review Request 13950: Tez backend layout

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13950/#review25860 --- This looks like a good layout to start with. There are a couple thin

[jira] [Commented] (PIG-3288) Kill jobs if the number of output files is over a configurable limit

[ https://issues.apache.org/jira/browse/PIG-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757136#comment-13757136 ] Aniket Mokashi commented on PIG-3288: - [~cheolsoo], I am attempting to solve this with U

[jira] [Updated] (PIG-3448) Tez backend layout

[ https://issues.apache.org/jira/browse/PIG-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3448: --- Attachment: PIG-3448-1.patch Uploading a first draft. The patch includes skeleton classes that I think

[jira] [Commented] (PIG-2315) Make as clause work in generate

[ https://issues.apache.org/jira/browse/PIG-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757098#comment-13757098 ] Koji Noguchi commented on PIG-2315: --- > because it is not working anyway. > There's at leas

Review Request 13950: Tez backend layout

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13950/ --- Review request for pig. Bugs: PIG-3448 https://issues.apache.org/jira/brows

[jira] [Updated] (PIG-3434) Null subexpression in bincond nullifies outer tuple (or bag)

[ https://issues.apache.org/jira/browse/PIG-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated PIG-3434: - Release Note: Nevermind, I was confused. This is ready to be reviewed. Status: Patch Available (w

[jira] [Updated] (PIG-3434) Null subexpression in bincond nullifies outer tuple (or bag)

[ https://issues.apache.org/jira/browse/PIG-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated PIG-3434: - Status: Open (was: Patch Available) Some of my other work sneaked in here. I'll upload a clean patch.

[jira] [Updated] (PIG-3117) A debug mode in which pig does not delete temporary files

[ https://issues.apache.org/jira/browse/PIG-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3117: --- Fix Version/s: (was: 0.12) Status: Open (was: Patch Available) Canceling the patch sinc

[jira] [Updated] (PIG-3449) Move JobCreationException to org.apache.pig.backend.hadoop.executionengine

[ https://issues.apache.org/jira/browse/PIG-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3449: --- Affects Version/s: tez-branch > Move JobCreationException to org.apache.pig.backend.hadoop.executio

[jira] [Created] (PIG-3449) Move JobCreationException to org.apache.pig.backend.hadoop.executionengine

Cheolsoo Park created PIG-3449: -- Summary: Move JobCreationException to org.apache.pig.backend.hadoop.executionengine Key: PIG-3449 URL: https://issues.apache.org/jira/browse/PIG-3449 Project: Pig

[jira] [Created] (PIG-3448) Tez backend layout

Cheolsoo Park created PIG-3448: -- Summary: Tez backend layout Key: PIG-3448 URL: https://issues.apache.org/jira/browse/PIG-3448 Project: Pig Issue Type: Sub-task Components: tez Aff

[jira] [Commented] (PIG-3434) Null subexpression in bincond nullifies outer tuple (or bag)

[ https://issues.apache.org/jira/browse/PIG-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13757057#comment-13757057 ] Mark Wagner commented on PIG-3434: -- Nevermind, I was confused. This is ready to be reviewed

[jira] [Updated] (PIG-3278) Fix javadoc warnings

[ https://issues.apache.org/jira/browse/PIG-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3278: --- Fix Version/s: (was: 0.12) > Fix javadoc warnings > > > Ke

[jira] [Updated] (PIG-3434) Null subexpression in bincond nullifies outer tuple (or bag)

[ https://issues.apache.org/jira/browse/PIG-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated PIG-3434: - Release Note: (was: Nevermind, I was confused. This is ready to be reviewed.) > Null subexpression i

[jira] [Assigned] (PIG-2417) Streaming UDFs - allow users to easily write UDFs in scripting languages with no JVM implementation.

[ https://issues.apache.org/jira/browse/PIG-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Karn reassigned PIG-2417: Assignee: (was: Jeremy Karn) > Streaming UDFs - allow users to easily write UDFs in scriptin

[jira] [Updated] (PIG-3447) Compiler warning message dropped for CastLineageSetter and others with no enum kind

[ https://issues.apache.org/jira/browse/PIG-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3447: -- Attachment: pig-3447-v01.txt With the patch, it'll print out {noformat} 2013-09-03 13:58:20,625 [main]

[jira] [Assigned] (PIG-3426) Add support for removing s3 files

[ https://issues.apache.org/jira/browse/PIG-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Karn reassigned PIG-3426: Assignee: (was: Jeremy Karn) > Add support for removing s3 files > --

[jira] [Updated] (PIG-3426) Add support for removing s3 files

[ https://issues.apache.org/jira/browse/PIG-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Karn updated PIG-3426: - Attachment: PIG-3426-2.patch > Add support for removing s3 files > - >

[jira] [Commented] (PIG-3426) Add support for removing s3 files

[ https://issues.apache.org/jira/browse/PIG-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756994#comment-13756994 ] Jeremy Karn commented on PIG-3426: -- That makes sense. I've attached a new patch that does

Are we ready for Pig 0.12.0 release?

Hi, All, It has been more than half a year since initial Pig 0.11 release. I'd like roll a Pig 0.12 release around the end of September or the beginning of October. Let me know if it is possible. Proposed schedule: 1. Commit all major features (1-2 weeks) 2. Branching Pig 0.12 3. Commit remaining

[jira] [Updated] (PIG-3175) Refactor PigServer and GruntServer

[ https://issues.apache.org/jira/browse/PIG-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Kommireddi updated PIG-3175: - Fix Version/s: (was: 0.12) > Refactor PigServer and GruntServer > -

[jira] [Updated] (PIG-3259) Optimize byte to Long/Integer conversions

[ https://issues.apache.org/jira/browse/PIG-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Kommireddi updated PIG-3259: - Fix Version/s: (was: 0.12) 0.13.0 > Optimize byte to Long/Intege

[jira] [Updated] (PIG-2672) Optimize the use of DistributedCache

[ https://issues.apache.org/jira/browse/PIG-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-2672: Fix Version/s: 0.12 > Optimize the use of DistributedCache >

[jira] [Updated] (PIG-3367) Add assert keyword (operator) in pig

[ https://issues.apache.org/jira/browse/PIG-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3367: Fix Version/s: 0.12 > Add assert keyword (operator) in pig >

[jira] [Updated] (PIG-3421) Script jars should be added to extra jars instead of pig's job.jar

[ https://issues.apache.org/jira/browse/PIG-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3421: Fix Version/s: 0.12 > Script jars should be added to extra jars instead of pig's job.jar > --

[jira] [Updated] (PIG-3367) Add assert keyword (operator) in pig

[ https://issues.apache.org/jira/browse/PIG-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3367: Labels: 0 (was: ) > Add assert keyword (operator) in pig >

[jira] [Updated] (PIG-3367) Add assert keyword (operator) in pig

[ https://issues.apache.org/jira/browse/PIG-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3367: Labels: (was: 0) > Add assert keyword (operator) in pig >

Re: Are we ready for Pig 0.12.0 release?

+1 for a 0.12 release. I have one outstanding JIRA https://issues.apache.org/jira/browse/PIG-3199. Cheolsoo was fine with the patch (except for a typo which I will correct) but wanted a second opinion. Can someone please take a look? On Tue, Sep 3, 2013 at 11:08 AM, Jarek Jarcec Cecho wrote: >

[jira] [Commented] (PIG-3430) Add xml format for explaining MapReduce Plan.

[ https://issues.apache.org/jira/browse/PIG-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756911#comment-13756911 ] Jeremy Karn commented on PIG-3430: -- 1. I only added support for the mapreduce plan since th

[jira] [Created] (PIG-3447) Compiler warning message dropped for CastLineageSetter and others with no enum kind

Koji Noguchi created PIG-3447: - Summary: Compiler warning message dropped for CastLineageSetter and others with no enum kind Key: PIG-3447 URL: https://issues.apache.org/jira/browse/PIG-3447 Project: Pig

Re: Are we ready for Pig 0.12.0 release?

I'll try to clean up and finish PIG-3390 (HBase 0.95 support) this week, to see if it can be included. Jarcec On Tue, Sep 03, 2013 at 10:56:42AM -0700, Cheolsoo Park wrote: > +1. I will go through my jiras this week. > > > On Tue, Sep 3, 2013 at 10:34 AM, Daniel Dai wrote: > > > Hi, All, > >

Re: Are we ready for Pig 0.12.0 release?

+1. I will go through my jiras this week. On Tue, Sep 3, 2013 at 10:34 AM, Daniel Dai wrote: > Hi, All, > It has been more than half a year since initial Pig 0.11 release. I'd like > roll a Pig 0.12 release around the end of September or the beginning of > October. Let me know if it is possible

[jira] [Updated] (PIG-3295) Casting from bytearray failing after Union (even when each field is from a single Loader)

[ https://issues.apache.org/jira/browse/PIG-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3295: -- Attachment: pig-3295-v02.patch Just noticed my previous patch wasn't created with '--no-prefix' option.

dev@pig.apache.org

[ https://issues.apache.org/jira/browse/PIG-3293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13756712#comment-13756712 ] Koji Noguchi commented on PIG-3293: --- bq. Also improve the error message to indicate possib

[jira] [Updated] (PIG-2417) Streaming UDFs - allow users to easily write UDFs in scripting languages with no JVM implementation.

[ https://issues.apache.org/jira/browse/PIG-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Karn updated PIG-2417: - Affects Version/s: (was: 0.11) 0.12 Status: Patch Available (was: