[jira] [Commented] (PIG-3404) Improve Pig to ignore bad files or inaccessible files or folders

[ https://issues.apache.org/jira/browse/PIG-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777186#comment-13777186 ] Jerry Chen commented on PIG-3404: - Hi Park, sorry for the late response and I am glad that w

[jira] [Updated] (PIG-3404) Improve Pig to ignore bad files or inaccessible files or folders

[ https://issues.apache.org/jira/browse/PIG-3404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry Chen updated PIG-3404: Attachment: PIG-3404.patch Patch for reference > Improve Pig to ignore bad files or inaccess

[jira] [Commented] (PIG-3477) *Nix commands do not execute sequentially

[ https://issues.apache.org/jira/browse/PIG-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777138#comment-13777138 ] Daniel Dai commented on PIG-3477: - This is expected. Load is a statement which will run unti

Re: Review Request 14274: PIG-2672 Optimize the use of DistributedCache

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14274/#review26370 --- trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLa

[jira] [Commented] (PIG-3453) Implement a Storm backend to Pig

[ https://issues.apache.org/jira/browse/PIG-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777052#comment-13777052 ] Brian ONeill commented on PIG-3453: --- I've been looking at SummingBird as well, and have ha

[jira] [Commented] (PIG-2672) Optimize the use of DistributedCache

[ https://issues.apache.org/jira/browse/PIG-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777040#comment-13777040 ] Koji Noguchi commented on PIG-2672: --- bq. In fact, any misconfiguration is still protected

[jira] Subscription: PIG patch available

Issue Subscription Filter: PIG patch available (14 issues) Subscriber: pigdaily Key Summary PIG-3470Print configuration variables in grunt https://issues.apache.org/jira/browse/PIG-3470 PIG-3458ScalarExpression lost with multiquery optimization http

[jira] [Commented] (PIG-3295) Casting from bytearray failing after Union (even when each field is from a single Loader)

[ https://issues.apache.org/jira/browse/PIG-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777011#comment-13777011 ] Daniel Dai commented on PIG-3295: - +1. > Casting from bytearray failing aft

[jira] [Commented] (PIG-2672) Optimize the use of DistributedCache

[ https://issues.apache.org/jira/browse/PIG-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13777001#comment-13777001 ] Rohini Palaniswamy commented on PIG-2672: - I guess you don't have to check for permi

Re: Review Request 14274: PIG-2672 Optimize the use of DistributedCache

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14274/#review26369 --- trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLa

Re: Review Request 14274: PIG-2672 Optimize the use of DistributedCache

> On Sept. 25, 2013, 12:13 a.m., Rohini Palaniswamy wrote: > > trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java, > > line 1495 > > > > > > Since the name of the file on h

[jira] [Commented] (PIG-2672) Optimize the use of DistributedCache

[ https://issues.apache.org/jira/browse/PIG-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776988#comment-13776988 ] Rohini Palaniswamy commented on PIG-2672: - [~aniket486], bq. Currently jars get cop

Re: Review Request 14274: PIG-2672 Optimize the use of DistributedCache

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/14274/#review26364 --- There are some white spaces in the patch and code is not formatted.

[jira] [Resolved] (PIG-3360) Some intermittent negative e2e tests fail on hadoop 2

[ https://issues.apache.org/jira/browse/PIG-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-3360. - Resolution: Fixed Hadoop Flags: Reviewed Patch committed to both 0.12 branch and trunk.

[jira] [Resolved] (PIG-3468) PIG-3123 breaks e2e test Jython_Diagnostics_2

[ https://issues.apache.org/jira/browse/PIG-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-3468. - Resolution: Fixed Hadoop Flags: Reviewed Patch committed to both 0.12 branch and trunk.

[jira] [Commented] (PIG-3360) Some intermittent negative e2e tests fail on hadoop 2

[ https://issues.apache.org/jira/browse/PIG-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776935#comment-13776935 ] Thejas M Nair commented on PIG-3360: Looks good. +1 > Some intermitten

[jira] [Commented] (PIG-2417) Streaming UDFs - allow users to easily write UDFs in scripting languages with no JVM implementation.

[ https://issues.apache.org/jira/browse/PIG-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776931#comment-13776931 ] Daniel Dai commented on PIG-2417: - I compiled on both my RHEL6 and Windows, seems fine for m

[jira] [Commented] (PIG-3468) PIG-3123 breaks e2e test Jython_Diagnostics_2

[ https://issues.apache.org/jira/browse/PIG-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776917#comment-13776917 ] Alan Gates commented on PIG-3468: - +1 > PIG-3123 breaks e2e test Jython_Dia

[jira] [Commented] (PIG-2672) Optimize the use of DistributedCache

[ https://issues.apache.org/jira/browse/PIG-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776910#comment-13776910 ] Aniket Mokashi commented on PIG-2672: - [~cheolsoo], thanks for your comments. I will wor

[jira] [Commented] (PIG-2417) Streaming UDFs - allow users to easily write UDFs in scripting languages with no JVM implementation.

[ https://issues.apache.org/jira/browse/PIG-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776884#comment-13776884 ] Rohini Palaniswamy commented on PIG-2417: - I see compilation fails with 4 similar er

[jira] [Commented] (PIG-3445) Make Parquet format available out of the box in Pig

[ https://issues.apache.org/jira/browse/PIG-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776879#comment-13776879 ] Dmitriy V. Ryaboy commented on PIG-3445: Other loaders like csv, avro, json, xml, et

[jira] [Updated] (PIG-3479) Fix BigInt, BigDec, Date serialization. Improve perf of PigNullableWritable deserilization

[ https://issues.apache.org/jira/browse/PIG-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy V. Ryaboy updated PIG-3479: --- Resolution: Fixed Release Note: Skewed join internals improved to get 10% or better impro

[jira] [Updated] (PIG-3479) Fix BigInt, BigDec, Date serialization. Improve perf of PigNullableWritable deserilization

[ https://issues.apache.org/jira/browse/PIG-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy V. Ryaboy updated PIG-3479: --- Attachment: PIG-3479.whitespace.patch Same patch, but with whitespace changes. Committing this.

[jira] [Commented] (PIG-3458) ScalarExpression lost with multiquery optimization

[ https://issues.apache.org/jira/browse/PIG-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776795#comment-13776795 ] Daniel Dai commented on PIG-3458: - +1. Please commit to trunk and 0.12 branch.

[jira] [Updated] (PIG-3458) ScalarExpression lost with multiquery optimization

[ https://issues.apache.org/jira/browse/PIG-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3458: Fix Version/s: 0.12 > ScalarExpression lost with multiquery optimization > --

[jira] [Commented] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776769#comment-13776769 ] Rohini Palaniswamy commented on PIG-3480: - We do have complex types like bag of maps

[jira] [Commented] (PIG-3445) Make Parquet format available out of the box in Pig

[ https://issues.apache.org/jira/browse/PIG-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776764#comment-13776764 ] Daniel Dai commented on PIG-3445: - Size maybe one thing, but still, doing a favor for Parque

[jira] [Commented] (PIG-3479) Fix BigInt, BigDec, Date serialization. Improve perf of PigNullableWritable deserilization

[ https://issues.apache.org/jira/browse/PIG-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776752#comment-13776752 ] Jonathan Coveney commented on PIG-3479: --- +1 > Fix BigInt, BigDec, Dat

[jira] [Commented] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776745#comment-13776745 ] Olga Natkovich commented on PIG-3480: - Could this be related to Hadoop version?

[jira] [Commented] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776728#comment-13776728 ] Dmitriy V. Ryaboy commented on PIG-3480: Rohini I suspect this might be something ab

[jira] [Commented] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776732#comment-13776732 ] Dmitriy V. Ryaboy commented on PIG-3480: Rohini, do you guys use lzo or gz compressi

[jira] [Updated] (PIG-3360) Some intermittent negative e2e tests fail on hadoop 2

[ https://issues.apache.org/jira/browse/PIG-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3360: Attachment: PIG-3360-2.patch The new patch put the logic in shims layer. The API TaskReport.getCurrentStatus

[jira] [Commented] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776681#comment-13776681 ] Rohini Palaniswamy commented on PIG-3480: - [~dvryaboy], We have been running it w

[jira] [Commented] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1377#comment-1377 ] Dmitriy V. Ryaboy commented on PIG-3480: [~knoguchi] yeah, I'm not sure the stack tr

[jira] [Commented] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776644#comment-13776644 ] Koji Noguchi commented on PIG-3480: --- Dmitriy, isn't your stacktrace failing at mapred.IFil

[jira] [Commented] (PIG-3461) Rewrite PartitionFilterOptimizer to make it work for all the cases

[ https://issues.apache.org/jira/browse/PIG-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776636#comment-13776636 ] Aniket Mokashi commented on PIG-3461: - Committed to trunk and 0.12 branch. Thanks again

[jira] [Updated] (PIG-3461) Rewrite PartitionFilterOptimizer to make it work for all the cases

[ https://issues.apache.org/jira/browse/PIG-3461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3461: Resolution: Fixed Status: Resolved (was: Patch Available) > Rewrite PartitionFilterOptim

[jira] [Updated] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy V. Ryaboy updated PIG-3480: --- Attachment: PIG-3480.patch Attaching a rough patch which replaces use of TFile with SequenceFil

[jira] [Comment Edited] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776602#comment-13776602 ] Dmitriy V. Ryaboy edited comment on PIG-3480 at 9/24/13 6:36 PM: -

[jira] [Commented] (PIG-3480) TFile-based tmpfile compression crashes in some cases

[ https://issues.apache.org/jira/browse/PIG-3480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776602#comment-13776602 ] Dmitriy V. Ryaboy commented on PIG-3480: For most of the tasks that fail, no stack t

[jira] [Created] (PIG-3480) TFile-based tmpfile compression crashes in some cases

Dmitriy V. Ryaboy created PIG-3480: -- Summary: TFile-based tmpfile compression crashes in some cases Key: PIG-3480 URL: https://issues.apache.org/jira/browse/PIG-3480 Project: Pig Issue Type:

[jira] [Commented] (PIG-2672) Optimize the use of DistributedCache

[ https://issues.apache.org/jira/browse/PIG-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776452#comment-13776452 ] Koji Noguchi commented on PIG-2672: --- On the secure hadoop environment, this patch would ba

[jira] [Commented] (PIG-2672) Optimize the use of DistributedCache

[ https://issues.apache.org/jira/browse/PIG-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776429#comment-13776429 ] Cheolsoo Park commented on PIG-2672: Forgot to mention. [~aniket486], can you please doc

[jira] [Updated] (PIG-3445) Make Parquet format available out of the box in Pig

[ https://issues.apache.org/jira/browse/PIG-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy V. Ryaboy updated PIG-3445: --- Fix Version/s: 0.12 > Make Parquet format available out of the box in Pig > ---

[jira] [Commented] (PIG-3445) Make Parquet format available out of the box in Pig

[ https://issues.apache.org/jira/browse/PIG-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13776172#comment-13776172 ] Dmitriy V. Ryaboy commented on PIG-3445: The size of the dependency introduced by th

[jira] [Updated] (PIG-3325) Adding a tuple to a bag is slow

[ https://issues.apache.org/jira/browse/PIG-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy V. Ryaboy updated PIG-3325: --- Fix Version/s: 0.12 > Adding a tuple to a bag is slow > --- > >

[jira] [Updated] (PIG-3479) Fix BigInt, BigDec, Date serialization. Improve perf of PigNullableWritable deserilization

[ https://issues.apache.org/jira/browse/PIG-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy V. Ryaboy updated PIG-3479: --- Fix Version/s: 0.12 > Fix BigInt, BigDec, Date serialization. Improve perf of PigNullableWr

[jira] [Updated] (PIG-3325) Adding a tuple to a bag is slow

[ https://issues.apache.org/jira/browse/PIG-3325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy V. Ryaboy updated PIG-3325: --- Affects Version/s: 0.12 > Adding a tuple to a bag is slow > ---