[jira] [Comment Edited] (PIG-3906) ant site errors out

2016-07-17 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381207#comment-15381207 ] Daniel Dai edited comment on PIG-3906 at 7/17/16 8:00 AM: -- This patch needs to be

[jira] [Commented] (PIG-3906) ant site errors out

2016-07-17 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381207#comment-15381207 ] Daniel Dai commented on PIG-3906: - This patch needs to be reverted. We can get false alarm when the plugin

[jira] [Updated] (PIG-4942) Fix mvn-install ant target

2016-07-17 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4942: Assignee: Peter Slawski > Fix mvn-install ant target > -- > > Key:

[jira] [Updated] (PIG-4942) Fix mvn-install ant target

2016-07-17 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4942: Fix Version/s: 0.16.1 > Fix mvn-install ant target > -- > > Key:

[jira] [Updated] (PIG-4942) Fix mvn-install ant target

2016-07-17 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4942: Attachment: PIG-4942.3.patch Yes mvn-install is broken, we shall fix it. Since mvn-deploy does more than

[jira] [Resolved] (PIG-4945) Update document for conflicting macro params

2016-07-16 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-4945. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 0.16.1 0.17.0 Patch

[jira] [Resolved] (PIG-4752) XPath - ignoreNamespace cannot be overridden

2016-07-16 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-4752. - Resolution: Duplicate Fixed as part of PIG-4938. > XPath - ignoreNamespace cannot be overridden >

[jira] [Resolved] (PIG-4938) [PiggyBank] XPath returns empty values when using aggregation method

2016-07-16 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-4938. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 0.17.0 +1. Patch committed to trunk.

[jira] [Resolved] (PIG-4949) Fix registering jar in S3 which was broken by PIG-4417 in Pig 0.16

2016-07-15 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-4949. - Resolution: Fixed Assignee: Yishan Yang Hadoop Flags: Reviewed +1. Patch committed to both

[jira] [Updated] (PIG-4948) Pig on Tez AM use too much memory on a small cluster

2016-07-15 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4948: Status: Patch Available (was: Open) > Pig on Tez AM use too much memory on a small cluster >

[jira] [Comment Edited] (PIG-4948) Pig on Tez AM use too much memory on a small cluster

2016-07-15 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15378576#comment-15378576 ] Daniel Dai edited comment on PIG-4948 at 7/15/16 6:14 PM: -- I am going to introduce

[jira] [Updated] (PIG-4948) Pig on Tez AM use too much memory on a small cluster

2016-07-15 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4948: Attachment: PIG-4948-1.patch > Pig on Tez AM use too much memory on a small cluster >

[jira] [Updated] (PIG-4948) Pig on Tez AM use too much memory on a small cluster

2016-07-14 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4948: Description: Pig will use 1024M for AM memory even if "tez.am.resource.memory.mb" is smaller. On a small

[jira] [Commented] (PIG-4948) Pig on Tez AM use too much memory on a small cluster

2016-07-14 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15378576#comment-15378576 ] Daniel Dai commented on PIG-4948: - I am going to introduce an option to respect tez.am.resource.memory.mb is

[jira] [Created] (PIG-4948) Pig on Tez AM use too much memory on a small cluster

2016-07-14 Thread Daniel Dai (JIRA)
Daniel Dai created PIG-4948: --- Summary: Pig on Tez AM use too much memory on a small cluster Key: PIG-4948 URL: https://issues.apache.org/jira/browse/PIG-4948 Project: Pig Issue Type: Bug

[jira] [Updated] (PIG-4947) LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards results in empty maps for the pure wildcarded column families.

2016-07-14 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4947: Status: Patch Available (was: Open) > LOAD with HBaseStorage using a mix of pure wildcards and prefixed

[jira] [Updated] (PIG-4947) LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards results in empty maps for the pure wildcarded column families.

2016-07-14 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4947: Attachment: PIG-4947-1.patch > LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards >

[jira] [Assigned] (PIG-4947) LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards results in empty maps for the pure wildcarded column families.

2016-07-14 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reassigned PIG-4947: --- Assignee: Daniel Dai > LOAD with HBaseStorage using a mix of pure wildcards and prefixed wildcards >

[jira] [Commented] (PIG-4940) Predicate push-down filtering unary expressions can be pushed.

2016-07-01 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359503#comment-15359503 ] Daniel Dai commented on PIG-4940: - Can you attach patch instead? Would you mind a test case? > Predicate

[jira] [Updated] (PIG-4940) Predicate push-down filtering unary expressions can be pushed.

2016-07-01 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4940: Assignee: Ryan Blue > Predicate push-down filtering unary expressions can be pushed. >

[jira] [Commented] (PIG-4938) [PiggyBank] XPath returns empty values when using aggregation method

2016-07-01 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359479#comment-15359479 ] Daniel Dai commented on PIG-4938: - Can you attach patch instead? Is your patch address both PIG-4751 and

[jira] [Commented] (PIG-4933) TestDataBagAccess.testBagConstantFlatten1/TestLogicalPlanBuilder.testQuery90 broken after PIG-2315

2016-06-24 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348808#comment-15348808 ] Daniel Dai commented on PIG-4933: - The statement doesn't mean to cast to bytearray, it means don't cast. We

[jira] [Resolved] (PIG-2348) Bloom should be able to take a relation or a file

2016-06-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-2348. - Resolution: Duplicate > Bloom should be able to take a relation or a file >

[jira] [Updated] (PIG-4906) Add Bigdecimal functions in Over function

2016-06-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4906: Resolution: Fixed Assignee: Cristian Galán Hadoop Flags: Reviewed Fix Version/s:

[jira] [Commented] (PIG-4925) Support for passing the bloom filter to the Bloom UDF

2016-06-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346845#comment-15346845 ] Daniel Dai commented on PIG-4925: - +1 > Support for passing the bloom filter to the Bloom UDF >

[jira] [Commented] (PIG-4911) Provide option to disable DAG recovery

2016-06-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346831#comment-15346831 ] Daniel Dai commented on PIG-4911: - +1 > Provide option to disable DAG recovery >

[jira] [Assigned] (PIG-4930) Skewed Join Breaks On Empty Sampled Input When Key is From Map

2016-06-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reassigned PIG-4930: --- Assignee: Daniel Dai > Skewed Join Breaks On Empty Sampled Input When Key is From Map >

[jira] [Updated] (PIG-4930) Skewed Join Breaks On Empty Sampled Input When Key is From Map

2016-06-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4930: Fix Version/s: 0.17.0 > Skewed Join Breaks On Empty Sampled Input When Key is From Map >

[jira] [Updated] (PIG-4932) Cache files not loaded when using 'limit' operator

2016-06-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4932: Fix Version/s: 0.17.0 > Cache files not loaded when using 'limit' operator >

[jira] [Assigned] (PIG-4932) Cache files not loaded when using 'limit' operator

2016-06-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reassigned PIG-4932: --- Assignee: Daniel Dai > Cache files not loaded when using 'limit' operator >

[jira] [Commented] (PIG-4911) Provide option to disable DAG recovery

2016-06-08 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321480#comment-15321480 ] Daniel Dai commented on PIG-4911: - Yes, you only need one entry (or several for future extension). >

[jira] [Commented] (PIG-4911) Provide option to disable DAG recovery

2016-06-08 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15321382#comment-15321382 ] Daniel Dai commented on PIG-4911: - Can we not copy TezConfiguration? Seems a smaller data structure

[jira] [Commented] (PIG-4921) Kill running jobs on InterruptedException

2016-06-08 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320189#comment-15320189 ] Daniel Dai commented on PIG-4921: - You mentioned TezSessionManager.shutdown is not effective in Oozie. If

[jira] [Commented] (PIG-4921) Kill running jobs on InterruptedException

2016-06-07 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318964#comment-15318964 ] Daniel Dai commented on PIG-4921: - Then in case of Oozie interruption, Pig call HangingJobKiller; In case of

[jira] [Commented] (PIG-4921) Kill running jobs on InterruptedException

2016-06-07 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318870#comment-15318870 ] Daniel Dai commented on PIG-4921: - It kills active session but not idle sessions. Idle session will go away

[jira] [Commented] (PIG-4922) Deadlock between SpillableMemoryManager and InternalSortedBag$SortedDataBagIterator

2016-06-07 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317968#comment-15317968 ] Daniel Dai commented on PIG-4922: - +1 > Deadlock between SpillableMemoryManager and >

[jira] [Commented] (PIG-4921) Kill running jobs on InterruptedException

2016-06-07 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317951#comment-15317951 ] Daniel Dai commented on PIG-4921: - Thanks for taking care of shutdown hook with priority. In case of

[jira] [Commented] (PIG-4923) Drop Hadoop 1.x support in Pig 0.17

2016-06-06 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317828#comment-15317828 ] Daniel Dai commented on PIG-4923: - I would say no need, we shall focus on Hadoop 2 and save time, [~rohini],

[jira] [Commented] (PIG-4916) Pig on Tez fail to remove temporary HDFS files in some cases

2016-06-06 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317236#comment-15317236 ] Daniel Dai commented on PIG-4916: - Good catch on Hadoop 1 compilation failure. In that sense, I'd only put

[jira] [Updated] (PIG-4916) Pig on Tez fail to remove temporary HDFS files in some cases

2016-06-06 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4916: Fix Version/s: (was: 0.16.1) > Pig on Tez fail to remove temporary HDFS files in some cases >

[jira] [Commented] (PIG-4921) Kill running jobs on InterruptedException

2016-06-03 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15315120#comment-15315120 ] Daniel Dai commented on PIG-4921: - TezLaucher.kill still kill individual job, is that redundant as well?

[jira] [Updated] (PIG-4918) Pig on Tez cannot switch pig.temp.dir to another fs

2016-06-01 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4918: Status: Patch Available (was: Open) Don't find an easy way to write a test since we need different fs to

[jira] [Updated] (PIG-4918) Pig on Tez cannot switch pig.temp.dir to another fs

2016-06-01 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4918: Attachment: PIG-4918-1.patch > Pig on Tez cannot switch pig.temp.dir to another fs >

[jira] [Created] (PIG-4918) Pig on Tez cannot switch pig.temp.dir to another fs

2016-06-01 Thread Daniel Dai (JIRA)
Daniel Dai created PIG-4918: --- Summary: Pig on Tez cannot switch pig.temp.dir to another fs Key: PIG-4918 URL: https://issues.apache.org/jira/browse/PIG-4918 Project: Pig Issue Type: Bug

[jira] [Updated] (PIG-4916) Pig on Tez fail to remove temporary HDFS files in some cases

2016-06-01 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4916: Status: Patch Available (was: Open) I don't see a way to write a test case since this is non-deterministic

[jira] [Updated] (PIG-4916) Pig on Tez fail to remove temporary HDFS files in some cases

2016-06-01 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4916: Attachment: PIG-4916-1.patch > Pig on Tez fail to remove temporary HDFS files in some cases >

[jira] [Created] (PIG-4916) Pig on Tez fail to remove temporary HDFS files in some cases

2016-06-01 Thread Daniel Dai (JIRA)
Daniel Dai created PIG-4916: --- Summary: Pig on Tez fail to remove temporary HDFS files in some cases Key: PIG-4916 URL: https://issues.apache.org/jira/browse/PIG-4916 Project: Pig Issue Type: Bug

[jira] [Commented] (PIG-2315) Make as clause work in generate

2016-06-01 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15311296#comment-15311296 ] Daniel Dai commented on PIG-2315: - +1. Also note there is a performance regression in some cases. For

[jira] [Updated] (PIG-4719) Documentation for PIG-4704: Customizable Error Handling for Storers in Pig

2016-06-01 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4719: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 0.17.0 Status: Resolved (was:

[jira] [Updated] (PIG-4719) Documentation for PIG-4704: Customizable Error Handling for Storers in Pig

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4719: Attachment: PIG-4719-2.patch New patch with suggested changes. > Documentation for PIG-4704: Customizable

[jira] [Commented] (PIG-4821) Pig chararray field with special UTF-8 chars as part of tuple join key produces wrong results in Tez

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308774#comment-15308774 ] Daniel Dai commented on PIG-4821: - +1 > Pig chararray field with special UTF-8 chars as part of tuple join

[jira] [Updated] (PIG-4719) Documentation for PIG-4704: Customizable Error Handling for Storers in Pig

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4719: Attachment: PIG-4719-1.patch > Documentation for PIG-4704: Customizable Error Handling for Storers in Pig >

[jira] [Assigned] (PIG-4719) Documentation for PIG-4704: Customizable Error Handling for Storers in Pig

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reassigned PIG-4719: --- Assignee: Daniel Dai > Documentation for PIG-4704: Customizable Error Handling for Storers in Pig >

[jira] [Updated] (PIG-4719) Documentation for PIG-4704: Customizable Error Handling for Storers in Pig

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4719: Status: Patch Available (was: Open) > Documentation for PIG-4704: Customizable Error Handling for Storers in

[jira] [Updated] (PIG-4734) TOMAP schema inferring breaks some scripts in type checking for bincond

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4734: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: (was: 0.15.1)

[jira] [Updated] (PIG-4786) CROSS will not work correctly with Grace Parallelism

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4786: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 0.17.0 Status: Resolved (was:

[jira] [Commented] (PIG-4908) JythonFunction refers to Oozie launcher script absolute path

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308454#comment-15308454 ] Daniel Dai commented on PIG-4908: - +1. Further, I don't see a reason to make relative path different. We

[jira] [Commented] (PIG-3228) SearchEngineExtractor throws an exception on a malformed URL

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15307290#comment-15307290 ] Daniel Dai commented on PIG-3228: - Shall we use a counter for malformed URL? > SearchEngineExtractor throws

[jira] [Updated] (PIG-3228) SearchEngineExtractor throws an exception on a malformed URL

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3228: Fix Version/s: (was: 0.16.0) 0.17.0 > SearchEngineExtractor throws an exception on a

[jira] [Resolved] (PIG-3227) SearchEngineExtractor does not work for bing

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-3227. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 0.17.0 Patch committed to 0.16 branch

[jira] [Updated] (PIG-3227) SearchEngineExtractor does not work for bing

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-3227: Assignee: Danny Antonetti > SearchEngineExtractor does not work for bing >

[jira] [Updated] (PIG-4786) CROSS will not work correctly with Grace Parallelism

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4786: Status: Patch Available (was: Open) > CROSS will not work correctly with Grace Parallelism >

[jira] [Updated] (PIG-4786) CROSS will not work correctly with Grace Parallelism

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4786: Attachment: PIG-4786-1.patch > CROSS will not work correctly with Grace Parallelism >

[jira] [Updated] (PIG-4734) TOMAP schema inferring breaks some scripts in type checking for bincond

2016-05-31 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4734: Attachment: PIG-4734-3.patch The issue is not limited to map. Tuple/bag has similar issue. Attach a patch to

[jira] [Updated] (PIG-4786) CROSS will not work correctly with Grace Parallelism

2016-05-28 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4786: Fix Version/s: (was: 0.17.0) 0.16.0 > CROSS will not work correctly with Grace

[jira] [Updated] (PIG-4734) TOMAP schema inferring breaks some scripts in type checking for bincond

2016-05-28 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4734: Attachment: PIG-4734-2.patch Use a restrictive way to do the check. GenerateMapFloat is no special in terms

[jira] [Commented] (PIG-3911) Define unique fields with @OutputSchema

2016-05-26 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303097#comment-15303097 ] Daniel Dai commented on PIG-3911: - I think it introduces some backward-incompatible changes, right? >

[jira] [Commented] (PIG-4906) Add Bigdecimal functions in Over function

2016-05-26 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303071#comment-15303071 ] Daniel Dai commented on PIG-4906: - I remember once you've test case in your patch, is that right? > Add

[jira] [Created] (PIG-4910) Assert wrongly pushed up in optimizer

2016-05-24 Thread Daniel Dai (JIRA)
Daniel Dai created PIG-4910: --- Summary: Assert wrongly pushed up in optimizer Key: PIG-4910 URL: https://issues.apache.org/jira/browse/PIG-4910 Project: Pig Issue Type: Bug Reporter:

[jira] [Commented] (PIG-4892) removing /tmp/output before UT

2016-05-24 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298761#comment-15298761 ] Daniel Dai commented on PIG-4892: - PIG-4892-TestLoad.patch committed to both trunk and 0.16 branch. >

[jira] [Updated] (PIG-4892) removing /tmp/output before UT

2016-05-24 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4892: Attachment: PIG-4892-TestLoad.patch Another case in TestLoad. > removing /tmp/output before UT >

[jira] [Updated] (PIG-4902) Fix UT failures on 0.16 branch: TestTezGraceParallelism, TestPigScriptParser

2016-05-24 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4902: Attachment: PIG-4902-3.patch Committed to both 0.16 branch and trunk with a format change. Thanks Rohini for

[jira] [Resolved] (PIG-4714) Improve logging across multiple components with callerId

2016-05-24 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai resolved PIG-4714. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 0.17.0 Patch committed to both trunk

[jira] [Updated] (PIG-4902) Fix UT failures on 0.16 branch: TestTezGraceParallelism, TestPigScriptParser

2016-05-24 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4902: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Fix UT

[jira] [Updated] (PIG-4902) Fix UT failures on 0.16 branch: TestTezGraceParallelism, TestPigScriptParser

2016-05-24 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4902: Attachment: PIG-4902-2.patch You are right, it is not the runtime data size change. It is due to a missing

[jira] [Commented] (PIG-4905) Input of empty dir does not produce empty output part file in Tez

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297301#comment-15297301 ] Daniel Dai commented on PIG-4905: - I misunderstood the issue initially. So you are talking about the part

[jira] [Updated] (PIG-4905) Input of empty dir does not produce empty output part file in Tez

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4905: Summary: Input of empty dir does not produce empty output part file in Tez (was: Input of empty dir does not

[jira] [Commented] (PIG-4900) Using PIG 0.13.0 with HBase 1.1.4(pseudo distributed) in Hadoop 2.5.1. Pig error 1200 encountered while using store command in pig to hbase. Before this load and dump com

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297201#comment-15297201 ] Daniel Dai commented on PIG-4900: - Most likely your ivy-2.2.0.jar is corrupted. Can you remove the jar and

[jira] [Commented] (PIG-4905) Input of empty dir does not produce empty output file in Tez

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297192#comment-15297192 ] Daniel Dai commented on PIG-4905: - I already commented that's fine. One thing still mysterious to me is the

[jira] [Commented] (PIG-4905) Input of empty dir does not produce empty output file in Tez

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297132#comment-15297132 ] Daniel Dai commented on PIG-4905: - Ok, other tests also only check #map, that's fine. > Input of empty dir

[jira] [Commented] (PIG-4905) Input of empty dir does not produce empty output file in Tez

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297115#comment-15297115 ] Daniel Dai commented on PIG-4905: - One more comment, since you already check #maps in MR mode in

[jira] [Commented] (PIG-4908) JythonFunction refers to Oozie launcher script absolute path

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297083#comment-15297083 ] Daniel Dai commented on PIG-4908: - +1. The "starts with cwd" check is very buggy. There are other situations

[jira] [Commented] (PIG-4885) Turn off union optimizer if there is PARALLEL clause in union in Tez

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296925#comment-15296925 ] Daniel Dai commented on PIG-4885: - +1 > Turn off union optimizer if there is PARALLEL clause in union in

[jira] [Issue Comment Deleted] (PIG-4379) Make RoundRobinPartitioner public

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4379: Comment: was deleted (was: +1) > Make RoundRobinPartitioner public > - > >

[jira] [Commented] (PIG-4905) Input of empty dir does not produce empty output file in Tez

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296836#comment-15296836 ] Daniel Dai commented on PIG-4905: - Never mind, it does fail. +1 once address Koji's comment. > Input of

[jira] [Commented] (PIG-4379) Make RoundRobinPartitioner public

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296823#comment-15296823 ] Daniel Dai commented on PIG-4379: - +1 > Make RoundRobinPartitioner public >

[jira] [Commented] (PIG-4905) Input of empty dir does not produce empty output file in Tez

2016-05-23 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296800#comment-15296800 ] Daniel Dai commented on PIG-4905: - The test case pass for me even without change. Is that for a specific

[jira] [Updated] (PIG-4902) Fix UT failures on 0.16 branch: TestTezGraceParallelism, TestPigScriptParser

2016-05-22 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4902: Status: Patch Available (was: Open) > Fix UT failures on 0.16 branch: TestTezGraceParallelism,

[jira] [Updated] (PIG-4902) Fix UT failures on 0.16 branch: TestTezGraceParallelism, TestPigScriptParser

2016-05-22 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4902: Attachment: PIG-4902-1.patch > Fix UT failures on 0.16 branch: TestTezGraceParallelism, TestPigScriptParser >

[jira] [Created] (PIG-4902) Fix UT failures on 0.16 branch: TestTezGraceParallelism, TestPigScriptParser

2016-05-22 Thread Daniel Dai (JIRA)
Daniel Dai created PIG-4902: --- Summary: Fix UT failures on 0.16 branch: TestTezGraceParallelism, TestPigScriptParser Key: PIG-4902 URL: https://issues.apache.org/jira/browse/PIG-4902 Project: Pig

[jira] [Commented] (PIG-4714) Improve logging across multiple components with callerId

2016-05-20 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294338#comment-15294338 ] Daniel Dai commented on PIG-4714: - New patch attached. > Improve logging across multiple components with

[jira] [Updated] (PIG-4714) Improve logging across multiple components with callerId

2016-05-20 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4714: Attachment: PIG-4714-5.patch > Improve logging across multiple components with callerId >

[jira] [Commented] (PIG-4900) Using PIG 0.13.0 with HBase 1.1.4(pseudo distributed) in Hadoop 2.5.1. Pig error 1200 encountered while using store command in pig to hbase. Before this load and dump com

2016-05-20 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293921#comment-15293921 ] Daniel Dai commented on PIG-4900: - Saw /usr/lib/pig/pig-0.13.0/lib/hbase-0.94.1.jar in your classpath, can

[jira] [Commented] (PIG-4892) removing /tmp/output before UT

2016-05-19 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292250#comment-15292250 ] Daniel Dai commented on PIG-4892: - +1 for PIG-4892-fixtest.patch. > removing /tmp/output before UT >

[jira] [Commented] (PIG-4900) Using PIG 0.13.0 with HBase 1.1.4(pseudo distributed) in Hadoop 2.5.1. Pig error 1200 encountered while using store command in pig to hbase. Before this load and dump com

2016-05-19 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292247#comment-15292247 ] Daniel Dai commented on PIG-4900: - It sounds a runtime jar conflict. Let's take a look what hbase in your

[jira] [Updated] (PIG-4714) Improve logging across multiple components with callerId

2016-05-19 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4714: Attachment: PIG-4714-4.patch Fixed Yarn property. Setting hdfs context should be invoked even if the ATS is

[jira] [Commented] (PIG-2315) Make as clause work in generate

2016-05-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15289890#comment-15289890 ] Daniel Dai commented on PIG-2315: - I will try this Friday. > Make as clause work in generate >

[jira] [Updated] (PIG-4714) Improve logging across multiple components with callerId

2016-05-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4714: Attachment: PIG-4714-3.patch bq. 1) pig.ats.enabled=true is the default and this code affects PigServer. With

[jira] [Updated] (PIG-4416) Fix or comment piggybank tests with ExecType.LOCAL

2016-05-17 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4416: Fix Version/s: (was: 0.16.0) 0.17.0 > Fix or comment piggybank tests with

[jira] [Updated] (PIG-4579) casting of primitive datetime data type should work

2016-05-17 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4579: Fix Version/s: (was: 0.16.0) 0.17.0 > casting of primitive datetime data type should

<    1   2   3   4   5   6   7   8   9   10   >