[jira] [Commented] (PIG-4417) Pig's register command should support automatic fetching of jars from repo.

[ https://issues.apache.org/jira/browse/PIG-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344711#comment-14344711 ] Ratandeep Ratti commented on PIG-4417: -- I think we should refrain from configuring the

[jira] [Commented] (PIG-4417) Pig's register command should support automatic fetching of jars from repo.

[ https://issues.apache.org/jira/browse/PIG-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344709#comment-14344709 ] Akshay Rai commented on PIG-4417: - RB Link: https://reviews.apache.org/r/31662/ Pig's

Re:Re:Re: use hcatalog in eclipse/pig

2015-03-03 19:54:43,072 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1421806758143_0252_01 2015-03-03 19:54:43,623 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter:

Re: use hcatalog in eclipse/pig

HCatSchema is in hcatalog-core.jar and it is missing in the backend. You will need to register several jars in pig script: hcatalog-core.jar hive-metastore.jar libthrift.jar hive-core.jar libfb303.jar jdo*-api.jar slf4j-api.jar When you run in command line, bin/pig script does that for

[jira] [Commented] (PIG-4442) Eliminate redundant RPC call to get file information in HPath.

[ https://issues.apache.org/jira/browse/PIG-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345522#comment-14345522 ] Chris Nauroth commented on PIG-4442: Thanks for the reviews and commit, Daniel and

[jira] [Commented] (PIG-4412) Race condition in writing multiple outputs from STREAM op

[ https://issues.apache.org/jira/browse/PIG-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345566#comment-14345566 ] Daniel Dai commented on PIG-4412: - How about making ExecutableManager.close synchronized?

[jira] [Commented] (PIG-4408) Merge join should support replicated join as a predecessor

[ https://issues.apache.org/jira/browse/PIG-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345402#comment-14345402 ] Daniel Dai commented on PIG-4408: - The test failure is due to PIG-4431 not this patch. I

[jira] [Updated] (PIG-4431) ReadToEndLoader does not close the record reader for the last input split

[ https://issues.apache.org/jira/browse/PIG-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4431: Attachment: PIG-4431-1.patch This patch breaks TestCollectedGroup. Attach a fix. ReadToEndLoader does not

[jira] [Commented] (PIG-4431) ReadToEndLoader does not close the record reader for the last input split

[ https://issues.apache.org/jira/browse/PIG-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345451#comment-14345451 ] Daniel Dai commented on PIG-4431: - PIG-4431-1.patch committed. ReadToEndLoader does not

Re: a question about TestAccumulator#testAccumBasic

In first case Accumulator is get used and the second case is not (believe due to the second UDF BagCount in the plan). There are certain conditions whether Accumulator can be used in the plan or not, the logic is in AccumulatorOptimizer. Daniel On 3/3/15, 12:40 AM, Zhang, Liyun

newbie unable to compile pig svn + eclipse + IvyDE

I am a newbie trying to get unmodified pig trunk to compile in eclipse. My issues must be a FAQ, but I have not found solution in the mailing list archive. I followed simple instructions at: https://cwiki.apache.org/confluence/display/PIG/How+to+set+up+Eclipse+environment I used svn to check

[jira] Subscription: PIG patch available

Issue Subscription Filter: PIG patch available (21 issues) Subscriber: pigdaily Key Summary PIG-Fix unit test failure TestTezAutoParallelism https://issues.apache.org/jira/browse/PIG- PIG-4377Skewed outer join produce wrong result in some cases

[jira] [Commented] (PIG-4444) Fix unit test failure TestTezAutoParallelism

[ https://issues.apache.org/jira/browse/PIG-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346459#comment-14346459 ] Daniel Dai commented on PIG-: - The size of the special mark is 144 bytes. However, there

[jira] [Commented] (PIG-4444) Fix unit test failure TestTezAutoParallelism

[ https://issues.apache.org/jira/browse/PIG-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346199#comment-14346199 ] Rohini Palaniswamy commented on PIG-: - How much was the special tuple size? If

[jira] [Updated] (PIG-4445) VALUELIST and VALUESET outputSchema does not match actual schema of data returned when map value schema is complex

[ https://issues.apache.org/jira/browse/PIG-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-4445: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Patch

Re:Re: use hcatalog in eclipse/pig

I have added all needed jars to eclipse class path including hive-metastore.jar libthrift.jar hive-core.jar libfb303.jar jdo*-api.jar slf4j-api.jar ,can you tell me how can I register these jars in pig script? use setProperty? or others? At 2015-03-04 02:34:05, Daniel Dai

Re:Re: use hcatalog in eclipse/pig

I have added all needed jars to eclipse class path including hive-metastore.jar libthrift.jar hive-core.jar libfb303.jar jdo*-api.jar slf4j-api.jar ,can you tell me how can I register these jars in pig script? use setProperty? or others? At 2015-03-04 02:34:05, Daniel Dai

[jira] [Commented] (PIG-4193) Make collected group work with Spark

[ https://issues.apache.org/jira/browse/PIG-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346127#comment-14346127 ] liyunzhang_intel commented on PIG-4193: --- Hi [~praveenr019]: If the only problem for

[jira] [Updated] (PIG-4444) Fix unit test failure TestTezAutoParallelism

[ https://issues.apache.org/jira/browse/PIG-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-: Attachment: PIG--1.patch Fix unit test failure TestTezAutoParallelism

[jira] [Created] (PIG-4444) Fix unit test failure TestTezAutoParallelism

Daniel Dai created PIG-: --- Summary: Fix unit test failure TestTezAutoParallelism Key: PIG- URL: https://issues.apache.org/jira/browse/PIG- Project: Pig Issue Type: Bug

RE: a question about TestAccumulator#testAccumBasic

Hi Daniel: Thanks for your comment and I have figured out why org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator#isAccumulative() is false in second case. Best regards Zhang,Liyun -Original Message- From: Daniel Dai [mailto:da...@hortonworks.com] Sent:

[jira] [Commented] (PIG-4442) Eliminate redundant RPC call to get file information in HPath.

[ https://issues.apache.org/jira/browse/PIG-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346251#comment-14346251 ] Prashant Kommireddi commented on PIG-4442: -- Schumi, I shall get you one day!

[jira] [Updated] (PIG-4445) VALUELIST and VALUESET outputSchema does not match actual schema of data returned when map value schema is complex

[ https://issues.apache.org/jira/browse/PIG-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Hsu updated PIG-4445: - Fix Version/s: 0.15.0 Affects Version/s: 0.11.1 Status: Patch Available (was: Open)

[jira] [Created] (PIG-4445) VALUELIST and VALUESET outputSchema does not match actual schema of data returned when map value schema is complex

Anthony Hsu created PIG-4445: Summary: VALUELIST and VALUESET outputSchema does not match actual schema of data returned when map value schema is complex Key: PIG-4445 URL:

[jira] [Updated] (PIG-4445) VALUELIST and VALUESET outputSchema does not match actual schema of data returned when map value schema is complex

[ https://issues.apache.org/jira/browse/PIG-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Hsu updated PIG-4445: - Attachment: PIG-4445.1.patch VALUELIST and VALUESET outputSchema does not match actual schema of data

[jira] [Commented] (PIG-4408) Merge join should support replicated join as a predecessor

[ https://issues.apache.org/jira/browse/PIG-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14345115#comment-14345115 ] Brian Johnson commented on PIG-4408: That's very strange. Daniel's patch looks like it's

[jira] [Updated] (PIG-4193) Make collected group work with Spark

[ https://issues.apache.org/jira/browse/PIG-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Praveen Rachabattuni updated PIG-4193: -- Status: Open (was: Patch Available) Make collected group work with Spark

[jira] [Commented] (PIG-4193) Make collected group work with Spark

[ https://issues.apache.org/jira/browse/PIG-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14344812#comment-14344812 ] Praveen Rachabattuni commented on PIG-4193: --- Currently, PIG-4193-1.patch makes a

RE: a question about TestAccumulator#testAccumBasic

Hi Daniel: Thanks for your reply! According to your comment: The first case will use Accumulator, so accumulate - cleanup will be called, but no exec. The second case will not use Accumulator, exec will be called instead of accumulate - cleanup. I guess what you mean is that If