[jira] Subscription: PIG patch available

[jira]

Issue Subscription
Filter: PIG patch available (33 issues)

Subscriber: pigdaily

Key Summary
PIG-4524Pig Minicluster unit tests broken by TEZ-2333
https://issues.apache.org/jira/browse/PIG-4524
PIG-4506binstorage fails to write biginteger
https://issues.apache.org/jira/browse/PIG-4506
PIG-4504Enable Secondary key sort feature in spark mode
https://issues.apache.org/jira/browse/PIG-4504
PIG-4496Fix CBZip2InputStream to close underlying stream
https://issues.apache.org/jira/browse/PIG-4496
PIG-4490MIN/MAX builtin UDFs return wrong results when accumulating for 
strings
https://issues.apache.org/jira/browse/PIG-4490
PIG-4481e2e tests ComputeSpec_1, ComputeSpec_2, StreamingPerformance_3 and  
StreamingPerformance_4 produce different result on Windows
https://issues.apache.org/jira/browse/PIG-4481
PIG-4468Pig's jackson version conflicts with that of hadoop 2.6.0
https://issues.apache.org/jira/browse/PIG-4468
PIG-4455Should use DependencyOrderWalker instead of DepthFirstWalker in 
MRPrinter
https://issues.apache.org/jira/browse/PIG-4455
PIG-4452Embedded SQL using "SQL" instead of "sql" fails with string index 
out of range: -1 error
https://issues.apache.org/jira/browse/PIG-4452
PIG-4422Implement visitMergeJoin in SparkCompiler
https://issues.apache.org/jira/browse/PIG-4422
PIG-4418NullPointerException in JVMReuseImpl
https://issues.apache.org/jira/browse/PIG-4418
PIG-4417Pig's register command should support automatic fetching of jars 
from repo.
https://issues.apache.org/jira/browse/PIG-4417
PIG-4377Skewed outer join produce wrong result in some cases
https://issues.apache.org/jira/browse/PIG-4377
PIG-4365TOP udf should implement Accumulator interface
https://issues.apache.org/jira/browse/PIG-4365
PIG-4341Add CMX support to pig.tmpfilecompression.codec
https://issues.apache.org/jira/browse/PIG-4341
PIG-4323PackageConverter hanging in Spark
https://issues.apache.org/jira/browse/PIG-4323
PIG-4313StackOverflowError in LIMIT operation on Spark
https://issues.apache.org/jira/browse/PIG-4313
PIG-4295Enable unit test "TestPigContext" for spark
https://issues.apache.org/jira/browse/PIG-4295
PIG-4276Fix ordering related failures in TestEvalPipeline for Spark
https://issues.apache.org/jira/browse/PIG-4276
PIG-4251Pig on Storm
https://issues.apache.org/jira/browse/PIG-4251
PIG-4193Make collected group work with Spark
https://issues.apache.org/jira/browse/PIG-4193
PIG-4111Make Pig compiles with avro-1.7.7
https://issues.apache.org/jira/browse/PIG-4111
PIG-4004Upgrade the Pigmix queries from the (old) mapred API to mapreduce
https://issues.apache.org/jira/browse/PIG-4004
PIG-4002Disable combiner when map-side aggregation is used
https://issues.apache.org/jira/browse/PIG-4002
PIG-3952PigStorage accepts '-tagSplit' to return full split information
https://issues.apache.org/jira/browse/PIG-3952
PIG-3911Define unique fields with @OutputSchema
https://issues.apache.org/jira/browse/PIG-3911
PIG-3877Getting Geo Latitude/Longitude from Address Lines
https://issues.apache.org/jira/browse/PIG-3877
PIG-3873Geo distance calculation using Haversine
https://issues.apache.org/jira/browse/PIG-3873
PIG-3866Create ThreadLocal classloader per PigContext
https://issues.apache.org/jira/browse/PIG-3866
PIG-3851Upgrade jline to 2.11
https://issues.apache.org/jira/browse/PIG-3851
PIG-3668COR built-in function when atleast one of the coefficient values is 
NaN
https://issues.apache.org/jira/browse/PIG-3668
PIG-3635Fix e2e tests for Hadoop 2.X on Windows
https://issues.apache.org/jira/browse/PIG-3635
PIG-3587add functionality for rolling over dates
https://issues.apache.org/jira/browse/PIG-3587

You may edit this subscription at:
https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=16328&filterId=12322384

[jira] [Updated] (PIG-4525) Clarify "Scalar has more than one row in the output."

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-4525:

Fix Version/s: (was: 0.15.0)
   0.16.0

> Clarify "Scalar has more than one row in the output."
> -
>
> Key: PIG-4525
> URL: https://issues.apache.org/jira/browse/PIG-4525
> Project: Pig
>  Issue Type: Improvement
>Reporter: Niels Basjes
>Assignee: Niels Basjes
>Priority: Trivial
> Fix For: 0.16.0
>
> Attachments: PIG-4525-2015-04-30-1115.patch
>
>
> The exception "Scalar has more than one row in the output." is correct yet is 
> reason for many (starting) pig developers to search the internet for a 
> solution.
> I propose (and I'll include a patch) to simply extend the exception message 
> with a hint towards the right solution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PIG-4525) Clarify "Scalar has more than one row in the output."

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated PIG-4525:

   Resolution: Fixed
Fix Version/s: 0.15.0
   Status: Resolved  (was: Patch Available)

Patch committed.  Thanks Niels.

> Clarify "Scalar has more than one row in the output."
> -
>
> Key: PIG-4525
> URL: https://issues.apache.org/jira/browse/PIG-4525
> Project: Pig
>  Issue Type: Improvement
>Reporter: Niels Basjes
>Assignee: Niels Basjes
>Priority: Trivial
> Fix For: 0.15.0
>
> Attachments: PIG-4525-2015-04-30-1115.patch
>
>
> The exception "Scalar has more than one row in the output." is correct yet is 
> reason for many (starting) pig developers to search the internet for a 
> solution.
> I propose (and I'll include a patch) to simply extend the exception message 
> with a hint towards the right solution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Jenkins build is back to normal : Pig-trunk #1787

[Apache Jenkins Server]

See 

[jira] [Updated] (PIG-4527) NON-ASCII Characters in Javadoc break 'ant docs'

[Daniel Dai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated PIG-4527:

   Resolution: Fixed
Fix Version/s: 0.15.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Patch committed to both 0.15 branch and trunk. Thanks Niels!

> NON-ASCII Characters in Javadoc break 'ant docs'
> 
>
> Key: PIG-4527
> URL: https://issues.apache.org/jira/browse/PIG-4527
> Project: Pig
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Fix For: 0.15.0
>
> Attachments: PIG-4527-2015-04-30-1657.patch
>
>
> Found while testing PIG-4526
> {code}contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/IsInt.java
>  {code}
> contains
> {code}- * Note this function checks for Integer range 
> <88><92>2,147,483,648 to 2,147,483,647.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PIG-4315) MergeJoin followed by order by gives NPE in Tez

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-4315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521867#comment-14521867
 ] 

Daniel Dai commented on PIG-4315:
-

Oh, yes, sorry, I just notice you skip this test in Tez.

> MergeJoin followed by order by gives NPE in Tez
> ---
>
> Key: PIG-4315
> URL: https://issues.apache.org/jira/browse/PIG-4315
> Project: Pig
>  Issue Type: Bug
>  Components: tez
>Reporter: Rohini Palaniswamy
> Fix For: 0.15.0
>
>
> TestHBaseStorage.testMergeJoin() fails. connectingLR is null in 
> CombinerOptimizer and throws NPE in 
> {code}
> from.plan.getOperator(connectingLR.getOperatorKey())
> {code}
> When splitting DAGs and doing moveTree TezCompilerUtil.connect() overwrites 
> the output key of sampler LocalRearrange to the partitioner vertex.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: Pig-trunk-commit #2112

[Apache Jenkins Server]

See 

--
[...truncated 4400 lines...]
[junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
35.195 sec
[junit] Running org.apache.pig.test.TestNewPlanListener
[junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.369 sec
[junit] Running org.apache.pig.test.TestNewPlanLogToPhyTranslationVisitor
[junit] Tests run: 27, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
5.139 sec
[junit] Running org.apache.pig.test.TestNewPlanLogicalOptimizer
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
1.449 sec
[junit] Running org.apache.pig.test.TestNewPlanOperatorPlan
[junit] Tests run: 47, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
4.443 sec
[junit] Running org.apache.pig.test.TestNewPlanPruneMapKeys
[junit] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
2.995 sec
[junit] Running org.apache.pig.test.TestNewPlanPushDownForeachFlatten
[junit] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
7.562 sec
[junit] Running org.apache.pig.test.TestNewPlanPushUpFilter
[junit] Tests run: 46, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
7.475 sec
[junit] Running org.apache.pig.test.TestNewPlanRule
[junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.334 sec
[junit] Running org.apache.pig.test.TestNotEqualTo
[junit] Tests run: 28, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.451 sec
[junit] Running org.apache.pig.test.TestNull
[junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.441 sec
[junit] Running org.apache.pig.test.TestNullConstant
[junit] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
24.999 sec
[junit] Running org.apache.pig.test.TestNumberOfReducers
[junit] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
514.19 sec
[junit] Running org.apache.pig.test.TestOptimizeLimit
[junit] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
3.493 sec
[junit] Running org.apache.pig.test.TestOrderBy3
[junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
14.49 sec
[junit] Running org.apache.pig.test.TestPOBinCond
[junit] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.464 sec
[junit] Running org.apache.pig.test.TestPOCast
[junit] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.881 sec
[junit] Running org.apache.pig.test.TestPODistinct
[junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.398 sec
[junit] Running org.apache.pig.test.TestPOGenerate
[junit] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.37 sec
[junit] Running org.apache.pig.test.TestPOMapLookUp
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.364 sec
[junit] Running org.apache.pig.test.TestPONegative
[junit] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
5.766 sec
[junit] Running org.apache.pig.test.TestPOPartialAgg
[junit] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
4.11 sec
[junit] Running org.apache.pig.test.TestPOPartialAggPlan
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 
0.245 sec
[junit] Running org.apache.pig.test.TestPORegexp
[junit] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.381 sec
[junit] Running org.apache.pig.test.TestPOSort
[junit] Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.469 sec
[junit] Running org.apache.pig.test.TestPOSplit
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.376 sec
[junit] Running org.apache.pig.test.TestPOUserFunc
[junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.462 sec
[junit] Running org.apache.pig.test.TestPackage
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
7.679 sec
[junit] Running org.apache.pig.test.TestParamSubPreproc
[junit] Tests run: 36, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
4.714 sec
[junit] Running org.apache.pig.test.TestParser
[junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
5.195 sec
[junit] Running org.apache.pig.test.TestPhyOp
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.434 sec
[junit] Running org.apache.pig.test.TestPhyPatternMatch
[junit] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
0.36 sec
[junit] Running org.apache.pig.test.TestPigContext
[junit] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 
60.939 sec
[junit] Running org.apache.pig.test.TestPigContextClassCache
[junit] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed

[jira] [Updated] (PIG-4527) NON-ASCII Characters in Javadoc break 'ant docs'

[Niels Basjes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated PIG-4527:
--
Description: 
Found while testing PIG-4526
{code}contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/IsInt.java
 {code}
contains
{code}- * Note this function checks for Integer range <88><92>2,147,483,648 
to 2,147,483,647.
{code}

  was:
{code}contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/IsInt.java
 {code}
contains
{code}- * Note this function checks for Integer range <88><92>2,147,483,648 
to 2,147,483,647.
{code}


> NON-ASCII Characters in Javadoc break 'ant docs'
> 
>
> Key: PIG-4527
> URL: https://issues.apache.org/jira/browse/PIG-4527
> Project: Pig
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: PIG-4527-2015-04-30-1657.patch
>
>
> Found while testing PIG-4526
> {code}contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/IsInt.java
>  {code}
> contains
> {code}- * Note this function checks for Integer range 
> <88><92>2,147,483,648 to 2,147,483,647.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PIG-4526) Make setting up the build environment easier

[Niels Basjes (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521607#comment-14521607
 ] 

Niels Basjes commented on PIG-4526:
---

Submitted bug report and patch for this issue: PIG-4527

> Make setting up the build environment easier
> 
>
> Key: PIG-4526
> URL: https://issues.apache.org/jira/browse/PIG-4526
> Project: Pig
>  Issue Type: New Feature
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: PIG-4526-2015-04-30-1632.patch
>
>
> In AVRO-1537 and HADOOP-11843 a docker based solution was created to setup 
> all the tools for doing a full build. This enables much easier reproduction 
> of any issues and getting up and running for new developers.
> This issue is to 'copy/port' that setup into the pig project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PIG-4527) NON-ASCII Characters in Javadoc break 'ant docs'

[Niels Basjes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated PIG-4527:
--
Attachment: PIG-4527-2015-04-30-1657.patch

> NON-ASCII Characters in Javadoc break 'ant docs'
> 
>
> Key: PIG-4527
> URL: https://issues.apache.org/jira/browse/PIG-4527
> Project: Pig
>  Issue Type: Bug
>Reporter: Niels Basjes
> Attachments: PIG-4527-2015-04-30-1657.patch
>
>
> {code}contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/IsInt.java
>  {code}
> contains
> {code}- * Note this function checks for Integer range 
> <88><92>2,147,483,648 to 2,147,483,647.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PIG-4527) NON-ASCII Characters in Javadoc break 'ant docs'

[Niels Basjes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated PIG-4527:
--
Assignee: Niels Basjes
  Status: Patch Available  (was: Open)

> NON-ASCII Characters in Javadoc break 'ant docs'
> 
>
> Key: PIG-4527
> URL: https://issues.apache.org/jira/browse/PIG-4527
> Project: Pig
>  Issue Type: Bug
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: PIG-4527-2015-04-30-1657.patch
>
>
> {code}contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/IsInt.java
>  {code}
> contains
> {code}- * Note this function checks for Integer range 
> <88><92>2,147,483,648 to 2,147,483,647.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PIG-4527) NON-ASCII Characters in Javadoc break 'ant docs'

[Niels Basjes (JIRA)]

Niels Basjes created PIG-4527:
-

 Summary: NON-ASCII Characters in Javadoc break 'ant docs'
 Key: PIG-4527
 URL: https://issues.apache.org/jira/browse/PIG-4527
 Project: Pig
  Issue Type: Bug
Reporter: Niels Basjes


{code}contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/IsInt.java
 {code}
contains
{code}- * Note this function checks for Integer range <88><92>2,147,483,648 
to 2,147,483,647.
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PIG-4526) Make setting up the build environment easier

[Niels Basjes (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521603#comment-14521603
 ] 

Niels Basjes commented on PIG-4526:
---

Please test and post your findings.
I just found that running {code}ant docs{code} shows non-ACSII characters in 
the Javadoc in 
{code}
contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/IsInt.java

* Note this function checks for Integer range <88><92>2,147,483,648 to 
2,147,483,647.
{code}

> Make setting up the build environment easier
> 
>
> Key: PIG-4526
> URL: https://issues.apache.org/jira/browse/PIG-4526
> Project: Pig
>  Issue Type: New Feature
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: PIG-4526-2015-04-30-1632.patch
>
>
> In AVRO-1537 and HADOOP-11843 a docker based solution was created to setup 
> all the tools for doing a full build. This enables much easier reproduction 
> of any issues and getting up and running for new developers.
> This issue is to 'copy/port' that setup into the pig project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PIG-4526) Make setting up the build environment easier

[Michael Howard (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521599#comment-14521599
 ] 

Michael Howard commented on PIG-4526:
-

Niels,
I am a pig newbie who has recently struggled with getting a dev environment set 
up. 
Thank you for your work to make it easier for developers to get started. 


> Make setting up the build environment easier
> 
>
> Key: PIG-4526
> URL: https://issues.apache.org/jira/browse/PIG-4526
> Project: Pig
>  Issue Type: New Feature
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: PIG-4526-2015-04-30-1632.patch
>
>
> In AVRO-1537 and HADOOP-11843 a docker based solution was created to setup 
> all the tools for doing a full build. This enables much easier reproduction 
> of any issues and getting up and running for new developers.
> This issue is to 'copy/port' that setup into the pig project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PIG-4526) Make setting up the build environment easier

[Niels Basjes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated PIG-4526:
--
Attachment: PIG-4526-2015-04-30-1632.patch

First draft patch that can build the entire project.
Needs further testing.

> Make setting up the build environment easier
> 
>
> Key: PIG-4526
> URL: https://issues.apache.org/jira/browse/PIG-4526
> Project: Pig
>  Issue Type: New Feature
>Reporter: Niels Basjes
>Assignee: Niels Basjes
> Attachments: PIG-4526-2015-04-30-1632.patch
>
>
> In AVRO-1537 and HADOOP-11843 a docker based solution was created to setup 
> all the tools for doing a full build. This enables much easier reproduction 
> of any issues and getting up and running for new developers.
> This issue is to 'copy/port' that setup into the pig project.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PIG-4526) Make setting up the build environment easier

[Niels Basjes (JIRA)]

Niels Basjes created PIG-4526:
-

 Summary: Make setting up the build environment easier
 Key: PIG-4526
 URL: https://issues.apache.org/jira/browse/PIG-4526
 Project: Pig
  Issue Type: New Feature
Reporter: Niels Basjes
Assignee: Niels Basjes


In AVRO-1537 and HADOOP-11843 a docker based solution was created to setup all 
the tools for doing a full build. This enables much easier reproduction of any 
issues and getting up and running for new developers.

This issue is to 'copy/port' that setup into the pig project.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PIG-4525) Clarify "Scalar has more than one row in the output."

[Niels Basjes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated PIG-4525:
--
Attachment: PIG-4525-2015-04-30-1115.patch

This patch simply changes the exception message from 
{code}java.lang.Exception: 
org.apache.pig.backend.executionengine.ExecException: ERROR 0: Scalar has more 
than one row in the output. 1st : (key,1), 2nd :(key,2){code}
to
{code}java.lang.Exception: 
org.apache.pig.backend.executionengine.ExecException: ERROR 0: Scalar has more 
than one row in the output. 1st : (key,1), 2nd :(key,2) (common cause: "JOIN" 
then "FOREACH ... GENERATE foo.bar" should be "foo::bar" )  
  
{code}

> Clarify "Scalar has more than one row in the output."
> -
>
> Key: PIG-4525
> URL: https://issues.apache.org/jira/browse/PIG-4525
> Project: Pig
>  Issue Type: Improvement
>Reporter: Niels Basjes
>Assignee: Niels Basjes
>Priority: Trivial
> Attachments: PIG-4525-2015-04-30-1115.patch
>
>
> The exception "Scalar has more than one row in the output." is correct yet is 
> reason for many (starting) pig developers to search the internet for a 
> solution.
> I propose (and I'll include a patch) to simply extend the exception message 
> with a hint towards the right solution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PIG-4525) Clarify "Scalar has more than one row in the output."

[Niels Basjes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Niels Basjes updated PIG-4525:
--
Status: Patch Available  (was: Open)

> Clarify "Scalar has more than one row in the output."
> -
>
> Key: PIG-4525
> URL: https://issues.apache.org/jira/browse/PIG-4525
> Project: Pig
>  Issue Type: Improvement
>Reporter: Niels Basjes
>Assignee: Niels Basjes
>Priority: Trivial
> Attachments: PIG-4525-2015-04-30-1115.patch
>
>
> The exception "Scalar has more than one row in the output." is correct yet is 
> reason for many (starting) pig developers to search the internet for a 
> solution.
> I propose (and I'll include a patch) to simply extend the exception message 
> with a hint towards the right solution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PIG-4525) Clarify "Scalar has more than one row in the output."

[Niels Basjes (JIRA)]

Niels Basjes created PIG-4525:
-

 Summary: Clarify "Scalar has more than one row in the output."
 Key: PIG-4525
 URL: https://issues.apache.org/jira/browse/PIG-4525
 Project: Pig
  Issue Type: Improvement
Reporter: Niels Basjes
Assignee: Niels Basjes
Priority: Trivial


The exception "Scalar has more than one row in the output." is correct yet is 
reason for many (starting) pig developers to search the internet for a solution.

I propose (and I'll include a patch) to simply extend the exception message 
with a hint towards the right solution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PIG-4421) implement visitSkewedJoin in SparkCompiler

[liyunzhang_intel (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PIG-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

liyunzhang_intel updated PIG-4421:
--
Attachment: PIG-4421_4.patch

after PIG-4518(SparkOperator should correspond to complete Spark job). we can 
directly add POSkewedJoin to sparkPlan not divide it to  LRA, GRA, PKG and 
FOREACH to fix TestSkewedJoin#testSkewedJoinWithGroup. 

in PIG-4421_4.patch, directly add skewedjoin to sparkplan. After testing, 
following unit tests about TestSkewedJoin fails:
TestSkewedJoin.testSkewedJoinKeyPartition (this fails in PIG-4421_3.patch)

TestSkewedJoin.testSkewedJoinManyReducers( this does not fail in 
PIG-4421_3.patch, need to be investigated)

> implement visitSkewedJoin in SparkCompiler
> --
>
> Key: PIG-4421
> URL: https://issues.apache.org/jira/browse/PIG-4421
> Project: Pig
>  Issue Type: Sub-task
>  Components: spark
>Reporter: liyunzhang_intel
>Assignee: liyunzhang_intel
> Fix For: spark-branch
>
> Attachments: PIG-4421.patch, PIG-4421_2.patch, PIG-4421_3.patch, 
> PIG-4421_4.patch
>
>
> If visitSkewedJoin is not implemented, following unittests will fail.
> org.apache.pig.test.TestSkewedJoin.testSkewedJoinWithGroup
> org.apache.pig.test.TestSkewedJoin.testSkewedJoinMapKey
> org.apache.pig.test.TestSkewedJoin.testSkewedJoinManyReducers
> org.apache.pig.test.TestSkewedJoin.testNonExistingInputPathInSkewJoin
> org.apache.pig.test.TestSkewedJoin.testSkewedJoinOneValue
> org.apache.pig.test.TestSkewedJoin.testSkewedJoinWithNoProperties
> org.apache.pig.test.TestSkewedJoin.testSkewedJoinEmptyInput
> org.apache.pig.test.TestSkewedJoin.testSkewedJoinNullKeys
> org.apache.pig.test.TestSkewedJoin.testSkewedJoinOuter
> org.apache.pig.test.TestSkewedJoin.testRecursiveFileListing
> org.apache.pig.test.TestSkewedJoin.testSkewedJoinReducers
> org.apache.pig.test.TestJoinSmoke.testSkewedJoinWithGroup
> org.apache.pig.test.TestJoinSmoke.testSkewedJoinOuter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)