-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32036/
-----------------------------------------------------------
Review request for pig, liyun zhang and Mohit Sabharwal.
Bugs: PIG-4422
https://issues.apache.org/jira/browse/PIG-4422
Repository: pig-git
Description
-------
POMergeJoin operator is added as parent to load operators and a regular join is
performed as part of the initial implementation and the MergeJoinConverter
should later be modified to achieve the specialized join.
TODO:
- Perform join considering the input data is sorted.
- Fix failing test cases in TestMergeJoin
Diffs
-----
src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POMergeJoin.java
87249e4c9d6c890e8ac864c3faea32e3d6aa872d
src/org/apache/pig/backend/hadoop/executionengine/spark/SparkLauncher.java
ca7a45f33320064e22628b40b34be7b9f7b07c36
src/org/apache/pig/backend/hadoop/executionengine/spark/converter/MergeJoinConverter.java
PRE-CREATION
src/org/apache/pig/backend/hadoop/executionengine/spark/plan/SparkCompiler.java
PRE-CREATION
Diff: https://reviews.apache.org/r/32036/diff/
Testing
-------
Tested TestMergeJoin and we now have all tests passing except the following:
- testMergeJoinWithCommaSeperatedFilePaths
- testMergeJoinEmptyIndex
- testMergeJoinOutPipeline
- testExpressionFail
Thanks,
Praveen Rachabattuni
