[
https://issues.apache.org/jira/browse/CALCITE-3122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912781#comment-16912781
]
Khai Tran commented on CALCITE-3122:
------------------------------------
Two reasons I create PigRelPlanner: for use a custom set of rules and to
override getCost method to set preference when apply some rules. Second reason
is not needed to pass all local tests (I might need it sometime for other Pig
scripts at LinkedIn, but I forget those cases now) while we can use some tricks
for the first reason. So now for passing all local tests, PigRelPlanner is not
needed.
There is only one item left, adding unit tests for ToLogicalConverter, which I
will complete tomorrow morning. Hope that you can go over all changes again
since last point to see if code looks good for merging. I'm excited to see this
in Calcite 21.
> Convert Pig Latin scripts into Calcite logical plan
> ----------------------------------------------------
>
> Key: CALCITE-3122
> URL: https://issues.apache.org/jira/browse/CALCITE-3122
> Project: Calcite
> Issue Type: New Feature
> Components: core, piglet
> Reporter: Khai Tran
> Assignee: Julian Hyde
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> We create an internal Calcite repo at LinkedIn and develop APIs to parse any
> Pig Latin scripts into Calcite logical plan. The code was tested in nearly
> ~1000 Pig scripts written at LinkedIn.
> Changes:
> 1. piglet: main conversion code live there, include:
> * APIs to convert any Pig scripts into RelNode plans or SQL statements
> * Use Pig Grunt parser to parse Pig Latin scripts into Pig logical plan
> (DAGs)
> * Convert Pig schemas into RelDatatype
> * Traverse through Pig expression plan and convert Pig expressions into
> RexNodes
> * Map some basic Pig UDFs to Calcite SQL operators
> * Build Calcite UDFs for any other Pig UDFs, including UDFs written in both
> Java and Python
> * Traverse (DFS) through Pig logical plans to convert each Pig logical nodes
> to RelNodes
> * Have an optimizer rule to optimize Pig group/cogroup into Aggregate
> operators
> 2. core:
> * Implement other RelNode in Rel2Sql so that Pig can be translated into SQL
> * Other minor changes in a few other classes to make Pig to Calcite works
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
