GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/1743
STORM-2148 [Storm SQL] Trident mode: back to code generate and compile
Trident topology
This is on top of STORM-1446 (#1736).
Instead of fully generating topology code, I just generate classes for each
expression, and let EvaluateXXX calls instance's method. This is much simpler
and nearly no overhead added, and also much faster than evaluating code block.
I did a [simple
benchmark](https://issues.apache.org/jira/browse/STORM-2148?focusedCommentId=15601152&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15601152)
between evaluating code block and calling instance's method. Please refer the
link if you are interested.
Also manually tested.
You can narrow the change to only last commit. It's 700 lines of change.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/storm
STORM-2148-on-top-of-STORM-1446
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1743.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1743
----
commit e1da468267d73c50ddc8c11ff8b3444949543c63
Author: Jungtaek Lim <kabh...@gmail.com>
Date: 2016-10-13T10:00:10Z
STORM-1446 Compile the Calcite logical plan to Storm Trident logical plan
* Port SamzaSQL implementation to Storm
* https://github.com/milinda/samza-sql
* Apply some rules to optimize
* optimize Calc
* merge filter and projection scripts into one
* also applying short circuit
* Modify Trident unit tests to use new query planner
* arrange some files
* Move some files which are only used from standalone
* Remove some files which are no longer used
* guard the possibility of stack overflow error on explaining
* just leave error logs, and print out empty plan and continue
* reported this behavior to Calcite community
* leave some comments to clarify what it means
commit 68b80fae752f5f7d2824f50dec195d03f3520d6f
Author: Jungtaek Lim <kabh...@gmail.com>
Date: 2016-10-24T12:27:07Z
STORM-2148 [Storm SQL] Trident mode: back to code generate and compile
Trident topology
* Change RexNodeToBlockStatementCompiler (also rename)
* For standalone mode, it still returns BlockStatement which contains
code block
* For Trident mode, it returns String which is a "class" implementation
* Replace evaluation of code block to instance method call
* Package compiled classes to be included to jar
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
