[GitHub] storm issue #1703: [STORM-1057] Add throughput metrics to spouts/bolts and d...

2016-09-26 Thread wangli1426
Github user wangli1426 commented on the issue: https://github.com/apache/storm/pull/1703 @HeartSaVioR Can you review this PR? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] storm issue #1714: STORM-2125 Use Calcite's implementation of Rex Compiler

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1714 @arunmahadevan Other than that I addressed your review comments. Thanks for the thoughtful review. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] storm issue #1714: STORM-2125 Use Calcite's implementation of Rex Compiler

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1714 @arunmahadevan Let me summary regarding nested map/array support. * accessing nested element in nested map/array works without modifying Calcite. * it should be defined as

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80594627 --- Diff: pom.xml --- @@ -264,7 +264,7 @@ 4.11 2.5.1 2.1.7 -1.4.0-incubating +1.9.0

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80593580 --- Diff: external/sql/storm-sql-runtime/src/jvm/org/apache/calcite/interpreter/StormDataContext.java --- @@ -0,0 +1,79 @@ +/** + * Licensed to t

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80591597 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/compiler/TestCompilerUtils.java --- @@ -153,6 +158,29 @@ public static CalciteState s

Re: [DISCUSS] Plan for merge SQE and Storm SQL

2016-09-26 Thread Jungtaek Lim
Great! For storm-redis, we might need to modify key/value mapper to use byte[] rather than String. When I co-authored storm-redis, I forgot considering binary format of serde. If we want to address that part, we can also address it. 2016년 9월 27일 (화) 오전 7:19, Morrigan Jones 님이 작성: > Sure, when I

Re: [DISCUSS] Plan for merge SQE and Storm SQL

2016-09-26 Thread Morrigan Jones
Sure, when I can. Storm-kafka should be pretty easy. The storm-redis one will require more work to make it more complete. On Mon, Sep 26, 2016 at 6:09 PM, P. Taylor Goetz wrote: > Thanks for the explanation Morrigan! > > Would you be willing to provide a pull request or patch so the community >

[GitHub] storm issue #1689: STORM-2099 Introduce new sql external module: storm-sql-r...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1689 Rebased to reflect merged STORM-2089. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

Re: [DISCUSS] Plan for merge SQE and Storm SQL

2016-09-26 Thread P. Taylor Goetz
Thanks for the explanation Morrigan! Would you be willing to provide a pull request or patch so the community can review? It sounds like at least some of the changes you mention could be useful to the broader community (beyond the SQL effort). Thanks again, -Taylor > On Sep 26, 2016, at 4:40

[GitHub] storm pull request #1682: STORM-2089 Replace Consumer of ISqlTridentDataSour...

2016-09-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/1682 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] storm issue #1710: STORM-1546: Adding Read and Write Aggregations for Pacema...

2016-09-26 Thread knusbaum
Github user knusbaum commented on the issue: https://github.com/apache/storm/pull/1710 Addressed your comments. Not sure why they have not disappeared. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

Re: [DISCUSS] Plan for merge SQE and Storm SQL

2016-09-26 Thread Morrigan Jones
storm-kafka - This is needed because storm-kafka does not provide a scheme class that gives you the key, value (payload), partition, and offset. MessageMetadataScheme.java comes comes closest, but is missing the key. This was a pretty simple change on my part. storm-redis - This is needed for prop

Re: [DISCUSS] Plan for merge SQE and Storm SQL

2016-09-26 Thread P. Taylor Goetz
Sounds good. I’ll find out if it builds against 2.x. If so I’ll go that direction. Otherwise I’ll come back with my findings and we can discuss it further. I notice there are jars in the git repo that we obviously can’t import. They look like they might be custom JWPlayer builds of storm-kafka

Re: [DISCUSS] Plan for merge SQE and Storm SQL

2016-09-26 Thread Bobby Evans
Does it compile against 2.X?  If so I would prefer to have it go there, and then possibly 1.x if people what it there too. - Bobby On Monday, September 26, 2016 12:47 PM, P. Taylor Goetz wrote: The IP Clearance vote has passed and we are now able to import the SQE code. The question n

Re: [DISCUSS] Plan for merge SQE and Storm SQL

2016-09-26 Thread P. Taylor Goetz
The IP Clearance vote has passed and we are now able to import the SQE code. The question now is to where do we want to import the code? My inclination is to import it to “external” in the 1.x branch. It can be ported to other branches as necessary/if desired. An alternative would be to treat i

[GitHub] storm issue #1713: Storm 2124 show requested cpu mem for each component

2016-09-26 Thread raghavgautam
Github user raghavgautam commented on the issue: https://github.com/apache/storm/pull/1713 @HeartSaVioR I have started a conversation with travis-ci support, to see what we can do. I will update you on the progress. --- If your project is set up for it, you can reply to this email an

Re: [DISCUSS] Plan for merge SQE and Storm SQL

2016-09-26 Thread Morrigan Jones
Thanks Jungtaek for the analysis. We've been super busy the past week, but I will perform my own analysis this week. On Wed, Sep 21, 2016 at 9:14 PM, P. Taylor Goetz wrote: > Thank you for the analysis Jungtaek. I particularly appreciate your > admitted bias as a big contributor to storm-SQL. >

[GitHub] storm pull request #1710: STORM-1546: Adding Read and Write Aggregations for...

2016-09-26 Thread knusbaum
Github user knusbaum commented on a diff in the pull request: https://github.com/apache/storm/pull/1710#discussion_r80508806 --- Diff: storm-core/src/jvm/org/apache/storm/pacemaker/PacemakerClientHandler.java --- @@ -69,7 +69,7 @@ else if(evm instanceof HBMessage) {

[GitHub] storm issue #1697: STORM-2018: Supervisor V2

2016-09-26 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/1697 The test failures look unrelated. Some are rat failures caused by test logs not being excluded. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] storm issue #1714: STORM-2125 Use Calcite's implementation of Rex Compiler

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/storm/pull/1714 I don't want to get blocked by non-SQL standard support unless we have clear use case or clear way to implement it. If my understanding is right, both Spark and Flink seems not support this as we

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80478720 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/compiler/TestCompilerUtils.java --- @@ -153,6 +158,29 @@ public static CalciteState s

[GitHub] storm issue #1697: STORM-2018: Supervisor V2

2016-09-26 Thread revans2
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/1697 Just pulled in the latest set of bug fixes from master. All known issues have been addressed and we have been running in staging with various versions of this patch for over a week now. Expect to r

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80444655 --- Diff: external/sql/storm-sql-runtime/src/jvm/org/apache/calcite/interpreter/StormDataContext.java --- @@ -0,0 +1,79 @@ +/** + * Licensed to t

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80444322 --- Diff: pom.xml --- @@ -264,7 +264,7 @@ 4.11 2.5.1 2.1.7 -1.4.0-incubating +1.9.0

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80444129 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/compiler/backends/trident/TestPlanCompiler.java --- @@ -255,6 +265,198 @@ public void

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80443829 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/compiler/backends/standalone/TestPlanCompiler.java --- @@ -71,9 +71,9 @@ public void

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80442515 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/TestStormSql.java --- @@ -174,26 +173,42 @@ public void testExternalDataSource() throw

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80442067 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/TestStormSql.java --- @@ -174,26 +173,42 @@ public void testExternalDataSource() throw

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread HeartSaVioR
Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80439168 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/TestStormSql.java --- @@ -174,26 +173,42 @@ public void testExternalDataSource() throw

[GitHub] storm issue #1714: STORM-2125 Use Calcite's implementation of Rex Compiler

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/1714 Can you also document the list of functions that are now available since we are directly using calicite to generate the java code? Does it support all the standard SQL functions? --- If your

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80425461 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/compiler/backends/trident/TestPlanCompiler.java --- @@ -255,6 +265,198 @@ public vo

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80416617 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/compiler/backends/standalone/TestPlanCompiler.java --- @@ -71,9 +71,9 @@ public voi

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80418615 --- Diff: external/sql/storm-sql-runtime/src/jvm/org/apache/calcite/interpreter/StormDataContext.java --- @@ -0,0 +1,79 @@ +/** + * Licensed to

[GitHub] storm issue #1682: STORM-2089 Replace Consumer of ISqlTridentDataSource with...

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/storm/pull/1682 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80414692 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/TestStormSql.java --- @@ -174,26 +173,42 @@ public void testExternalDataSource() thr

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80425683 --- Diff: pom.xml --- @@ -264,7 +264,7 @@ 4.11 2.5.1 2.1.7 -1.4.0-incubating +1.9.0

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80415245 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/TestStormSql.java --- @@ -174,26 +173,42 @@ public void testExternalDataSource() thr

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80414653 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/TestStormSql.java --- @@ -174,26 +173,42 @@ public void testExternalDataSource() thr

[GitHub] storm pull request #1714: STORM-2125 Use Calcite's implementation of Rex Com...

2016-09-26 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/storm/pull/1714#discussion_r80416478 --- Diff: external/sql/storm-sql-core/src/test/org/apache/storm/sql/compiler/TestCompilerUtils.java --- @@ -153,6 +158,29 @@ public static CalciteState