[jira] [Commented] (GEARPUMP-217) Add SQL support
[
https://issues.apache.org/jira/browse/GEARPUMP-217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927517#comment-15927517
]
Manu Zhang commented on GEARPUMP-217:
-
[~lvkr10],
I've touched a bit about design in the description. My thought is to leverage
Apache Calcite to translate a SQL query to Gearpump high level DSL. You may
have your own preference in your design, and list pros and cons for each
solution.
[~buddhiayesha2013],
Thank you for your interest. The scope is to support simple streaming SQL
queries like {{select * from kafka}} with {{filter}} and {{groupBy}}.
Manu
> Add SQL support
>
>
> Key: GEARPUMP-217
> URL: https://issues.apache.org/jira/browse/GEARPUMP-217
> Project: Apache Gearpump
> Issue Type: New Feature
> Components: streaming
>Affects Versions: 0.8.1
>Reporter: Manu Zhang
>Assignee: Manu Zhang
> Labels: gsoc2017, mentor
>
> SQL support will allow those unfamiliar with Scala/Java to use Gearpump. I
> propose to build SQL layer with Apache Calcite because
> 1. It has done the hard job of parsing, translating logical plan to physical
> plan and optimizing.
> 2. It is under active development and has a great community.
> 3. It's proved to be a good solution since Apache Storm, Apache Flink and
> Apache Samza all build their Streaming SQL with Apache Calcite
> Note SQL here actually means Streaming SQL dialects supported by Calcite
> (check reference 1 and 2). Please checkout reference 3 for an example
> implementation.
> The physical plan will be translated to Gearpump high level DSL.
> References:
> 1. http://www.slideshare.net/julianhyde/streaming-sql
> 2. https://calcite.apache.org/docs/stream.html
> 3. https://github.com/milinda/samza-sql
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (GEARPUMP-217) Add SQL support
[ https://issues.apache.org/jira/browse/GEARPUMP-217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang updated GEARPUMP-217: Component/s: streaming > Add SQL support > > > Key: GEARPUMP-217 > URL: https://issues.apache.org/jira/browse/GEARPUMP-217 > Project: Apache Gearpump > Issue Type: New Feature > Components: streaming >Affects Versions: 0.8.1 >Reporter: Manu Zhang >Assignee: Manu Zhang > Labels: gsoc2017, mentor > > SQL support will allow those unfamiliar with Scala/Java to use Gearpump. I > propose to build SQL layer with Apache Calcite because > 1. It has done the hard job of parsing, translating logical plan to physical > plan and optimizing. > 2. It is under active development and has a great community. > 3. It's proved to be a good solution since Apache Storm, Apache Flink and > Apache Samza all build their Streaming SQL with Apache Calcite > Note SQL here actually means Streaming SQL dialects supported by Calcite > (check reference 1 and 2). Please checkout reference 3 for an example > implementation. > The physical plan will be translated to Gearpump high level DSL. > References: > 1. http://www.slideshare.net/julianhyde/streaming-sql > 2. https://calcite.apache.org/docs/stream.html > 3. https://github.com/milinda/samza-sql -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (GEARPUMP-23) Add DSL window (time series) support
[ https://issues.apache.org/jira/browse/GEARPUMP-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved GEARPUMP-23. Resolution: Fixed Fix Version/s: 0.8.3 > Add DSL window (time series) support > - > > Key: GEARPUMP-23 > URL: https://issues.apache.org/jira/browse/GEARPUMP-23 > Project: Apache Gearpump > Issue Type: Sub-task > Components: streaming >Affects Versions: 0.8.0 >Reporter: Kam Kasravi >Assignee: Manu Zhang > Fix For: 0.8.3 > > > Time series support is need for both GEARPUMP-21 and GEARPUMP-22. This was > originally github issue > [1917|https://github.com/gearpump/gearpump/issues/1917]. Other realtime > streaming engines support time windows such as > [Flink|https://flink.apache.org/news/2015/12/04/Introducing-windows.html], > [Beam|https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-101] and > a future release of > [Spark|https://issues.apache.org/jira/secure/attachment/12793410/StructuredStreamingProgrammingAbstractionSemanticsandAPIs-ApacheJIRA.pdf]. > A comparison matrix is found > [here|https://docs.google.com/spreadsheets/d/1OM077lZBARrtUi6g0X0O0PHaIbFKCD6v0djRefQRE1I/edit#gid=995660187] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (GEARPUMP-213) Documentation improvement
[ https://issues.apache.org/jira/browse/GEARPUMP-213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved GEARPUMP-213. - Resolution: Fixed Fix Version/s: 0.8.3 > Documentation improvement > - > > Key: GEARPUMP-213 > URL: https://issues.apache.org/jira/browse/GEARPUMP-213 > Project: Apache Gearpump > Issue Type: Improvement > Components: doc >Affects Versions: 0.8.1 >Reporter: Manu Zhang >Assignee: Manu Zhang > Fix For: 0.8.3 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (GEARPUMP-282) Allow user to configure wait duration for a running application
[ https://issues.apache.org/jira/browse/GEARPUMP-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved GEARPUMP-282. - Resolution: Fixed Fix Version/s: 0.8.3 > Allow user to configure wait duration for a running application > --- > > Key: GEARPUMP-282 > URL: https://issues.apache.org/jira/browse/GEARPUMP-282 > Project: Apache Gearpump > Issue Type: Improvement > Components: client >Reporter: Manu Zhang >Assignee: Huafeng Wang >Priority: Minor > Fix For: 0.8.3 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (GEARPUMP-192) refactor example sources task to use DataSource API
[ https://issues.apache.org/jira/browse/GEARPUMP-192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manu Zhang resolved GEARPUMP-192. - Resolution: Fixed Fix Version/s: 0.8.3 > refactor example sources task to use DataSource API > --- > > Key: GEARPUMP-192 > URL: https://issues.apache.org/jira/browse/GEARPUMP-192 > Project: Apache Gearpump > Issue Type: Improvement > Components: streaming >Affects Versions: 0.8.1 >Reporter: Manu Zhang >Assignee: Kaifang Bao > Fix For: 0.8.3 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (GEARPUMP-252) return application status to client
[ https://issues.apache.org/jira/browse/GEARPUMP-252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karol Brejna updated GEARPUMP-252: -- Fix Version/s: 0.8.3 > return application status to client > --- > > Key: GEARPUMP-252 > URL: https://issues.apache.org/jira/browse/GEARPUMP-252 > Project: Apache Gearpump > Issue Type: Improvement >Affects Versions: 0.8.2 >Reporter: Manu Zhang >Assignee: Huafeng Wang > Fix For: 0.8.3 > > > Currently, client exits after submitting applications and has no idea of the > application status. It will be valuable to return such information to client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (GEARPUMP-267) Kafka integration test failure due to docker-kafka version incompatibility
[ https://issues.apache.org/jira/browse/GEARPUMP-267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karol Brejna resolved GEARPUMP-267. --- Resolution: Duplicate > Kafka integration test failure due to docker-kafka version incompatibility > -- > > Key: GEARPUMP-267 > URL: https://issues.apache.org/jira/browse/GEARPUMP-267 > Project: Apache Gearpump > Issue Type: Bug > Components: integrationtest >Reporter: Manu Zhang >Assignee: Karol Brejna > > The current "spotify/kafka" has been updated to kafka 0.10 while Gearpump > depends on kafka 0.8, and the author doesn't provide history tags. There is > an open [issue|https://github.com/spotify/docker-kafka/issues/61] but the > repo doesn't seem to be under active development. An alternative is > https://github.com/wurstmeister/kafka-docker -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (GEARPUMP-281) Put dockerfiles used in interation test into integrationtest/docker
[ https://issues.apache.org/jira/browse/GEARPUMP-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karol Brejna resolved GEARPUMP-281. --- Resolution: Fixed > Put dockerfiles used in interation test into integrationtest/docker > --- > > Key: GEARPUMP-281 > URL: https://issues.apache.org/jira/browse/GEARPUMP-281 > Project: Apache Gearpump > Issue Type: Task > Components: integrationtest > Environment: N/A >Reporter: Karol Brejna >Assignee: Karol Brejna >Priority: Minor > Fix For: 0.8.3 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (GEARPUMP-255) application id is not returned to user in Java Stream DSL
[ https://issues.apache.org/jira/browse/GEARPUMP-255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karol Brejna updated GEARPUMP-255: -- Fix Version/s: 0.8.3 > application id is not returned to user in Java Stream DSL > - > > Key: GEARPUMP-255 > URL: https://issues.apache.org/jira/browse/GEARPUMP-255 > Project: Apache Gearpump > Issue Type: Bug > Components: streaming >Affects Versions: 0.8.2 >Reporter: Manu Zhang >Assignee: Manu Zhang > Fix For: 0.8.3 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (GEARPUMP-265) remove AppMasterRuntimeInfo from AppMasterContext
[ https://issues.apache.org/jira/browse/GEARPUMP-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karol Brejna updated GEARPUMP-265: -- Fix Version/s: 0.8.3 > remove AppMasterRuntimeInfo from AppMasterContext > - > > Key: GEARPUMP-265 > URL: https://issues.apache.org/jira/browse/GEARPUMP-265 > Project: Apache Gearpump > Issue Type: Sub-task >Affects Versions: 0.8.2 >Reporter: Huafeng Wang >Assignee: Huafeng Wang > Fix For: 0.8.3 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (GEARPUMP-264) refactor Application status implementation
[ https://issues.apache.org/jira/browse/GEARPUMP-264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karol Brejna updated GEARPUMP-264: -- Fix Version/s: 0.8.3 > refactor Application status implementation > -- > > Key: GEARPUMP-264 > URL: https://issues.apache.org/jira/browse/GEARPUMP-264 > Project: Apache Gearpump > Issue Type: Sub-task >Reporter: Huafeng Wang >Assignee: Huafeng Wang > Fix For: 0.8.3 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
