[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16526264#comment-16526264
]
Jackey Lee commented on SPARK-24630:
------------------------------------
Main Goal:
* SQL API for StructStreaming
Benefits:
* Users, who are unfamiliar with streaming, can easily use SQL to run
StructStreaming especially when migrating offline tasks to real time processing
tasks.
* Support SQL API in StructStreaming can also combine StructStreaming with
hive. Users can store the source/sink metadata in a table and use hive
metastore to manage it. The users, who want to read this data, can easily
create a stream by accessing the table, which can greatly reduce the
development cost and maintenance costs of StructStreaming.
* Easy to achieve unified management and authority control of source and sink,
and more controllable in the management of some private data, especially in
some financial or security area.
> SPIP: Support SQLStreaming in Spark
> -----------------------------------
>
> Key: SPARK-24630
> URL: https://issues.apache.org/jira/browse/SPARK-24630
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 2.2.0, 2.2.1
> Reporter: Jackey Lee
> Priority: Minor
> Labels: SQLStreaming
> Attachments: SQLStreaming SPIP.pdf
>
>
> At present, KafkaSQL, Flink SQL(which is actually based on Calcite),
> SQLStream, StormSQL all provide a stream type SQL interface, with which users
> with little knowledge about streaming, can easily develop a flow system
> processing model. In Spark, we can also support SQL API based on
> StructStreamig.
> To support for SQL Streaming, there are two key points:
> 1, Analysis should be able to parse streaming type SQL.
> 2, Analyzer should be able to map metadata information to the corresponding
> Relation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
