Unsuscribe 2018-06-15 9:20 GMT+02:00 stc <qcsd2...@163.com>:
> The repo you give may solve some of SqlStreaming problems, but not > friendly enough, user need to learn this new syntax. > > -- > Jacky Lee > Mail：qcsd2...@163.com > > At 2018-06-15 11:48:01, "Bowden, Chris" <chris.bow...@microfocus.com> > wrote: > > Not sure if there is a question in here, but if you are hinting that > structured streaming should support a sql interface, spark has appropriate > extensibility hooks to make it possible. However, the most powerful > construct in structured streaming is quite difficult to find a sql > equivalent for (e.g., flatMapGroupsWithState). This repo could use some > cleanup but is an example of providing a sql interface to a subset of > structured streaming's functionality: https://github. > com/vertica/pstl/blob/master/pstl/src/main/antlr4/org/ > apache/spark/sql/catalyst/parser/pstl/PstlSqlBase.g4. > > ------------------------------ > *From:* JackyLee <qcsd2...@163.com> > *Sent:* Thursday, June 14, 2018 7:06:17 PM > *To:* dev@spark.apache.org > *Subject:* Support SqlStreaming in spark > > Hello > > Nowadays, more and more streaming products begin to support SQL streaming, > such as KafaSQL, Flink SQL and Storm SQL. To support SQL Streaming can not > only reduce the threshold of streaming, but also make streaming easier to > be > accepted by everyone. > > At present, StructStreaming is relatively mature, and the StructStreaming > is > based on DataSet API, which make it possibal to provide a SQL portal for > structstreaming and run structstreaming in SQL. > > To support for SQL Streaming, there are two key points: > 1, Analysis should be able to parse streaming type SQL. > 2, Analyzer should be able to map metadata information to the corresponding > Relation. > > Running StructStreaming in SQL can bring some benefits. > 1, Reduce the entry threshold of StructStreaming and attract users more > easily. > 2, Encapsulate the meta information of source or sink into table, maintain > and manage uniformly, and make users more accessible. > 3. Metadata permissions management, which is based on hive, can control > StructStreaming's overall authority management scheme more closely. > > We have found some ways to solve this problem. It's a pleasure to discuss > it > with you. > > Thanks, > > Jackey Lee > > > > -- > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
