Hello Nowadays, more and more streaming products begin to support SQL streaming, such as KafaSQL, Flink SQL and Storm SQL. To support SQL Streaming can not only reduce the threshold of streaming, but also make streaming easier to be accepted by everyone.
At present, StructStreaming is relatively mature, and the StructStreaming is based on DataSet API, which make it possibal to provide a SQL portal for structstreaming and run structstreaming in SQL. To support for SQL Streaming, there are two key points: 1, Analysis should be able to parse streaming type SQL. 2, Analyzer should be able to map metadata information to the corresponding Relation. Running StructStreaming in SQL can bring some benefits. 1, Reduce the entry threshold of StructStreaming and attract users more easily. 2, Encapsulate the meta information of source or sink into table, maintain and manage uniformly, and make users more accessible. 3. Metadata permissions management, which is based on hive, can control StructStreaming's overall authority management scheme more closely. We have found some ways to solve this problem. It's a pleasure to discuss it with you. Thanks, Jackey Lee -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org