Hi Egor, Structured Streaming handles all of its metadata itself, which files are actually valid, etc. You may use the "create table" syntax in SQL to treat it like a hive table, but it will handle all partitioning information in its own metadata log. Is there a specific reason that you want to store the information in the Hive Metastore?
Best, Burak On Mon, Feb 6, 2017 at 11:39 AM, Egor Pahomov <pahomov.e...@gmail.com> wrote: > Hi, I'm thinking of using Structured Streaming instead of old streaming, > but I need to be able to save results to Hive table. Documentation for file > sink says(http://spark.apache.org/docs/latest/structured- > streaming-programming-guide.html#output-sinks): "Supports writes to > partitioned tables. ". But being able to write to partitioned directories > is not enough to write to the table: someone needs to write to Hive > metastore. How can I use Structured Streaming and write to Hive table? > > -- > > > *Sincerely yoursEgor Pakhomov* >
