Re: How to print plan of Structured Streaming DataFrame

2017-11-21 Thread Chang Chen
Yes, I switched to query.explain, though it did't fail, I still can't make it print executed plan, is there any example? I first try to placed query.explain() exactly after start, but spark reports no data. Hera are my codes: while (query.awaitTermination(2)) { // 1 query.explain()

Re: How to print plan of Structured Streaming DataFrame

2017-11-20 Thread Liang-Chi Hsieh
wordCounts.explain() -> query.explain()? Chang Chen wrote > Hi Guys > > I modified StructuredNetworkWordCount to see what the executed plan is, > here are my codes: > > val wordCounts = words.groupBy("value").count() > > // Start running the query that prints the running counts to the

How to print plan of Structured Streaming DataFrame

2017-11-20 Thread Chang Chen
Hi Guys I modified StructuredNetworkWordCount to see what the executed plan is, here are my codes: val wordCounts = words.groupBy("value").count() // Start running the query that prints the running counts to the console val query = wordCounts.writeStream .outputMode("complete")