Structured Streaming UI similar to Spark Streaming

Hi, Today, Spark Streaming exposes an extensive, detailed graphs of input rate, processing time and delay. I was wondering, is there any plan to integrate such a graph for Structured Streaming? Now with Kafka support and implementation of stateful aggregations in Spark 2.2, it's becoming a very

Re: How to reduce the amount of data that is getting written to the checkpoint from Spark Streaming

You can't. Spark doesn't let you fiddle with the data being checkpoint, as it's an internal implementation detail. -- View this message in context:

Re: about broadcast join of base table in spark sql

Then you need to tell us the spark version, and post the execution plan here, so we can help you better. Yong From: Paley Louie Sent: Sunday, July 2, 2017 12:36 AM To: Yong Zhang Cc: Bryan Jeffrey; d...@spark.org; user@spark.apache.org

Re: about broadcast join of base table in spark sql

Thank you for your reply, but when I remove the left join option(like A = A.join(B,A("key1") === B("key2"))), it can be broadcast out. there is no reason spark cannot get table size when left join option is chosen on. On Sun, Jul 2, 2017 at 1:55 PM, Xiaoye Sun wrote: >