It scales down with yarn. Not sure how you've tested. On Fri, 24 May 2019, 19:10 Stavros Kontopoulos, < stavros.kontopou...@lightbend.com> wrote:
> Yes nothing happens. In this case it could propagate info to the resource > manager to scale down the number of executors no? Just a thought. > > Στις Παρ, 24 Μαΐ 2019, 7:17 μ.μ. ο χρήστης Gabor Somogyi < > gabor.g.somo...@gmail.com> έγραψε: > >> Structured Streaming works differently. If no data arrives no tasks are >> executed (just had a case in this area). >> >> BR, >> G >> >> >> On Fri, 24 May 2019, 18:14 Stavros Kontopoulos, < >> stavros.kontopou...@lightbend.com> wrote: >> >>> Hi, >>> >>> Some while ago the streaming dynamic allocation part was added in >>> DStreams(https://issues.apache.org/jira/browse/SPARK-12133) to improve >>> the issues with the batch based one. Should this be ported to >>> structured streaming? Thoughts? >>> AFAIK there is no support in SS for it. >>> >>> Best, >>> Stavros >>> >>>
