Re: Re: spark+kafka+dynamic resource allocation

2023-01-29 Thread Lingzhe Sun
Hi Mich, Thanks for the information. I myself think this open issue should have higher priority as more streaming application is being built using spark. Hope this becomes a feature in the coming future. BR Lingzhe Sun From: Mich Talebzadeh Date: 2023-01-30 02:14 To: Lingzhe Sun CC:

Re: Re: spark+kafka+dynamic resource allocation

2023-01-29 Thread Mich Talebzadeh
Hi, Spark Structured Streaming currently does not support dynamic allocation (see SPARK-24815: Structured Streaming should support dynamic allocation ). which is still open Autoscaling in Cloud offerings