Re: Removing Kinesis in Spark 4

2024-01-20 Thread Nicholas Chammas
Oh, that’s a very interesting dashboard. I was familiar with the Matomo snippet but never looked up where exactly those metrics were going. I see that the Kinesis docs do indeed have around 650 views in the past month, but for Kafka I see 11K and 1.3K views for the Structured Streaming and

Re: Removing Kinesis in Spark 4

2024-01-20 Thread Sean Owen
I'm not aware of much usage. but that doesn't mean a lot. FWIW, in the past month or so, the Kinesis docs page got about 700 views, compared to about 1400 for Kafka

Removing Kinesis in Spark 4

2024-01-20 Thread Nicholas Chammas
From the dev thread: What else could be removed in Spark 4? > On Aug 17, 2023, at 1:44 AM, Yang Jie wrote: > > I would like to know how we should handle the two Kinesis-related modules in > Spark 4.0. They have a very low

Re: Dynamic resource allocation for structured streaming [SPARK-24815]

2024-01-20 Thread Pavan Kotikalapudi
Here is the link to the voting thread https://lists.apache.org/thread/rlwqrw6ddxdkbvkp78kpd0zgvglgbbp8. Thank you, Pavan On Wed, Jan 17, 2024 at 7:15 PM Pavan Kotikalapudi wrote: > Thanks for the +1, I will propose voting in a new thread now. > > - Pavan > > On Wed, Jan 17, 2024 at 5:28 PM