I'm not aware of much usage. but that doesn't mean a lot. FWIW, in the past month or so, the Kinesis docs page got about 700 views, compared to about 1400 for Kafka https://analytics.apache.org/index.php?module=CoreHome&action=index&date=yesterday&period=day&idSite=40#?idSite=40&period=range&date=2023-12-15,2024-01-20&category=General_Actions&subcategory=Actions_SubmenuPageTitles
Those are "low" in general, compared to the views for streaming pages, which got tens of thousands of views. I do feel like it's unmaintained, and do feel like it might be a stretch to leave it lying around until Spark 5. It's not exactly unused though. I would not object to removing it unless there is some voice of support here. On Sat, Jan 20, 2024 at 10:38 AM Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > From the dev thread: What else could be removed in Spark 4? > <https://lists.apache.org/thread/shxj7qmrtqbxqf85lrlsv6510892ktnz> > > On Aug 17, 2023, at 1:44 AM, Yang Jie <yangji...@apache.org> wrote: > > I would like to know how we should handle the two Kinesis-related modules > in Spark 4.0. They have a very low frequency of code updates, and because > the corresponding tests are not continuously executed in any GitHub Actions > pipeline, so I think they significantly lack quality assurance. On top of > that, I am not certain if the test cases, which require AWS credentials in > these modules, get verified during each Spark version release. > > > Did we ever reach a decision about removing Kinesis in Spark 4? > > I was cleaning up some docs related to Kinesis and came across a reference > to some Java API docs that I could not find > <https://github.com/apache/spark/pull/44802#discussion_r1459337001>. And > looking around I came across both this email thread and this thread on > JIRA > <https://issues.apache.org/jira/browse/SPARK-45720?focusedCommentId=17787227&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17787227> > about > potentially removing Kinesis. > > But as far as I can tell we haven’t made a clear decision one way or the > other. > > Nick > >
