[ https://issues.apache.org/jira/browse/SPARK-43036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jungtaek Lim resolved SPARK-43036. ---------------------------------- Resolution: Invalid Please leverage the user mailing list for questions. > Spark structuring streaming with kinesis and rocksdb integration > ---------------------------------------------------------------- > > Key: SPARK-43036 > URL: https://issues.apache.org/jira/browse/SPARK-43036 > Project: Spark > Issue Type: Question > Components: Structured Streaming > Affects Versions: 3.3.1 > Reporter: Rajesh > Priority: Blocker > > Hi Spark Team, > We need to read/write the kinesis streams using spark streaming. > We checked the official documentation - > [https://spark.apache.org/docs/latest/streaming-kinesis-integration.html] > It does not mention about kinesis connector. Alternative is - > [https://github.com/qubole/kinesis-sql] which is not active now. This is now > handed over here - [https://github.com/roncemer/spark-sql-kinesis] > Also according to SPARK-18165 , Spark officially do not have any kinesis > connector > We have few below questions , It would be great if you can answer > # Does Spark provides officially any kinesis connector which have > readstream/writestream and endorse any connector for production use cases ? > # [https://spark.apache.org/docs/latest/streaming-kinesis-integration.html] > This documentation does not mention about how to write to kinesis. This > method have default dynamodb as checkpoint, can we override it ? > # We have rocksdb as state store but when we ran application using official > [https://spark.apache.org/docs/latest/streaming-kinesis-integration.html] > rocksdb configurations were not effective. Can you please confirm if rocksdb > is not applicable in these cases? > # rocksdb however works with qubole connector , do you have any plan to > release kinesis connector? > # Please help/recommend us for any good stable kinesis connector or some > pointers around it -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org