Re: [Spark Structured Streaming] Could we apply new options of readStream/writeStream without stopping spark application (zero downtime)?

t; >> HTH >> >> >>view my Linkedin profile >> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >> >> >> https://en.everybodywiki.com/Mich_Talebzadeh >> >> >> >> *Disclaimer:* Use it at your own risk. Any a

[Spark] How to find which type of key is illegal during from_json() function

*Component*: Spark *Level*: Advanced *Scenario*: How-to - *Problems Description* I have nested json string value in someone field of spark dataframe, and I would like to use from_json() to parse json object. Especially, if one of key type is not match with our defined

[Spark Structured Streaming] Do spark structured streaming is support sink to AWS Kinesis currently and how to handle if achieve quotas of kinesis?

*Component*: Spark Structured Streaming *Level*: Advanced *Scenario*: How-to *Problems Description* 1. I currently would like to use pyspark structured streaming to write data to kinesis. But it seems like does not have corresponding connector can use. I would confirm

[Spark Structured Streaming] Do spark structured streaming is support sink to AWS Kinesis currently?

*Component*: Spark Structured Streaming *Level*: Advanced *Scenario*: How-to *Problems Description* I would like to implement witeStream data to AWS Kinesis with Spark structured Streaming, but I do not find related connector jar can be used. I want to check whether fully

[Spark Structured Streaming] Could we apply new options of readStream/writeStream without stopping spark application (zero downtime)?

*Component*: Spark Structured Streaming *Level*: Advanced *Scenario*: How-to - *Problems Description* I would like to confirm could we directly apply new options of readStream/writeStream without stopping current running spark structured streaming applications? For