Re: Null pointer exception while replying WAL

2024-02-12 Thread Mich Talebzadeh
OK Getting Null pointer exception while replying WAL! One possible reason is that the messages RDD might contain null elements, and attempting to read JSON from null values can result in an NPE. To handle this, you can add a filter before processing the RDD to remove null elements

Re: Null pointer exception while replying WAL

2024-02-12 Thread nayan sharma
Please find below code def main(args: Array[String]): Unit = { val config: Config = ConfigFactory.load() val streamC = StreamingContext.getOrCreate( checkpointDirectory, () => functionToCreateContext(config, checkpointDirectory) ) streamC.start()

Re: Null pointer exception while replying WAL

2024-02-11 Thread Mich Talebzadeh
Hi, It is challenging to make a recommendation without further details. I am guessing you are trying to build a fault-tolerant spark application (spark structured streaming) that consumes messages from Solace? To address *NullPointerException* in the context of the provided information, you need

Null pointer exception while replying WAL

2024-02-09 Thread nayan sharma
Hi Users, I am trying to build fault tolerant spark solace consumer. Issue :- we have to take restart of the job due to multiple issue load average is one of them. At that time whatever spark is processing or batches in the queue is lost. We can't replay it because we already had send ack while