OK Spark Structured Streaming. How are you getting messages into Spark? Is it Kafka?
This to me index that the message is incomplete or having another value in Json HTH Mich Talebzadeh, Lead Solutions Architect/Engineering Lead Palantir Technologies London United Kingdom view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Wed, 5 Apr 2023 at 12:58, me <m...@rsi2m.dev> wrote: > Dear Apache Spark users, > I have a long running Spark application that is encountering an > ArrayIndexOutOfBoundsException once every two weeks. The exception does not > disrupt the operation of my app, but I'm still concerned about it and would > like to find a solution. > > Here's some additional information about my setup: > > Spark is running in standalone mode > Spark version is 3.3.1 > Scala version is 2.12.15 > I'm using Spark in Structured Streaming > > Here's the relevant error message: > java.lang.ArrayIndexOutOfBoundsException Index 59 out of bounds for length > 16 > I've reviewed the code and searched online, but I'm still unable to find a > solution. The full stacktrace can be found at this link: > https://gist.github.com/rsi2m/ae54eccac93ae602d04d383e56c1a737 > I would appreciate any insights or suggestions on how to resolve this > issue. Thank you in advance for your help. > > Best regards, > rsi2m > > >
