Currently my job fails even on a single failure. In other words, even if one incoming message is malformed the job fails. I believe there's a property that allows us to set an acceptable number of failures. I Googled but couldn't find the answer. Can someone please help? Thanks.
- Failure Threshold in Spark Structured Streaming? Eric Beabes
- Re: Failure Threshold in Spark Structured Streaming? Jungtaek Lim