Hello Irakli, The error is due to the fact that the Adaptive Scheduler doesn’t support batch jobs, as detailed in the Flink documentation[1]. When operating in reactive mode, Flink automatically decides the type of scheduler to use. For batch execution, the default scheduler is AdaptiveBatchScheduler, not AdaptiveScheduler as in the streaming case.
[1] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/elastic_scaling/#limitations Best regards, Junrui lorenzo.affetti.ververica.com via user <user@flink.apache.org> 于2024年3月4日周一 23:24写道： > Hello Irakli and thank you for your question. > > I guess that somehow Flink enters the "reactive" mode while the adaptive > scheduler is not configured. > > I would go with 2 options to isolate your issue: > > - Try with forcing the scheduling mode ( > > https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/elastic_scaling/#adaptive-scheduler) > in your configuration: `jobmanager.scheduler: adaptive` > - Let Flink decide for the runtime mode: if the source is bounded, > you don't need `env.setRuntimeMode(RuntimeExecutionMode.BATCH)`, as Flink > will understand that correctly. > > > Can you try one of the two and see if that helps? > > For the rest: "running it in the "BATCH" mode was better as I don't have > to deal with the Watermark Strategy". Still, you could opt for a simple > watermarking strategy and stay with the streaming mode (for example, > 'BoundedOutOfOrcerness': > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/event-time/generating_watermarks/#watermark-strategies-and-the-kafka-connector > ). > On Mar 4, 2024 at 15:54 +0100, irakli.keshel...@sony.com < > irakli.keshel...@sony.com>, wrote: > > Hello, > > I have a Flink job which is processing bounded number of events. > Initially, I was running the job in the "STREAMING" mode, but I realized > that running it in the "BATCH" mode was better as I don't have to deal with > the Watermark Strategy. The job is reading the data from the Kafka topic > and was running fine in the "STREAMING" mode. > I switched the job to the "BATCH" mode by setting > "env.setRuntimeMode(RuntimeExecutionMode.BATCH)". I changed the Kafka > Source to be bounded by setting ".setBounded(OffsetsInitializer.latest())" > to the source builder and I updated the Watermark strategy to be > "WatermarkStrategy.noWatermarks()". > After making these changes and deploying the job I end up with following > error: "java.lang.IllegalStateException","error.message":"Adaptive > Scheduler is required for reactive mode". I couldn't find any documentation > online which is connecting "Adaptive Scheduler" to the "BATCH" processing. > Does anyone know where this error is coming from and how I can deal with it? > > Cheers, > Irakli > >
