[
https://issues.apache.org/jira/browse/FLINK-29762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17624011#comment-17624011
]
Gyula Fora commented on FLINK-29762:
------------------------------------
As you can see from the error message: "reactive mode is only supported by
standalone application clusters"
You forgot to specify the job spec in your FlinkDeployment yaml. You need that
to create an Application cluster otherwise you get an empty Session cluster .
Only Application clusters support reactive mode
> Can not create a standalone cluster with reactive mode using the operator
> -------------------------------------------------------------------------
>
> Key: FLINK-29762
> URL: https://issues.apache.org/jira/browse/FLINK-29762
> Project: Flink
> Issue Type: Bug
> Components: Kubernetes Operator
> Environment: Kubernetes Version 1.22 on EKS.
> Flink Operator veresion 1.2.0
> Flink Veresion 1.15 (errors in 1.14 too)
> Reporter: yuvipanda
> Priority: Major
>
> I'm trying to create a minimal running flink cluster with reactive scaling
> using the kubernetes operator (running v1.2.0), with the following YAML:
>
> {{
> kind: FlinkDeployment
> metadata:
> name: test-flink-cluster
> spec:
> flinkConfiguration:
> scheduler-mode: reactive
> flinkVersion: v1_15
> image: flink:1.15
> jobManager:
> replicas: 1
> resource:
> cpu: 0.2
> memory: 1024m
> mode: standalone
> serviceAccount: flink
> taskManager:
> replicas: 1
> resource:
> cpu: 0.2
> memory: 1024m}}
>
> However, this causes the jobmanager to crash with the following:
>
> {{sed: couldn't open temporary file /opt/flink/conf/sedLX7Jx8: Read-only file
> system}}
> {{sed: couldn't open temporary file /opt/flink/conf/sed1vva8t: Read-only file
> system}}
> {{/docker-entrypoint.sh: line 73: /opt/flink/conf/flink-conf.yaml: Read-only
> file system}}
> {{/docker-entrypoint.sh: line 89: /opt/flink/conf/flink-conf.yaml.tmp:
> Read-only file system}}
> {{Starting Job Manager}}
> {{Starting standalonesession as a console application on host
> test-flink-cluster-58cd584fdd-xwbtf.}}
> {{2022-10-25 18:32:00,422 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> --------------------------------------------------------------------------------}}
> {{2022-10-25 18:32:00,510 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> Preconfiguration: }}
> {{2022-10-25 18:32:00,512 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - }}
> {{RESOURCE_PARAMS extraction logs:}}
> {{jvm_params: -Xmx469762048 -Xms469762048 -XX:MaxMetaspaceSize=268435456}}
> {{dynamic_configs: -D jobmanager.memory.off-heap.size=134217728b -D
> jobmanager.memory.jvm-overhead.min=201326592b -D
> jobmanager.memory.jvm-metaspace.size=268435456b -D
> jobmanager.memory.heap.size=469762048b -D
> jobmanager.memory.jvm-overhead.max=201326592b}}
> {{logs: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This
> will impact performance.}}
> {{INFO [] - Loading configuration property: blob.server.port, 6124}}
> {{INFO [] - Loading configuration property:
> kubernetes.jobmanager.annotations,
> flinkdeployment.flink.apache.org/generation:1}}
> {{INFO [] - Loading configuration property: kubernetes.jobmanager.replicas,
> 1}}
> {{INFO [] - Loading configuration property: scheduler-mode, reactive}}
> {{INFO [] - Loading configuration property:
> "kubernetes.operator.metrics.reporter.prom.port", "9999"}}
> {{INFO [] - Loading configuration property: jobmanager.rpc.address,
> test-flink-cluster.default}}
> {{INFO [] - Loading configuration property: kubernetes.taskmanager.cpu, 0.2}}
> {{INFO [] - Loading configuration property: "prometheus.io/port", "9999"}}
> {{INFO [] - Loading configuration property: kubernetes.service-account,
> flink}}
> {{INFO [] - Loading configuration property: kubernetes.cluster-id,
> test-flink-cluster}}
> {{INFO [] - Loading configuration property: kubernetes.container.image,
> flink:1.15}}
> {{INFO [] - Loading configuration property: parallelism.default, 2}}
> {{INFO [] - Loading configuration property: kubernetes.namespace, default}}
> {{INFO [] - Loading configuration property: taskmanager.numberOfTaskSlots,
> 2}}
> {{INFO [] - Loading configuration property:
> kubernetes.rest-service.exposed.type, ClusterIP}}
> {{INFO [] - Loading configuration property: "prometheus.io/scrape", "true"}}
> {{INFO [] - Loading configuration property: taskmanager.memory.process.size,
> 1024m}}
> {{INFO [] - Loading configuration property:
> "kubernetes.operator.metrics.reporter.prom.class",
> "org.apache.flink.metrics.prometheus.PrometheusReporter"}}
> {{INFO [] - Loading configuration property: web.cancel.enable, false}}
> {{INFO [] - Loading configuration property: execution.target, remote}}
> {{INFO [] - Loading configuration property: jobmanager.memory.process.size,
> 1024m}}
> {{INFO [] - Loading configuration property: taskmanager.rpc.port, 6122}}
> {{INFO [] - Loading configuration property:
> kubernetes.internal.cluster-mode, SESSION}}
> {{INFO [] - Loading configuration property: kubernetes.jobmanager.cpu, 0.2}}
> {{INFO [] - Loading configuration property: $internal.flink.version, v1_15}}
> {{INFO [] - The derived from fraction jvm overhead memory (102.400mb
> (107374184 bytes)) is less than its min value 192.000mb (201326592 bytes),
> min value will be used instead}}
> {{INFO [] - Final Master Memory configuration:}}
> {{INFO [] - Total Process Memory: 1024.000mb (1073741824 bytes)}}
> {{INFO [] - Total Flink Memory: 576.000mb (603979776 bytes)}}
> {{INFO [] - JVM Heap: 448.000mb (469762048 bytes)}}
> {{INFO [] - Off-heap: 128.000mb (134217728 bytes)}}
> {{INFO [] - JVM Metaspace: 256.000mb (268435456 bytes)}}
> {{INFO [] - JVM Overhead: 192.000mb (201326592 bytes)}}
> {{2022-10-25 18:32:00,514 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> --------------------------------------------------------------------------------}}
> {{2022-10-25 18:32:00,516 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Starting
> StandaloneSessionClusterEntrypoint (Version: 1.15.1, Scala: 2.12,
> Rev:f494be6, Date:2022-06-20T14:40:28+02:00)}}
> {{2022-10-25 18:32:00,516 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - OS current
> user: flink}}
> {{2022-10-25 18:32:00,517 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Current
> Hadoop/Kerberos user: <no hadoop dependency found>}}
> {{2022-10-25 18:32:00,517 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM:
> OpenJDK 64-Bit Server VM - Oracle Corporation - 11/11.0.16+8}}
> {{2022-10-25 18:32:00,519 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Arch:
> amd64}}
> {{2022-10-25 18:32:00,519 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Maximum
> heap size: 433 MiBytes}}
> {{2022-10-25 18:32:00,520 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JAVA_HOME:
> /usr/local/openjdk-11}}
> {{2022-10-25 18:32:00,520 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - No Hadoop
> Dependency available}}
> {{2022-10-25 18:32:00,522 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - JVM
> Options:}}
> {{2022-10-25 18:32:00,523 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Xmx469762048}}
> {{2022-10-25 18:32:00,523 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Xms469762048}}
> {{2022-10-25 18:32:00,523 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -XX:MaxMetaspaceSize=268435456}}
> {{2022-10-25 18:32:00,524 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Dlog.file=/opt/flink/log/flink--standalonesession-0-test-flink-cluster-58cd584fdd-xwbtf.log}}
> {{2022-10-25 18:32:00,524 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties}}
> {{2022-10-25 18:32:00,524 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties}}
> {{2022-10-25 18:32:00,524 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> -Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml}}
> {{2022-10-25 18:32:00,524 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Program
> Arguments:}}
> {{2022-10-25 18:32:00,526 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> --configDir}}
> {{2022-10-25 18:32:00,528 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> /opt/flink/conf}}
> {{2022-10-25 18:32:00,528 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> --executionMode}}
> {{2022-10-25 18:32:00,528 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> cluster}}
> {{2022-10-25 18:32:00,529 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
> {{2022-10-25 18:32:00,529 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> jobmanager.memory.off-heap.size=134217728b}}
> {{2022-10-25 18:32:00,529 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
> {{2022-10-25 18:32:00,533 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> jobmanager.memory.jvm-overhead.min=201326592b}}
> {{2022-10-25 18:32:00,533 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
> {{2022-10-25 18:32:00,534 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> jobmanager.memory.jvm-metaspace.size=268435456b}}
> {{2022-10-25 18:32:00,534 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
> {{2022-10-25 18:32:00,534 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> jobmanager.memory.heap.size=469762048b}}
> {{2022-10-25 18:32:00,534 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -D}}
> {{2022-10-25 18:32:00,534 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> jobmanager.memory.jvm-overhead.max=201326592b}}
> {{2022-10-25 18:32:00,534 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Classpath:
> /opt/flink/lib/flink-cep-1.15.1.jar:/opt/flink/lib/flink-connector-files-1.15.1.jar:/opt/flink/lib/flink-csv-1.15.1.jar:/opt/flink/lib/flink-json-1.15.1.jar:/opt/flink/lib/flink-scala_2.12-1.15.1.jar:/opt/flink/lib/flink-shaded-zookeeper-3.5.9.jar:/opt/flink/lib/flink-table-api-java-uber-1.15.1.jar:/opt/flink/lib/flink-table-planner-loader-1.15.1.jar:/opt/flink/lib/flink-table-runtime-1.15.1.jar:/opt/flink/lib/log4j-1.2-api-2.17.1.jar:/opt/flink/lib/log4j-api-2.17.1.jar:/opt/flink/lib/log4j-core-2.17.1.jar:/opt/flink/lib/log4j-slf4j-impl-2.17.1.jar:/opt/flink/lib/flink-dist-1.15.1.jar:::}}
> {{2022-10-25 18:32:00,535 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] -
> --------------------------------------------------------------------------------}}
> {{2022-10-25 18:32:00,611 INFO
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Registered
> UNIX signal handlers for [TERM, HUP, INT]}}
> {{2022-10-25 18:32:00,635 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: blob.server.port, 6124}}
> {{2022-10-25 18:32:00,715 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.jobmanager.annotations,
> flinkdeployment.flink.apache.org/generation:1}}
> {{2022-10-25 18:32:00,716 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.jobmanager.replicas, 1}}
> {{2022-10-25 18:32:00,716 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: scheduler-mode, reactive}}
> {{2022-10-25 18:32:00,716 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: "kubernetes.operator.metrics.reporter.prom.port",
> "9999"}}
> {{2022-10-25 18:32:00,717 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: jobmanager.rpc.address, test-flink-cluster.default}}
> {{2022-10-25 18:32:00,717 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.taskmanager.cpu, 0.2}}
> {{2022-10-25 18:32:00,717 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: "prometheus.io/port", "9999"}}
> {{2022-10-25 18:32:00,717 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.service-account, flink}}
> {{2022-10-25 18:32:00,717 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.cluster-id, test-flink-cluster}}
> {{2022-10-25 18:32:00,718 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.container.image, flink:1.15}}
> {{2022-10-25 18:32:00,718 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: parallelism.default, 2}}
> {{2022-10-25 18:32:00,718 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.namespace, default}}
> {{2022-10-25 18:32:00,718 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: taskmanager.numberOfTaskSlots, 2}}
> {{2022-10-25 18:32:00,718 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.rest-service.exposed.type, ClusterIP}}
> {{2022-10-25 18:32:00,718 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: "prometheus.io/scrape", "true"}}
> {{2022-10-25 18:32:00,719 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: taskmanager.memory.process.size, 1024m}}
> {{2022-10-25 18:32:00,719 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: "kubernetes.operator.metrics.reporter.prom.class",
> "org.apache.flink.metrics.prometheus.PrometheusReporter"}}
> {{2022-10-25 18:32:00,719 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: web.cancel.enable, false}}
> {{2022-10-25 18:32:00,719 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: execution.target, remote}}
> {{2022-10-25 18:32:00,720 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: jobmanager.memory.process.size, 1024m}}
> {{2022-10-25 18:32:00,720 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: taskmanager.rpc.port, 6122}}
> {{2022-10-25 18:32:00,720 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.internal.cluster-mode, SESSION}}
> {{2022-10-25 18:32:00,720 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: kubernetes.jobmanager.cpu, 0.2}}
> {{2022-10-25 18:32:00,720 INFO
> org.apache.flink.configuration.GlobalConfiguration [] - Loading
> configuration property: $internal.flink.version, v1_15}}
> {{2022-10-25 18:32:00,924 ERROR
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Reactive
> mode is configured for an unsupported cluster type. At the moment, reactive
> mode is only supported by standalone application clusters
> (bin/standalone-job.sh).}}
> {{Exception in thread "main"
> org.apache.flink.configuration.IllegalConfigurationException: Reactive mode
> is configured for an unsupported cluster type. At the moment, reactive mode
> is only supported by standalone application clusters
> (bin/standalone-job.sh).}}
> {{ at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.<init>(ClusterEntrypoint.java:177)}}
> {{ at
> org.apache.flink.runtime.entrypoint.SessionClusterEntrypoint.<init>(SessionClusterEntrypoint.java:39)}}
> {{ at
> org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint.<init>(StandaloneSessionClusterEntrypoint.java:32)}}
> {{ at
> org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint.main(StandaloneSessionClusterEntrypoint.java:56)}}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
