各位好,我在flink1.13版本 的flink配置文件里配置checkpoint和savepoint参数时,相关配置项并没有生效,现将我的配置文件信息和日志放在下文。
*配置文件* #============================================================================== # Fault tolerance and checkpointing #============================================================================== # The backend that will be used to store operator state checkpoints if # checkpointing is enabled. # # Supported backends are 'jobmanager', 'filesystem', 'rocksdb', or the # <class-name-of-factory>. # # Directory for checkpoints filesystem, when using any of the default bundled # state backends. # # state.checkpoints.dir: hdfs://namenode-host:port/flink-checkpoints execution.checkpointing.interval: 10000 state.backend: filesystem state.checkpoints.dir: file:///opt/xxx/flink-1.13.0/savepoint/checkpoints state.savepoints.dir: file:///opt/xxx/flink-1.13.0/savepoint/savepoints # Default target directory for savepoints, optional. # # state.savepoints.dir: hdfs://namenode-host:port/flink-savepoints # Flag to enable/disable incremental checkpoints for backends that # support incremental checkpoints (like the RocksDB state backend). # state.backend.incremental: false # The failover strategy, i.e., how the job computation recovers from task failures. # Only restart tasks that may have been affected by the task failure, which typically includes # downstream tasks and potentially upstream tasks if their produced data is no longer available for consumption. jobmanager.execution.failover-strategy: region #============================================================================== *服务器log日志:* flink-root-sql-client-xxx03.log 2021-06-02 11:29:57,566 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: execution.checkpointing.interval, 10000 2021-06-02 11:29:57,566 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: state.backend, filesystem 2021-06-02 11:29:57,566 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: state.checkpoints.dir, file:///opt/xxx/flink-1.13.0/savepoint/checkpoints 2021-06-02 11:29:57,566 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: state.savepoints.dir, file:///opt/xxx/flink-1.13.0/savepoint/savepoints 2021-06-02 11:29:57,566 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: state.backend.incremental, false 2021-06-02 11:29:57,566 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.execution.failover-strategy, region 2021-06-02 11:29:57,567 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: rest.bind-port, 8086 2021-06-02 11:29:57,596 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli [] - Found Yarn properties file under /tmp/.yarn-properties-root. 2021-06-02 11:29:57,998 INFO org.apache.flink.table.client.config.entries.ExecutionEntry [] - Property 'execution.restart-strategy.type' not specified. Using default value: fallback 2021-06-02 11:29:58,028 INFO org.apache.flink.table.client.gateway.context.DefaultContext [] - Executor config: {execution.savepoint.ignore-unclaimed-state=false, execution.attached=true, yarn.application.id=xxx, execution.shutdown-on-attached-exit=false, pipeline.jars=[file:/opt/xxx/flink-1.13.0/opt/flink-sql-client_2.11-1.13.0.jar], high-availability.cluster-id=application_1620482572059_3697, pipeline.classpaths=[], execution.target=yarn-session, $internal.deployment.config-dir=/opt/xxx/flink-1.13.0/conf} 2021-06-02 11:30:02,947 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory ('/opt/xxx/flink-1.13.0/conf') already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file. 2021-06-02 11:30:02,986 WARN org.apache.flink.runtime.util.HadoopUtils [] - Could not find Hadoop configuration via any of the supported methods (Flink configuration, environment variables). 2021-06-02 11:30:03,391 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar 2021-06-02 11:30:03,393 WARN org.apache.flink.yarn.YarnClusterDescriptor [] - Neither the HADOOP_CONF_DIR nor the YARN_CONF_DIR environment variable is set.The Flink YARN Client needs one of these to be set to properly load the Hadoop configuration for accessing YARN. 2021-06-02 11:30:03,445 INFO org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider [] - Failing over to rm236 2021-06-02 11:30:03,502 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Found Web Interface xxx:54194 of application 'xxx *通过web ui观察到的日志:* 2021-05-28 15:32:30,673 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - No state backend has been configured, using default (HashMap) org.apache.flink.runtime.state.hashmap.HashMapStateBackend@b376db1 2021-05-28 15:32:30,673 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Checkpoint storage is set to 'jobmanager' 2021-05-28 15:32:30,674 INFO org.apache.flink.runtime.checkpoint. CheckpointCoordinator [] - No checkpoint found during restore. 还请各位大佬帮忙看一下。 配置文件参考网址: https://stackoverflow.com/questions/65674619/how-to-configure-flink-sql-client-to-submit-job-with-checkpointing-enabled