Re: flink state.savepoints.dir 目录配置问题
Hi 这个你可以尝试把这个信息记录到哪里,或者在启动的时候从这个 jobId 的目录下去查找所有的 chk-xxx 然后选择一个合适的 目录进行恢复 Best, Congxian marble.zh...@coinflex.com.INVALID 于2020年10月27日周二 下午4:54写道: > 刚钉钉群里建议我把路径指到jobId/chk-xx目录,这样就可以恢复了。 > > 但是如果这样,这个xx随着checkpoint的变化而变化,这样怎么做到自动提交job? > > > > -- > Sent from: http://apache-flink.147419.n8.nabble.com/
Re: flink state.savepoints.dir 目录配置问题
刚钉钉群里建议我把路径指到jobId/chk-xx目录,这样就可以恢复了。 但是如果这样,这个xx随着checkpoint的变化而变化,这样怎么做到自动提交job? -- Sent from: http://apache-flink.147419.n8.nabble.com/
Re: flink state.savepoints.dir 目录配置问题
谢谢, 我把这个folder 设置为一个755就可以了。 但现在我遇到一个问题,我目前的环境是用docker 创建了一个jobmanager, 二个taskmanager, 这三个container都map到了主机上的一个地址, 用于放checkpoints/savepoints,理论上这三个container都可以访问得到。 但尝试用这个命令恢复state启动job时报以下错误, /opt/flink/bin/flink run -d -s /opt/flink/savepoints -c com.coinflex.flink.ohlc.kafka.OrderTickCandleView /home/service-ohlc-*-SNAPSHOT.jar job启动不了。 Caused by: org.apache.flink.runtime.client.JobExecutionException: Could not instantiate JobManager. at org.apache.flink.runtime.dispatcher.Dispatcher.lambda$createJobManagerRunner$6(Dispatcher.java:398) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604) ... 6 more Caused by: java.io.FileNotFoundException: Cannot find meta data file '_metadata' in directory '/opt/flink/savepoints'. Please try to load the checkpoint/savepoint directly from the metadata file instead of the directory. -- Sent from: http://apache-flink.147419.n8.nabble.com/
Re: flink state.savepoints.dir 目录配置问题
Hi 从报错来看是无法在 "/opt/flink/savepoints" 这个路径下创建目录,这个错误下面应该应该一些异常信息,可以看下具体是啥原因。 另外,使用本地路径存储 checkpoint/savepoint 的话,那么恢复的时候,需要确保该文件能被新的 JM/TM 所访问到 Best, Congxian marble.zh...@coinflex.com.INVALID 于2020年10月26日周一 下午3:32写道: > 你好, > 我在flink jobmanager里的flink-conf.yaml添加了以加三个个关的state配置参数, > state.backend: filesystem > state.checkpoints.dir: file:///opt/flink/savepoints > state.savepoints.dir: file:///opt/flink/savepoints > > > 但在做./flink savepoint 时还是报以下的错, > > Caused by: java.io.IOException: Failed to create savepoint directory at > file:/opt/flink/savepoints > at > > org.apache.flink.runtime.state.filesystem.AbstractFsCheckpointStorage.initializeLocationForSavepoint(AbstractFsCheckpointStorage.java:167) > at > > org.apache.flink.runtime.checkpoint.CheckpointCoordinator.lambda$initializeCheckpoint$8(CheckpointCoordinator.java:633) > ... 8 more > > > 我需要做哪些调整,是只能支持hdfs吗? 多谢, 急。 > > > > -- > Sent from: http://apache-flink.147419.n8.nabble.com/ >
flink state.savepoints.dir 目录配置问题
你好, 我在flink jobmanager里的flink-conf.yaml添加了以加三个个关的state配置参数, state.backend: filesystem state.checkpoints.dir: file:///opt/flink/savepoints state.savepoints.dir: file:///opt/flink/savepoints 但在做./flink savepoint 时还是报以下的错, Caused by: java.io.IOException: Failed to create savepoint directory at file:/opt/flink/savepoints at org.apache.flink.runtime.state.filesystem.AbstractFsCheckpointStorage.initializeLocationForSavepoint(AbstractFsCheckpointStorage.java:167) at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.lambda$initializeCheckpoint$8(CheckpointCoordinator.java:633) ... 8 more 我需要做哪些调整,是只能支持hdfs吗? 多谢, 急。 -- Sent from: http://apache-flink.147419.n8.nabble.com/