我使用 ZooKeeper High-Availability with Standalone 集群模式,使用 sql-client.sh 提交任务
On 2022/04/20 10:43:16 Zhanghao Chen wrote: > 你是用的什么 Flink 集群部署模式提交的 SQL 任务呢? > > Best, > Zhanghao Chen > ________________________________ > From: 段晓雄 <xi...@zznode.com> > Sent: Saturday, April 16, 2022 19:52 > To: user-zh@flink.apache.org <us...@flink.apache.org> > Subject: RE: Re: Re: 日志里提示 The state is cleared because of state ttl. This > will result in incorrect result 如何解决? > > Zhanghao, > > 感谢帮助!我在 sql-client.sh 里设置了 table.exec.state.ttl= > 129600000,但现在我不知道如何确认是否设置成功,我从web UI 和 rest api 返回的任务状态和checkpoint状态中都没有找到 > state ttl 的值,我发现任务状态还是不断增大,我如何能确认任务 state ttl 的情况呢? > > > On 2022/04/12 04:15:37 Zhanghao Chen wrote: > > 你可以用 SQL client SET 'table.exec.state.ttl' = 'xxx'; 的语法来实现哈,具体可以参考下: > > > > > > 1. > > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sqlclient/#running-sql-queries > > SQL Client | Apache > > Flink<https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/sqlclient/#running-sql-queries> > > The SET command allows you to tune the job execution and the sql client > > behaviour. See SQL Client Configuration below for more details.. After a > > query is defined, it can be submitted to the cluster as a long-running, > > detached Flink job. The configuration section explains how to declare table > > sources for reading data, how to declare table sinks for writing data, and > > how to configure other table ... > > nightlies.apache.org > > 2. > > 3. > > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/config/#table-exec-state-ttl > > Configuration | Apache > > Flink<https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/table/config/#table-exec-state-ttl> > > Configuration # By default, the Table & SQL API is preconfigured for > > producing accurate results with acceptable performance. Depending on the > > requirements of a table program, it might be necessary to adjust certain > > parameters for optimization. For example, unbounded streaming programs may > > need to ensure that the required state size is capped (see streaming > > concepts). > > nightlies.apache.org > > > > > > Best, > > Zhanghao Chen > > ________________________________ > > From: 段晓雄 <ma...@gmail.com> > > Sent: Monday, April 11, 2022 20:23 > > To: user-zh@flink.apache.org <us...@flink.apache.org> > > Subject: RE: Re: 日志里提示 The state is cleared because of state ttl. This will > > result in incorrect result 如何解决? > > > > 我使用sql-client.sh 执行sql创建的任务,如何设置 ttl 的时长? > > > > On 2022/04/11 11:14:36 yidan zhao wrote: > > > You can increase the state ttl to avoid this. > > > 这个已经明确了方法了,增加ttl的时长。 > > > > > > 段晓雄 <ma...@gmail.com> 于2022年4月11日周一 09:52写道: > > > > > > > > 各位老大, > > > > > > > > 现在是 Flink 1.14.4 集群,通过 pyflink执行 sql 做流处理, > > > > > > > > taskmanager日志大量 The state is cleared because of state ttl. This will > > > > result in incorrect result. You can increase the state ttl to avoid > > > > this. 为什么出现?如何解决? > > > > > > > > > > > > 2022-04-09 17:08:54,672 INFO > > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler > > > > [] - Committing the state for checkpoint 284 > > > > 2022-04-09 17:08:54,672 INFO > > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler > > > > [] - Committing the state for checkpoint 284 > > > > 2022-04-09 17:08:54,852 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:54,852 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:54,852 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:54,922 INFO > > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler > > > > [] - Committing the state for checkpoint 277 > > > > 2022-04-09 17:08:54,922 INFO > > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler > > > > [] - Committing the state for checkpoint 277 > > > > 2022-04-09 17:08:54,952 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:54,952 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:54,952 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:54,952 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:54,952 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:54,952 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:55,052 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:55,052 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:55,152 INFO > > > > org.apache.flink.table.runtime.operators.sink.SinkUpsertMaterializer [] > > > > - The state is cleared because of state ttl. This will result in > > > > incorrect result. You can increase the state ttl to avoid this. > > > > 2022-04-09 17:08:55,330 INFO > > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler > > > > [] - Committing the state for checkpoint 282 > > > > 2022-04-09 17:08:55,330 INFO > > > > org.apache.flink.streaming.runtime.operators.sink.AbstractStreamingCommitterHandler > > > > [] - Committing the state for checkpoint 282 > > > > > > > > > >