Re: 关于savepoint恢复问题咨询

2021-05-26 文章 LakeShen
看下你的 flink 命令对不对,然后去 Flink Web UI  Checkpoint 界面,看下是否从 Savepoint 恢复(下面有个
restore path).
之后再看下你的窗口时间类型用的是什么。

Best,
LakeShen

王春浩  于2021年5月27日周四 上午9:26写道:

> hi, 社区
> ​
> 版本flink 1.7
> ​
>
> 我正在尝试从保存点(或检查点)还原flink作业,该作业的工作是从kafka读取->执行30分钟的窗口聚合(只是AggregationFunction,就像一个计数器)->下沉到kafka。
> ​
> 我使用rocksdb和启用检查点。
> ​
> 现在我尝试手动触发一个保存点。 每个汇总的期望值是30(1个数据/每分钟)。 但是,当我从保存点还原时(flink运行-d -s
> {savepoint的url}),聚合值不是30(小于30,取决于我取消flink作业并还原的时间)。 但是当作业正常运行时,它将达到30。
> ​
> 我不知道为什么有些数据似乎会丢失?
> ​
> 日志显示``No restore state for FlinkKafkaConsumer''
> ​
> ​​
> ​
> 四川省成都市高新区天府三街199号 太平洋保险金融大厦A区11楼
> 11th Floor,Tower A,Pacific insurance finance Building, No.199 TianFu 3rd
> Street, GaoXin District, Chengdu, Sichuan Province
> Mobile +86 15817382279
> Email  wangchun...@navercorp.com
>
> NCloud
>
> -Original Message-
> From: "王春浩"
> To: ;
> Cc:
> Sent: 2021/5/26周三 17:03 (GMT+08:00)
> Subject: inquire about restore from savepoint
>
> Hi Community,
> ​
> version flink 1.7
> im trying to make a flink job restore from a savepoint(or checkpoint),
> what the job do is reading from kafka -> do a 30-minutes-window
> aggregation(just AggregationFunction, acts like a counter) -> sink to kafka.
> i use rocksdb and enabled checkpoint.
> now i try to trigger a savepoint manually. the expected value of each
> aggregated one is 30(1 data/per minute). but when i restore from a
> savepoint(flink run -d -s {savepoint's url}), the aggregated value is not
> 30(less than 30, depends on the time i cancel flink job and restore). but
> when the job run normally, it gets 30.
> i don't know why could some data seems to be lost?
> and a log shows "No restore state for FlinkKafkaConsumer"​
> ​
> ​
> ​
> 四川省成都市高新区天府三街199号 太平洋保险金融大厦A区11楼
> 11th Floor,Tower A,Pacific insurance finance Building, No.199 TianFu 3rd
> Street, GaoXin District, Chengdu, Sichuan Province
> Mobile +86 15817382279
> Email  wangchun...@navercorp.com
>
> NCloud
>


关于savepoint恢复问题咨询

2021-05-26 文章 王春浩
hi, 社区
​
版本flink 1.7
​
我正在尝试从保存点(或检查点)还原flink作业,该作业的工作是从kafka读取->执行30分钟的窗口聚合(只是AggregationFunction,就像一个计数器)->下沉到kafka。
​
我使用rocksdb和启用检查点。
​
现在我尝试手动触发一个保存点。 每个汇总的期望值是30(1个数据/每分钟)。 但是,当我从保存点还原时(flink运行-d -s 
{savepoint的url}),聚合值不是30(小于30,取决于我取消flink作业并还原的时间)。 但是当作业正常运行时,它将达到30。
​
我不知道为什么有些数据似乎会丢失?
​
日志显示``No restore state for FlinkKafkaConsumer''
​
​​
​
四川省成都市高新区天府三街199号 太平洋保险金融大厦A区11楼
11th Floor,Tower A,Pacific insurance finance Building, No.199 TianFu 3rd 
Street, GaoXin District, Chengdu, Sichuan Province
Mobile +86 15817382279
Email  wangchun...@navercorp.com  

NCloud

-Original Message-
From: "王春浩"
To: ;
Cc:
Sent: 2021/5/26周三 17:03 (GMT+08:00)
Subject: inquire about restore from savepoint
 
Hi Community,
​
version flink 1.7
im trying to make a flink job restore from a savepoint(or checkpoint), what the 
job do is reading from kafka -> do a 30-minutes-window aggregation(just 
AggregationFunction, acts like a counter) -> sink to kafka.
i use rocksdb and enabled checkpoint.
now i try to trigger a savepoint manually. the expected value of each 
aggregated one is 30(1 data/per minute). but when i restore from a 
savepoint(flink run -d -s {savepoint's url}), the aggregated value is not 
30(less than 30, depends on the time i cancel flink job and restore). but when 
the job run normally, it gets 30.
i don't know why could some data seems to be lost?
and a log shows "No restore state for FlinkKafkaConsumer"​
​
​
​
四川省成都市高新区天府三街199号 太平洋保险金融大厦A区11楼
11th Floor,Tower A,Pacific insurance finance Building, No.199 TianFu 3rd 
Street, GaoXin District, Chengdu, Sichuan Province
Mobile +86 15817382279
Email  wangchun...@navercorp.com  

NCloud