Re: We found a deadlock problem when a single piece of data is too large on Flink1.13.2, do not continue to process the data, which more understand the data transmission piece, welcome to comment.

ask.java:779) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566) at java.lang.Thread.run(Thread.java:748) rui chen 于2024年10月9日周三 11:38写道： > We found a deadlock problem when a single piece of data is too large on > Flink1.13.2, do not continue to process the data, which more understand the > data trans

We found a deadlock problem when a single piece of data is too large on Flink1.13.2, do not continue to process the data, which more understand the data transmission piece, welcome to comment.

We found a deadlock problem when a single piece of data is too large on Flink1.13.2, do not continue to process the data, which more understand the data transmission piece, welcome to comment.

Re: Flink has a large number of data and appears to be suspended after restarting.

TM上没有报错，在监控上看不处理数据了，cp也超时。 Xuyang 于2024年9月30日周一 10:18写道： > Hi, chen. > > 你可以把TM日志报错的栈和上下文贴上来吗？ > > > > > -- > > Best！ > Xuyang > > > > > > At 2024-09-29 10:00:44, "rui chen" wrote: > >1.A single piece of data is 500kb > >2.The job restarts after a tm fails >

Flink has a large number of data and appears to be suspended after restarting.

1.A single piece of data is 500kb 2.The job restarts after a tm fails

Re: 这绝对算是bug

建议检查一下JM的GC情况。 wjw_bigd...@163.com 于2024年7月1日周一 17:18写道： > 退订 > > > > 回复的原邮件 > | 发件人 | wjw_bigd...@163.com | > | 日期 | 2024年07月01日 17:13 | > | 收件人 | user-zh | > | 抄送至 | | > | 主题 | 回复：这绝对算是bug | > 退订 > > > > 回复的原邮件 > | 发件人 | 星海<2278179...@qq.com.INVALID> | > | 日期 | 2024年06月29日

FLINK-24035, how can this issue be repeated?

We encountered similar problems in production, and we want to integrate FLINK-24035 to solve them, but we don't know how to repeat the problem.

Re: Problems with the state.backend.fs.memory-threshold parameter

(1.19). > > > Best, > Zakelly > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-306%3A+Unified+File+Merging+Mechanism+for+Checkpoints > [2] https://issues.apache.org/jira/browse/FLINK-32070 > > On Fri, Oct 13, 2023 at 6:28 PM rui chen wrote: > > >

Real-time task blocking problem

After the task restart of our 1.13 version, kakfa consumption zero problem occurred. Have you ever encountered it?

Problems with the state.backend.fs.memory-threshold parameter

We found that for some tasks, the JM memory continued to increase. I set the parameter of state.backend.fs.memory-threshold to 0, and the JM memory would no longer increase, but many small files might be written in this way. Does the community have any optimization plan for this area?

Re: Cannot find metata file metadats in directory

ems that you are trying to restore from a not completed or failed > checkpoint. > > On Thu, Sep 28, 2023 at 6:09 PM rui chen wrote: > > > When we use 1.13.2,we have the following error: > > FileNotFoundException: Cannot find metata file metadats in directory > > '

Cannot find metata file metadats in directory

When we use 1.13.2,we have the following error: FileNotFoundException: Cannot find metata file metadats in directory 'hdfs://xx/f408dbe327f9e5053e76d7b5323d6e81/chk-173'.

在使用使用jemalloc内存分配器一段时间后，出现checkpoint 超时，任务卡住的情况

在使用使用jemalloc内存分配器一段时间后，出现checkpoint 超时，任务卡住的情况，哪位遇到过呢？flink版本：flink-1.13.2，jiemalloc版本：5.3.0

After using the jemalloc memory allocator for a period of time, checkpoint timeout occurs and tasks are stuck

After using the jemalloc memory allocator for a period of time, checkpoint timeout occurs and tasks are stuck. Who has encountered this? flink version:1.13.2, jiemalloc version: 5.3.0