Re: Flink Web UI

2024-08-31 Thread Yu Chen
Hi Kartik, The time for complete job erxpired in SessionCluster was controlled by conf `jobstore.expiration-time`[1]. Best, Yu Chen [1] https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/deployment/config/#jobstore-expiration-time 发件人

Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-18 Thread Yu Chen
Congratulations! Thanks to release managers and everyone involved! Best, Yu Chen > 2024年3月19日 01:01,Jeyhun Karimov 写道: > > Congrats! > Thanks to release managers and everyone involved. > > Regards, > Jeyhun > > On Mon, Mar 18, 2024 at 9:25 AM Lincoln Lee wr

Re: Flink autoscaler scaling report

2024-01-18 Thread Yu Chen
r/blob/48df9d35ed55ae8bb513d9153e9f6f668da9e1c3/flink-autoscaler/src/main/java/org/apache/flink/autoscaler/event/LoggingEventHandler.java#L43C18-L43C18 Best, Yu Chen > 2024年1月18日 18:20,Yang LI 写道: > > Hello dear flink community, > > I noticed that there's a scaling report feature

Re: How to monitor changes in the existing files using flink 1.17.2

2024-01-09 Thread Yu Chen
/file/src/impl/ContinuousFileSplitEnumerator.java#L62C33-L62C54 [2] Overview | Apache Paimon <https://paimon.apache.org/docs/master/concepts/overview/> Best, Yu Chen > 2024年1月10日 04:31,Nitin Saini 写道: > > Hi Flink Community, > > I was using flink 1.12.7 readFile to read fil

Re: oomkill issue

2023-12-04 Thread Yu Chen
#heading=h.f5wfmsmpemd0> Best, Yu Chen > 2023年12月5日 04:42,prashant parbhane 写道: > > Hi Yu, > > Thanks for your reply. > > When i run below script > > ``` > jeprof --show_bytes -svg `which java` /tmp/jeprof.out.301.1009.i1009.heap > > 1009.svg > ``` &

回复: oomkill issue

2023-11-27 Thread Yu Chen
t=b,file=/tmp/heap.hprof ``` [1] Using jemalloc to Optimize Memory Allocation ― Sentieon Appnotes 202308.01 documentation<https://support.sentieon.com/appnotes/jemalloc/> Best, Yu Chen 发件人: prashant parbhane 发送时间: 2023年11月28日 1:42 收件人: user@flink.apach

回复: Operator ids

2023-11-26 Thread Yu Chen
to add such interface, you can follow the ticket FLINK-33230[1] [1] [FLINK-33230] Support Expanding ExecutionGraph to StreamGraph in Web UI - ASF JIRA (apache.org)<https://issues.apache.org/jira/browse/FLINK-33230> Best, Yu Chen 发件人: rania duni 发送时间: 2023年

回复: Doubts about state and table API

2023-11-26 Thread Yu Chen
the exception of the failed restore operator id. However, the lost of the operator state would only produce some erroneous results and would not result in `not able to return any row`. It would be better to provide logs after restoring to locate a more specific problem. Best, Yu Chen

回复: Operator ids

2023-11-26 Thread Yu Chen
Hi rania, If you means the Job Vertex ID of the JobGraph, you can try this: http://localhost:8081/jobs/ Best, Yu Chen 发件人: Zhanghao Chen 发送时间: 2023年11月26日 11:02 收件人: rania duni ; user@flink.apache.org 主题: Re: Operator ids It is not supported yet. Curious why

Re: 配置了state.checkpoints.num-retained为1,但taskmanger 中checkpoints数量越来越多,占用内存,如何解决?

2023-11-07 Thread Yu Chen
-externalized-checkpoint-retention Best, Yu Chen > 2023年11月8日 13:08,梁嘉贤 写道: > > Hi, 我纠正一下我的问题,是taskmanager中checkpoints数量越来越多占用磁盘。同时,补充一下以下信息: > 我通过把task manager的checkpoint路径挂载到本地,采用du > -h命令查看checkpoint中的结果,发现任务中会持续增加chk,导致占用磁盘越来越大,如下图 > 我的疑问是,如何把这些历史chk文件删掉? > <2dfdf...@8

Re: 配置了state.checkpoints.num-retained为1,但taskmanger 中checkpoints数量越来越多,占用内存,如何解决?

2023-11-07 Thread Yu Chen
Checkpoint导致(例如全局窗口聚合且未设置State TTL的场景),如果要定位内存上涨的原因还需要更多的作业信息。 另外,如果你希望确认参数是否生效,可以在JobManager的Configuration一栏查看。 Best, Yu Chen > 2023年11月8日 11:56,梁嘉贤 写道: > > 您好, > 采用Flink 1.14 > 版本,用docker分别建立了jobmanger和taskmanager两个容器,docker-compose.yml信息如下图1所示。 > 在配置中,设置了sta

Re: Error in /jars/upload curl request

2023-11-07 Thread Yu Chen
kend: serviceName: xxx servicePort: 8081 ``` Please let me know if there are any other problems. Best,Yu Chen > 2023年11月7日 18:40,Tauseef Janvekar 写道: > > Hi Chen, > > We are not using nginx anywhere on the server(kubernetes cluster) or on my > client(my local

Re: Handling Schema Variability and Applying Regex Patterns in Flink Job Configuration

2023-11-07 Thread Yu Chen
Hi Arjun, As stated in the document, 'This regex pattern should be matched with the absolute file path.' Therefore, you should adjust your regular expression to match absolute paths. Please let me know if there are any other problems. Best, Yu Chen > 2023年11月7日 18:11,arjun s 写道:

Re: Error in /jars/upload curl request

2023-11-07 Thread Yu Chen
Hi Tauseef, The error was caused by the nginx configuration and was not a flink problem. You can find many related solutions on the web [1]. Best, Yu Chen [1] https://stackoverflow.com/questions/24306335/413-request-entity-too-large-file-upload-issue > 2023年11月7日 15:14,Tauseef Janvekar

Re: Inquiry about ActiveResourceManager and StandaloneResourceManager in Flink

2023-11-03 Thread Yu Chen
erent resource managers to deal with different usage scenarios. Please feel free to correct me if there are any misunderstandings. Best regards, Yu Chen Steven Chen 于2023年11月3日周五 13:28写道: > Dear Flink Community, > > > I am currently using Flink for my project and have a q