Re: Job Cancellation Failing

2023-02-20 Thread Matthias Pohl via user
What do you mean by "earlier it used to fail due to ExecutionGraphStore not existing in /tmp" folder? Did you get the error message "Could not create executionGraphStorage directory in /tmp." and creating this folder fixed the issue? It also looks like the stacktrace doesn't match any of the 1.15

Calculation of UI's maximum non-heap memory

2023-02-20 Thread Alexis Sarda-Espinosa
Hello, I have configured a job manager with the following settings (Flink 1.16.1): jobmanager.memory.process.size: 1024m jobmanager.memory.jvm-metaspace.size: 150m jobmanager.memory.off-heap.size: 64m jobmanager.memory.jvm-overhead.min: 168m jobmanager.memory.jvm-overhead.max: 168m jobmanager.mem

Re:Re: Flink SQL support array transform function

2023-02-20 Thread Xuekui
Hi YuXia, Thanks for your advice. By adding the hint, the type validation can pass. But still I can't pass the function to this udf Here is my query select array_transform(ids, id -> id +1) from tmp_table The lambda function  id -> id +1 can't be passed because "->" is not supported in cal

Re: Calculation of UI's maximum non-heap memory

2023-02-20 Thread Weihua Hu
Hi, Alexis 1. With those configuration, Flink will set JVM parameters -Xms and -Xmx to 673185792(642m),-XX:MaxDirectMemorySize to 67108864(64m),-XX:MaxMetaspaceSize to 157286400(150m), you can find more information from [1] 2. As the hint in Flink UI: "The maximum heap displayed might differ from

Re: Metrics or runtimeContext in global commit

2023-02-20 Thread Tobias Fröhlich
Dear Yuxia, thank you for your answer! This is also our conclusion and my colleague has already proposed this feature. Best regards, Tobias - Ursprüngliche Mail - Von: "yuxia" An: "Dr. Tobias Fröhlich" CC: "User" , "dev" Gesendet: Montag, 20. Februar 2023 03:31:22 Betreff: Re: Metri

Whether Flink SQL window operations support "Allow Lateness and SideOutput"?

2023-02-20 Thread wang
Hi dear engineers, One question as title: Whether Flink SQL window operations support "Allow Lateness and SideOutput"? Just as supported in Datastream api (allowedLateness and sideOutputLateData) like: SingleOutputStreamOperator<>sumStream = dataStream.keyBy().timeWindow()

Re: Calculation of UI's maximum non-heap memory

2023-02-20 Thread Alexis Sarda-Espinosa
Hi Weihua, Thanks for your response, I am familiar with those calculations, the one I don't understand is the Maximum Non-Heap value. Regards, Alexis. On Tue, 21 Feb 2023, 04:45 Weihua Hu, wrote: > Hi, Alexis > > 1. With those configuration, Flink will set JVM parameters -Xms and -Xmx > to 673