Flink plugin can't get access to some of the classes in /opt/flink/lib folder

2022-03-22 Thread Fan Xie
Hi Flink Community, Recently I implemented a new metrics reporter to report Flink runtime metrics to a Kafka topic. After building the project, I move my reporter jar file from my build folder to the plugin folder: COPY build/flink-diagnostics-message-reporter-${VERSION}.jar ${FLINK_HOME}/plug

Flink TaskManager container got restarted by K8S very frequently

2021-07-21 Thread Fan Xie
Hi Flink Community, Recently I deployed a Flink cluster(1 JM, 1TM) with k8s standalone mode. Later on I notice that the pod which the TM is running on got restarted by k8s very frequently (3 times within 10 minutes). And I didn't see any error log for this pod. I tried to increase the containe

Re: Customized Metric Reporter can not be found by Flink

2021-05-11 Thread Fan Xie
Hi Yangze, I add that file and it works! Thanks for helping with this! Best, Fan From: Yangze Guo Sent: Tuesday, May 11, 2021 7:12 PM To: Fan Xie Cc: user@flink.apache.org Subject: Re: Customized Metric Reporter can not be found by Flink Hi, Fan Flink loaded

Customized Metric Reporter can not be found by Flink

2021-05-11 Thread Fan Xie
Hi Flink Community, Recently I implemented a customized metric reporter (named: DiagnosticsMessageReporter) to report Flink metrics to a Kafka topic. I built this reporter into a jar file and copy it to /opt/flink/plugins/DiagnosticsMessageReporter/DiagnosticsMessageReporter.jar for both the J

Re: Question regarding cpu limit config in Flink standalone mode

2021-05-07 Thread Fan Xie
the native kubernetes deployment, TM pods are requested by Flink's ResourceManager. Thus, the configuration option `kubernets.taskmanager.cpu` controls the cpu resource of pods Flink requests from Kubernetes. Thank you~ Xintong Song On Fri, May 7, 2021 at 10:35 AM Fan Xie mailto:f...@linkedin.

Question regarding cpu limit config in Flink standalone mode

2021-05-06 Thread Fan Xie
Hi Flink Community, Recently I am working on an auto-scaling project that needs to dynamically adjust the cpu config of Flink standalone jobs . Our jobs will be running on standalone mode in a k8s cluster. After going through the configuration doc: https://ci.apache.org/projects/flink/flink-doc