Add a space between -yD and the param should do the trick. Best, Zhanghao Chen ________________________________ From: Lei Wang <[email protected]> Sent: Thursday, April 11, 2024 19:40 To: Zhanghao Chen <[email protected]> Cc: Biao Geng <[email protected]>; user <[email protected]> Subject: Re: How to enable RocksDB native metrics？
Hi Zhanghao， flink run -m yarn-cluster -ys 4 -ynm EventCleaning_wl -yjm 2G -ytm 16G -yqu default -p 8 -yDstate.backend.latency-track.keyed-state-enabled=true -c com.zkj.task.EventCleaningTask SourceDataCleaning-wl_0410.jar --sourceTopic dwd_audio_record --groupId clean_wl_ --sourceServers x.x.x.x:9092 Tried， it doesn't work, the error is: Could not get job jar and dependencies from JAR file: JAR file does not exist: -yDstate.backend.latency-track.keyed-state-enabled=true Thanks， Lei On Thu, Apr 11, 2024 at 5:19 PM Zhanghao Chen <[email protected]<mailto:[email protected]>> wrote: Hi Lei, You are using an old-styled CLI for YARN jobs where "-yD" instead of "-D" should be used. ________________________________ From: Lei Wang <[email protected]<mailto:[email protected]>> Sent: Thursday, April 11, 2024 12:39 To: Biao Geng <[email protected]<mailto:[email protected]>> Cc: user <[email protected]<mailto:[email protected]>> Subject: Re: How to enable RocksDB native metrics？ Hi Biao, I tried, it doesn't work. The cmd is: flink run -m yarn-cluster -ys 4 -ynm EventCleaning_wl -yjm 2G -ytm 16G -yqu default -p 8 -Dstate.backend.latency-track.keyed-state-enabled=true -c com.zkj.task.EventCleaningTask SourceDataCleaning-wl_0410.jar --sourceTopic dwd_audio_record --groupId clean_wl_ --sourceServers x.x.x.x:9092 Seems the -D param is ignored. Even i change the param to a wrong spelling, the job is submitted successfully. Any suggestions on this? Thanks, Lei On Mon, Apr 8, 2024 at 9:48 AM Biao Geng <[email protected]<mailto:[email protected]>> wrote: Hi Lei, You can use the "-D" option in the command line to set configs for a specific job. E.g, `flink run-application -t yarn-application -Djobmanager.memory.process.size=1024m .... `. See https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/deployment/cli/ for more details. Best, Biao Geng Marco Villalobos <[email protected]<mailto:[email protected]>> 于2024年4月8日周一 09:22写道： Hi Lei, Have you tried enabling these Flink configuration properties? <https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/deployment/config/#rocksdb-native-metrics> Configuration<https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/deployment/config/#rocksdb-native-metrics> nightlies.apache.org<https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/deployment/config/#rocksdb-native-metrics> [favicon.png]<https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/deployment/config/#rocksdb-native-metrics> Sent from my iPhone On Apr 7, 2024, at 6:03 PM, Lei Wang <[email protected]<mailto:[email protected]>> wrote: I want to enable it only for specified jobs, how can I specify the configurations on cmd line when submitting a job? Thanks， Lei On Sun, Apr 7, 2024 at 4:59 PM Zakelly Lan <[email protected]<mailto:[email protected]>> wrote: Hi Lei, You can enable it by some configurations listed in: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/#rocksdb-native-metrics (RocksDB Native Metrics) Best, Zakelly On Sun, Apr 7, 2024 at 4:59 PM Zakelly Lan <[email protected]<mailto:[email protected]>> wrote: Hi Lei, You can enable it by some configurations listed in: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/#rocksdb-native-metrics (RocksDB Native Metrics) Best, Zakelly On Sun, Apr 7, 2024 at 4:47 PM Lei Wang <[email protected]<mailto:[email protected]>> wrote: Using big state and want to do some performance tuning, how can i enable RocksDB native metrics？ I am using Flink 1.14.4 Thanks， Lei
