davidradl commented on code in PR #28692: URL: https://github.com/apache/flink/pull/28692#discussion_r3585909784
########## docs/content/docs/ops/metrics.md: ########## @@ -2441,6 +2469,22 @@ It is recommended to only use them for debugging purposes. If state.ttl is enabled, the size of the value will include the size of the TTL-related timestamp. The value size of AggregatingState is not accounted for because AggregatingState returns a result processed by a user-defined AggregateFunction, whereas currently, only the actual stored data size in the state can be tracked. +## UDF metrics + +Flink can track the processing time and exceptions of SQL/Table user-defined functions on a per-operator basis. This feature is disabled by default. +To enable it you must set `table.exec.udf-metric-enabled` to `true`. When disabled, nothing is registered and there is no overhead. + +Once enabled, each user-defined function gets two metrics on the operator that runs it, scoped under `udf.<udf_name>`: a `udfProcessingTime` histogram and a `udfExceptionCount` counter. + +Flink samples the processing time every `N` invocations, in which `N` is defined by `table.exec.udf-metric.sample-interval`. +This configuration has a default value of 100. A smaller value will get more accurate results but have a higher performance impact since it is sampled more frequently. +Exceptions are counted on every invocation and are not sampled. + +The metrics cover synchronous and asynchronous scalar and table functions. Process table functions (`PROCESS_TABLE`), aggregate functions, and functions registered through the deprecated `registerFunction` API are not covered. + +<span class="label label-danger">Warning</span> Enabling UDF metrics may impact the performance, both from the sampled timing and from the metric reporter. +It is recommended to only use them for debugging purposes. Review Comment: is this advice the same for all metrics - or is this metric particularly performance adverse. Do we have an idea of what the performance overhead is when switching this on? I would expect metrics overhead would/should be minimal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
