lhotari opened a new issue, #23723: URL: https://github.com/apache/pulsar/issues/23723
### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Motivation I haven't found a way to tune the memory limit for the Pulsar client used in Pulsar Functions. The memory limit is disabled in Pulsar Functions: https://github.com/apache/pulsar/blob/82237d3684fe506bcb6426b3b23f413422e6e4fb/pulsar-functions/instance/src/main/java/org/apache/pulsar/functions/instance/InstanceUtils.java#L161 There's code for configuring it: https://github.com/apache/pulsar/blob/05e57dd3a443c5b99c21054c56a1b497455fa867/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/thread/ThreadRuntimeFactory.java#L106-L168 but it doesn't get called in the JavaInstanceStarter case. The JavaInstanceStarter is used when running Java functions in process / k8s / function mesh runtimes (other than thread runtime): https://github.com/apache/pulsar/blob/567174f43528c0f7ae917bfb5166213973c62c29/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/JavaInstanceStarter.java#L236-L245 `Optional.empty()` is passed to memory limit. https://github.com/apache/pulsar/blob/05e57dd3a443c5b99c21054c56a1b497455fa867/pulsar-functions/runtime/src/main/java/org/apache/pulsar/functions/runtime/thread/ThreadRuntimeFactory.java#L92-L104 It looks like the memory limit solution has been only implemented for the ThreadRuntimeFactory in #9320. When using ThreadRuntimeFactory, it's possible to set the limit in `conf/functions_worker.yml`. This isn't helpful for other runtime types (process, k8s). ### Solution Add the Pulsar client memory limit configuration to `org.apache.pulsar.common.functions.FunctionConfig`, `org.apache.pulsar.functions.instance.InstanceConfig` and `org.apache.pulsar.functions.worker.WorkerConfig` and make it configurable in the same way as other instance config parameters such as `maxPendingAsyncRequests`. The default would be configured in `WorkerConfig` for all functions and the function configuration would be in `InstanceConfig` and configurable with `FunctionConfig` in yaml and command line. It's not fully consistent how configs should be handled. There's also the protobuf `org.apache.pulsar.functions.proto.Function.FunctionDetails` where configs are passed. ### Alternatives _No response_ ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
