xintongsong commented on code in PR #72: URL: https://github.com/apache/flink-agents/pull/72#discussion_r2238884187
########## python/flink_agents/plan/function.py: ########## @@ -147,6 +231,71 @@ def check_signature(self, *args: Tuple[Any, ...]) -> None: def call_python_function(module: str, qualname: str, func_args: Tuple[Any, ...]) -> Any: - """Used to call a Python function in the Pemja environment.""" - func = PythonFunction(module=module, qualname=qualname) + """Used to call a Python function in the Pemja environment. + + Uses selective caching to reuse PythonFunction instances for identical + (module, qualname) pairs to improve performance during frequent invocations. + Only caches functions that are safe to cache (no closures, generators, etc.). + + Parameters + ---------- + module : str + Name of the Python module where the function is defined. + qualname : str + Qualified name of the function (e.g., 'ClassName.method' for class methods). + func_args : Tuple[Any, ...] + Arguments to pass to the function. + + Returns: + ------- + Any + The result of calling the function with the provided arguments. + """ + cache_key = (module, qualname) + + if cache_key not in _PYTHON_FUNCTION_CACHE: + python_func = PythonFunction(module=module, qualname=qualname) + try: + actual_func = python_func._PythonFunction__get_func() + if _is_function_cacheable(actual_func): + _PYTHON_FUNCTION_CACHE[cache_key] = python_func + else: + return python_func(*func_args) + except Exception: + return python_func(*func_args) Review Comment: This means for non-cacheable functions, we will go through the `_is_function_cacheable` inspection every time it gets called. Might be better to check it only once on initialization. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
