hamzamazhar opened a new issue, #77137: URL: https://github.com/apache/airflow/issues/77137
### Under which category would you file this issue? Task SDK ### Apache Airflow version 3.x.x ### What happened and how to reproduce it? ## Issue I have a custom operator that inherits from `PythonOperator` and passes arguments to its callable through `op_kwargs`. Following the custom-operator testing pattern in the [documentation](https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html), I instantiate the operator and call: ```operator.execute(context={})``` This works with Airflow `2.11.0`, but fails with Airflow `3.0.0` (also tested with `Airflow 3.3.1`) because the callable receives none of its required arguments. Error: ``` TypeError: MyCustomPythonOperator.print_arg() missing 1 required positional arguments: 'arg_1' ``` Passing a non-empty context makes the same test pass: ```operator.execute(context={"test_mode": True})``` Calling the callable directly with the configured arguments also works: ```operator.python_callable(**operator.op_kwargs)``` ## How to reproduce ### Custom Python Operator ``` # Change the import path for airflow 2.11 from airflow.providers.standard.operators.python import PythonOperator class MyCustomPythonOperator(PythonOperator): def __init__( self, task_id: str, arg_1: str, **kwargs, ): super().__init__( task_id=task_id, python_callable=print_arg, op_kwargs={ "arg_1": arg_1, }, **kwargs, ) def print_arg(arg_1): print(arg_1) return 1 ``` ### Test file ``` from my_custom_operators import MyCustomPythonOperator def test_my_custom_operator_execute(): op = MyCustomPythonOperator(task_id="my_custom_operator_task", arg_1="abc") assert op.execute(context={}) == 1 ``` ## Debugging I looked at the source code for both [Airflow 2.11](https://github.com/apache/airflow/blob/2.11.2/airflow/utils/context.py) and [main](https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/definitions/context.py) and found out a difference in the `context_merge` function for the `execute()` method of the operator. In Airflow 3.x, there is an additional check, which doesn't exist on Airflow 2.11. I suspect this ignores any args and kwargs if the context is empty: ``` if not context: context = Context() ``` ### What you think should happen instead? This should work as described in the example [documentation here](https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html) for Airflow 3 as well: ``` def test_my_custom_operator_execute(): op = MyCustomOperator(task_id="my_custom_operator_task", prefix="s3://bucket/some/prefix") assert op.execute(context={}) == "expected return value" ``` ### Operating System _No response_ ### Deployment None ### Apache Airflow Provider(s) _No response_ ### Versions of Apache Airflow Providers _No response_ ### Official Helm Chart version Not Applicable ### Kubernetes Version _No response_ ### Helm Chart configuration _No response_ ### Docker Image customizations _No response_ ### Anything else? _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
