Ferdinanddb commented on issue #50915:
URL: https://github.com/apache/airflow/issues/50915#issuecomment-3033724211
Hi @valentinDruzhinin ,
Did you try to use this feature with an actual operator?
I am asking because I only see tests related to the S3Hook in your merge
request, and I am currently testing the provider version 9.9.0 (where this
change is included) and the following does not work.
```python
my_task_name = S3ToGCSOperator(
task_id="my_task_name",
bucket=S3_EQUITY_LOOKUP_BUCKET_NAME,
prefix="somefile.csv",
gcp_conn_id="google_cloud_default",
apply_gcs_prefix=True,
dest_gcs=f"gs://{GCS_LANDING_ZONE_BUCKET}/{dag.dag_id}/{{{{
ds_nodash }}}}/somefile.csv",
replace=True,
aws_conn_id="aws_conn",
requester_pays=True,
)
```
I get the following error when the DAG gets parsed by the dag processor:
```
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/operator.py",
line 502, in apply_defaults
result = func(self, **kwargs, default_args=default_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/operator.py",
line 1026, in __init__
raise TypeError(
TypeError: Invalid arguments were passed to S3ToGCSOperator (task_id:
my_task_name). Invalid arguments were:
**kwargs: {'requester_pays': True}
```
This[ `S3ToGCSOperator`
operator](https://airflow.apache.org/docs/apache-airflow-providers-google/stable/_modules/airflow/providers/google/cloud/transfers/s3_to_gcs.html#S3ToGCSOperator)
is from the google provider, but it relies on `S3ListOperator` as you can see
in the link, so I don't really understand why it does not work since it accepts
a `**kwargs` and that the kwargs is passed to the `super().__init__()` so it is
passed to `S3ListOperator`..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
