dabla commented on code in PR #76408:
URL: https://github.com/apache/airflow/pull/76408#discussion_r4487065048
##########
providers/microsoft/azure/src/airflow/providers/microsoft/azure/transfers/s3_to_wasb.py:
##########
@@ -193,6 +193,29 @@ def execute(self, context: Context) -> list[str]:
# Return a list of the files that were moved
return files_to_move
+ @staticmethod
+ def _strip_overlapping_folder_markers(keys: list[str]) -> tuple[list[str],
list[str]]:
Review Comment:
I think this shows a deeper issue in Airflow, as for each transfer between
S3 (but not only S3) and another hook or vice versa, an operator gets created.
I'm thinking of reviving
[AIP-14](https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/103091837/AIP-14+Create+composable+operators+using+common+interfaces),
I'll start a discussion soon in the devlist as we are having the same story
again we had in the past with dedicated SQL operators per provider package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
