yuseok89 commented on code in PR #72892:
URL: https://github.com/apache/airflow/pull/72892#discussion_r4031981873
##########
providers/amazon/src/airflow/providers/amazon/aws/transfers/azure_blob_to_s3.py:
##########
@@ -116,6 +116,29 @@ def __init__(
self.wasb_extra_args = wasb_extra_args or {}
self.s3_extra_args = s3_extra_args or {}
+ @staticmethod
+ def _strip_overlapping_folder_markers(keys: list[str]) -> tuple[list[str],
list[str]]:
Review Comment:
Thanks for the suggestion. I moved the folder-marker filtering into a shared
Amazon transfer utility, and both GCSToS3Operator and
AzureBlobStorageToS3Operator now use it. I also moved the relevant cases into
focused tests for the shared utility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
