Lee-W commented on code in PR #36578:
URL: https://github.com/apache/airflow/pull/36578#discussion_r1445594656
##########
airflow/providers/amazon/aws/sensors/s3.py:
##########
@@ -121,6 +124,11 @@ def _check_key(self, key):
# Reduce the set of metadata to size only
files = [{"Size": f["Size"]} for f in key_matches]
+ elif self.use_regex:
+ keys = self.hook.get_file_metadata("", bucket_name)
+ key_matches = [k for k in keys if re.match(pattern=key,
string=k["Key"])]
+ if not key_matches:
+ return False
Review Comment:
Yep, we might want to rewrite sync and async _check_key. 🤔 I think it might
be better to create another PR for it to make it easier to review
##########
airflow/providers/amazon/aws/triggers/s3.py:
##########
@@ -39,7 +39,9 @@ class S3KeyTrigger(BaseTrigger):
:param wildcard_match: whether the bucket_key should be interpreted as a
Unix wildcard pattern
:param aws_conn_id: reference to the s3 connection
+ :param use_regex: whether to use regex to check bucket
:param hook_params: params for hook its optional
+ :param use_regex: whether to use regex to check bucket
Review Comment:
Yep, it's accidentally added 🤦♂️ Just remove it. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
