o-nikolas commented on code in PR #36578:
URL: https://github.com/apache/airflow/pull/36578#discussion_r1445417180
##########
airflow/providers/amazon/aws/triggers/s3.py:
##########
@@ -39,7 +39,9 @@ class S3KeyTrigger(BaseTrigger):
:param wildcard_match: whether the bucket_key should be interpreted as a
Unix wildcard pattern
:param aws_conn_id: reference to the s3 connection
+ :param use_regex: whether to use regex to check bucket
:param hook_params: params for hook its optional
+ :param use_regex: whether to use regex to check bucket
Review Comment:
added twice?
##########
airflow/providers/amazon/aws/sensors/s3.py:
##########
@@ -121,6 +124,11 @@ def _check_key(self, key):
# Reduce the set of metadata to size only
files = [{"Size": f["Size"]} for f in key_matches]
+ elif self.use_regex:
+ keys = self.hook.get_file_metadata("", bucket_name)
+ key_matches = [k for k in keys if re.match(pattern=key,
string=k["Key"])]
+ if not key_matches:
+ return False
Review Comment:
it's unfortunate that this is re-implemented in the sensor and was also in
the hook above, but I suppose it's out of scope to fix that in this PR :cry:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
