john-jac commented on code in PR #65789:
URL: https://github.com/apache/airflow/pull/65789#discussion_r3139370655
##########
providers/amazon/docs/operators/s3/s3.rst:
##########
@@ -178,6 +178,28 @@ for more information about Amazon S3 prefixes.
:start-after: [START howto_operator_s3_list_prefixes]
:end-before: [END howto_operator_s3_list_prefixes]
+.. _howto/operator:S3ReadObjectOperator:
+
+Read an Amazon S3 object
+========================
+
+To read the content of an Amazon S3 object and return it as a string you can
use
+:class:`~airflow.providers.amazon.aws.operators.s3.S3ReadObjectOperator`.
+
+.. exampleinclude::
/../../amazon/tests/system/amazon/aws/example_s3_read_object.py
+ :language: python
+ :dedent: 4
+ :start-after: [START howto_operator_s3_read_object]
+ :end-before: [END howto_operator_s3_read_object]
+
+You can also pass a full ``s3://`` URL as the key:
Review Comment:
Thanks! Fixed.
##########
providers/amazon/src/airflow/providers/amazon/aws/operators/s3.py:
##########
@@ -930,3 +930,67 @@ def execute(self, context: Context):
)
return self.hook.list_prefixes(bucket_name=self.bucket,
prefix=self.prefix, delimiter=self.delimiter)
+
+
+class S3ReadObjectOperator(AwsBaseOperator[S3Hook]):
+ """
+ Read an S3 object and return its content as a string.
+
+ Uses ``S3Hook.read_key`` to fetch the object body, decode it as UTF-8,
+ and return the resulting string. The return value is automatically pushed
+ to XCom so downstream tasks can consume it.
+
+ .. seealso::
+ For more information on how to use this operator, take a look at the
guide:
+ :ref:`howto/operator:S3ReadObjectOperator`
+
+ :param s3_bucket: Name of the S3 bucket. (templated)
+ It should be omitted when ``s3_key`` is provided as a full ``s3://``
URL.
+ :param s3_key: The key of the object to read. (templated)
+ It can be either a full ``s3://`` style URL or a relative path from
root level.
+ When specified as a full ``s3://`` URL, omit ``s3_bucket``.
+ :param aws_conn_id: The Airflow connection used for AWS credentials.
+ If this is ``None`` or empty then the default boto3 behaviour is used.
If
+ running Airflow in a distributed manner and aws_conn_id is None or
+ empty, then default boto3 configuration would be used (and must be
+ maintained on each worker node).
+ :param region_name: AWS region_name. If not specified then the default
boto3 behaviour is used.
+ :param verify: Whether or not to verify SSL certificates. See:
+
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
+ :param botocore_config: Configuration dictionary (key-values) for botocore
client. See:
+
https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html
+ """
+
+ template_fields: Sequence[str] = aws_template_fields("s3_bucket", "s3_key")
+ aws_hook_class = S3Hook
+
+ def __init__(
+ self,
+ *,
+ s3_key: str,
Review Comment:
Thanks! Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
