Kengo Seki created AIRFLOW-2299: ----------------------------------- Summary: Add S3 Select functionarity to S3FileTransformOperator Key: AIRFLOW-2299 URL: https://issues.apache.org/jira/browse/AIRFLOW-2299 Project: Apache Airflow Issue Type: Improvement Components: aws, operators Reporter: Kengo Seki
S3FileTransformOperator downloads the whole file from S3 before transforming and uploading it, but it's inefficient if the original file is large but the necessary part is small. S3 Select, [which became GA recently|https://aws.amazon.com/about-aws/whats-new/2018/04/amazon-s3-select-is-now-generally-available/], can improve its efficiency and usablitily. -- This message was sent by Atlassian JIRA (v7.6.3#76005)