Have started to push our group to standardizing on airflow. We still have a few large Redshift clusters.
The s3_to_redshift_operator.py only appears to be written to authenticate via secret/access-keys. We no longer use Key Based authentication and rely upon Role Based, therefore IAM groups. What would the recommended way - in airflow - to rely upon IAM for authentication for COPY Commands to Redshift? Should I rely on a plugin? I'd be interested to contribute to the project by updating/extending s3_to_redshift_operator.py should that be welcome. References: * https://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html * https://airflow.readthedocs.io/en/latest/_modules/airflow/operators/s3_to_redshift_operator.html * https://github.com/apache/airflow/blob/master/airflow/operators/s3_to_redshift_operator.py
