Taragolis commented on PR #25541: URL: https://github.com/apache/airflow/pull/25541#issuecomment-1207385297
@blcksrx Personal thoughts just copy code from [S3ToSnowflakeOperator](https://github.com/apache/airflow/blob/main/airflow/providers/snowflake/transfers/s3_to_snowflake.py#L26-L28) create new operator based, and inherit S3ToSnowflakeOperator by new operator (with deprecation warnings) Everything except copyOptions and VALIDATION_MODE already implemented there. And it already works with all Cloud Providers which already has configured `externalStage` for appropriate cloud provider (current and futures) @potiuk When I mention that I agreed with abstract class I meant that right now current `S3ToSnowflakeOperator` basically just a query builder for Snowflake `COPY INTO <table>` and only one piece of query might be changed depend on location (external stage, internal stage, external location), which could be implemented per appropriate class, same as it done in [BaseSQLToGCSOperator](https://github.com/apache/airflow/blob/ae7bf474109410fa838ab2728ae6d581cdd41808/airflow/providers/google/cloud/transfers/sql_to_gcs.py#L316-L326). I absolutely agree with the fact that it should be part of Snowflake Provider. And last but not least I do not sure that this operator (current and in this PR) should be a placed in `transfers` it only generates SQL Statement and everything transfers between Cloud (as well as local stages) complete internally in Snowflake -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
