Taragolis commented on PR #25541:
URL: https://github.com/apache/airflow/pull/25541#issuecomment-1207392425
> Crossed my mind too. I think it shoudl indeed be in the regular operators
package.
The only one case when it could be as `transfers` when query use credentials
(instead of stages or integrations) for AWS or Azure (GCS not supported), like
```sql
COPY INTO FOO.BAR
FROM s3://source-bucket-123/key/path/location
CREDENTIALS = ( AWS_KEY_ID='xxxx' AWS_SECRET_KEY='xxxxx'
AWS_TOKEN='xxxxxx' )
FILE_FORMAT = ( TYPE = PARQUET )
;
```
```sql
COPY INTO FOO.BAR
FROM
azure://{account}.blob.core.windows.net/source-container-123/key/path/location
CREDENTIALS = ( AZURE_SAS_TOKEN = 'xxx' )
FILE_FORMAT = ( TYPE = PARQUET )
;
```
But **I don't think this a good idea** to implement operator which pass
credentials directly to SQL query (not as part Snowflake providers and any
others). This credentials could expose to logs and also it not recommended by
Snowflake docs:
```
We highly recommend the use of storage integrations.
This option avoids the need to supply cloud storage credentials using the
CREDENTIALS parameter when creating stages or loading data.
```
However if we implement `COPY INTO <table>` as some kind of abstract
operator than end users might create their own custom operator to fit their
requirements and provide credentials as part of SQL query if they do not won't
follow recommendation and security practice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
