mik-laj commented on issue #24456: URL: https://github.com/apache/airflow/issues/24456#issuecomment-1166519231
@sfc-gh-madkins I wonder if I should set [`QUERY_TAG`](https://docs.snowflake.com/en/sql-reference/parameters.html#query-tag) and pass the following values: - [Instance name](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#instance-name) - DAG ID - Task ID - Run ID Does Snowflake have any recommendations for ISV on how to use the `query_tag` that we can use here? Unfortunately, the public documentation is quite poor on the description of this parameter and I do not know the best practices on how it should be used. I also wonder what to do to make Snowflake customers know about this integration. Currently, support for Snowflake in Airflow is not mentioned in the end-customer Snowflake documentation. For example, we can add Apache Airflow to the [ecosystem section](https://docs.snowflake.com/en/user-guide/ecosystem.html). There is also integration for Apache Beam added, but under the commercial name - Google Dataflow, so adding Apache Airflow under the commercial name - [Cloud Composer](https://cloud.google.com/composer) is one solution. <img width="1080" alt="Screenshot 2022-06-26 at 14 22 46" src="https://user-images.githubusercontent.com/12058428/175813909-ca28f942-1832-4bb2-8c42-b281bc88c3c0.png";> But Apache Airflow has more service providers, e.g. [Amazon Managed Workflows for Apache Airflow (MWAA)](https://docs.aws.amazon.com/mwaa/latest/userguide/what-is-mwaa.html) or [Astronomer](https://www.astronomer.io/). In addition, Apache Beam can also be run on-premises when using Apache Spark runner instead of Google Dataflow Runner, which also allows us to use SnowflakeIO for Apache Beam without Google Dataflow. For comparison, dbt has explicitly described Apache Airflow, and they even published a guide on how to use Apache Airflow with dbt cloud. https://docs.getdbt.com/guides/orchestration/airflow-and-dbt-cloud/1-airflow-and-dbt-cloud https://docs.getdbt.com/docs/running-a-dbt-project/running-dbt-in-production CC: @sfc-gh-pgancz, @sfc-gh-sleslie -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
![https://user-images.githubusercontent.com/12058428/175813909-ca28f942-1832-4bb2-8c42-b281bc88c3c0.png"](https://user-images.githubusercontent.com/12058428/175813909-ca28f942-1832-4bb2-8c42-b281bc88c3c0.png"){kind=link}