[GitHub] [airflow] Bowrna commented on a change in pull request #21145: enter the shell breeze2 environment

2022-02-12 Thread GitBox
Bowrna commented on a change in pull request #21145: URL: https://github.com/apache/airflow/pull/21145#discussion_r805295828 ## File path: dev/breeze/src/airflow_breeze/shell/enter_shell.py ## @@ -0,0 +1,176 @@ +# Licensed to the Apache Software Foundation (ASF) under one +#

[GitHub] [airflow] sungpeo commented on pull request #21538: py files doesn't have to be checked is_zipfiles in process_files

2022-02-12 Thread GitBox
sungpeo commented on pull request #21538: URL: https://github.com/apache/airflow/pull/21538#issuecomment-1037900877 Without extra changes, I rebased my PR branch. > The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However,

[GitHub] [airflow] Bowrna commented on a change in pull request #21145: enter the shell breeze2 environment

2022-02-12 Thread GitBox
Bowrna commented on a change in pull request #21145: URL: https://github.com/apache/airflow/pull/21145#discussion_r805291826 ## File path: dev/breeze/src/airflow_breeze/shell/enter_shell.py ## @@ -0,0 +1,176 @@ +# Licensed to the Apache Software Foundation (ASF) under one +#

[GitHub] [airflow] besenthil commented on issue #21377: Databricks: add SQL endpoint operators

2022-02-12 Thread GitBox
besenthil commented on issue #21377: URL: https://github.com/apache/airflow/issues/21377#issuecomment-1037626693 @dinowernli Is there a test stub that I can use for testing the API call? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] github-actions[bot] commented on pull request #18938: Jinja templates should be rendered in dict keys

2022-02-12 Thread GitBox
github-actions[bot] commented on pull request #18938: URL: https://github.com/apache/airflow/pull/18938#issuecomment-1037598181 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank

[GitHub] [airflow] ac1997 commented on pull request #21351: Add and use supports_celery attribute for cli celery command validation

2022-02-12 Thread GitBox
ac1997 commented on pull request #21351: URL: https://github.com/apache/airflow/pull/21351#issuecomment-1037536124 Just want to resurface this PR... @potiuk Do you have any thoughts on this? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] ac1997 opened a new pull request #21541: Add str to xcom_pull task_ids typing

2022-02-12 Thread GitBox
ac1997 opened a new pull request #21541: URL: https://github.com/apache/airflow/pull/21541 `TaskInstance.xcom_pull` supports both `str` and `list` but the typing has only `List` in it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[airflow] branch main updated (0a2d0d1 -> cca2f94)

2022-02-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 0a2d0d1 Added template_ext = ('.json') to databricks operators #18925 (#21530) add cca2f94 Add mssql-cli to

[GitHub] [airflow] potiuk merged pull request #21520: Add mssql-cli to devel extra in Airflow

2022-02-12 Thread GitBox
potiuk merged pull request #21520: URL: https://github.com/apache/airflow/pull/21520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #21443: Status of testing Providers that were prepared on February 09, 2022

2022-02-12 Thread GitBox
potiuk commented on issue #21443: URL: https://github.com/apache/airflow/issues/21443#issuecomment-1037453412 Cool! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] thcidale0808 commented on pull request #21223: Added Feature: Create Pools from Env Vars.

2022-02-12 Thread GitBox
thcidale0808 commented on pull request #21223: URL: https://github.com/apache/airflow/pull/21223#issuecomment-1037443067 @potiuk , please find below a proposed solution and answers to your questions: ## Solution: Create a state file and a pool config sync cli and api to sync

[GitHub] [airflow] pateash edited a comment on issue #21443: Status of testing Providers that were prepared on February 09, 2022

2022-02-12 Thread GitBox
pateash edited a comment on issue #21443: URL: https://github.com/apache/airflow/issues/21443#issuecomment-1037411412 Hi @potiuk @eladkal , **apache-airflow-provider-github v1.0.0rc2** working as expected, I have validated against **airflow v2.2.3**

[GitHub] [airflow] pateash edited a comment on issue #21443: Status of testing Providers that were prepared on February 09, 2022

2022-02-12 Thread GitBox
pateash edited a comment on issue #21443: URL: https://github.com/apache/airflow/issues/21443#issuecomment-1037411412 Hi @potiuk @eladkal , **apache-airflow-provider-github v1.0.0rc2** working as expected, I have validated against **airflow v2.2.3**

[GitHub] [airflow] pateash removed a comment on issue #21443: Status of testing Providers that were prepared on February 09, 2022

2022-02-12 Thread GitBox
pateash removed a comment on issue #21443: URL: https://github.com/apache/airflow/issues/21443#issuecomment-1037409252 > k int @potiuk, **apache-airflow-providers-GitHub** tested and working as expected.

[GitHub] [airflow] pateash commented on issue #21443: Status of testing Providers that were prepared on February 09, 2022

2022-02-12 Thread GitBox
pateash commented on issue #21443: URL: https://github.com/apache/airflow/issues/21443#issuecomment-1037411412 @potiuk @eladkal , **apache-airflow-provider-github v1.0.0rc2** working as expected, I have validated against **airflow v2.2.3**

[GitHub] [airflow] pateash commented on issue #21443: Status of testing Providers that were prepared on February 09, 2022

2022-02-12 Thread GitBox
pateash commented on issue #21443: URL: https://github.com/apache/airflow/issues/21443#issuecomment-1037409252 > k int @potiuk, **apache-airflow-providers-GitHub** tested and working as expected.

[GitHub] [airflow] Acehaidrey commented on pull request #20733: Add Audit Log View to Dag View

2022-02-12 Thread GitBox
Acehaidrey commented on pull request #20733: URL: https://github.com/apache/airflow/pull/20733#issuecomment-1037384199 Thank you so much! Glad can help and had a lot of help from Sam + Brent here. I'll start the discussion in the slack about the improvements discussed. -- This is an

[GitHub] [airflow] hubert-pietron opened a new pull request #21540: Fix logging JDBC SQL error when task fails

2022-02-12 Thread GitBox
hubert-pietron opened a new pull request #21540: URL: https://github.com/apache/airflow/pull/21540 closes: #16295 closes: #18482 JDBC operator could not log errors when task failed. For specific type of error (jpype.java.sql.SQLException, jaydebeapi.DatabaseError,

[GitHub] [airflow] xyu opened a new pull request #21539: Filter out default configs when overrides exist.

2022-02-12 Thread GitBox
xyu opened a new pull request #21539: URL: https://github.com/apache/airflow/pull/21539 When sending configs to Airflow workers we materialize a temp config file. In #18772 a feature was added so that `_cmd` generated secrets are not written to the files in some cases instead favoring

[GitHub] [airflow] potiuk closed pull request #21520: Add mssql-cli to devel extra in Airflow

2022-02-12 Thread GitBox
potiuk closed pull request #21520: URL: https://github.com/apache/airflow/pull/21520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #21443: Status of testing Providers that were prepared on February 09, 2022

2022-02-12 Thread GitBox
potiuk commented on issue #21443: URL: https://github.com/apache/airflow/issues/21443#issuecomment-1037340722 Still some chances for testing as we are waiting for votes. So feel free :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] potiuk commented on pull request #21356: Standardize approach to dependencies

2022-02-12 Thread GitBox
potiuk commented on pull request #21356: URL: https://github.com/apache/airflow/pull/21356#issuecomment-1037338257 I think that one is pretty read if we will be ok with introducing the policies as I proposed . -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] potiuk edited a comment on pull request #21520: Add mssql-cli to devel extra in Airflow

2022-02-12 Thread GitBox
potiuk edited a comment on pull request #21520: URL: https://github.com/apache/airflow/pull/21520#issuecomment-1037334254 Hey @ephraimbuddy - I found a way of adding the mssql-cli without impacting our dependencies (I am using pipx to install it). @uranusjr - re pipx - is it often

[GitHub] [airflow] potiuk commented on pull request #21520: Add mssql-cli to devel extra in Airflow

2022-02-12 Thread GitBox
potiuk commented on pull request #21520: URL: https://github.com/apache/airflow/pull/21520#issuecomment-1037334254 Hey @ephraimbuddy - I found a way of adding the mssql-cli without impacting our dependencies (I am using pipx to install it). @uranusjr - re pipx - is it often the

[GitHub] [airflow] potiuk commented on a change in pull request #21145: enter the shell breeze2 environment

2022-02-12 Thread GitBox
potiuk commented on a change in pull request #21145: URL: https://github.com/apache/airflow/pull/21145#discussion_r805176319 ## File path: dev/breeze/src/airflow_breeze/shell/enter_shell.py ## @@ -0,0 +1,176 @@ +# Licensed to the Apache Software Foundation (ASF) under one +#

[GitHub] [airflow] github-actions[bot] commented on pull request #21538: py files doesn't have to be checked is_zipfiles in process_files

2022-02-12 Thread GitBox
github-actions[bot] commented on pull request #21538: URL: https://github.com/apache/airflow/pull/21538#issuecomment-1037266688 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] Bowrna commented on a change in pull request #21145: enter the shell breeze2 environment

2022-02-12 Thread GitBox
Bowrna commented on a change in pull request #21145: URL: https://github.com/apache/airflow/pull/21145#discussion_r805174798 ## File path: dev/breeze/src/airflow_breeze/shell/enter_shell.py ## @@ -0,0 +1,176 @@ +# Licensed to the Apache Software Foundation (ASF) under one +#

[GitHub] [airflow] potiuk commented on pull request #21145: enter the shell breeze2 environment

2022-02-12 Thread GitBox
potiuk commented on pull request #21145: URL: https://github.com/apache/airflow/pull/21145#issuecomment-1037260947 > @potiuk I am trying to run docker-compose command via subprocess and I face error when I pass env variables as -e. It works properly when set as os.environ variable and

[GitHub] [airflow] Bowrna commented on pull request #21145: enter the shell breeze2 environment

2022-02-12 Thread GitBox
Bowrna commented on pull request #21145: URL: https://github.com/apache/airflow/pull/21145#issuecomment-1037259920 @potiuk I am trying to run docker-compose command via subprocess and I face error when I pass env variables as -e. It works properly when set as os.environ variable and

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #21538: py files doesn't have to be checked is_zipfiles in process_files

2022-02-12 Thread GitBox
boring-cyborg[bot] commented on pull request #21538: URL: https://github.com/apache/airflow/pull/21538#issuecomment-1037251194 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] sungpeo opened a new pull request #21538: py files doesn't have to be checked is_zipfiles in process_files

2022-02-12 Thread GitBox
sungpeo opened a new pull request #21538: URL: https://github.com/apache/airflow/pull/21538 py files doesn't have to be checked with is_zipfiles in process_files like find_dag_file_paths in file.

[airflow] branch main updated (5590e98 -> 0a2d0d1)

2022-02-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 5590e98 Add Audit Log View to Dag View (#20733) add 0a2d0d1 Added template_ext = ('.json') to databricks

[GitHub] [airflow] potiuk closed issue #18925: Add template_ext = ('.json') to databricks operators

2022-02-12 Thread GitBox
potiuk closed issue #18925: URL: https://github.com/apache/airflow/issues/18925 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #21530: Added template_ext = ('.json') to databricks operators #18925

2022-02-12 Thread GitBox
boring-cyborg[bot] commented on pull request #21530: URL: https://github.com/apache/airflow/pull/21530#issuecomment-1037217272 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] potiuk merged pull request #21530: Added template_ext = ('.json') to databricks operators #18925

2022-02-12 Thread GitBox
potiuk merged pull request #21530: URL: https://github.com/apache/airflow/pull/21530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (a08140e -> 5590e98)

2022-02-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from a08140e fixup! Prepare to switch to debian bullseye (#21522) (#21536) add 5590e98 Add Audit Log View to Dag

[GitHub] [airflow] potiuk commented on pull request #20733: Add Audit Log View to Dag View

2022-02-12 Thread GitBox
potiuk commented on pull request #20733: URL: https://github.com/apache/airflow/pull/20733#issuecomment-1037216950 > Finally thank you @potiuk it is good :) Do I need to do anything else for this to be merged? Nope. Just merged! Great job. Audit log view has been a highly requested

[GitHub] [airflow] potiuk merged pull request #20733: Add Audit Log View to Dag View

2022-02-12 Thread GitBox
potiuk merged pull request #20733: URL: https://github.com/apache/airflow/pull/20733 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on issue #21537: airflow/providers/google/cloud/transfers/sql_to_gcs.py partition parquet files

2022-02-12 Thread GitBox
boring-cyborg[bot] commented on issue #21537: URL: https://github.com/apache/airflow/issues/21537#issuecomment-1037135852 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] manlouk opened a new issue #21537: airflow/providers/google/cloud/transfers/sql_to_gcs.py partition parquet files

2022-02-12 Thread GitBox
manlouk opened a new issue #21537: URL: https://github.com/apache/airflow/issues/21537 ### Description Add the ability to partition parquet files by columns. Right now you can partition files only size. ### Use case/motivation _No response_ ### Related issues

[GitHub] [airflow] pierrejeambrun edited a comment on pull request #20386: Add support for BeamGoPipelineOperator

2022-02-12 Thread GitBox
pierrejeambrun edited a comment on pull request #20386: URL: https://github.com/apache/airflow/pull/20386#issuecomment-1037129646 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] pierrejeambrun commented on pull request #20386: Add support for BeamGoPipelineOperator

2022-02-12 Thread GitBox
pierrejeambrun commented on pull request #20386: URL: https://github.com/apache/airflow/pull/20386#issuecomment-1037129646 > Unfortunately - static checks you will have to fix by pushing new code I am afraid :smile: @potiuk that new docstring-param hook got me I guess :joy:

[GitHub] [airflow] potiuk commented on pull request #21378: Switch to Debian 11 (bullseye) as base for our dockerfiles

2022-02-12 Thread GitBox
potiuk commented on pull request #21378: URL: https://github.com/apache/airflow/pull/21378#issuecomment-1037097959 Unfortuntely we cannot move to bullseye (yet). The problem is that MSSQL ODBC driver (which is the only "reliable" and stable drirver for MSSQL for SQLAlchemy) does not

[GitHub] [airflow] potiuk edited a comment on issue #18190: Docker image - Migrate to 3.x-slim-bullseye from 3.x-slim-buster

2022-02-12 Thread GitBox
potiuk edited a comment on issue #18190: URL: https://github.com/apache/airflow/issues/18190#issuecomment-1037097619 I asked the maintainers for the expected timeline https://github.com/MicrosoftDocs/sql-docs/issues/7255#issuecomment-1037097131 -- This is an automated message from the

[GitHub] [airflow] potiuk commented on issue #18190: Docker image - Migrate to 3.x-slim-bullseye from 3.x-slim-buster

2022-02-12 Thread GitBox
potiuk commented on issue #18190: URL: https://github.com/apache/airflow/issues/18190#issuecomment-1037097619 I asked the maintainer for the expected timeline https://github.com/MicrosoftDocs/sql-docs/issues/7255#issuecomment-1037097131 -- This is an automated message from the Apache

[GitHub] [airflow] uranusjr edited a comment on issue #20991: Elasticsearch remote log will not fetch task logs from manual dagruns before 2.2 upgrade

2022-02-12 Thread GitBox
uranusjr edited a comment on issue #20991: URL: https://github.com/apache/airflow/issues/20991#issuecomment-1037092707 The core “fix” is actually overriden when the ElasticSearch provider’s implementation is used, so just releasing the provider would be enough to resolve the title issue

[GitHub] [airflow] uranusjr commented on issue #20991: Elasticsearch remote log will not fetch task logs from manual dagruns before 2.2 upgrade

2022-02-12 Thread GitBox
uranusjr commented on issue #20991: URL: https://github.com/apache/airflow/issues/20991#issuecomment-1037092707 The core “fix” is actually overriden when the ElasticSearch provider’s implementation is used, so just releasing the provider would be enough to resolve the issue in Airflow

[GitHub] [airflow] potiuk commented on issue #18190: Docker image - Migrate to 3.x-slim-bullseye from 3.x-slim-buster

2022-02-12 Thread GitBox
potiuk commented on issue #18190: URL: https://github.com/apache/airflow/issues/18190#issuecomment-1037085507 Unfortuntely we cannot move to bullseye (yet). The problem is that MSSQL ODBC driver (which is the only "reliable" and stable drirver for MSSQL for SQLAlchemy) does not support

[airflow] branch main updated (44bd211 -> a08140e)

2022-02-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 44bd211 Use compat data interval shim in log handlers (#21289) add a08140e fixup! Prepare to switch to debian

[GitHub] [airflow] potiuk merged pull request #21536: fixup! Prepare to switch to debian bullseye (#21522)

2022-02-12 Thread GitBox
potiuk merged pull request #21536: URL: https://github.com/apache/airflow/pull/21536 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] Acehaidrey commented on pull request #20733: Add Audit Log View to Dag View

2022-02-12 Thread GitBox
Acehaidrey commented on pull request #20733: URL: https://github.com/apache/airflow/pull/20733#issuecomment-1037056388 Finally thank you @potiuk it is good :) Do I need to do anything else for this to be merged? -- This is an automated message from the Apache Git Service. To respond to