[GitHub] [airflow] potiuk closed pull request #17576: Add pre/post execution hooks

2021-08-23 Thread GitBox
potiuk closed pull request #17576: URL: https://github.com/apache/airflow/pull/17576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow-site] kaxil merged pull request #468: Add documentation for Apache Airflow 2.1.3

2021-08-23 Thread GitBox
kaxil merged pull request #468: URL: https://github.com/apache/airflow-site/pull/468 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow-site] kaxil opened a new pull request #468: Add documentation for Apache Airflow 2.1.3

2021-08-23 Thread GitBox
kaxil opened a new pull request #468: URL: https://github.com/apache/airflow-site/pull/468 Add docs for Airflow and Docker images -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[airflow-site] 02/03: Remove log files

2021-08-23 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch airflow-2.1.3-docker in repository https://gitbox.apache.org/repos/asf/airflow-site.git commit 260873efa12d4d008ef95a85538774a14aa48be7 Author: Kaxil Naik AuthorDate: Mon Aug 23 18:56:44 2021

[airflow-site] branch airflow-2.1.3-docker created (now 545bbb5)

2021-08-23 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch airflow-2.1.3-docker in repository https://gitbox.apache.org/repos/asf/airflow-site.git. at 545bbb5 Remove Airflow Summit banner This branch includes the following new commits: n

[GitHub] [airflow] iblaine commented on pull request #17777: get_pandas_df() fails when it tries to read an empty table

2021-08-23 Thread GitBox
iblaine commented on pull request #1: URL: https://github.com/apache/airflow/pull/1#issuecomment-903993858 Setting this [line](https://github.com/apache/airflow/blob/main/tests/test_utils/mock_process.py#L59) to `self.iterable = []` will trigger this bug. -- This is an automated

[GitHub] [airflow] iblaine commented on pull request #17777: get_pandas_df() fails when it tries to read an empty table

2021-08-23 Thread GitBox
iblaine commented on pull request #1: URL: https://github.com/apache/airflow/pull/1#issuecomment-903981341 > I wonder if it’d work to do `SELECT * FROM table LIMIT 0` Mock seems to return a result set regardless of the SQL. -- This is an automated message from the Apache Gi

[GitHub] [airflow] iblaine commented on issue #17765: get_pandas_df() fails when it tries to read an empty table

2021-08-23 Thread GitBox
iblaine commented on issue #17765: URL: https://github.com/apache/airflow/issues/17765#issuecomment-903980110 PR for this issue is https://github.com/apache/airflow/pull/1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [airflow] subkanthi opened a new pull request #17790: Gcp ai hyperparameter tuning

2021-08-23 Thread GitBox
subkanthi opened a new pull request #17790: URL: https://github.com/apache/airflow/pull/17790 closes: #17348 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[GitHub] [airflow] dimberman commented on a change in pull request #15330: Add a Docker Taskflow decorator

2021-08-23 Thread GitBox
dimberman commented on a change in pull request #15330: URL: https://github.com/apache/airflow/pull/15330#discussion_r694154586 ## File path: airflow/providers_manager.py ## @@ -278,6 +309,51 @@ def _get_attr(obj: Any, attr_name: str): return None return g

[GitHub] [airflow] jedcunningham commented on a change in pull request #17757: Improves documentation about modules management

2021-08-23 Thread GitBox
jedcunningham commented on a change in pull request #17757: URL: https://github.com/apache/airflow/pull/17757#discussion_r694153747 ## File path: docs/apache-airflow/modules_management.rst ## @@ -68,99 +81,192 @@ In the next section, you will learn how to create your own simpl

[GitHub] [airflow] jedcunningham commented on a change in pull request #17757: Improves documentation about modules management

2021-08-23 Thread GitBox
jedcunningham commented on a change in pull request #17757: URL: https://github.com/apache/airflow/pull/17757#discussion_r694114919 ## File path: docs/apache-airflow/modules_management.rst ## @@ -25,9 +25,21 @@ Airflow configuration. The following article will describe how you

[airflow] branch main updated: Fix failing Helm Chart docs test (#17789)

2021-08-23 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d97e50f Fix failing Helm Chart docs test (#1778

[GitHub] [airflow] kaxil merged pull request #17789: Fix failing Helm Chart docs test

2021-08-23 Thread GitBox
kaxil merged pull request #17789: URL: https://github.com/apache/airflow/pull/17789 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [airflow] kaxil opened a new pull request #17789: Fix failing Helm Chart docs test

2021-08-23 Thread GitBox
kaxil opened a new pull request #17789: URL: https://github.com/apache/airflow/pull/17789 Fixes: https://github.com/apache/airflow/runs/3402579247#step:7:718 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/ai

[airflow] annotated tag 2.1.3 updated (595fb4c -> e785a18)

2021-08-23 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to annotated tag 2.1.3 in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag 2.1.3 was modified! *** from 595fb4c (commit) to e785a18 (tag) tagging 595fb4c95f91ee

[airflow] annotated tag constraints-2.1.3 updated (595fb4c -> 4e7a847)

2021-08-23 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to annotated tag constraints-2.1.3 in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag constraints-2.1.3 was modified! *** from 595fb4c (commit) to 4e7a847 (tag)

svn commit: r49607 - /release/airflow/2.1.2/

2021-08-23 Thread kaxilnaik
Author: kaxilnaik Date: Mon Aug 23 16:30:20 2021 New Revision: 49607 Log: Remove old release: 2.1.2 Removed: release/airflow/2.1.2/

svn commit: r49606 - /release/airflow/2.1.3/

2021-08-23 Thread kaxilnaik
Author: kaxilnaik Date: Mon Aug 23 16:30:03 2021 New Revision: 49606 Log: Release Airflow 2.1.3 from 2.1.3rc1 Added: release/airflow/2.1.3/ release/airflow/2.1.3/apache-airflow-2.1.3-source.tar.gz - copied unchanged from r49605, dev/airflow/2.1.3rc1/apache-airflow-2.1.3-source.tar.

[GitHub] [airflow] ManiBharataraju commented on issue #17752: Skip Scheduled Runs

2021-08-23 Thread GitBox
ManiBharataraju commented on issue #17752: URL: https://github.com/apache/airflow/issues/17752#issuecomment-903926318 @sou-joshi - Can't you add a BranchPythonOperator as the first task and in the function, query the DagRun table and check if the dag has already run and based on the result

[airflow] branch main updated (1a929f7 -> 32bfb2f)

2021-08-23 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 1a929f7 Fix dag_processing.last_duration metric random holes (#17769) add 32bfb2f Add Triggerer to Helm Char

[GitHub] [airflow] kaxil merged pull request #17743: Add Triggerer to Helm Chart

2021-08-23 Thread GitBox
kaxil merged pull request #17743: URL: https://github.com/apache/airflow/pull/17743 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [airflow] suhanovv commented on pull request #17769: Fix dag_processing.last_duration metric random holes

2021-08-23 Thread GitBox
suhanovv commented on pull request #17769: URL: https://github.com/apache/airflow/pull/17769#issuecomment-903922435 @xinbinhuang good, thnx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [airflow] xinbinhuang commented on pull request #17769: Fix dag_processing.last_duration metric random holes

2021-08-23 Thread GitBox
xinbinhuang commented on pull request #17769: URL: https://github.com/apache/airflow/pull/17769#issuecomment-903921667 I think the test is just flaky. Merging now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[airflow] branch main updated: Fix dag_processing.last_duration metric random holes (#17769)

2021-08-23 Thread binh
This is an automated email from the ASF dual-hosted git repository. binh pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1a929f7 Fix dag_processing.last_duration metric rand

[GitHub] [airflow] xinbinhuang merged pull request #17769: Fix dag_processing.last_duration metric random holes

2021-08-23 Thread GitBox
xinbinhuang merged pull request #17769: URL: https://github.com/apache/airflow/pull/17769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-

[GitHub] [airflow] potiuk closed issue #17785: getting authlib error while using keycloak with airflow 2.1.1

2021-08-23 Thread GitBox
potiuk closed issue #17785: URL: https://github.com/apache/airflow/issues/17785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[GitHub] [airflow] potiuk commented on issue #17785: getting authlib error while using keycloak with airflow 2.1.1

2021-08-23 Thread GitBox
potiuk commented on issue #17785: URL: https://github.com/apache/airflow/issues/17785#issuecomment-903913434 This is not Airflow issue so it should not be open as such. It is likely problem with your configuration/network that you should try to debug (and at the worst case raise as F

[GitHub] [airflow] ShakaibKhan commented on issue #17476: Sensitive variables don't get masked when rendered with airflow tasks test

2021-08-23 Thread GitBox
ShakaibKhan commented on issue #17476: URL: https://github.com/apache/airflow/issues/17476#issuecomment-903909215 I repeated above instructions and was able reproduce. So it looks like the airflow/util/log/secrets_masker.py:should_hide_value_for_key() is where the check for hiding a sensit

[airflow] branch constraints-main updated: Updating constraints. Build id:1159137427

2021-08-23 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new e270b7f Updating const

[GitHub] [airflow] kaxil commented on a change in pull request #17786: Improve dag/task concurrency check

2021-08-23 Thread GitBox
kaxil commented on a change in pull request #17786: URL: https://github.com/apache/airflow/pull/17786#discussion_r694081982 ## File path: airflow/jobs/scheduler_job.py ## @@ -322,6 +322,7 @@ def _executable_task_instances_to_queued(self, max_tis: int, session: Session =

[GitHub] [airflow] Jorricks commented on a change in pull request #17207: WIP: Fix external_executor_id not being set for manually run jobs.

2021-08-23 Thread GitBox
Jorricks commented on a change in pull request #17207: URL: https://github.com/apache/airflow/pull/17207#discussion_r694076466 ## File path: airflow/executors/celery_executor.py ## @@ -125,8 +129,10 @@ def _execute_in_fork(command_to_exec: CommandType) -> None: os._exi

[GitHub] [airflow] msumit commented on a change in pull request #17100: WIP: Advanced Params using json-schema

2021-08-23 Thread GitBox
msumit commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r694062958 ## File path: airflow/serialization/serialized_objects.py ## @@ -789,6 +825,27 @@ def from_dict(cls, serialized_obj: dict) -> 'SerializedDAG':

[GitHub] [airflow] suhanovv edited a comment on pull request #17769: Fix dag_processing.last_duration metric random holes

2021-08-23 Thread GitBox
suhanovv edited a comment on pull request #17769: URL: https://github.com/apache/airflow/pull/17769#issuecomment-903859306 @ashb, i did, but tests for mssql fail in random places. =( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [airflow] suhanovv commented on pull request #17769: Fix dag_processing.last_duration metric random holes

2021-08-23 Thread GitBox
suhanovv commented on pull request #17769: URL: https://github.com/apache/airflow/pull/17769#issuecomment-903859306 @ashb, i did, but tests for mssql fail in random places. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[airflow] branch main updated (5f159cb -> ffb81ea)

2021-08-23 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 5f159cb Ensure that airflow modules are not imported too early in conftest (#17779) add ffb81ea Impro

[GitHub] [airflow] ephraimbuddy merged pull request #17786: Improve dag/task concurrency check

2021-08-23 Thread GitBox
ephraimbuddy merged pull request #17786: URL: https://github.com/apache/airflow/pull/17786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits

[GitHub] [airflow] jedcunningham commented on a change in pull request #17759: Show all import_errors from zip files

2021-08-23 Thread GitBox
jedcunningham commented on a change in pull request #17759: URL: https://github.com/apache/airflow/pull/17759#discussion_r694041319 ## File path: airflow/dag_processing/processor.py ## @@ -513,7 +513,12 @@ def update_import_errors(session: Session, dagbag: DagBag) -> None:

[GitHub] [airflow] kosteev commented on pull request #17450: Add missing menu access for dag dependencies and configurations pages

2021-08-23 Thread GitBox
kosteev commented on pull request #17450: URL: https://github.com/apache/airflow/pull/17450#issuecomment-903834351 @mik-laj Done. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [airflow] cocampbe opened a new issue #17788: livy operator print sparkUiUrl in log output

2021-08-23 Thread GitBox
cocampbe opened a new issue #17788: URL: https://github.com/apache/airflow/issues/17788 **Description** Adding the sparkUiUrl to the log output of the livy operator. **Use case / motivation** It would be useful for our end users to be able to go straight to the s

[GitHub] [airflow] github-actions[bot] commented on pull request #17743: Add Triggerer to Helm Chart

2021-08-23 Thread GitBox
github-actions[bot] commented on pull request #17743: URL: https://github.com/apache/airflow/pull/17743#issuecomment-903795854 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] jedcunningham commented on a change in pull request #17743: Add Triggerer to Helm Chart

2021-08-23 Thread GitBox
jedcunningham commented on a change in pull request #17743: URL: https://github.com/apache/airflow/pull/17743#discussion_r693995426 ## File path: chart/values.yaml ## @@ -719,6 +719,78 @@ webserver: topologyKey: "kubernetes.io/hostname" tolerations: [] +# Airf

[GitHub] [airflow] lwyszomi opened a new pull request #17787: [AIRFLOW-9300] Add DatafusionPipelineStateSensor and aync option to the CloudDataFusionStartPipelineOperator

2021-08-23 Thread GitBox
lwyszomi opened a new pull request #17787: URL: https://github.com/apache/airflow/pull/17787 I added new sensor for the Google Data Fusion service, and I added the possibility to run the `CloudDataFusionStartPipelineOperator` in the async way. related: https://github.com/apache/airf

[GitHub] [airflow] mik-laj commented on a change in pull request #17775: Improve discoverability of Provider packages' functionality

2021-08-23 Thread GitBox
mik-laj commented on a change in pull request #17775: URL: https://github.com/apache/airflow/pull/17775#discussion_r693965741 ## File path: docs/apache-airflow-providers-microsoft-azure/logging/index.rst ## @@ -17,7 +17,7 @@ Review comment: Missing backward compatibil

[GitHub] [airflow] mik-laj commented on a change in pull request #17775: Improve discoverability of Provider packages' functionality

2021-08-23 Thread GitBox
mik-laj commented on a change in pull request #17775: URL: https://github.com/apache/airflow/pull/17775#discussion_r693965147 ## File path: docs/apache-airflow-providers-elasticsearch/logging/index.rst ## @@ -17,7 +17,7 @@ Review comment: Changing the filename may bre

[GitHub] [airflow] github-actions[bot] commented on pull request #17786: Improve dag/task concurrency check

2021-08-23 Thread GitBox
github-actions[bot] commented on pull request #17786: URL: https://github.com/apache/airflow/pull/17786#issuecomment-903739397 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[airflow] branch main updated: Ensure that airflow modules are not imported too early in conftest (#17779)

2021-08-23 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5f159cb Ensure that airflow modules are not imported

[GitHub] [airflow] ashb merged pull request #17779: Ensure that airflow modules are not imported too early in conftest

2021-08-23 Thread GitBox
ashb merged pull request #17779: URL: https://github.com/apache/airflow/pull/17779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [airflow] ashb commented on pull request #17576: Add pre/post execution hooks

2021-08-23 Thread GitBox
ashb commented on pull request #17576: URL: https://github.com/apache/airflow/pull/17576#issuecomment-903721414 These two new fields you add don't need to be serialized (and callables can't generally be anyway) -- the general rule is that things needed by the Scheduler should be serialized

[GitHub] [airflow] ashb commented on pull request #17769: Fix dag_processing.last_duration metric random holes

2021-08-23 Thread GitBox
ashb commented on pull request #17769: URL: https://github.com/apache/airflow/pull/17769#issuecomment-903716491 Yup, sounds good @suhanovv -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [airflow] ephraimbuddy opened a new pull request #17786: Improve dag/task concurrency check

2021-08-23 Thread GitBox
ephraimbuddy opened a new pull request #17786: URL: https://github.com/apache/airflow/pull/17786 Currently, tasks can be run even if the `dagrun` is queued. Task instances of queued `dagruns` should only be run when the `dagrun` is in a running state. This PR makes sure `tis` of queued

[airflow] branch main updated (4ced69b -> 6301faa)

2021-08-23 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 4ced69b Remove deprecated metrics from metrics.rst (#17772) add 6301faa Enable Connection creation from Vault para

[GitHub] [airflow] ashb merged pull request #15013: Enable Connection creation from Vault parameters

2021-08-23 Thread GitBox
ashb merged pull request #15013: URL: https://github.com/apache/airflow/pull/15013 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubsc

[GitHub] [airflow] ashb commented on pull request #17779: Ensure that airflow modules are not imported too early in conftest

2021-08-23 Thread GitBox
ashb commented on pull request #17779: URL: https://github.com/apache/airflow/pull/17779#issuecomment-903699807 See https://issues.apache.org/jira/browse/AIRFLOW-1931 :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [airflow] ashb commented on pull request #17779: Ensure that airflow modules are not imported too early in conftest

2021-08-23 Thread GitBox
ashb commented on pull request #17779: URL: https://github.com/apache/airflow/pull/17779#issuecomment-903697418 Yeah, the amount that happens from `import airflow` is a big problem -- both for tests and for "import time" -- This is an automated message from the Apache Git Service. To res

[GitHub] [airflow] kaxil commented on a change in pull request #17743: Add Triggerer to Helm Chart

2021-08-23 Thread GitBox
kaxil commented on a change in pull request #17743: URL: https://github.com/apache/airflow/pull/17743#discussion_r693900410 ## File path: chart/templates/triggerer/triggerer-deployment.yaml ## @@ -0,0 +1,181 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or

[GitHub] [airflow] debimishra89 opened a new issue #17785: getting authlib error while using keycloak with airflow 2.1.1

2021-08-23 Thread GitBox
debimishra89 opened a new issue #17785: URL: https://github.com/apache/airflow/issues/17785 ### Discussed in https://github.com/apache/airflow/discussions/17784 Originally posted by **debimishra89** August 23, 2021 I am getting below error while logging in airflow 2.1.1 wit

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17785: getting authlib error while using keycloak with airflow 2.1.1

2021-08-23 Thread GitBox
boring-cyborg[bot] commented on issue #17785: URL: https://github.com/apache/airflow/issues/17785#issuecomment-903689818 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[airflow] branch constraints-main updated: Updating constraints. Build id:1158295194

2021-08-23 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 4e37e99 Updating const

[GitHub] [airflow] bbovenzi commented on a change in pull request #17732: Add Next Run to UI

2021-08-23 Thread GitBox
bbovenzi commented on a change in pull request #17732: URL: https://github.com/apache/airflow/pull/17732#discussion_r693876177 ## File path: airflow/www/templates/airflow/dags.html ## @@ -98,7 +98,10 @@ {{ page_title }} Schedule Las

[GitHub] [airflow] potiuk edited a comment on pull request #17779: Ensure that airflow modules are not imported too early in conftest

2021-08-23 Thread GitBox
potiuk edited a comment on pull request #17779: URL: https://github.com/apache/airflow/pull/17779#issuecomment-903662619 > This seems to break MySQL’s collation settings for some reason I thought it could be because it needs to be rebased to latest main. I just merged yesterday a cha

[GitHub] [airflow] potiuk commented on pull request #17779: Ensure that airflow modules are not imported too early in conftest

2021-08-23 Thread GitBox
potiuk commented on pull request #17779: URL: https://github.com/apache/airflow/pull/17779#issuecomment-903662619 > This seems to break MySQL’s collation settings for some reason I thought it could be because it needs to be rebased to latest main. I just merged yesterday a change tha

[GitHub] [airflow] potiuk opened a new pull request #17783: Add warning about https configuration in SimpleHttpOperator

2021-08-23 Thread GitBox
potiuk opened a new pull request #17783: URL: https://github.com/apache/airflow/pull/17783 For historical reasons, configuring ``https`` via SimpleHttpOperator is well, complex. This PR adds warning which informs the users about it as well provides explanation why it is like tha

[airflow] branch main updated (ccaff5a -> 4ced69b)

2021-08-23 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from ccaff5a Automatically use utf8mb3_general_ci collation for mysql (#17729) add 4ced69b Remove deprecat

[GitHub] [airflow] ephraimbuddy merged pull request #17772: Remove deprecated metrics from metrics.rst

2021-08-23 Thread GitBox
ephraimbuddy merged pull request #17772: URL: https://github.com/apache/airflow/pull/17772 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #17207: WIP: Fix external_executor_id not being set for manually run jobs.

2021-08-23 Thread GitBox
ephraimbuddy commented on a change in pull request #17207: URL: https://github.com/apache/airflow/pull/17207#discussion_r693832397 ## File path: airflow/executors/celery_executor.py ## @@ -125,8 +129,10 @@ def _execute_in_fork(command_to_exec: CommandType) -> None: os.

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #17207: WIP: Fix external_executor_id not being set for manually run jobs.

2021-08-23 Thread GitBox
ephraimbuddy commented on a change in pull request #17207: URL: https://github.com/apache/airflow/pull/17207#discussion_r693832397 ## File path: airflow/executors/celery_executor.py ## @@ -125,8 +129,10 @@ def _execute_in_fork(command_to_exec: CommandType) -> None: os.

[GitHub] [airflow] LionelZhao28 commented on a change in pull request #17751: add `cwd` in `BashOperator`

2021-08-23 Thread GitBox
LionelZhao28 commented on a change in pull request #17751: URL: https://github.com/apache/airflow/pull/17751#discussion_r693828558 ## File path: docs/spelling_wordlist.txt ## @@ -44,6 +44,7 @@ Booleans Boto BounceX Bq +cwd Review comment: OK. committed the suggestion

[GitHub] [airflow] github-actions[bot] commented on pull request #17772: Remove deprecated metrics from metrics.rst

2021-08-23 Thread GitBox
github-actions[bot] commented on pull request #17772: URL: https://github.com/apache/airflow/pull/17772#issuecomment-903615060 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide

[GitHub] [airflow] uranusjr commented on a change in pull request #17751: add `cwd` in `BashOperator`

2021-08-23 Thread GitBox
uranusjr commented on a change in pull request #17751: URL: https://github.com/apache/airflow/pull/17751#discussion_r693824611 ## File path: airflow/hooks/subprocess.py ## @@ -35,23 +35,31 @@ def __init__(self) -> None: super().__init__() def run_command( -

[GitHub] [airflow] Jorricks commented on a change in pull request #17207: WIP: Fix external_executor_id not being set for manually run jobs.

2021-08-23 Thread GitBox
Jorricks commented on a change in pull request #17207: URL: https://github.com/apache/airflow/pull/17207#discussion_r693824593 ## File path: airflow/executors/celery_executor.py ## @@ -125,8 +129,10 @@ def _execute_in_fork(command_to_exec: CommandType) -> None: os._exi

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #17207: WIP: Fix external_executor_id not being set for manually run jobs.

2021-08-23 Thread GitBox
ephraimbuddy commented on a change in pull request #17207: URL: https://github.com/apache/airflow/pull/17207#discussion_r693819227 ## File path: airflow/cli/commands/task_command.py ## @@ -174,6 +175,12 @@ def _run_raw_task(args, ti: TaskInstance) -> None: ) +def _extr

[GitHub] [airflow] potiuk opened a new pull request #17782: Move instructions of constraint/image refreshing to dev

2021-08-23 Thread GitBox
potiuk opened a new pull request #17782: URL: https://github.com/apache/airflow/pull/17782 When we have a prolonged issue with flaky tests or Github runners instabilities, our automated constraint and image refresh might not work, so we might need to manually refresh the constraints

[GitHub] [airflow] ashb commented on pull request #17779: Ensure that airflow modules are not imported too early in conftest

2021-08-23 Thread GitBox
ashb commented on pull request #17779: URL: https://github.com/apache/airflow/pull/17779#issuecomment-903597055 It _shouldn't_ be able to do that -- all it's doing is checking that something isn't imported too early (which it isn't, otherwise the assert would trigger and the tests would fa

[GitHub] [airflow] suhanovv commented on pull request #17769: Fix dag_processing.last_duration metric random holes

2021-08-23 Thread GitBox
suhanovv commented on pull request #17769: URL: https://github.com/apache/airflow/pull/17769#issuecomment-903592983 > Is there a reason we don't emit the metric when each file is finished parsing instead? Tying metrics emission to how often we _print_ them in the logs seems slightly unexpe

[GitHub] [airflow] uranusjr commented on pull request #17777: get_pandas_df() fails when it tries to read an empty table

2021-08-23 Thread GitBox
uranusjr commented on pull request #1: URL: https://github.com/apache/airflow/pull/1#issuecomment-903567253 I wonder if it’d work to do `SELECT * FROM table LIMIT 0` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] github-actions[bot] commented on pull request #17749: Example xcom update

2021-08-23 Thread GitBox
github-actions[bot] commented on pull request #17749: URL: https://github.com/apache/airflow/pull/17749#issuecomment-903563690 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] uranusjr commented on pull request #17779: Ensure that airflow modules are not imported too early in conftest

2021-08-23 Thread GitBox
uranusjr commented on pull request #17779: URL: https://github.com/apache/airflow/pull/17779#issuecomment-903560897 This seems to break MySQL’s collation settings for some reason -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [airflow] github-actions[bot] commented on pull request #17779: Ensure that airflow modules are not imported too early in conftest

2021-08-23 Thread GitBox
github-actions[bot] commented on pull request #17779: URL: https://github.com/apache/airflow/pull/17779#issuecomment-903554985 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] ashb opened a new pull request #17779: Ensure that airflow modules are not imported too early in conftest

2021-08-23 Thread GitBox
ashb opened a new pull request #17779: URL: https://github.com/apache/airflow/pull/17779 If someone adds `import airflow` or any submodule in to conftest it will cause airflow.settings and airflow.configuration to be imported and initialized before we set the test env vars -- this leads to

[GitHub] [airflow] ashb commented on pull request #17769: Fix dag_processing.last_duration metric random holes

2021-08-23 Thread GitBox
ashb commented on pull request #17769: URL: https://github.com/apache/airflow/pull/17769#issuecomment-903551658 Is there a reason we don't emit the metric when each file is finished parsing instead? Tying metrics emission to how often we _print_ them in the logs seems slightly unexpected.

[GitHub] [airflow] potiuk commented on pull request #17321: Enable specifying dictionary paths in `template_fields_renderers`

2021-08-23 Thread GitBox
potiuk commented on pull request #17321: URL: https://github.com/apache/airflow/pull/17321#issuecomment-903548833 > @potiuk I'm more than happy to I'm not sure what you had in mind exactly or where to put it. At the moment, the only operator that has a path which can be rendered is BigQuer

[GitHub] [airflow] potiuk commented on issue #17778: Add Arango hook

2021-08-23 Thread GitBox
potiuk commented on issue #17778: URL: https://github.com/apache/airflow/issues/17778#issuecomment-903543234 Would you be willing to add it ? I think if you have the need and use Rango, you might be the best person to add it . -- This is an automated message from the Apache Git Service.

[GitHub] [airflow] nathadfield commented on pull request #17321: Enable specifying dictionary paths in `template_fields_renderers`

2021-08-23 Thread GitBox
nathadfield commented on pull request #17321: URL: https://github.com/apache/airflow/pull/17321#issuecomment-903542861 @potiuk I'm more than happy to I'm not sure what you had in mind exactly or where to put it. At the moment, the only operator that has a path which can be rendered is Big

[GitHub] [airflow] LionelZhao28 commented on pull request #17751: add `cwd` in `BashOperator`

2021-08-23 Thread GitBox
LionelZhao28 commented on pull request #17751: URL: https://github.com/apache/airflow/pull/17751#issuecomment-903498163 @uranusjr This is the first time that I create the PR to airflow. As there are many different coding habits, there are lots of updates for this review. I appreciate yo

[GitHub] [airflow] github-actions[bot] commented on pull request #17751: add `cwd` in `BashOperator`

2021-08-23 Thread GitBox
github-actions[bot] commented on pull request #17751: URL: https://github.com/apache/airflow/pull/17751#issuecomment-903496095 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

<    1   2