[GitHub] [airflow] xinbinhuang commented on a change in pull request #15104: Get connection URI with AWS Secrets Manager

2021-04-01 Thread GitBox
xinbinhuang commented on a change in pull request #15104: URL: https://github.com/apache/airflow/pull/15104#discussion_r606082085 ## File path: airflow/providers/amazon/aws/secrets/secrets_manager.py ## @@ -102,17 +102,66 @@ def client(self): ) return

[GitHub] [airflow] potiuk opened a new pull request #15148: Fixes failing docs upload on master

2021-04-01 Thread GitBox
potiuk opened a new pull request #15148: URL: https://github.com/apache/airflow/pull/15148 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] uranusjr edited a comment on issue #8421: Hide sensitive data in UI

2021-04-01 Thread GitBox
uranusjr edited a comment on issue #8421: URL: https://github.com/apache/airflow/issues/8421#issuecomment-812291421 I’m thinking about a combination of the two approaches: 1. `airflow.cfg` will gain a flag (`show_templated_fields`) to control the behaviour on this. 2. The value

[GitHub] [airflow] potiuk commented on pull request #15081: Finish quarantine for test_should_force_kill_process

2021-04-01 Thread GitBox
potiuk commented on pull request #15081: URL: https://github.com/apache/airflow/pull/15081#issuecomment-812328110 Anyone :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[airflow] branch master updated (535e1a8 -> 4d1b2e9)

2021-04-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 535e1a8 Better handling of docker command (#15080) add 4d1b2e9 Mark the test_scheduler_task_start_date as

[GitHub] [airflow] potiuk merged pull request #15086: Mark the test_scheduler_task_start_date as quarantined

2021-04-01 Thread GitBox
potiuk merged pull request #15086: URL: https://github.com/apache/airflow/pull/15086 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[airflow] branch master updated: Better handling of docker command (#15080)

2021-04-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 535e1a8 Better handling of docker command

[GitHub] [airflow] potiuk merged pull request #15080: Better handling of docker command

2021-04-01 Thread GitBox
potiuk merged pull request #15080: URL: https://github.com/apache/airflow/pull/15080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [airflow] xinbinhuang edited a comment on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
xinbinhuang edited a comment on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812326424 > If we ever want to add edge information beyond just a label (e.g. add a longer description), it means we have to go back to this pattern Can you give an

[GitHub] [airflow] xinbinhuang edited a comment on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
xinbinhuang edited a comment on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812326424 How about `task1 >> "this is a label" | task2`? This is the exact opposite of the Beam Python SDK [example

[GitHub] [airflow] xinbinhuang commented on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
xinbinhuang commented on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812326424 How about `task1 >> "this is a label" | task2`? This is the exact opposite of the Beam Python SDK [example

[GitHub] [airflow] uranusjr commented on issue #15108: PostgresHook do not use the specified cursor

2021-04-01 Thread GitBox
uranusjr commented on issue #15108: URL: https://github.com/apache/airflow/issues/15108#issuecomment-812294111 I think this depends on whether @Davide95 wants to turn this into a feature request (instead of a bug report) on setting the cursor class (and persuambly other extras) on a

[GitHub] [airflow] uranusjr commented on issue #8421: Hide sensitive data in UI

2021-04-01 Thread GitBox
uranusjr commented on issue #8421: URL: https://github.com/apache/airflow/issues/8421#issuecomment-812291421 I’m thinking about a combination of the two approaches: 1. `airflow.cfg` will gain a flag (`hide_secrets`) to control the behaviour on this. 2. The value can also be set

[GitHub] [airflow] uranusjr commented on pull request #15141: Share app instance between Kerberos tests

2021-04-01 Thread GitBox
uranusjr commented on pull request #15141: URL: https://github.com/apache/airflow/pull/15141#issuecomment-812288990 Both `TestApiKerberos._set_attrs` and `dagbag_to_db` are [autouse

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15147: Dag created with a dag decorator in a packaged dag file (zip) has incorrect filepath property

2021-04-01 Thread GitBox
boring-cyborg[bot] commented on issue #15147: URL: https://github.com/apache/airflow/issues/15147#issuecomment-812278953 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] mejran opened a new issue #15147: Dag created with a dag decorator in a packaged dag file (zip) has incorrect filepath property

2021-04-01 Thread GitBox
mejran opened a new issue #15147: URL: https://github.com/apache/airflow/issues/15147 **Apache Airflow version**: 2.0.1 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): 1.19.6 **Environment**: - **Cloud provider or hardware

[GitHub] [airflow] dstandish commented on a change in pull request #15013: Enable Connection creation from Vault parameters

2021-04-01 Thread GitBox
dstandish commented on a change in pull request #15013: URL: https://github.com/apache/airflow/pull/15013#discussion_r605975880 ## File path: airflow/providers/hashicorp/secrets/vault.py ## @@ -188,12 +204,28 @@ def get_conn_uri(self, conn_id: str) -> Optional[str]:

[GitHub] [airflow] dstandish commented on a change in pull request #15013: Enable Connection creation from Vault parameters

2021-04-01 Thread GitBox
dstandish commented on a change in pull request #15013: URL: https://github.com/apache/airflow/pull/15013#discussion_r605975880 ## File path: airflow/providers/hashicorp/secrets/vault.py ## @@ -188,12 +204,28 @@ def get_conn_uri(self, conn_id: str) -> Optional[str]:

[GitHub] [airflow] jhtimmins commented on issue #15144: Support of Timestamp for Cloudwatch-based remote logging

2021-04-01 Thread GitBox
jhtimmins commented on issue #15144: URL: https://github.com/apache/airflow/issues/15144#issuecomment-812258584 @codenamestif Thanks for including the output. Would you like to submit a PR for this? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] jhtimmins commented on pull request #15134: add loggin.info when poke in SFTP,FTP and Filesystem sensor to show last modified

2021-04-01 Thread GitBox
jhtimmins commented on pull request #15134: URL: https://github.com/apache/airflow/pull/15134#issuecomment-812257634 Can you make the changes required by the failing static tests? https://github.com/apache/airflow/pull/15134/checks?check_run_id=2246371510#step:9:93 -- This is an

[GitHub] [airflow] jhtimmins commented on a change in pull request #15134: add loggin.info when poke in SFTP,FTP and Filesystem sensor to show last modified

2021-04-01 Thread GitBox
jhtimmins commented on a change in pull request #15134: URL: https://github.com/apache/airflow/pull/15134#discussion_r606019412 ## File path: airflow/sensors/filesystem.py ## @@ -57,6 +58,9 @@ def poke(self, context): for path in glob(full_path): if

[GitHub] [airflow] jhtimmins commented on a change in pull request #15141: Share app instance between Kerberos tests

2021-04-01 Thread GitBox
jhtimmins commented on a change in pull request #15141: URL: https://github.com/apache/airflow/pull/15141#discussion_r606018061 ## File path: tests/api/auth/backend/test_kerberos_auth.py ## @@ -19,42 +19,46 @@ import json import os import socket -import unittest from

[GitHub] [airflow] github-actions[bot] closed pull request #13080: KubernetesExecutor overrides should only append to lists

2021-04-01 Thread GitBox
github-actions[bot] closed pull request #13080: URL: https://github.com/apache/airflow/pull/13080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [airflow] github-actions[bot] closed pull request #11769: Update charts to follow Helm RBAC best practices

2021-04-01 Thread GitBox
github-actions[bot] closed pull request #11769: URL: https://github.com/apache/airflow/pull/11769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [airflow] RNHTTR commented on a change in pull request #14883: Fix celery executor bug trying to call len on map

2021-04-01 Thread GitBox
RNHTTR commented on a change in pull request #14883: URL: https://github.com/apache/airflow/pull/14883#discussion_r606009786 ## File path: airflow/executors/celery_executor.py ## @@ -543,20 +539,27 @@ def __init__(self, sync_parallelism=None): super().__init__()

[GitHub] [airflow] vikramkoka commented on issue #14593: airflow2.0 - Run the task with the specified user will report an error

2021-04-01 Thread GitBox
vikramkoka commented on issue #14593: URL: https://github.com/apache/airflow/issues/14593#issuecomment-812237289 @zqd759592942 Based on the conversation above, there is nothing to indicate this is a bug. Please do reopen or report a new issue if you continue to encounter a problem. --

[GitHub] [airflow] vikramkoka closed issue #14593: airflow2.0 - Run the task with the specified user will report an error

2021-04-01 Thread GitBox
vikramkoka closed issue #14593: URL: https://github.com/apache/airflow/issues/14593 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [airflow] vikramkoka closed issue #14580: Request for sub process

2021-04-01 Thread GitBox
vikramkoka closed issue #14580: URL: https://github.com/apache/airflow/issues/14580 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [airflow] vikramkoka commented on issue #14580: Request for sub process

2021-04-01 Thread GitBox
vikramkoka commented on issue #14580: URL: https://github.com/apache/airflow/issues/14580#issuecomment-812236746 @snehlata08 Based on the conversation, this does not seem like a bug, therefore closing this issue. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] vikramkoka commented on issue #15145: Big Query to MS SQL operator

2021-04-01 Thread GitBox
vikramkoka commented on issue #15145: URL: https://github.com/apache/airflow/issues/15145#issuecomment-812225455 @subkanthi Please do submit a PR if you believe this is needed. One of committers will surely help review it when it is ready. -- This is an automated message from the

[GitHub] [airflow] vikramkoka commented on issue #15066: try to open admin->xcom and get error message

2021-04-01 Thread GitBox
vikramkoka commented on issue #15066: URL: https://github.com/apache/airflow/issues/15066#issuecomment-812224323 @bintocher What are the values in your airflow.cfg with respect to Xcom? Specifically: 1. Do you have pickling set to True of False, and 2. Are you using a

[GitHub] [airflow] chutcheson commented on issue #9387: Use enum for task_states/dag_states

2021-04-01 Thread GitBox
chutcheson commented on issue #9387: URL: https://github.com/apache/airflow/issues/9387#issuecomment-812223959 Is this still an open issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] chutcheson commented on issue #9855: Import connections/variables from a file

2021-04-01 Thread GitBox
chutcheson commented on issue #9855: URL: https://github.com/apache/airflow/issues/9855#issuecomment-81602 I'm looking for a good first issue to work on. Is this a correct account of the feature request? > When using the airflow command line interface tool, I want to be

[GitHub] [airflow] vikramkoka commented on issue #15108: PostgresHook do not use the specified cursor

2021-04-01 Thread GitBox
vikramkoka commented on issue #15108: URL: https://github.com/apache/airflow/issues/15108#issuecomment-812220739 @uranusjr and @Davide95, Can this issue be closed now? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] wolfier commented on issue #15136: curl REST authentication fails if variable value hidden from the UI

2021-04-01 Thread GitBox
wolfier commented on issue #15136: URL: https://github.com/apache/airflow/issues/15136#issuecomment-812218246 Based on #8421, the render template view is not be capable of hiding sensitive information. Quoting @kaxil, > Yes the Rendered UI Field currently exposes everything.

[GitHub] [airflow] dstandish commented on a change in pull request #15013: Enable Connection creation from Vault parameters

2021-04-01 Thread GitBox
dstandish commented on a change in pull request #15013: URL: https://github.com/apache/airflow/pull/15013#discussion_r605975880 ## File path: airflow/providers/hashicorp/secrets/vault.py ## @@ -188,12 +204,28 @@ def get_conn_uri(self, conn_id: str) -> Optional[str]:

[GitHub] [airflow] dstandish commented on a change in pull request #15013: Enable Connection creation from Vault parameters

2021-04-01 Thread GitBox
dstandish commented on a change in pull request #15013: URL: https://github.com/apache/airflow/pull/15013#discussion_r605979696 ## File path: tests/providers/hashicorp/secrets/test_vault.py ## @@ -59,6 +59,48 @@ def test_get_conn_uri(self, mock_hvac): returned_uri =

[GitHub] [airflow] dstandish commented on a change in pull request #15013: Enable Connection creation from Vault parameters

2021-04-01 Thread GitBox
dstandish commented on a change in pull request #15013: URL: https://github.com/apache/airflow/pull/15013#discussion_r605975880 ## File path: airflow/providers/hashicorp/secrets/vault.py ## @@ -188,12 +204,28 @@ def get_conn_uri(self, conn_id: str) -> Optional[str]:

[GitHub] [airflow] SamWheating commented on a change in pull request #15137: Adding retries when starting kubernetes pods

2021-04-01 Thread GitBox
SamWheating commented on a change in pull request #15137: URL: https://github.com/apache/airflow/pull/15137#discussion_r605972319 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -98,6 +98,12 @@ def delete_pod(self, pod: V1Pod): if e.status != 404:

[GitHub] [airflow] SamWheating commented on a change in pull request #15137: Adding retries when starting kubernetes pods

2021-04-01 Thread GitBox
SamWheating commented on a change in pull request #15137: URL: https://github.com/apache/airflow/pull/15137#discussion_r605972319 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -98,6 +98,12 @@ def delete_pod(self, pod: V1Pod): if e.status != 404:

[airflow] branch constraints-master updated: Updating constraints. Build id:709827947

2021-04-01 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-master by this push: new e5eebb0 Updating

[GitHub] [airflow] dimberman commented on a change in pull request #15137: Adding retries when starting kubernetes pods

2021-04-01 Thread GitBox
dimberman commented on a change in pull request #15137: URL: https://github.com/apache/airflow/pull/15137#discussion_r605948923 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -98,6 +98,12 @@ def delete_pod(self, pod: V1Pod): if e.status != 404:

[airflow] branch master updated (6822665 -> 486b764)

2021-04-01 Thread xddeng
This is an automated email from the ASF dual-hosted git repository. xddeng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 6822665 Send region_name into parant class of AwsGlueJobHook (#14251) add 486b764 Fix password masking in

[GitHub] [airflow] XD-DENG merged pull request #15143: Fix password masking in CLI action_logging

2021-04-01 Thread GitBox
XD-DENG merged pull request #15143: URL: https://github.com/apache/airflow/pull/15143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [airflow] XD-DENG closed issue #15131: airflow scheduler -p command not working in airflow 2.0.1

2021-04-01 Thread GitBox
XD-DENG closed issue #15131: URL: https://github.com/apache/airflow/issues/15131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] davlum commented on a change in pull request #15013: Enable Connection creation from Vault parameters

2021-04-01 Thread GitBox
davlum commented on a change in pull request #15013: URL: https://github.com/apache/airflow/pull/15013#discussion_r605942070 ## File path: tests/providers/hashicorp/secrets/test_vault.py ## @@ -59,6 +59,48 @@ def test_get_conn_uri(self, mock_hvac): returned_uri =

[GitHub] [airflow] davlum opened a new pull request #15146: Validate that the extra parameter is parseable as JSON

2021-04-01 Thread GitBox
davlum opened a new pull request #15146: URL: https://github.com/apache/airflow/pull/15146 Before this commit there was a documented but unenforced limitation that the extra parameter be encoded JSON. In #15013 this issue garnered attention and motivated this PR. ---

[GitHub] [airflow] codenamestif commented on issue #15144: Support of Timestamp for Cloudwatch-based remote logging

2021-04-01 Thread GitBox
codenamestif commented on issue #15144: URL: https://github.com/apache/airflow/issues/15144#issuecomment-812167706 Here is an example of the current behaviour: ``` *** Reading remote log from Cloudwatch log_group: my_log_group log_stream:

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15145: Big Query to MS SQL operator

2021-04-01 Thread GitBox
boring-cyborg[bot] commented on issue #15145: URL: https://github.com/apache/airflow/issues/15145#issuecomment-812157646 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] subkanthi opened a new issue #15145: Big Query to MS SQL operator

2021-04-01 Thread GitBox
subkanthi opened a new issue #15145: URL: https://github.com/apache/airflow/issues/15145 **Description** A new transfer operator for transferring records from Big Query to MSSQL. **Use case / motivation** Very similar to Bigquery to mysql, this will be an operator for

[GitHub] [airflow] potiuk edited a comment on issue #15139: Add documentation on how to easily add new python packages to the docker-compose development setup

2021-04-01 Thread GitBox
potiuk edited a comment on issue #15139: URL: https://github.com/apache/airflow/issues/15139#issuecomment-812151337 You need to create your own image. Extending/customizing image, including adding your own python packages are described in

[GitHub] [airflow] potiuk closed issue #15139: Add documentation on how to easily add new python packages to the docker-compose development setup

2021-04-01 Thread GitBox
potiuk closed issue #15139: URL: https://github.com/apache/airflow/issues/15139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] potiuk commented on issue #15139: Add documentation on how to easily add new python packages to the docker-compose development setup

2021-04-01 Thread GitBox
potiuk commented on issue #15139: URL: https://github.com/apache/airflow/issues/15139#issuecomment-812151337 You need to create your own image. Extending/customizing image, including adding your own python packages are described in

[GitHub] [airflow] XD-DENG commented on pull request #15143: Fix password masking in CLI action_logging

2021-04-01 Thread GitBox
XD-DENG commented on pull request #15143: URL: https://github.com/apache/airflow/pull/15143#issuecomment-812145316 > Worth adding tests do you think? Added in https://github.com/apache/airflow/pull/15143/commits/6b1f641b0fd44d38fb7d745a6bb3e6a903aab887 -- This is an automated

[GitHub] [airflow] andrewgodwin commented on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
andrewgodwin commented on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812144541 > > `from airflow.models import Label` > > How about hiding the `Label` class from the user entirely? Something like `task1 << "Label" << task2` and create the

[GitHub] [airflow] XD-DENG commented on pull request #15143: Fix password masking in CLI action_logging

2021-04-01 Thread GitBox
XD-DENG commented on pull request #15143: URL: https://github.com/apache/airflow/pull/15143#issuecomment-812142141 Ha, I think the adding more cases to `test_cli_create_user_supplied_password_is_masked` would be sufficient. I overlooked it just now. -- This is an automated message from

[GitHub] [airflow] jhtimmins commented on issue #15144: Support of Timestamp for Cloudwatch-based remote logging

2021-04-01 Thread GitBox
jhtimmins commented on issue #15144: URL: https://github.com/apache/airflow/issues/15144#issuecomment-812141160 @codenamestif could you include log output and/or images of what you see? If you're interested in taking on this PR, it sounds like a solid first one. I'm happy to answer

[GitHub] [airflow] XD-DENG commented on pull request #15143: Fix password masking in CLI action_logging

2021-04-01 Thread GitBox
XD-DENG commented on pull request #15143: URL: https://github.com/apache/airflow/pull/15143#issuecomment-812140247 @ashb yep, I agree it's worth adding test(s). I was thinking about it when I was preparing the PR, but still trying figure out how to do it in the most proper way. -- This

[GitHub] [airflow] ashb commented on a change in pull request #15137: Adding retries when starting kubernetes pods

2021-04-01 Thread GitBox
ashb commented on a change in pull request #15137: URL: https://github.com/apache/airflow/pull/15137#discussion_r605912371 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -98,6 +98,12 @@ def delete_pod(self, pod: V1Pod): if e.status != 404:

[airflow] branch master updated: Send region_name into parant class of AwsGlueJobHook (#14251)

2021-04-01 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 6822665 Send region_name into parant class of

[GitHub] [airflow] ashb merged pull request #14251: Send region_name into parant class of AwsGlueJobHook

2021-04-01 Thread GitBox
ashb merged pull request #14251: URL: https://github.com/apache/airflow/pull/14251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] codenamestif opened a new issue #15144: Support of Timestamp for Cloudwatch-based remote logging

2021-04-01 Thread GitBox
codenamestif opened a new issue #15144: URL: https://github.com/apache/airflow/issues/15144 **Description** `CloudwatchTaskHandler` doesn't parse log event timestamp. As result if Airflow uses `Cloudwatch` for remote logging, logs in Airflow UI miss timestamp. They just contain

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15144: Support of Timestamp for Cloudwatch-based remote logging

2021-04-01 Thread GitBox
boring-cyborg[bot] commented on issue #15144: URL: https://github.com/apache/airflow/issues/15144#issuecomment-812128682 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] uranusjr commented on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
uranusjr commented on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812113661 > `from airflow.models import Label` How about hiding the `Label` class from the user entirely? Something like `task1 << "Label" << task2` and create the label

[GitHub] [airflow] uranusjr removed a comment on issue #15140: Allow labelling of task dependencies

2021-04-01 Thread GitBox
uranusjr removed a comment on issue #15140: URL: https://github.com/apache/airflow/issues/15140#issuecomment-812113086 How about hiding the `Label` class from the user entirely? Something like `task1 << "Label" << task2` and create the label automatically in `_set_relatives()`. -- This

[GitHub] [airflow] uranusjr commented on issue #15140: Allow labelling of task dependencies

2021-04-01 Thread GitBox
uranusjr commented on issue #15140: URL: https://github.com/apache/airflow/issues/15140#issuecomment-812113086 How about hiding the `Label` class from the user entirely? Something like `task1 << "Label" << task2` and create the label automatically in `_set_relatives()`. -- This is an

[GitHub] [airflow] XD-DENG opened a new pull request #15143: Fix password masking in CLI action_logging

2021-04-01 Thread GitBox
XD-DENG opened a new pull request #15143: URL: https://github.com/apache/airflow/pull/15143 closes: #15131 Currently as long as argument '-p' if present, code tries to mask it. However, '-p' may mean something else (not password), like a boolean flag. Such cases may result in

[GitHub] [airflow] XD-DENG commented on issue #15131: airflow scheduler -p command not working in airflow 2.0.1

2021-04-01 Thread GitBox
XD-DENG commented on issue #15131: URL: https://github.com/apache/airflow/issues/15131#issuecomment-812109409 I can confirm this issue. That's because `_build_metrics()` checks the arg, and tries to mask value for password. When it sees `-p`, it assumes there is another value

[GitHub] [airflow] SamWheating commented on a change in pull request #15137: Adding retries when starting kubernetes pods

2021-04-01 Thread GitBox
SamWheating commented on a change in pull request #15137: URL: https://github.com/apache/airflow/pull/15137#discussion_r605870761 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -98,6 +98,12 @@ def delete_pod(self, pod: V1Pod): if e.status != 404:

[GitHub] [airflow] eladkal commented on issue #15140: Allow labelling of task dependencies

2021-04-01 Thread GitBox
eladkal commented on issue #15140: URL: https://github.com/apache/airflow/issues/15140#issuecomment-812096724 This is really cool. I'm glad to see that you thought of it. This is something that i'm thinking about for a while and I believe to be one of the features that will make a big

[GitHub] [airflow] ashb commented on a change in pull request #15137: Adding retries when starting kubernetes pods

2021-04-01 Thread GitBox
ashb commented on a change in pull request #15137: URL: https://github.com/apache/airflow/pull/15137#discussion_r605860870 ## File path: airflow/kubernetes/pod_launcher.py ## @@ -98,6 +98,12 @@ def delete_pod(self, pod: V1Pod): if e.status != 404:

[GitHub] [airflow] SamWheating commented on pull request #15137: Adding retries when starting kubernetes pods

2021-04-01 Thread GitBox
SamWheating commented on pull request #15137: URL: https://github.com/apache/airflow/pull/15137#issuecomment-812088826 Since this requires changes to the pod_launcher and not the CNCF provider package, would it be possible to get this into the 2.0.2 release @ashb ? -- This is an

[GitHub] [airflow] ashb commented on a change in pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
ashb commented on a change in pull request #15142: URL: https://github.com/apache/airflow/pull/15142#discussion_r605859767 ## File path: airflow/serialization/schema.json ## @@ -102,7 +102,8 @@ "_task_group": {"anyOf": [ { "type": "null" }, {

[GitHub] [airflow] ashb commented on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
ashb commented on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812087623 > > To add to your Todo list: update the SerialisedDag representations to include this > > That's actually done already, or it wouldn't make it across to the UI. Unless

[GitHub] [airflow] andrewgodwin commented on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
andrewgodwin commented on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812086673 > To add to your Todo list: update the SerialisedDag representations to include this That's actually done already, or it wouldn't make it across to the UI. Unless

[GitHub] [airflow] ashb commented on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
ashb commented on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812082998 To add to your Todo list: update the SerialisedDag representations to include this -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] ashb edited a comment on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
ashb edited a comment on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812077444 ```from airflow.models import Label``` I'm not a fan of this import for in user dags (they generally don't import from models currently) but I don't have an

[GitHub] [airflow] andrewgodwin commented on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
andrewgodwin commented on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812078553 > `from airflow.models import Label` > > I'm not a fan of this import for in user dags (they generally don't import from models currently) but I don't have an

[GitHub] [airflow] ashb commented on pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
ashb commented on pull request #15142: URL: https://github.com/apache/airflow/pull/15142#issuecomment-812077444 ```from airflow.models import Label``` I'm not a fan of this import for in user dags (they generally don't import from models currently) but I don't have an immediate

[GitHub] [airflow] ashb commented on issue #15140: Allow labelling of task dependencies

2021-04-01 Thread GitBox
ashb commented on issue #15140: URL: https://github.com/apache/airflow/issues/15140#issuecomment-812076510 Cc @eladkal -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] github-actions[bot] commented on pull request #15100: Add support for arbitrary json in conn uri format

2021-04-01 Thread GitBox
github-actions[bot] commented on pull request #15100: URL: https://github.com/apache/airflow/pull/15100#issuecomment-812074683 [The Workflow run](https://github.com/apache/airflow/actions/runs/709342034) is cancelling this PR. Building image for the PR has been cancelled -- This is an

[GitHub] [airflow] github-actions[bot] commented on pull request #15100: Add support for arbitrary json in conn uri format

2021-04-01 Thread GitBox
github-actions[bot] commented on pull request #15100: URL: https://github.com/apache/airflow/pull/15100#issuecomment-812074200 [The Workflow run](https://github.com/apache/airflow/actions/runs/709381633) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static

[GitHub] [airflow] andrewgodwin opened a new pull request #15142: Add support for labelling DAG edges (WIP)

2021-04-01 Thread GitBox
andrewgodwin opened a new pull request #15142: URL: https://github.com/apache/airflow/pull/15142 This adds support for putting human-readable labels on edges in the DAG between Tasks, as well as making the underlying framework for that generic enough that future metadata could be

[GitHub] [airflow] dstandish edited a comment on pull request #11350: Allow execution of multiple sql statements in SnowflakeHook

2021-04-01 Thread GitBox
dstandish edited a comment on pull request #11350: URL: https://github.com/apache/airflow/pull/11350#issuecomment-812038431 Any reason why this one needs to be abandoned? I can imagine circumstances where you just want to start over and go in a different direction. I'm not

[GitHub] [airflow] dstandish edited a comment on pull request #11350: Allow execution of multiple sql statements in SnowflakeHook

2021-04-01 Thread GitBox
dstandish edited a comment on pull request #11350: URL: https://github.com/apache/airflow/pull/11350#issuecomment-812038431 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] github-actions[bot] commented on pull request #15100: Add support for arbitrary json in conn uri format

2021-04-01 Thread GitBox
github-actions[bot] commented on pull request #15100: URL: https://github.com/apache/airflow/pull/15100#issuecomment-812057210 [The Workflow run](https://github.com/apache/airflow/actions/runs/709302929) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static

[GitHub] [airflow] uranusjr opened a new pull request #15141: Share app instance between Kerberos tests

2021-04-01 Thread GitBox
uranusjr opened a new pull request #15141: URL: https://github.com/apache/airflow/pull/15141 Continuing #14878. I don’t have Kerberos set up to run the tests, but this should save a slight bit of time. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] eladkal closed issue #13570: `ImportChangesRule` raises false positives

2021-04-01 Thread GitBox
eladkal closed issue #13570: URL: https://github.com/apache/airflow/issues/13570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] andrewgodwin opened a new issue #15140: Allow labelling of task dependencies

2021-04-01 Thread GitBox
andrewgodwin opened a new issue #15140: URL: https://github.com/apache/airflow/issues/15140 It would be beneficial to larger/more complex DAGs to be able to label the dependency edges between tasks so that, in the graph view (and potentially other places), we can show this information to

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15140: Allow labelling of task dependencies

2021-04-01 Thread GitBox
boring-cyborg[bot] commented on issue #15140: URL: https://github.com/apache/airflow/issues/15140#issuecomment-812050020 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] iameugenejo commented on issue #15036: Scheduler is skipping a day sometimes

2021-04-01 Thread GitBox
iameugenejo commented on issue #15036: URL: https://github.com/apache/airflow/issues/15036#issuecomment-812048076 @eladkal , it happened 5 times so far since 2/20. It's happening to 1 specific dag. The dag itself is static but the tasks the dag executes are generated

[GitHub] [airflow] dstandish edited a comment on pull request #11350: Allow execution of multiple sql statements in SnowflakeHook

2021-04-01 Thread GitBox
dstandish edited a comment on pull request #11350: URL: https://github.com/apache/airflow/pull/11350#issuecomment-812038431 Hi @JavierLopezT I think it would be better to keep the same PR rather than creating a new one. Any reason why this one needs to be abandoned? -- This is an

[GitHub] [airflow] dstandish commented on pull request #11350: Allow execution of multiple sql statements in SnowflakeHook

2021-04-01 Thread GitBox
dstandish commented on pull request #11350: URL: https://github.com/apache/airflow/pull/11350#issuecomment-812038431 Hi @JavierLopezT I think it would be better to keep the same PR rather than creating a new one. Any reason why this one needs to be abandoned. -- This is an

[GitHub] [airflow] turbaszek commented on pull request #15087: Show dagbag import errors in dag list command

2021-04-01 Thread GitBox
turbaszek commented on pull request #15087: URL: https://github.com/apache/airflow/pull/15087#issuecomment-812036692 > I like this idea! It will be helpful for the users. Thanks @mik-laj ! I'm just wondering about adding `"ERROR"` as a `dag_id` for dagfiles with errors so it's even

[airflow] branch master updated (ee2d6c5 -> da897c9)

2021-04-01 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from ee2d6c5 Make app creation session- or class-wide when possible to make tests faster (#14878) add da897c9

[GitHub] [airflow] dimberman closed issue #11870: Add @taskgroup decorator

2021-04-01 Thread GitBox
dimberman closed issue #11870: URL: https://github.com/apache/airflow/issues/11870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [airflow] dimberman merged pull request #15034: Taskgroup decorator

2021-04-01 Thread GitBox
dimberman merged pull request #15034: URL: https://github.com/apache/airflow/pull/15034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [airflow] bbovenzi commented on a change in pull request #15042: Add REST API auth flow for the new UI

2021-04-01 Thread GitBox
bbovenzi commented on a change in pull request #15042: URL: https://github.com/apache/airflow/pull/15042#discussion_r605784949 ## File path: airflow/api_connexion/openapi/v1.yaml ## @@ -1490,6 +1490,142 @@ paths: '404': $ref:

[GitHub] [airflow] boring-cyborg[bot] commented on issue #15139: Add documentation on how to easily add new python packages to the docker-compose development setup

2021-04-01 Thread GitBox
boring-cyborg[bot] commented on issue #15139: URL: https://github.com/apache/airflow/issues/15139#issuecomment-812013911 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] CharlesWinter opened a new issue #15139: Add documentation on how to easily add new python packages to the docker-compose development setup

2021-04-01 Thread GitBox
CharlesWinter opened a new issue #15139: URL: https://github.com/apache/airflow/issues/15139 **Description** It is currently not clear how one should use 3rd party python packages when running apache airflow locally using docker-compose as shown here.

  1   2   3   >