[airflow] tag nightly-master updated (096f5c5 -> c420dbd)

2020-06-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from 096f5c5 (commit) to c420dbd (commit) from

[GitHub] [airflow] r-richmond commented on pull request #8793: Expose option: look_for_keys in ssh_hook via extras

2020-06-27 Thread GitBox
r-richmond commented on pull request #8793: URL: https://github.com/apache/airflow/pull/8793#issuecomment-650644169 I've made both of your requested changes (docs & additional test). It looks like one of the tests failed for an unrelated issue. ```

[jira] [Commented] (AIRFLOW-5391) Clearing a task skipped by BranchPythonOperator will cause the task to execute

2020-06-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147102#comment-17147102 ] ASF GitHub Bot commented on AIRFLOW-5391: - kaxil commented on pull request #8992: URL:

[GitHub] [airflow] kaxil commented on pull request #8992: [AIRFLOW-5391] Do not re-run skipped tasks when they are cleared (#7276)

2020-06-27 Thread GitBox
kaxil commented on pull request #8992: URL: https://github.com/apache/airflow/pull/8992#issuecomment-650620677 > > @yuqian90 tests are failing, can you take a look please > > Hi @kaxil thank you. I looked into it. I think `tests/operators/test_latest_only_operator.py:test_run` is

[GitHub] [airflow] j-y-matsubara commented on a change in pull request #9531: Support .airflowignore for plugins : .pluginingore

2020-06-27 Thread GitBox
j-y-matsubara commented on a change in pull request #9531: URL: https://github.com/apache/airflow/pull/9531#discussion_r446508436 ## File path: airflow/plugins_manager.py ## @@ -164,8 +163,28 @@ def load_plugins_from_plugin_directory(): global plugins # pylint:

[GitHub] [airflow] kaxil commented on pull request #9550: Enable Black - Python Auto Formmatter

2020-06-27 Thread GitBox
kaxil commented on pull request #9550: URL: https://github.com/apache/airflow/pull/9550#issuecomment-650599014 Do you guys have any opinion if we should delay "Black" or do it now? We can also enable this on `v1-10-test`. so shouldn't make backporting difficult.

[GitHub] [airflow] kaxil opened a new pull request #9550: Enable Black - Python Auto Formmatter

2020-06-27 Thread GitBox
kaxil opened a new pull request #9550: URL: https://github.com/apache/airflow/pull/9550 Enable Black using pre-commit --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for

[jira] [Commented] (AIRFLOW-6719) Introduce pyupgrade to enforce latest syntax

2020-06-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147048#comment-17147048 ] ASF GitHub Bot commented on AIRFLOW-6719: - potiuk commented on pull request #7573: URL:

[GitHub] [airflow] potiuk commented on pull request #7573: [AIRFLOW-6719][WIP] Introduce pyupgrade to enforce latest syntax

2020-06-27 Thread GitBox
potiuk commented on pull request #7573: URL: https://github.com/apache/airflow/pull/7573#issuecomment-650598003 @zhongjiajie -> backports are fiinished. You can proceed with that :) This is an automated message from the

[airflow] branch master updated: Bump Pylint to 2.5.3 (#9294)

2020-06-27 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new c420dbd Bump Pylint to 2.5.3 (#9294)

[GitHub] [airflow] kaxil merged pull request #9294: Bump Pylint to 2.5.3

2020-06-27 Thread GitBox
kaxil merged pull request #9294: URL: https://github.com/apache/airflow/pull/9294 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] stale[bot] commented on pull request #7573: [AIRFLOW-6719][WIP] Introduce pyupgrade to enforce latest syntax

2020-06-27 Thread GitBox
stale[bot] commented on pull request #7573: URL: https://github.com/apache/airflow/pull/7573#issuecomment-650584653 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jira] [Commented] (AIRFLOW-6719) Introduce pyupgrade to enforce latest syntax

2020-06-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147032#comment-17147032 ] ASF GitHub Bot commented on AIRFLOW-6719: - stale[bot] commented on pull request #7573: URL:

[GitHub] [airflow] mdediana commented on pull request #9354: Task logging handlers can provide custom log links

2020-06-27 Thread GitBox
mdediana commented on pull request #9354: URL: https://github.com/apache/airflow/pull/9354#issuecomment-650568904 @aneesh-joseph and @mik-laj I will do my best to address change requests as quick as possible. In general, I have more time on weekends, but I can do something on weekdays

[GitHub] [airflow] mdediana commented on a change in pull request #9354: Task logging handlers can provide custom log links

2020-06-27 Thread GitBox
mdediana commented on a change in pull request #9354: URL: https://github.com/apache/airflow/pull/9354#discussion_r446532544 ## File path: airflow/www/views.py ## @@ -1493,8 +1505,14 @@ def recurse_nodes(task, visited): form =

[GitHub] [airflow] mdediana commented on a change in pull request #9354: Task logging handlers can provide custom log links

2020-06-27 Thread GitBox
mdediana commented on a change in pull request #9354: URL: https://github.com/apache/airflow/pull/9354#discussion_r446532508 ## File path: airflow/utils/log/es_task_handler.py ## @@ -262,3 +267,24 @@ def close(self): super().close() self.closed = True + +

[GitHub] [airflow] mdediana commented on a change in pull request #9354: Task logging handlers can provide custom log links

2020-06-27 Thread GitBox
mdediana commented on a change in pull request #9354: URL: https://github.com/apache/airflow/pull/9354#discussion_r446532566 ## File path: airflow/www/views.py ## @@ -760,22 +761,33 @@ def log(self, session=None): execution_date=execution_date, form=form,

[GitHub] [airflow] mdediana commented on a change in pull request #9354: Task logging handlers can provide custom log links

2020-06-27 Thread GitBox
mdediana commented on a change in pull request #9354: URL: https://github.com/apache/airflow/pull/9354#discussion_r446532395 ## File path: airflow/www/views.py ## @@ -760,22 +761,33 @@ def log(self, session=None): execution_date=execution_date, form=form,

[GitHub] [airflow] VinayGb665 opened a new pull request #9549: YAML file supports extra json parameters

2020-06-27 Thread GitBox
VinayGb665 opened a new pull request #9549: URL: https://github.com/apache/airflow/pull/9549 --- LocalFileSystemBackend supports YAML files to add connections and the extra parameters ( like in here ) would have to be provided as raw JSON instead of default YAML. The PR is to resolve

[airflow] branch master updated (118ea2f -> e3e20e7)

2020-06-27 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 118ea2f show correct duration on graph view for running task (#8311) (#8675) add e3e20e7 Deprecate

[GitHub] [airflow] turbaszek merged pull request #9540: Contrib cleanup

2020-06-27 Thread GitBox
turbaszek merged pull request #9540: URL: https://github.com/apache/airflow/pull/9540 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] ashb merged pull request #8675: show correct duration on graph view for running task (#8311)

2020-06-27 Thread GitBox
ashb merged pull request #8675: URL: https://github.com/apache/airflow/pull/8675 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated: show correct duration on graph view for running task (#8311) (#8675)

2020-06-27 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 118ea2f show correct duration on graph view for

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8675: show correct duration on graph view for running task (#8311)

2020-06-27 Thread GitBox
boring-cyborg[bot] commented on pull request #8675: URL: https://github.com/apache/airflow/pull/8675#issuecomment-650562577 Awesome work, congrats on your first merged pull request! This is an automated message from the

[GitHub] [airflow] KumarManoj-S commented on issue #9548: Add support for snowflake on the sql_sensor

2020-06-27 Thread GitBox
KumarManoj-S commented on issue #9548: URL: https://github.com/apache/airflow/issues/9548#issuecomment-650562066 Hi, I would like to pick up this issue if it is a good feature to come in. This is an automated message from

[GitHub] [airflow] KumarManoj-S opened a new issue #9548: Add support for snowflake on the sql_sensor

2020-06-27 Thread GitBox
KumarManoj-S opened a new issue #9548: URL: https://github.com/apache/airflow/issues/9548 **Description** Enable the support for Snowflake on the SqlSensor. **Use case** Currently, the `SqlSensor` supports only the following connection types, -

[GitHub] [airflow] boring-cyborg[bot] commented on issue #9548: Add support for snowflake on the sql_sensor

2020-06-27 Thread GitBox
boring-cyborg[bot] commented on issue #9548: URL: https://github.com/apache/airflow/issues/9548#issuecomment-650561970 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] ipeluffo edited a comment on pull request #9540: Contrib cleanup

2020-06-27 Thread GitBox
ipeluffo edited a comment on pull request #9540: URL: https://github.com/apache/airflow/pull/9540#issuecomment-650561574 @potiuk , it's ready for review. Thanks This is an automated message from the Apache Git Service. To

[GitHub] [airflow] ipeluffo commented on pull request #9540: Contrib cleanup

2020-06-27 Thread GitBox
ipeluffo commented on pull request #9540: URL: https://github.com/apache/airflow/pull/9540#issuecomment-650561574 Ready for review. Thanks This is an automated message from the Apache Git Service. To respond to the message,

[airflow] branch master updated (cf510a3 -> 53c0f2d)

2020-06-27 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from cf510a3 Make Production Dockerfile OpenShift-compatible (#9545) add 53c0f2d Show "Task Reschedule" table

[GitHub] [airflow] kaxil merged pull request #9521: Show "Task Reschedule" table in Airflow Webserver

2020-06-27 Thread GitBox
kaxil merged pull request #9521: URL: https://github.com/apache/airflow/pull/9521 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk commented on pull request #9521: Show "Task Reschedule" table in Airflow Webserver

2020-06-27 Thread GitBox
potiuk commented on pull request #9521: URL: https://github.com/apache/airflow/pull/9521#issuecomment-650558975 NICE! This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on pull request #9540: Contrib cleanup

2020-06-27 Thread GitBox
potiuk commented on pull request #9540: URL: https://github.com/apache/airflow/pull/9540#issuecomment-650558873 Can you un-draft it please :) This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on pull request #9540: Contrib cleanup

2020-06-27 Thread GitBox
potiuk commented on pull request #9540: URL: https://github.com/apache/airflow/pull/9540#issuecomment-650558852 Looks good. I think it's ready to merge :) This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] potiuk opened a new pull request #9547: More sensible docker caching strategy for Prod images

2020-06-27 Thread GitBox
potiuk opened a new pull request #9547: URL: https://github.com/apache/airflow/pull/9547 Local caching is now default strategy when building the Production image. You can still change it to pulled - similar to CI builds by providing the right build flag and this is what is

[airflow] branch master updated: Make Production Dockerfile OpenShift-compatible (#9545)

2020-06-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new cf510a3 Make Production Dockerfile

[airflow] branch master updated: Make Production Dockerfile OpenShift-compatible (#9545)

2020-06-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new cf510a3 Make Production Dockerfile

[GitHub] [airflow] potiuk closed issue #9248: Docker Image Compatibility with Openshift/Kubernetes

2020-06-27 Thread GitBox
potiuk closed issue #9248: URL: https://github.com/apache/airflow/issues/9248 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk merged pull request #9545: Make Production Dockerfile OpenShift-compatible

2020-06-27 Thread GitBox
potiuk merged pull request #9545: URL: https://github.com/apache/airflow/pull/9545 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk closed issue #8706: User ’airflow' should be in group 'root' in the Docker image

2020-06-27 Thread GitBox
potiuk closed issue #8706: URL: https://github.com/apache/airflow/issues/8706 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] VinayGb665 closed pull request #9546: YAML file supporting extra json parameters for LocalFileSystemBackend

2020-06-27 Thread GitBox
VinayGb665 closed pull request #9546: URL: https://github.com/apache/airflow/pull/9546 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] VinayGb665 opened a new pull request #9546: YAML file supporting extra json parameters for LocalFileSystemBackend

2020-06-27 Thread GitBox
VinayGb665 opened a new pull request #9546: URL: https://github.com/apache/airflow/pull/9546 LocalFileSystemBackend supports YAML files to add connections and the extra parameters ( like in [here](https://airflow.readthedocs.io/en/latest/howto/connection/gcp.html) ) would

[GitHub] [airflow] fuyi commented on pull request #8675: show correct duration on graph view for running task (#8311)

2020-06-27 Thread GitBox
fuyi commented on pull request #8675: URL: https://github.com/apache/airflow/pull/8675#issuecomment-650548162 Thanks for comment @ashb , The screenshot is updated now This is an automated message from the Apache Git Service.

[GitHub] [airflow] potiuk edited a comment on pull request #9545: Make Production Dockerfile OpenShift-compatible

2020-06-27 Thread GitBox
potiuk edited a comment on pull request #9545: URL: https://github.com/apache/airflow/pull/9545#issuecomment-650543567 cc @altunbaratli @flymg This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on pull request #9545: Make Production Dockerfile OpenShift-compatible

2020-06-27 Thread GitBox
potiuk commented on pull request #9545: URL: https://github.com/apache/airflow/pull/9545#issuecomment-650543567 cc @altunbarati @flymg This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] potiuk opened a new pull request #9545: Make Production Dockerfile OpenShift-compatible

2020-06-27 Thread GitBox
potiuk opened a new pull request #9545: URL: https://github.com/apache/airflow/pull/9545 OpenShift (and other Kubernetes platforms) often use the approach that they start containers with random user and root group. This is described in the

[GitHub] [airflow] j-y-matsubara commented on a change in pull request #9531: Support .airflowignore for plugins : .pluginingore

2020-06-27 Thread GitBox
j-y-matsubara commented on a change in pull request #9531: URL: https://github.com/apache/airflow/pull/9531#discussion_r446508436 ## File path: airflow/plugins_manager.py ## @@ -164,8 +163,28 @@ def load_plugins_from_plugin_directory(): global plugins # pylint:

[jira] [Commented] (AIRFLOW-5391) Clearing a task skipped by BranchPythonOperator will cause the task to execute

2020-06-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146854#comment-17146854 ] ASF GitHub Bot commented on AIRFLOW-5391: - yuqian90 commented on pull request #8992: URL:

[GitHub] [airflow] yuqian90 commented on pull request #8992: [AIRFLOW-5391] Do not re-run skipped tasks when they are cleared (#7276)

2020-06-27 Thread GitBox
yuqian90 commented on pull request #8992: URL: https://github.com/apache/airflow/pull/8992#issuecomment-650531210 > @yuqian90 tests are failing, can you take a look please Hi @kaxil thank you. I looked into it. I think `tests/operators/test_latest_only_operator.py:test_run` is the

[GitHub] [airflow] potiuk commented on pull request #9502: generate go client from openapi spec

2020-06-27 Thread GitBox
potiuk commented on pull request #9502: URL: https://github.com/apache/airflow/pull/9502#issuecomment-650527361 One general comment - I think we should not commit the generated coded of client to the Airflow Code nor copied OpenAPI specification. It should be generated and ignored,

[GitHub] [airflow] potiuk edited a comment on pull request #9502: generate go client from openapi spec

2020-06-27 Thread GitBox
potiuk edited a comment on pull request #9502: URL: https://github.com/apache/airflow/pull/9502#issuecomment-650527361 One general comment - I think we should not commit the generated code of client to the Airflow Code nor copied OpenAPI specification. It should be generated and ignored,

[GitHub] [airflow] potiuk commented on pull request #8793: Expose option: look_for_keys in ssh_hook via extras

2020-06-27 Thread GitBox
potiuk commented on pull request #8793: URL: https://github.com/apache/airflow/pull/8793#issuecomment-650526549 And a test with "look_for_keys" = False would be nice as well. This is an automated message from the Apache Git

[GitHub] [airflow] Acehaidrey opened a new pull request #9544: Add metric for scheduling delay between first run task & expected start time

2020-06-27 Thread GitBox
Acehaidrey opened a new pull request #9544: URL: https://github.com/apache/airflow/pull/9544 This is to emit the true scheduling delay stats, which is defined as the time when the first task in DAG starts minus the expected DAG run datetime. This method will be used in

[GitHub] [airflow] Acehaidrey commented on pull request #9521: Show "Task Reschedule" table in Airflow Webserver

2020-06-27 Thread GitBox
Acehaidrey commented on pull request #9521: URL: https://github.com/apache/airflow/pull/9521#issuecomment-650518918 I would find this to be useful for us as well This is an automated message from the Apache Git Service. To

[GitHub] [airflow] Acehaidrey commented on pull request #9472: Add drop_partition functionality for HiveMetastoreHook

2020-06-27 Thread GitBox
Acehaidrey commented on pull request #9472: URL: https://github.com/apache/airflow/pull/9472#issuecomment-650518730 @vanka56 loook at the error logs ``` There were some pylint errors. Exiting

[GitHub] [airflow] potiuk commented on issue #9539: raise AirflowException("SSH operator error: {0}".format(str(e)))

2020-06-27 Thread GitBox
potiuk commented on issue #9539: URL: https://github.com/apache/airflow/issues/9539#issuecomment-650517639 Github issues are not ment for troubleshooting (see the templates we have) instead, please discuss your problem on slack/#troubleshooting channel. You will get all the information

[GitHub] [airflow] potiuk closed issue #9539: raise AirflowException("SSH operator error: {0}".format(str(e)))

2020-06-27 Thread GitBox
potiuk closed issue #9539: URL: https://github.com/apache/airflow/issues/9539 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on pull request #9540: Contrib cleanup

2020-06-27 Thread GitBox
potiuk commented on pull request #9540: URL: https://github.com/apache/airflow/pull/9540#issuecomment-650517082 Some wekday tests are failing :( This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] mik-laj commented on pull request #8793: Expose option: look_for_keys in ssh_hook via extras

2020-06-27 Thread GitBox
mik-laj commented on pull request #8793: URL: https://github.com/apache/airflow/pull/8793#issuecomment-650516935 > Relevant documentation is updated including usage instructions. Can you also update the docs? https://airflow.readthedocs.io/en/latest/howto/connection/ssh.html

[GitHub] [airflow] stale[bot] commented on pull request #8699: Scheduler just checks for task instances in 'running' state in execution.

2020-06-27 Thread GitBox
stale[bot] commented on pull request #8699: URL: https://github.com/apache/airflow/pull/8699#issuecomment-650516774 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] chandansingh1653 commented on issue #9543: Enable airflow to orchestrate python job on AWS glue

2020-06-27 Thread GitBox
chandansingh1653 commented on issue #9543: URL: https://github.com/apache/airflow/issues/9543#issuecomment-650512547 I can work on this if this feature looks valid This is an automated message from the Apache Git Service. To

[GitHub] [airflow] chandansingh1653 opened a new issue #9543: Enable airflow to orchestrate python job on AWS glue

2020-06-27 Thread GitBox
chandansingh1653 opened a new issue #9543: URL: https://github.com/apache/airflow/issues/9543 Description Enable airflow to trigger the python glue job --> Use case: **Related Issues**

[GitHub] [airflow] boring-cyborg[bot] commented on issue #9543: Enable airflow to orchestrate python job on AWS glue

2020-06-27 Thread GitBox
boring-cyborg[bot] commented on issue #9543: URL: https://github.com/apache/airflow/issues/9543#issuecomment-650511973 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated