[GitHub] [airflow] feng-tao commented on issue #5160: [AIRFLOW-4394] Don't double test behaviour of BackfillJob from CLI tests

2019-04-23 Thread GitBox
feng-tao commented on issue #5160: [AIRFLOW-4394] Don't double test behaviour of BackfillJob from CLI tests URL: https://github.com/apache/airflow/pull/5160#issuecomment-486077899 Nice, do we know how much we expect to save ?

[jira] [Commented] (AIRFLOW-4402) Update super() calls for nvd3

2019-04-23 Thread Tao Feng (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824830#comment-16824830 ] Tao Feng commented on AIRFLOW-4402: --- [~milton0825] didn't quite follow all the prs recently. But I

[GitHub] [airflow] codecov-io edited a comment on issue #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue

2019-04-23 Thread GitBox
codecov-io edited a comment on issue #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue URL: https://github.com/apache/airflow/pull/5167#issuecomment-486068982 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5167?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue

2019-04-23 Thread GitBox
codecov-io edited a comment on issue #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue URL: https://github.com/apache/airflow/pull/5167#issuecomment-486068982 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5167?src=pr=h1) Report > Merging

[GitHub] [airflow] feng-tao edited a comment on issue #5019: [AIRFLOW-4199][AIP-3-step3] Remove all sys version info

2019-04-23 Thread GitBox
feng-tao edited a comment on issue #5019: [AIRFLOW-4199][AIP-3-step3] Remove all sys version info URL: https://github.com/apache/airflow/pull/5019#issuecomment-486072340 @kaxil, @potiuk , @ashb, @bolkedebruin are we supposed to change the vendor files? I thought files under vendor dir

[GitHub] [airflow] feng-tao commented on issue #5019: [AIRFLOW-4199][AIP-3-step3] Remove all sys version info

2019-04-23 Thread GitBox
feng-tao commented on issue #5019: [AIRFLOW-4199][AIP-3-step3] Remove all sys version info URL: https://github.com/apache/airflow/pull/5019#issuecomment-486072369 cc @milton0825 This is an automated message from the Apache

[GitHub] [airflow] feng-tao commented on issue #5019: [AIRFLOW-4199][AIP-3-step3] Remove all sys version info

2019-04-23 Thread GitBox
feng-tao commented on issue #5019: [AIRFLOW-4199][AIP-3-step3] Remove all sys version info URL: https://github.com/apache/airflow/pull/5019#issuecomment-486072340 @kaxil, @potiuk , @ashb, @bolkedebruin are we supposed to change the vendor

[jira] [Commented] (AIRFLOW-4402) Update super() calls for nvd3

2019-04-23 Thread Chao-Han Tsai (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824823#comment-16824823 ] Chao-Han Tsai commented on AIRFLOW-4402: Hmm i have seen changes on those files. > Update

[GitHub] [airflow] codecov-io edited a comment on issue #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue

2019-04-23 Thread GitBox
codecov-io edited a comment on issue #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue URL: https://github.com/apache/airflow/pull/5167#issuecomment-486068982 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5167?src=pr=h1) Report > Merging

[GitHub] [airflow] milton0825 commented on issue #5143: [AIRFLOW-4204] Update super() calls to PY3

2019-04-23 Thread GitBox
milton0825 commented on issue #5143: [AIRFLOW-4204] Update super() calls to PY3 URL: https://github.com/apache/airflow/pull/5143#issuecomment-486068967 Removed the second commit and created a separate PR: https://github.com/apache/airflow/pull/5168

[GitHub] [airflow] codecov-io commented on issue #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue

2019-04-23 Thread GitBox
codecov-io commented on issue #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue URL: https://github.com/apache/airflow/pull/5167#issuecomment-486068982 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5167?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-4402) Update super() calls for nvd3

2019-04-23 Thread Tao Feng (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824820#comment-16824820 ] Tao Feng commented on AIRFLOW-4402: --- hold on, I don't think we should change any of the files under

[GitHub] [airflow] milton0825 opened a new pull request #5168: [AIRFLOW-4204] Update super() calls to PY3 for nvd3

2019-04-23 Thread GitBox
milton0825 opened a new pull request #5168: [AIRFLOW-4204] Update super() calls to PY3 for nvd3 URL: https://github.com/apache/airflow/pull/5168 (cherry picked from commit 52edaebf1a043120f8a9882b768fea0cf2270db1) Make sure you have checked _all_ steps below. ### Jira

[jira] [Updated] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4401: -- Description: After discussing with [~ash] and [~BasPH] potential reasons for flakiness of

[jira] [Updated] (AIRFLOW-4402) Update super() calls for nvd3

2019-04-23 Thread Chao-Han Tsai (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai updated AIRFLOW-4402: --- Description: In all classes under nvd3, replace {{super(__class__, self).__init__(...)}}

[jira] [Created] (AIRFLOW-4402) Update super() calls for nvd3

2019-04-23 Thread Chao-Han Tsai (JIRA)
Chao-Han Tsai created AIRFLOW-4402: -- Summary: Update super() calls for nvd3 Key: AIRFLOW-4402 URL: https://issues.apache.org/jira/browse/AIRFLOW-4402 Project: Apache Airflow Issue Type:

[jira] [Updated] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4401: -- Description: After discussing with [~ash] and [~BasPH] potential reasons for flakiness of

[GitHub] [airflow] milton0825 commented on issue #5041: [AIRFLOW-4208] Replace @abstractproperty by @abstractmethod

2019-04-23 Thread GitBox
milton0825 commented on issue #5041: [AIRFLOW-4208] Replace @abstractproperty by @abstractmethod URL: https://github.com/apache/airflow/pull/5041#issuecomment-486066778 Shall we merge this? This is an automated message from

[GitHub] [airflow] jmcarp commented on issue #4821: [AIRFLOW-3998] Use nested commands in cli.

2019-04-23 Thread GitBox
jmcarp commented on issue #4821: [AIRFLOW-3998] Use nested commands in cli. URL: https://github.com/apache/airflow/pull/4821#issuecomment-486065962 Updated documentation to match the new CLI structure. What's the next step here? A vote on the AIP?

[jira] [Updated] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4401: -- Description: After discussing with [~ash] and [~BasPH] potential reasons for flakiness of

[GitHub] [airflow] potiuk commented on a change in pull request #5166: [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor

2019-04-23 Thread GitBox
potiuk commented on a change in pull request #5166: [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor URL: https://github.com/apache/airflow/pull/5166#discussion_r277952601 ## File path: airflow/contrib/sensors/gcs_sensor.py ## @@ -160,3 +162,105 @@ def poke(self,

[GitHub] [airflow] jmcarp commented on issue #5030: [AIRFLOW-4227] Use python3-style type annotations.

2019-04-23 Thread GitBox
jmcarp commented on issue #5030: [AIRFLOW-4227] Use python3-style type annotations. URL: https://github.com/apache/airflow/pull/5030#issuecomment-486064887 This should be ready to go now that #5090 is finished. cc @Fokko @kaxil

[jira] [Created] (AIRFLOW-4401) multiprocessing.Queue is unreliable

2019-04-23 Thread Jarek Potiuk (JIRA)
Jarek Potiuk created AIRFLOW-4401: - Summary: multiprocessing.Queue is unreliable Key: AIRFLOW-4401 URL: https://issues.apache.org/jira/browse/AIRFLOW-4401 Project: Apache Airflow Issue Type:

[GitHub] [airflow] potiuk edited a comment on issue #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor

2019-04-23 Thread GitBox
potiuk edited a comment on issue #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor URL: https://github.com/apache/airflow/pull/5159#issuecomment-486063506 Hey @BasPH @ashb . I woke up this morning and could not stop thinking about that one and I took a dep dive. It

[GitHub] [airflow] potiuk commented on issue #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor

2019-04-23 Thread GitBox
potiuk commented on issue #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor URL: https://github.com/apache/airflow/pull/5159#issuecomment-486063506 Hey @BasPH @ashb . I woke up this morning and could not stop thinking about that one and I took a dep dive. It was great

[GitHub] [airflow] potiuk opened a new pull request #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue

2019-04-23 Thread GitBox
potiuk opened a new pull request #5167: Reliable SynchronizedQueue used instead of multiprocessing.Queue URL: https://github.com/apache/airflow/pull/5167 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[jira] [Updated] (AIRFLOW-4401) multiprocessing.Queue is unreliable

2019-04-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4401: -- Description: After discussing with [~ash] and [~BasPH] potential reasons for flakiness of

[jira] [Updated] (AIRFLOW-4401) multiprocessing.Queue.empty() is unreliable

2019-04-23 Thread Jarek Potiuk (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-4401: -- Summary: multiprocessing.Queue.empty() is unreliable (was: multiprocessing.Queue is

[GitHub] [airflow] codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli.

2019-04-23 Thread GitBox
codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli. URL: https://github.com/apache/airflow/pull/4821#issuecomment-468981700 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli.

2019-04-23 Thread GitBox
codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli. URL: https://github.com/apache/airflow/pull/4821#issuecomment-468981700 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=h1) Report > Merging

[GitHub] [airflow] jaketf opened a new pull request #5166: [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor

2019-04-23 Thread GitBox
jaketf opened a new pull request #5166: [AIRFLOW-4397] Add GCSUploadSessionCompleteSensor URL: https://github.com/apache/airflow/pull/5166 This commit add a GoogleCloudStorageUploadSessionCompleteSensor to address the use case of accepting files from a third party vendor who refuses

[GitHub] [airflow] codecov-io commented on issue #5165: [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag

2019-04-23 Thread GitBox
codecov-io commented on issue #5165: [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag URL: https://github.com/apache/airflow/pull/5165#issuecomment-486059506 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5165?src=pr=h1) Report > Merging

[jira] [Updated] (AIRFLOW-4400) Wrong filemode on three files

2019-04-23 Thread Beau Barker (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Beau Barker updated AIRFLOW-4400: - Description: Three files inside the models directory have the wrong file mode. __init__.py,

[jira] [Created] (AIRFLOW-4400) Wrong filemode on three files

2019-04-23 Thread Beau Barker (JIRA)
Beau Barker created AIRFLOW-4400: Summary: Wrong filemode on three files Key: AIRFLOW-4400 URL: https://issues.apache.org/jira/browse/AIRFLOW-4400 Project: Apache Airflow Issue Type: Bug

[jira] [Resolved] (AIRFLOW-4324) Partial search in RBAC UI no longer works

2019-04-23 Thread Xiaodong DENG (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaodong DENG resolved AIRFLOW-4324. Resolution: Resolved > Partial search in RBAC UI no longer works >

[GitHub] [airflow] XD-DENG edited a comment on issue #5131: [AIRFLOW-4324] fix search in RBAC UI

2019-04-23 Thread GitBox
XD-DENG edited a comment on issue #5131: [AIRFLOW-4324] fix search in RBAC UI URL: https://github.com/apache/airflow/pull/5131#issuecomment-486053307 On the other hand, we have the same issue in `master` branch https://github.com/apache/airflow/blob/master/airflow/www/views.py#L239

[jira] [Commented] (AIRFLOW-4324) Partial search in RBAC UI no longer works

2019-04-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824784#comment-16824784 ] ASF GitHub Bot commented on AIRFLOW-4324: - XD-DENG commented on pull request #5131:

[jira] [Commented] (AIRFLOW-4324) Partial search in RBAC UI no longer works

2019-04-23 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824785#comment-16824785 ] ASF subversion and git services commented on AIRFLOW-4324: -- Commit

[GitHub] [airflow] XD-DENG merged pull request #5131: [AIRFLOW-4324] fix search in RBAC UI

2019-04-23 Thread GitBox
XD-DENG merged pull request #5131: [AIRFLOW-4324] fix search in RBAC UI URL: https://github.com/apache/airflow/pull/5131 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Assigned] (AIRFLOW-3143) Support auto-zone in DataprocClusterCreateOperator

2019-04-23 Thread Joel Croteau (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Croteau reassigned AIRFLOW-3143: - Assignee: Joel Croteau (was: fengya li) > Support auto-zone in

[jira] [Commented] (AIRFLOW-4399) To avoid duplicated os.path.isfile(filepath) checks in models.dagbag

2019-04-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824766#comment-16824766 ] ASF GitHub Bot commented on AIRFLOW-4399: - XD-DENG commented on pull request #5165:

[GitHub] [airflow] XD-DENG opened a new pull request #5165: [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag

2019-04-23 Thread GitBox
XD-DENG opened a new pull request #5165: [AIRFLOW-4399] Avoid duplicated os.path.isfile() check in models.dagbag URL: https://github.com/apache/airflow/pull/5165 ### Jira - https://issues.apache.org/jira/browse/AIRFLOW-4399 ### Description The check which is removed

[jira] [Updated] (AIRFLOW-4399) To avoid duplicated os.path.isfile(filepath) checks in models.dagbag

2019-04-23 Thread Xiaodong DENG (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaodong DENG updated AIRFLOW-4399: --- Summary: To avoid duplicated os.path.isfile(filepath) checks in models.dagbag (was: To

[jira] [Created] (AIRFLOW-4399) To remove duplicated os.path.isfile(filepath) checks in models.dagbag

2019-04-23 Thread Xiaodong DENG (JIRA)
Xiaodong DENG created AIRFLOW-4399: -- Summary: To remove duplicated os.path.isfile(filepath) checks in models.dagbag Key: AIRFLOW-4399 URL: https://issues.apache.org/jira/browse/AIRFLOW-4399 Project:

[GitHub] [airflow] milton0825 commented on a change in pull request #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
milton0825 commented on a change in pull request #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#discussion_r277924790 ## File path: docs/plugins.rst ## @@ -202,6 +212,13 @@ definitions in Airflow. def

[GitHub] [airflow] feng-tao commented on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
feng-tao commented on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486027034 @KevinYang21 , not sure how to DM you, but you may want to check your email :)

[GitHub] [airflow] KevinYang21 edited a comment on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
KevinYang21 edited a comment on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486025570 @feng-tao Took a quick look at the code and I think that's because we start to construct the links from operator instead of from

[GitHub] [airflow] feng-tao edited a comment on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
feng-tao edited a comment on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486026307 yeah, FORM/Design/UI is always a tough question BTW @KevinYang21 , is yrql...@gmail.com your email ?

[GitHub] [airflow] feng-tao commented on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
feng-tao commented on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486026307 yeah, FORM/Design/UI is always a tough question BTW, @KevinYang21 , is yrql...@gmail.com your email ?

[GitHub] [airflow] zhongjiajie commented on a change in pull request #5153: [AIRFLOW-XXX] Name default config_file param in KubernetesPodOperator docstring

2019-04-23 Thread GitBox
zhongjiajie commented on a change in pull request #5153: [AIRFLOW-XXX] Name default config_file param in KubernetesPodOperator docstring URL: https://github.com/apache/airflow/pull/5153#discussion_r277922027 ## File path: airflow/contrib/operators/kubernetes_pod_operator.py

[GitHub] [airflow] KevinYang21 commented on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
KevinYang21 commented on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486025570 @feng-tao Took a quick look at the code and I think that's because we start to construct the links from operator instead of from task

[GitHub] [airflow] milton0825 commented on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
milton0825 commented on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486025327 @feng-tao we exposed the operator so all the following fields should be available to user:

[GitHub] [airflow] milton0825 commented on a change in pull request #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
milton0825 commented on a change in pull request #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#discussion_r277920779 ## File path: tests/plugins/test_plugin.py ## @@ -100,6 +114,10 @@ class

[GitHub] [airflow] milton0825 commented on a change in pull request #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
milton0825 commented on a change in pull request #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#discussion_r277920575 ## File path: airflow/models/baseoperator.py ## @@ -973,6 +986,25 @@ class BaseOperatorLink:

[jira] [Created] (AIRFLOW-4398) Extra links include try_number

2019-04-23 Thread Chao-Han Tsai (JIRA)
Chao-Han Tsai created AIRFLOW-4398: -- Summary: Extra links include try_number Key: AIRFLOW-4398 URL: https://issues.apache.org/jira/browse/AIRFLOW-4398 Project: Apache Airflow Issue Type:

[GitHub] [airflow] milton0825 edited a comment on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
milton0825 edited a comment on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486023240 @KevinYang21 @kunalyogenshah I think it totally make sense to include the try-number into the scope. I am thinking about

[GitHub] [airflow] milton0825 commented on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
milton0825 commented on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486023240 @KevinYang21 @kunalyogenshah I think it totally make sense to include the try-number into the scope. I am thinking about

[jira] [Created] (AIRFLOW-4397) Add GCSUploadSessionCompleteSensor

2019-04-23 Thread Jacob Ferriero (JIRA)
Jacob Ferriero created AIRFLOW-4397: --- Summary: Add GCSUploadSessionCompleteSensor Key: AIRFLOW-4397 URL: https://issues.apache.org/jira/browse/AIRFLOW-4397 Project: Apache Airflow Issue

[GitHub] [airflow] KevinYang21 commented on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
KevinYang21 commented on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486021540 Nice idea to make it pluggable. We do also have the need to put a Kibana link on the dialog but as @kunalyogenshah mentioned, for task

[GitHub] [airflow] kunalyogenshah commented on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
kunalyogenshah commented on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486019322 @milton0825 Got it. Constructing it this way is certainly doable. But somehow it does seem like introducing the Kibana log URLs in a

[GitHub] [airflow] milton0825 commented on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
milton0825 commented on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486015757 @kunalyogenshah so currently the operator instance and the `execution_time` are exposed to you like in:

[jira] [Commented] (AIRFLOW-4346) Kubernetes Executor Fails for Large Wide DAGs

2019-04-23 Thread Vincent Castaneda (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824629#comment-16824629 ] Vincent Castaneda commented on AIRFLOW-4346: Has anyone been looking into this? >

[GitHub] [airflow] kunalyogenshah commented on issue #5094: [AIRFLOW-4306] Global operator extra links

2019-04-23 Thread GitBox
kunalyogenshah commented on issue #5094: [AIRFLOW-4306] Global operator extra links URL: https://github.com/apache/airflow/pull/5094#issuecomment-486008366 Quick question about this implementation. Could you help provide an example usage for let's say Kibana logs that require arguments to

[GitHub] [airflow] kunalyogenshah commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI.

2019-04-23 Thread GitBox
kunalyogenshah commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. URL: https://github.com/apache/airflow/pull/5164#issuecomment-486007872 Perfect. Thanks @milton0825 . I'll move my questions over to that PR for better context :)

[GitHub] [airflow] milton0825 commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI.

2019-04-23 Thread GitBox
milton0825 commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. URL: https://github.com/apache/airflow/pull/5164#issuecomment-486006581 @kunalyogenshah I think this is what you are looking for: https://github.com/apache/airflow/pull/5094 We can

[GitHub] [airflow] kunalyogenshah commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI.

2019-04-23 Thread GitBox
kunalyogenshah commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. URL: https://github.com/apache/airflow/pull/5164#issuecomment-486006137 @feng-tao : A quick naive clarification question. Does this mean that the DAG operator itself must specify

[GitHub] [airflow] kunalyogenshah commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI.

2019-04-23 Thread GitBox
kunalyogenshah commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. URL: https://github.com/apache/airflow/pull/5164#issuecomment-486004575 Ah. Thanks @feng-tao . This seems to be exactly similar to what we've been trying to do. Let me look into

[GitHub] [airflow] feng-tao edited a comment on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI.

2019-04-23 Thread GitBox
feng-tao edited a comment on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. URL: https://github.com/apache/airflow/pull/5164#issuecomment-486001330 And we have already did that at Lyft in prod for quite some

[GitHub] [airflow] feng-tao commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI.

2019-04-23 Thread GitBox
feng-tao commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. URL: https://github.com/apache/airflow/pull/5164#issuecomment-486001330 And we have already did that at Lyft in prod for quite some time. cc @milton0825

[GitHub] [airflow] feng-tao commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI.

2019-04-23 Thread GitBox
feng-tao commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. URL: https://github.com/apache/airflow/pull/5164#issuecomment-486001239 @kunalyogenshah , not sure if this is the right way to show the UI. I think another approach(Lyft) is to leverage

[GitHub] [airflow] codecov-io commented on issue #5161: [AIRFLOW-4395] Remove pickle_info view

2019-04-23 Thread GitBox
codecov-io commented on issue #5161: [AIRFLOW-4395] Remove pickle_info view URL: https://github.com/apache/airflow/pull/5161#issuecomment-486000901 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5161?src=pr=h1) Report > Merging

[GitHub] [airflow] KevinYang21 commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI.

2019-04-23 Thread GitBox
KevinYang21 commented on issue #5164: [AIRFLOW-4396] Provide a link to external Elasticsearch logs in UI. URL: https://github.com/apache/airflow/pull/5164#issuecomment-485994653 This is nice, thank you @kunalyogenshah ! Users using Airflow with Elasticsearch backend do likely want to

[jira] [Commented] (AIRFLOW-3951) No easy way to get query_execution_id from AWSAthenaOperator

2019-04-23 Thread Rob Dinoff (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824570#comment-16824570 ] Rob Dinoff commented on AIRFLOW-3951: - yes you may close this ... thanks, Rob On Tue, Apr 23,

[jira] [Commented] (AIRFLOW-4396) Add a link to Elasticsearch frontend for logs if available

2019-04-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824571#comment-16824571 ] ASF GitHub Bot commented on AIRFLOW-4396: - kunalyogenshah commented on pull request #5164:

[jira] [Created] (AIRFLOW-4396) Add a link to Elasticsearch frontend for logs if available

2019-04-23 Thread Kunal Shah (JIRA)
Kunal Shah created AIRFLOW-4396: --- Summary: Add a link to Elasticsearch frontend for logs if available Key: AIRFLOW-4396 URL: https://issues.apache.org/jira/browse/AIRFLOW-4396 Project: Apache Airflow

[GitHub] [airflow] kaxil edited a comment on issue #5143: [AIRFLOW-4204] Update super() calls to PY3

2019-04-23 Thread GitBox
kaxil edited a comment on issue #5143: [AIRFLOW-4204] Update super() calls to PY3 URL: https://github.com/apache/airflow/pull/5143#issuecomment-485983744 We should have a separate PR for modifying nvd3 stuff - ideally with a new JIRA issue. It becomes easy to revert if need be.

[GitHub] [airflow] codecov-io commented on issue #5163: WIP: [AIRFLOW-XXX] Remove hard-coded logging level

2019-04-23 Thread GitBox
codecov-io commented on issue #5163: WIP: [AIRFLOW-XXX] Remove hard-coded logging level URL: https://github.com/apache/airflow/pull/5163#issuecomment-485984210 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5163?src=pr=h1) Report > Merging

[GitHub] [airflow] kaxil commented on issue #5143: [AIRFLOW-4204] Update super() calls to PY3

2019-04-23 Thread GitBox
kaxil commented on issue #5143: [AIRFLOW-4204] Update super() calls to PY3 URL: https://github.com/apache/airflow/pull/5143#issuecomment-485983744 @potiuk - We don't use tool any longer (at least I don't since we moved to GitBox). Use **Squash and merge** from the UI as shown in the image

[GitHub] [airflow] kaxil closed pull request #5163: WIP: [AIRFLOW-XXX] Remove hard-coded logging level

2019-04-23 Thread GitBox
kaxil closed pull request #5163: WIP: [AIRFLOW-XXX] Remove hard-coded logging level URL: https://github.com/apache/airflow/pull/5163 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] BasPH edited a comment on issue #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor

2019-04-23 Thread GitBox
BasPH edited a comment on issue #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor URL: https://github.com/apache/airflow/pull/5159#issuecomment-485977458 I have very limited time this week but tried digging into it. At least I managed to reproduce locally: I set an

[jira] [Commented] (AIRFLOW-3951) No easy way to get query_execution_id from AWSAthenaOperator

2019-04-23 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824525#comment-16824525 ] Ash Berlin-Taylor commented on AIRFLOW-3951: "but no way to tell if it succeeded or failed"

[GitHub] [airflow] kaxil opened a new pull request #5163: WIP: [AIRFLOW-XXX] Remove hard-coded logging level

2019-04-23 Thread GitBox
kaxil opened a new pull request #5163: WIP: [AIRFLOW-XXX] Remove hard-coded logging level URL: https://github.com/apache/airflow/pull/5163 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[jira] [Commented] (AIRFLOW-3951) No easy way to get query_execution_id from AWSAthenaOperator

2019-04-23 Thread Rob Dinoff (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824490#comment-16824490 ] Rob Dinoff commented on AIRFLOW-3951: - There is no way to get the query status returned from

[GitHub] [airflow] potiuk commented on a change in pull request #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor

2019-04-23 Thread GitBox
potiuk commented on a change in pull request #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor URL: https://github.com/apache/airflow/pull/5159#discussion_r277838641 ## File path: tests/executors/test_local_executor.py ## @@ -49,11 +48,7 @@ def

[GitHub] [airflow] codecov-io edited a comment on issue #5153: [AIRFLOW-XXX] Name default config_file param in KubernetesPodOperator docstring

2019-04-23 Thread GitBox
codecov-io edited a comment on issue #5153: [AIRFLOW-XXX] Name default config_file param in KubernetesPodOperator docstring URL: https://github.com/apache/airflow/pull/5153#issuecomment-485495821 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5153?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #5153: [AIRFLOW-XXX] Name default config_file param in KubernetesPodOperator docstring

2019-04-23 Thread GitBox
codecov-io edited a comment on issue #5153: [AIRFLOW-XXX] Name default config_file param in KubernetesPodOperator docstring URL: https://github.com/apache/airflow/pull/5153#issuecomment-485495821 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5153?src=pr=h1) Report > Merging

[GitHub] [airflow] Fokko commented on issue #5158: [AIRFLOW-XXX] Speed up tests for PythonSensor by 60s

2019-04-23 Thread GitBox
Fokko commented on issue #5158: [AIRFLOW-XXX] Speed up tests for PythonSensor by 60s URL: https://github.com/apache/airflow/pull/5158#issuecomment-485910763 Love it This is an automated message from the Apache Git Service.

[GitHub] [airflow] ashb commented on issue #5162: [AIRFLOW-4358] Speed up test_jobs by not running tasks

2019-04-23 Thread GitBox
ashb commented on issue #5162: [AIRFLOW-4358] Speed up test_jobs by not running tasks URL: https://github.com/apache/airflow/pull/5162#issuecomment-485898431 Looks like I might have some ordering in the tests to sort out.

[GitHub] [airflow] feng-tao commented on issue #5162: [AIRFLOW-4358] Speed up test_jobs by not running tasks

2019-04-23 Thread GitBox
feng-tao commented on issue #5162: [AIRFLOW-4358] Speed up test_jobs by not running tasks URL: https://github.com/apache/airflow/pull/5162#issuecomment-485891838 Nice! It is always good to speed up the CI! This is an

[jira] [Commented] (AIRFLOW-4358) Decouple running actual tasks from test_jobs.py

2019-04-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824325#comment-16824325 ] ASF GitHub Bot commented on AIRFLOW-4358: - ashb commented on pull request #5162: [AIRFLOW-4358]

[GitHub] [airflow] ashb opened a new pull request #5162: [AIRFLOW-4358] Speed up test_jobs by not running tasks

2019-04-23 Thread GitBox
ashb opened a new pull request #5162: [AIRFLOW-4358] Speed up test_jobs by not running tasks URL: https://github.com/apache/airflow/pull/5162 Make sure you have checked _all_ steps below. ### Jira - [x] https://issues.apache.org/jira/browse/AIRFLOW-4358 ### Description

[GitHub] [airflow] ashb commented on a change in pull request #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor

2019-04-23 Thread GitBox
ashb commented on a change in pull request #5159: [AIRFLOW-XXX] Attempt to remove flakeyness of LocalExecutor URL: https://github.com/apache/airflow/pull/5159#discussion_r23566 ## File path: tests/executors/test_local_executor.py ## @@ -49,11 +48,7 @@ def

[jira] [Work started] (AIRFLOW-4358) Decouple running actual tasks from test_jobs.py

2019-04-23 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4358 started by Ash Berlin-Taylor. -- > Decouple running actual tasks from test_jobs.py >

[jira] [Updated] (AIRFLOW-4358) Decouple running actual tasks from test_jobs.py

2019-04-23 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor updated AIRFLOW-4358: --- Issue Type: Sub-task (was: Test) Parent: AIRFLOW-2488 > Decouple running

[jira] [Updated] (AIRFLOW-2488) Speed up the CI

2019-04-23 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor updated AIRFLOW-2488: --- Description: Some of the tests take far to long: {noformat} [success] 10.48%

[jira] [Commented] (AIRFLOW-4395) Remove pickle_info view

2019-04-23 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-4395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16824298#comment-16824298 ] ASF GitHub Bot commented on AIRFLOW-4395: - mik-laj commented on pull request #5161:

[GitHub] [airflow] mik-laj opened a new pull request #5161: [AIRFLOW-4395] Remove pickle_info view

2019-04-23 Thread GitBox
mik-laj opened a new pull request #5161: [AIRFLOW-4395] Remove pickle_info view URL: https://github.com/apache/airflow/pull/5161 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[GitHub] [airflow] codecov-io commented on issue #5160: [AIRFLOW-4394] Don't double test behaviour of BackfillJob from CLI tests

2019-04-23 Thread GitBox
codecov-io commented on issue #5160: [AIRFLOW-4394] Don't double test behaviour of BackfillJob from CLI tests URL: https://github.com/apache/airflow/pull/5160#issuecomment-485876624 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5160?src=pr=h1) Report > Merging

[jira] [Created] (AIRFLOW-4395) Remove pickle_info view

2019-04-23 Thread Kamil Bregula (JIRA)
Kamil Bregula created AIRFLOW-4395: -- Summary: Remove pickle_info view Key: AIRFLOW-4395 URL: https://issues.apache.org/jira/browse/AIRFLOW-4395 Project: Apache Airflow Issue Type: New

[GitHub] [airflow] feluelle commented on a change in pull request #5127: [AIRFLOW-4343] Show warning in UI if scheduler is not running

2019-04-23 Thread GitBox
feluelle commented on a change in pull request #5127: [AIRFLOW-4343] Show warning in UI if scheduler is not running URL: https://github.com/apache/airflow/pull/5127#discussion_r277758349 ## File path: airflow/config_templates/default_airflow.cfg ## @@ -464,11 +464,6 @@

  1   2   3   >