[GitHub] feng-tao commented on issue #3478: [PTAL][AIRFLOW-2571] airflow backfill progress bar

2019-02-04 Thread GitBox
feng-tao commented on issue #3478: [PTAL][AIRFLOW-2571] airflow backfill progress bar URL: https://github.com/apache/airflow/pull/3478#issuecomment-460518449 there is a potential license concern with this pr. This is an

[jira] [Updated] (AIRFLOW-3806) Redirects use absolute URLs (and ignore base_url)

2019-02-04 Thread Stanislaw Pitucha (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislaw Pitucha updated AIRFLOW-3806: --- Summary: Redirects use absolute URLs (and ignore base_url) (was: Redirects use

[jira] [Created] (AIRFLOW-3806) Redirects use absolute URLs

2019-02-04 Thread Stanislaw Pitucha (JIRA)
Stanislaw Pitucha created AIRFLOW-3806: -- Summary: Redirects use absolute URLs Key: AIRFLOW-3806 URL: https://issues.apache.org/jira/browse/AIRFLOW-3806 Project: Apache Airflow Issue

[GitHub] codecov-io edited a comment on issue #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options …

2019-02-04 Thread GitBox
codecov-io edited a comment on issue #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options … URL: https://github.com/apache/airflow/pull/4648#issuecomment-460139330 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4648?src=pr=h1) Report > Merging

[GitHub] pgagnon commented on a change in pull request #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options …

2019-02-04 Thread GitBox
pgagnon commented on a change in pull request #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options … URL: https://github.com/apache/airflow/pull/4648#discussion_r253729845 ## File path: airflow/contrib/kubernetes/worker_configuration.py ## @@ -258,5

[GitHub] pgagnon commented on a change in pull request #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options …

2019-02-04 Thread GitBox
pgagnon commented on a change in pull request #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options … URL: https://github.com/apache/airflow/pull/4648#discussion_r253729866 ## File path: airflow/contrib/kubernetes/worker_configuration.py ## @@ -232,6

[GitHub] skudriashev commented on a change in pull request #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options …

2019-02-04 Thread GitBox
skudriashev commented on a change in pull request #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options … URL: https://github.com/apache/airflow/pull/4648#discussion_r253729154 ## File path: airflow/contrib/kubernetes/worker_configuration.py ## @@

[GitHub] skudriashev commented on a change in pull request #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options …

2019-02-04 Thread GitBox
skudriashev commented on a change in pull request #4648: [AIRFLOW-3274] Add run_as_user and fs_group security context options … URL: https://github.com/apache/airflow/pull/4648#discussion_r253729147 ## File path: airflow/contrib/kubernetes/worker_configuration.py ## @@

[GitHub] codecov-io edited a comment on issue #4637: [AIRFLOW-3793] Decommission configuration items for Flask-Admin web UI & related codes

2019-02-04 Thread GitBox
codecov-io edited a comment on issue #4637: [AIRFLOW-3793] Decommission configuration items for Flask-Admin web UI & related codes URL: https://github.com/apache/airflow/pull/4637#issuecomment-459930202 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4637?src=pr=h1) Report >

[GitHub] skudriashev commented on a change in pull request #4651: [AIRFLOW-XXX] Add exception details to warning log

2019-02-04 Thread GitBox
skudriashev commented on a change in pull request #4651: [AIRFLOW-XXX] Add exception details to warning log URL: https://github.com/apache/airflow/pull/4651#discussion_r253727191 ## File path: airflow/settings.py ## @@ -80,8 +80,8 @@ def timing(cls, stat, dt):

[GitHub] feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition

2019-02-04 Thread GitBox
feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition URL: https://github.com/apache/airflow/pull/4642#discussion_r253704652 ## File path: airflow/models/__init__.py ## @@ -4244,6 +4254,63 @@ def

[GitHub] feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition

2019-02-04 Thread GitBox
feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition URL: https://github.com/apache/airflow/pull/4642#discussion_r253704652 ## File path: airflow/models/__init__.py ## @@ -4244,6 +4254,63 @@ def

[GitHub] feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition

2019-02-04 Thread GitBox
feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition URL: https://github.com/apache/airflow/pull/4642#discussion_r253692717 ## File path: airflow/models/__init__.py ## @@ -4244,6 +4254,63 @@ def

[GitHub] astahlman commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition

2019-02-04 Thread GitBox
astahlman commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition URL: https://github.com/apache/airflow/pull/4642#discussion_r253678075 ## File path: airflow/models/__init__.py ## @@ -4244,6 +4254,63 @@ def

[GitHub] mik-laj closed pull request #4652: [PARKED][AIRFLOW-XXX] Simplfiy GCP decorators

2019-02-04 Thread GitBox
mik-laj closed pull request #4652: [PARKED][AIRFLOW-XXX] Simplfiy GCP decorators URL: https://github.com/apache/airflow/pull/4652 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] mik-laj opened a new pull request #4652: [PARKED][AIRFLOW-XXX] Simplfiy GCP decorators

2019-02-04 Thread GitBox
mik-laj opened a new pull request #4652: [PARKED][AIRFLOW-XXX] Simplfiy GCP decorators URL: https://github.com/apache/airflow/pull/4652 The decorator code is unnecessarily complicated. I simplified him. This is an automated

[GitHub] seelmann commented on issue #4646: AIRFLOW-3543: Fix deletion of DAG with rescheduled tasks

2019-02-04 Thread GitBox
seelmann commented on issue #4646: AIRFLOW-3543: Fix deletion of DAG with rescheduled tasks URL: https://github.com/apache/airflow/pull/4646#issuecomment-460369510 Good point, I'll try... This is an automated message from

[GitHub] feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition

2019-02-04 Thread GitBox
feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition URL: https://github.com/apache/airflow/pull/4642#discussion_r253585402 ## File path: airflow/models/__init__.py ## @@ -4244,6 +4254,63 @@ def

[GitHub] feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition

2019-02-04 Thread GitBox
feng-tao commented on a change in pull request #4642: [WIP] [AIRFLOW-2694] Declare permissions in DAG definition URL: https://github.com/apache/airflow/pull/4642#discussion_r253585402 ## File path: airflow/models/__init__.py ## @@ -4244,6 +4254,63 @@ def

[GitHub] feng-tao commented on issue #4645: [AIRFLOW-3801] Fix DagBag collect dags invocation to prevent examples…

2019-02-04 Thread GitBox
feng-tao commented on issue #4645: [AIRFLOW-3801] Fix DagBag collect dags invocation to prevent examples… URL: https://github.com/apache/airflow/pull/4645#issuecomment-460350778 Could you create the pr against master branch?

[GitHub] feng-tao commented on issue #4637: [AIRFLOW-3793] Decommission configuration items for Flask-Admin web UI & related codes

2019-02-04 Thread GitBox
feng-tao commented on issue #4637: [AIRFLOW-3793] Decommission configuration items for Flask-Admin web UI & related codes URL: https://github.com/apache/airflow/pull/4637#issuecomment-460346557 @XD-DENG , happy CNY~ This is

[GitHub] jmcarp commented on issue #4607: WIP: [AIRFLOW-1894] Google cloud bigquery

2019-02-04 Thread GitBox
jmcarp commented on issue #4607: WIP: [AIRFLOW-1894] Google cloud bigquery URL: https://github.com/apache/airflow/pull/4607#issuecomment-460328910 @kaxil @potiuk: tests are passing, so this is ready for review when you have time. A few differences to point out: * The current python

[GitHub] rhwang10 commented on a change in pull request #4303: [AIRFLOW-3370] Elasticsearch log task handler additional features

2019-02-04 Thread GitBox
rhwang10 commented on a change in pull request #4303: [AIRFLOW-3370] Elasticsearch log task handler additional features URL: https://github.com/apache/airflow/pull/4303#discussion_r253541579 ## File path: airflow/utils/log/es_task_handler.py ## @@ -68,12 +118,115 @@ def

[GitHub] rhwang10 commented on a change in pull request #4303: [AIRFLOW-3370] Elasticsearch log task handler additional features

2019-02-04 Thread GitBox
rhwang10 commented on a change in pull request #4303: [AIRFLOW-3370] Elasticsearch log task handler additional features URL: https://github.com/apache/airflow/pull/4303#discussion_r253536114 ## File path: airflow/utils/log/es_task_handler.py ## @@ -28,6 +32,43 @@ from

[jira] [Commented] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759965#comment-16759965 ] jack commented on AIRFLOW-3804: --- [~ashb] All codes I tested with the library generates the error so I

[GitHub] syucream opened a new pull request #4651: [AIRFLOW-XXX] Add exception details to warning log

2019-02-04 Thread GitBox
syucream opened a new pull request #4651: [AIRFLOW-XXX] Add exception details to warning log URL: https://github.com/apache/airflow/pull/4651 ### Jira - [ ] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references them in

[jira] [Issue Comment Deleted] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jack updated AIRFLOW-3804: -- Comment: was deleted (was: [~ashb] It does raise error. Just tested it with this code: {code:java} import

[jira] [Commented] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759939#comment-16759939 ] jack commented on AIRFLOW-3804: --- [~ashb] It does raise error. Just tested it with this code: {code:java}

[jira] [Commented] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759915#comment-16759915 ] Ash Berlin-Taylor commented on AIRFLOW-3804: I was expecting the the python mysql client

[jira] [Closed] (AIRFLOW-3805) `include_examples` flag is incorrectly passed to DagBag.collect_dags()

2019-02-04 Thread Samuel Wedge (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samuel Wedge closed AIRFLOW-3805. - Resolution: Duplicate Duplicate of AIRFLOW-3801 > `include_examples` flag is incorrectly

[jira] [Commented] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759910#comment-16759910 ] jack commented on AIRFLOW-3804: --- [~ashb] MySQL does raise error: If you run the query in MySQL you will

[jira] [Created] (AIRFLOW-3805) `include_examples` flag is incorrectly passed to DagBag.collect_dags()

2019-02-04 Thread Samuel Wedge (JIRA)
Samuel Wedge created AIRFLOW-3805: - Summary: `include_examples` flag is incorrectly passed to DagBag.collect_dags() Key: AIRFLOW-3805 URL: https://issues.apache.org/jira/browse/AIRFLOW-3805 Project:

[jira] [Commented] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759881#comment-16759881 ] Ash Berlin-Taylor commented on AIRFLOW-3804: Oh sorry I see, yes! No error checking in the

[jira] [Commented] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759876#comment-16759876 ] jack commented on AIRFLOW-3804: --- [~ashb] you misread :)  I know that I'm missing quoting. This isn't a

[jira] [Updated] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jack updated AIRFLOW-3804: -- Description: Testing the following query on MySqlToGoogleCloudStorageOperator.   {code:java} SELECT * FROM

[jira] [Commented] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759870#comment-16759870 ] Ash Berlin-Taylor commented on AIRFLOW-3804: Somewhere you are missing quoting. Please can

[jira] [Updated] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jack updated AIRFLOW-3804: -- Description: Testing the following query on MySqlToGoogleCloudStorageOperator.   {code:java} SELECT * FROM

[jira] [Created] (AIRFLOW-3804) MySqlToGoogleCloudStorageOperator success when it should fail

2019-02-04 Thread jack (JIRA)
jack created AIRFLOW-3804: - Summary: MySqlToGoogleCloudStorageOperator success when it should fail Key: AIRFLOW-3804 URL: https://issues.apache.org/jira/browse/AIRFLOW-3804 Project: Apache Airflow

[jira] [Assigned] (AIRFLOW-2999) S3_hook - add the ability to download file to local disk

2019-02-04 Thread Tanay Tummalapalli (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanay Tummalapalli reassigned AIRFLOW-2999: --- Assignee: Tanay Tummalapalli > S3_hook - add the ability to download

[GitHub] ttanay opened a new pull request #4650: [AIRFLOW-3643] Add shebang to docs/start_doc_server.sh

2019-02-04 Thread GitBox
ttanay opened a new pull request #4650: [AIRFLOW-3643] Add shebang to docs/start_doc_server.sh URL: https://github.com/apache/airflow/pull/4650 Since this script uses bash syntax, shebang needs to be added. Make sure you have checked _all_ steps below. ### Jira - [X]

[jira] [Commented] (AIRFLOW-3643) Add shebang to docs/start_doc_server.sh

2019-02-04 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759851#comment-16759851 ] ASF GitHub Bot commented on AIRFLOW-3643: - ttanay commented on pull request #4650:

[GitHub] gseva opened a new pull request #4649: [AIRFLOW-XXX] Fixed note in plugins.rst

2019-02-04 Thread GitBox
gseva opened a new pull request #4649: [AIRFLOW-XXX] Fixed note in plugins.rst URL: https://github.com/apache/airflow/pull/4649 Changing it to rst notation, so it stands out in read the docs. This is an automated message from

[jira] [Created] (AIRFLOW-3803) Airflow and Atlas integration

2019-02-04 Thread Ruslan Fialkovsky (JIRA)
Ruslan Fialkovsky created AIRFLOW-3803: -- Summary: Airflow and Atlas integration Key: AIRFLOW-3803 URL: https://issues.apache.org/jira/browse/AIRFLOW-3803 Project: Apache Airflow Issue

[jira] [Commented] (AIRFLOW-1632) MySQL to GCS fails for date/datetime before ~1850

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759767#comment-16759767 ] jack commented on AIRFLOW-1632: --- Update: This issue is Python 2.7 specific

[jira] [Reopened] (AIRFLOW-1632) MySQL to GCS fails for date/datetime before ~1850

2019-02-04 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor reopened AIRFLOW-1632: Reopening - different issue to the linked duplicate (which was about any dates,

[jira] [Comment Edited] (AIRFLOW-1632) MySQL to GCS fails for date/datetime before ~1850

2019-02-04 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759744#comment-16759744 ] Ash Berlin-Taylor edited comment on AIRFLOW-1632 at 2/4/19 10:18 AM:

[jira] [Commented] (AIRFLOW-1632) MySQL to GCS fails for date/datetime before ~1850

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-1632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759744#comment-16759744 ] jack commented on AIRFLOW-1632: --- [~ashb] [~mikeghen] This was not fixed. First it's not a clone of: 

[jira] [Assigned] (AIRFLOW-3643) Add shebang to docs/start_doc_server.sh

2019-02-04 Thread Tanay Tummalapalli (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanay Tummalapalli reassigned AIRFLOW-3643: --- Assignee: Tanay Tummalapalli > Add shebang to docs/start_doc_server.sh >

[jira] [Assigned] (AIRFLOW-3140) Add MongoDBToGoogleStorage Opearator

2019-02-04 Thread Tanay Tummalapalli (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanay Tummalapalli reassigned AIRFLOW-3140: --- Assignee: Tanay Tummalapalli > Add MongoDBToGoogleStorage Opearator >

[jira] [Resolved] (AIRFLOW-837) Clear task in the UI should state to None rather than deleting DB row

2019-02-04 Thread Ash Berlin-Taylor (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-837. --- Resolution: Fixed > Clear task in the UI should state to None rather than deleting

[GitHub] ashb commented on issue #4646: AIRFLOW-3543: Fix deletion of DAG with rescheduled tasks

2019-02-04 Thread GitBox
ashb commented on issue #4646: AIRFLOW-3543: Fix deletion of DAG with rescheduled tasks URL: https://github.com/apache/airflow/pull/4646#issuecomment-460177551 Could this not be fixed with less code by simply adding a `ON DELETE CASCADE` to the DB table?

[jira] [Commented] (AIRFLOW-3562) Remove DagBag dependency inside the webserver

2019-02-04 Thread jack (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16759700#comment-16759700 ] jack commented on AIRFLOW-3562: --- [~ffinfo] Could this also be related? 

[GitHub] drewsonne commented on a change in pull request #4640: Feature/flask restful

2019-02-04 Thread GitBox
drewsonne commented on a change in pull request #4640: Feature/flask restful URL: https://github.com/apache/airflow/pull/4640#discussion_r253375163 ## File path: setup.py ## @@ -333,6 +335,7 @@ def do_setup(): 'all': devel_all, 'devel_ci':