[GitHub] [airflow] baolsen opened a new pull request #6583: [AIRFLOW-XXX] Remove duplicate docs

2019-11-14 Thread GitBox
baolsen opened a new pull request #6583: [AIRFLOW-XXX] Remove duplicate docs URL: https://github.com/apache/airflow/pull/6583 Make sure you have checked _all_ steps below. ### Jira - [X] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW

[GitHub] [airflow] zhongjiajie commented on issue #6582: [AIRFLOW-5928] Hive hooks load_file short circuit

2019-11-14 Thread GitBox
zhongjiajie commented on issue #6582: [AIRFLOW-5928] Hive hooks load_file short circuit URL: https://github.com/apache/airflow/pull/6582#issuecomment-553773758 @BasPH @mik-laj PTAL, thanks This is an automated message from th

[GitHub] [airflow-site] mik-laj merged pull request #148: Update the integration list to point to the attached documentation

2019-11-14 Thread GitBox
mik-laj merged pull request #148: Update the integration list to point to the attached documentation URL: https://github.com/apache/airflow-site/pull/148 This is an automated message from the Apache Git Service. To respond t

[airflow-site] branch aip-11 updated: Update the integration list to point to the attached documentation (#148)

2019-11-14 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a commit to branch aip-11 in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/aip-11 by this push: new 4327eac Update the integration list

[GitHub] [airflow] baolsen closed pull request #6583: [AIRFLOW-XXX] Remove duplicate docs

2019-11-14 Thread GitBox
baolsen closed pull request #6583: [AIRFLOW-XXX] Remove duplicate docs URL: https://github.com/apache/airflow/pull/6583 This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [airflow] baolsen opened a new pull request #6584: Remove duplicate documentation

2019-11-14 Thread GitBox
baolsen opened a new pull request #6584: Remove duplicate documentation URL: https://github.com/apache/airflow/pull/6584 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) is

[GitHub] [airflow] zhongjiajie edited a comment on issue #6582: [AIRFLOW-5928] Hive hooks load_file short circuit

2019-11-14 Thread GitBox
zhongjiajie edited a comment on issue #6582: [AIRFLOW-5928] Hive hooks load_file short circuit URL: https://github.com/apache/airflow/pull/6582#issuecomment-553773758 @potiuk @BasPH @mik-laj PTAL, thanks This is an automated

[GitHub] [airflow] mik-laj commented on a change in pull request #6499: [AIRFLOW-5842] Switch to Debian buster image. Depends on [AIRFLOW-5704]

2019-11-14 Thread GitBox
mik-laj commented on a change in pull request #6499: [AIRFLOW-5842] Switch to Debian buster image. Depends on [AIRFLOW-5704] URL: https://github.com/apache/airflow/pull/6499#discussion_r346182152 ## File path: Dockerfile ## @@ -263,84 +255,70 @@ ENV AIRFLOW_REPO=${AIRFLOW_

[GitHub] [airflow] mik-laj commented on a change in pull request #6499: [AIRFLOW-5842] Switch to Debian buster image. Depends on [AIRFLOW-5704]

2019-11-14 Thread GitBox
mik-laj commented on a change in pull request #6499: [AIRFLOW-5842] Switch to Debian buster image. Depends on [AIRFLOW-5704] URL: https://github.com/apache/airflow/pull/6499#discussion_r346183619 ## File path: Dockerfile ## @@ -263,84 +255,70 @@ ENV AIRFLOW_REPO=${AIRFLOW_

[jira] [Commented] (AIRFLOW-5918) Group tests for the Pools command

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974062#comment-16974062 ] ASF GitHub Bot commented on AIRFLOW-5918: - potiuk commented on pull request #65

[GitHub] [airflow] potiuk merged pull request #6567: [AIRFLOW-5918][part of AIRFLOW-5893] Group tests for the Pools command

2019-11-14 Thread GitBox
potiuk merged pull request #6567: [AIRFLOW-5918][part of AIRFLOW-5893] Group tests for the Pools command URL: https://github.com/apache/airflow/pull/6567 This is an automated message from the Apache Git Service. To respond t

[jira] [Resolved] (AIRFLOW-5918) Group tests for the Pools command

2019-11-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5918. --- Fix Version/s: 2.0.0 Resolution: Fixed > Group tests for the Pools command > ---

[jira] [Commented] (AIRFLOW-5918) Group tests for the Pools command

2019-11-14 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974063#comment-16974063 ] ASF subversion and git services commented on AIRFLOW-5918: -- Co

[jira] [Commented] (AIRFLOW-5893) Refactor CLI tests

2019-11-14 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974064#comment-16974064 ] ASF subversion and git services commented on AIRFLOW-5893: -- Co

[GitHub] [airflow] OmerJog commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
OmerJog commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-55372 @feluelle if we want to support two libs shouldn't all tests run against the two libs ? ---

[GitHub] [airflow-site] kgabryje opened a new pull request #160: [depends on #151] Feature/shuffling commiters

2019-11-14 Thread GitBox
kgabryje opened a new pull request #160: [depends on #151] Feature/shuffling commiters URL: https://github.com/apache/airflow-site/pull/160 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow-site] kgabryje opened a new issue #161: Disable links check until CI is migrated to Gitlab CI

2019-11-14 Thread GitBox
kgabryje opened a new issue #161: Disable links check until CI is migrated to Gitlab CI URL: https://github.com/apache/airflow-site/issues/161 This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [airflow-site] kgabryje opened a new pull request #162: temporarily disabled links check on CI

2019-11-14 Thread GitBox
kgabryje opened a new pull request #162: temporarily disabled links check on CI URL: https://github.com/apache/airflow-site/pull/162 https://github.com/apache/airflow-site/issues/161 This is an automated message from the Apach

[GitHub] [airflow] feluelle commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
feluelle commented on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553804892 At least I should add tests for the `client` specification. But after you connected with one of these.

[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553804892 At least I should add tests for the `client` specification. But after you connected with one of

[jira] [Commented] (AIRFLOW-5859) Tasks locking and heartbeat warnings

2019-11-14 Thread Jacob Ward (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974088#comment-16974088 ] Jacob Ward commented on AIRFLOW-5859: - I'm still having this issue, it weirdly seem

[GitHub] [airflow] potiuk commented on issue #6584: Remove duplicate documentation

2019-11-14 Thread GitBox
potiuk commented on issue #6584: Remove duplicate documentation URL: https://github.com/apache/airflow/pull/6584#issuecomment-553807722 Just one problem with end of file @baolsen :) This is an automated message from the Apache

[jira] [Commented] (AIRFLOW-2999) Add S3DownloadOperator

2019-11-14 Thread Felix Uellendall (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974094#comment-16974094 ] Felix Uellendall commented on AIRFLOW-2999: --- In general if you think of the d

[jira] [Commented] (AIRFLOW-2999) Add S3DownloadOperator

2019-11-14 Thread Felix Uellendall (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974097#comment-16974097 ] Felix Uellendall commented on AIRFLOW-2999: --- If I would split this into "S3Do

[jira] [Comment Edited] (AIRFLOW-2999) Add S3DownloadOperator

2019-11-14 Thread Felix Uellendall (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974097#comment-16974097 ] Felix Uellendall edited comment on AIRFLOW-2999 at 11/14/19 9:50 AM:

[GitHub] [airflow] potiuk commented on issue #6576: [WIP][AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
potiuk commented on issue #6576: [WIP][AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553811419 Yeah @feluelle -> parameterising just one representative tests with all the options (no client, old

[GitHub] [airflow-site] kgabryje opened a new pull request #163: added disclaimer to footer

2019-11-14 Thread GitBox
kgabryje opened a new pull request #163: added disclaimer to footer URL: https://github.com/apache/airflow-site/pull/163 This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[jira] [Commented] (AIRFLOW-2572) Airflow /admin/configurationview/ Running Configuration not accurate

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974119#comment-16974119 ] jack commented on AIRFLOW-2572: --- [~joejasinski] can you recheck this with newer version o

[jira] [Resolved] (AIRFLOW-3235) Add list function in AzureDataLakeHook

2019-11-14 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-3235. Fix Version/s: 1.10.7 Resolution: Fixed > Add list function in AzureDataLa

[GitHub] [airflow] codecov-io edited a comment on issue #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer URL: https://github.com/apache/airflow/pull/6396#issuecomment-546875908 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6396?src=pr&el=h1) Report > Merging [#6396](https:/

[GitHub] [airflow] OmerJog commented on issue #6549: [AIRFLOW-5880] Enforce unique task ids

2019-11-14 Thread GitBox
OmerJog commented on issue #6549: [AIRFLOW-5880] Enforce unique task ids URL: https://github.com/apache/airflow/pull/6549#issuecomment-553837774 Great. I think we should also enforce unique dag_id / varibales This is an automa

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346251187 ## File path: airflow/lineage/__init__.py ## @@ -92,51 +115,43 @@ def prepare_lineage(func):

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346247195 ## File path: airflow/lineage/entity/dataset.py ## @@ -0,0 +1,25 @@ +# -*- coding: utf-8 -*-

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346248106 ## File path: docs/lineage.rst ## @@ -50,40 +51,43 @@ works. f_final = File("/tmp/fina

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346245600 ## File path: airflow/models/baseoperator.py ## @@ -412,25 +412,12 @@ def __init__(

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346245366 ## File path: airflow/models/baseoperator.py ## @@ -316,8 +316,8 @@ def __init__( ta

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346248832 ## File path: tests/lineage/test_lineage.py ## @@ -85,26 +85,26 @@ def test_lineage(self, _ge

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346246696 ## File path: airflow/models/baseoperator.py ## @@ -316,8 +316,8 @@ def __init__( ta

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346247319 ## File path: docs/lineage.rst ## @@ -50,40 +51,43 @@ works. f_final = File("/tmp/fina

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346248047 ## File path: docs/lineage.rst ## @@ -50,40 +51,43 @@ works. f_final = File("/tmp/fina

[jira] [Commented] (AIRFLOW-5627) transform function should be optional for s3_file_tranformation_operator

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974152#comment-16974152 ] jack commented on AIRFLOW-5627: --- If you don't want to transform then don't use transform

[jira] [Commented] (AIRFLOW-4066) dag_run failed while one operator is still running

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974156#comment-16974156 ] jack commented on AIRFLOW-4066: --- Which airflow version are you running? > dag_run failed

[GitHub] [airflow] ashb commented on issue #5787: [AIRFLOW-5172] Add choice of interval edge scheduling

2019-11-14 Thread GitBox
ashb commented on issue #5787: [AIRFLOW-5172] Add choice of interval edge scheduling URL: https://github.com/apache/airflow/pull/5787#issuecomment-553842789 @aoen That sounds sensible, so long as we clearly "sign-post" the change for users when upgrading somehow. The default could only cha

[GitHub] [airflow] ashb commented on a change in pull request #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer URL: https://github.com/apache/airflow/pull/6396#discussion_r346257108 ## File path: tests/operators/test_redshift_to_s3_operator.py ## @@ -51,23 +52,25 @@ de

[GitHub] [airflow] ashb commented on a change in pull request #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer URL: https://github.com/apache/airflow/pull/6396#discussion_r346257541 ## File path: tests/operators/test_redshift_to_s3_operator.py ## @@ -51,23 +52,25 @@ de

[GitHub] [airflow] codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553564200 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6576?src=pr&el=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553564200 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6576?src=pr&el=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553564200 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6576?src=pr&el=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #6573: [AIRFLOW-5904] "Trigger DAG" should redirect to the "dag_default_view" page

2019-11-14 Thread GitBox
codecov-io commented on issue #6573: [AIRFLOW-5904] "Trigger DAG" should redirect to the "dag_default_view" page URL: https://github.com/apache/airflow/pull/6573#issuecomment-553864434 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6573?src=pr&el=h1) Report > Merging [#6573](htt

[GitHub] [airflow] elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE]

2019-11-14 Thread GitBox
elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE] URL: https://github.com/apache/airflow/pull/5659#issuecomment-553870608 Update: my pull request for snakebite-py3 also includes support for SASL encryption for the Hadoop RPC protocol. This means that

[GitHub] [airflow] potiuk commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE]

2019-11-14 Thread GitBox
potiuk commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE] URL: https://github.com/apache/airflow/pull/5659#issuecomment-553878821 Happy to help @elukey -> please mark my github nickname when you ping them and i will also comment on that!

[GitHub] [airflow] ashb commented on a change in pull request #5913: [AIRFLOW-5073] Change SQLSensor to keep poking receving NULL, '0'

2019-11-14 Thread GitBox
ashb commented on a change in pull request #5913: [AIRFLOW-5073] Change SQLSensor to keep poking receving NULL, '0' URL: https://github.com/apache/airflow/pull/5913#discussion_r346311207 ## File path: tests/sensors/test_sql_sensor.py ## @@ -103,6 +103,18 @@ def test_sql_se

[jira] [Commented] (AIRFLOW-5818) Very bad webserver performance when defining many dags with many operators

2019-11-14 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974266#comment-16974266 ] Ash Berlin-Taylor commented on AIRFLOW-5818: [~gary@bidalgo] Ah, when runni

[jira] [Created] (AIRFLOW-5929) Improve Scheduler Performance

2019-11-14 Thread Ash Berlin-Taylor (Jira)
Ash Berlin-Taylor created AIRFLOW-5929: -- Summary: Improve Scheduler Performance Key: AIRFLOW-5929 URL: https://issues.apache.org/jira/browse/AIRFLOW-5929 Project: Apache Airflow Issue Ty

[jira] [Created] (AIRFLOW-5930) Reduce time spent building SQL strings

2019-11-14 Thread Ash Berlin-Taylor (Jira)
Ash Berlin-Taylor created AIRFLOW-5930: -- Summary: Reduce time spent building SQL strings Key: AIRFLOW-5930 URL: https://issues.apache.org/jira/browse/AIRFLOW-5930 Project: Apache Airflow

[jira] [Created] (AIRFLOW-5931) Spawning new python interpreter is slow

2019-11-14 Thread Ash Berlin-Taylor (Jira)
Ash Berlin-Taylor created AIRFLOW-5931: -- Summary: Spawning new python interpreter is slow Key: AIRFLOW-5931 URL: https://issues.apache.org/jira/browse/AIRFLOW-5931 Project: Apache Airflow

[jira] [Assigned] (AIRFLOW-5930) Reduce time spent building SQL strings

2019-11-14 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor reassigned AIRFLOW-5930: -- Assignee: Ash Berlin-Taylor > Reduce time spent building SQL strings > -

[GitHub] [airflow] potiuk opened a new pull request #6585: More GSOD improvements

2019-11-14 Thread GitBox
potiuk opened a new pull request #6585: More GSOD improvements URL: https://github.com/apache/airflow/pull/6585 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and

[GitHub] [airflow] potiuk commented on issue #6585: More GSOD improvements

2019-11-14 Thread GitBox
potiuk commented on issue #6585: More GSOD improvements URL: https://github.com/apache/airflow/pull/6585#issuecomment-553899400 Hey @efedotova -> I made a first quick attempt at separating TESTING.rst from Local/Breeze. I hope it might be a good start for you as discussed :). -

[jira] [Created] (AIRFLOW-5932) Unable to resolve backticks character

2019-11-14 Thread Badri (Jira)
Badri created AIRFLOW-5932: -- Summary: Unable to resolve backticks character Key: AIRFLOW-5932 URL: https://issues.apache.org/jira/browse/AIRFLOW-5932 Project: Apache Airflow Issue Type: Bug

[jira] [Commented] (AIRFLOW-5932) Unable to resolve backticks character

2019-11-14 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974289#comment-16974289 ] Ash Berlin-Taylor commented on AIRFLOW-5932: It's not quite clear what you

[jira] [Commented] (AIRFLOW-5932) Unable to resolve backticks character

2019-11-14 Thread Badri (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974297#comment-16974297 ] Badri commented on AIRFLOW-5932: Apologies for not adding many details. The aim of tha

[GitHub] [airflow-site] kgabryje opened a new pull request #164: [depends on #151] Feature/share social media

2019-11-14 Thread GitBox
kgabryje opened a new pull request #164: [depends on #151] Feature/share social media URL: https://github.com/apache/airflow-site/pull/164 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Assigned] (AIRFLOW-5929) Improve Scheduler Performance

2019-11-14 Thread Fokko Driesprong (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong reassigned AIRFLOW-5929: - Assignee: Ash Berlin-Taylor > Improve Scheduler Performance >

[jira] [Created] (AIRFLOW-5933) Unable to define schema for alembic version table

2019-11-14 Thread Axel Goblet (Jira)
Axel Goblet created AIRFLOW-5933: Summary: Unable to define schema for alembic version table Key: AIRFLOW-5933 URL: https://issues.apache.org/jira/browse/AIRFLOW-5933 Project: Apache Airflow

[GitHub] [airflow] OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553918541 @feluelle to my understanding bulk load is default to not be supported since MySQL 8 due to security reasons : https://dev.mysql.com

[jira] [Commented] (AIRFLOW-4848) MySQL warnings about aborted connections, missing engine disposal

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974325#comment-16974325 ] jack commented on AIRFLOW-4848: --- Duplicate of https://issues.apache.org/jira/browse/AIRFL

[jira] [Commented] (AIRFLOW-4789) Add Mysql Exception Handling while pushing data to XCom

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974327#comment-16974327 ] jack commented on AIRFLOW-4789: --- What is the exception? Can you provide example to reprod

[jira] [Created] (AIRFLOW-5934) TRAINING ISSUE! Add extra CC: to the emails sent by Airflow

2019-11-14 Thread Jarek Potiuk (Jira)
Jarek Potiuk created AIRFLOW-5934: - Summary: TRAINING ISSUE! Add extra CC: to the emails sent by Airflow Key: AIRFLOW-5934 URL: https://issues.apache.org/jira/browse/AIRFLOW-5934 Project: Apache Airfl

[jira] [Updated] (AIRFLOW-5934) TRAINING ISSUE! Add extra CC: to the emails sent by Airflow

2019-11-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-5934: -- Description: {quote}{color:#FF}NOTE! THIS IS A TEST issue that is used in Contributor's

[GitHub] [airflow] potiuk commented on issue #6571: [AIRFLOW-5923] Use absolute paths in GCP system tests

2019-11-14 Thread GitBox
potiuk commented on issue #6571: [AIRFLOW-5923] Use absolute paths in GCP system tests URL: https://github.com/apache/airflow/pull/6571#issuecomment-553950886 Travis is sad :( This is an automated message from the Apache Git

[GitHub] [airflow] potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready

2019-11-14 Thread GitBox
potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#discussion_r346408475 ## File path: docs/best-practices.rst ## @@ -0,0 +1,271 @@ + .. Licensed to the Apache Software

[GitHub] [airflow] potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready

2019-11-14 Thread GitBox
potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#discussion_r346409075 ## File path: docs/best-practices.rst ## @@ -0,0 +1,271 @@ + .. Licensed to the Apache Software

[GitHub] [airflow] potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready

2019-11-14 Thread GitBox
potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#discussion_r346408945 ## File path: docs/best-practices.rst ## @@ -0,0 +1,271 @@ + .. Licensed to the Apache Software

[jira] [Created] (AIRFLOW-5935) Logs are not sent S3 by S3TaskHandler

2019-11-14 Thread Aidar Mamytov (Jira)
Aidar Mamytov created AIRFLOW-5935: -- Summary: Logs are not sent S3 by S3TaskHandler Key: AIRFLOW-5935 URL: https://issues.apache.org/jira/browse/AIRFLOW-5935 Project: Apache Airflow Issue Ty

[GitHub] [airflow] potiuk commented on a change in pull request #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db

2019-11-14 Thread GitBox
potiuk commented on a change in pull request #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db URL: https://github.com/apache/airflow/pull/6440#discussion_r346421931 ## File path: tests/gcp/hooks/test_cloud_sql.py ## @@ -1213,235 +1173,144 @@ def t

[jira] [Commented] (AIRFLOW-5768) Google Cloud SQL - Don't store ephemeral connection object to database

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974414#comment-16974414 ] ASF GitHub Bot commented on AIRFLOW-5768: - potiuk commented on pull request #64

[GitHub] [airflow] potiuk merged pull request #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db

2019-11-14 Thread GitBox
potiuk merged pull request #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db URL: https://github.com/apache/airflow/pull/6440 This is an automated message from the Apache Git Service. To respond to t

[jira] [Resolved] (AIRFLOW-5768) Google Cloud SQL - Don't store ephemeral connection object to database

2019-11-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5768. --- Fix Version/s: 2.0.0 Resolution: Fixed > Google Cloud SQL - Don't store ephemeral co

[jira] [Commented] (AIRFLOW-5768) Google Cloud SQL - Don't store ephemeral connection object to database

2019-11-14 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974415#comment-16974415 ] ASF subversion and git services commented on AIRFLOW-5768: -- Co

[GitHub] [airflow] potiuk commented on issue #6447: [AIRFLOW-5792] Straighten out the migrations

2019-11-14 Thread GitBox
potiuk commented on issue #6447: [AIRFLOW-5792] Straighten out the migrations URL: https://github.com/apache/airflow/pull/6447#issuecomment-553974262 @Fokko :)? This is an automated message from the Apache Git Service. To res

[GitHub] [airflow] feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553976344 > @feluelle to my understanding bulk load is default to not be supported since MySQL 8 due to security reasons That's true it i

[GitHub] [airflow] feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553977530 Yes, exactly what you wrote in your jira ticket. This is an automated

[jira] [Created] (AIRFLOW-5936) Allow explicit get_pty in SSHOperator

2019-11-14 Thread Zikun Zhu (Jira)
Zikun Zhu created AIRFLOW-5936: -- Summary: Allow explicit get_pty in SSHOperator Key: AIRFLOW-5936 URL: https://issues.apache.org/jira/browse/AIRFLOW-5936 Project: Apache Airflow Issue Type: Impr

[GitHub] [airflow] codecov-io commented on issue #6572: [AIRFLOW-3489] Improve json data handling in PostgresToGcs operator

2019-11-14 Thread GitBox
codecov-io commented on issue #6572: [AIRFLOW-3489] Improve json data handling in PostgresToGcs operator URL: https://github.com/apache/airflow/pull/6572#issuecomment-553979724 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6572?src=pr&el=h1) Report > Merging [#6572](https://cod

[GitHub] [airflow] zzk-dev opened a new pull request #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator

2019-11-14 Thread GitBox
zzk-dev opened a new pull request #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator URL: https://github.com/apache/airflow/pull/6586 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/j

[jira] [Commented] (AIRFLOW-5936) Allow explicit get_pty in SSHOperator

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974435#comment-16974435 ] ASF GitHub Bot commented on AIRFLOW-5936: - zzk-dev commented on pull request #6

[GitHub] [airflow] OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553990916 @feluelle do you think it should be mentioned in the notes of the function? --

[GitHub] [airflow] feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553997657 I think I wouldn't do it because this can change again and again. It is really special to the mysql client library used. That's why th

[GitHub] [airflow] feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-554003580 But I am not 100 percent sure. If you guys agree I can add something like: ``` .. warning:: According to the mysql docs using thi

[GitHub] [airflow] ToxaZ commented on a change in pull request #5913: [AIRFLOW-5073] Change SQLSensor to keep poking receving NULL, '0'

2019-11-14 Thread GitBox
ToxaZ commented on a change in pull request #5913: [AIRFLOW-5073] Change SQLSensor to keep poking receving NULL, '0' URL: https://github.com/apache/airflow/pull/5913#discussion_r346462895 ## File path: tests/sensors/test_sql_sensor.py ## @@ -103,6 +103,18 @@ def test_sql_s

[GitHub] [airflow] codecov-io commented on issue #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator

2019-11-14 Thread GitBox
codecov-io commented on issue #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator URL: https://github.com/apache/airflow/pull/6586#issuecomment-554015076 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6586?src=pr&el=h1) Report > Merging [#6586](https://codecov.io/gh/apac

[jira] [Created] (AIRFLOW-5937) Group tests for the Version/Connection/Db command

2019-11-14 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-5937: -- Summary: Group tests for the Version/Connection/Db command Key: AIRFLOW-5937 URL: https://issues.apache.org/jira/browse/AIRFLOW-5937 Project: Apache Airflow

[GitHub] [airflow] mik-laj opened a new pull request #6587: [AIRFLOW-5937][part of AIRFLOW-5893][depends on AIRFLOW-5926][AIRFLOW-5919] Group tests for the version/connection/db commands

2019-11-14 Thread GitBox
mik-laj opened a new pull request #6587: [AIRFLOW-5937][part of AIRFLOW-5893][depends on AIRFLOW-5926][AIRFLOW-5919] Group tests for the version/connection/db commands URL: https://github.com/apache/airflow/pull/6587 Make sure you have checked _all_ steps below. ### Jira - [

[jira] [Commented] (AIRFLOW-5937) Group tests for the Version/Connection/Db command

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974519#comment-16974519 ] ASF GitHub Bot commented on AIRFLOW-5937: - mik-laj commented on pull request #6

[GitHub] [airflow] OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-554027473 > WDYT? You are correct. If `local_infile` it will raise warning. if not it will raise error: ``` ERROR 3950 (420

[GitHub] [airflow] codecov-io edited a comment on issue #6568: [AIRFLOW-5919][part of AIRFLOW-5893][depends on AIRFLOW-5926] Group tests for the Users/Roles/Perms commands

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6568: [AIRFLOW-5919][part of AIRFLOW-5893][depends on AIRFLOW-5926] Group tests for the Users/Roles/Perms commands URL: https://github.com/apache/airflow/pull/6568#issuecomment-553615632 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6568?src=pr

[GitHub] [airflow] dstandish commented on issue #6390: [AIRFLOW-5720] don't call _get_connections_from_db in cloud sql test

2019-11-14 Thread GitBox
dstandish commented on issue #6390: [AIRFLOW-5720] don't call _get_connections_from_db in cloud sql test URL: https://github.com/apache/airflow/pull/6390#issuecomment-554045581 closing as this is resolved in #6440 This is an

[GitHub] [airflow] dstandish closed pull request #6390: [AIRFLOW-5720] don't call _get_connections_from_db in cloud sql test

2019-11-14 Thread GitBox
dstandish closed pull request #6390: [AIRFLOW-5720] don't call _get_connections_from_db in cloud sql test URL: https://github.com/apache/airflow/pull/6390 This is an automated message from the Apache Git Service. To respond

[jira] [Commented] (AIRFLOW-5720) don't call _get_connections_from_db in TestCloudSqlDatabaseHook

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974561#comment-16974561 ] ASF GitHub Bot commented on AIRFLOW-5720: - dstandish commented on pull request

  1   2   >