[jira] [Comment Edited] (AIRFLOW-4593) Memory leak in Airflow scheduler

2019-11-14 Thread sison (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972109#comment-16972109 ] sison edited comment on AIRFLOW-4593 at 11/15/19 7:53 AM: -- I find this problem

[GitHub] [airflow] elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE]

2019-11-14 Thread GitBox
elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE] URL: https://github.com/apache/airflow/pull/5659#issuecomment-554245057 @potiuk there is a new version of snakebite-py3 on pypi! :)

[GitHub] [airflow] houqp commented on a change in pull request #6553: [AIRFLOW-5902] avoid unnecessary sleep to maintain local task job heart rate

2019-11-14 Thread GitBox
houqp commented on a change in pull request #6553: [AIRFLOW-5902] avoid unnecessary sleep to maintain local task job heart rate URL: https://github.com/apache/airflow/pull/6553#discussion_r346684260 ## File path: airflow/jobs/base_job.py ## @@ -171,17 +171,14 @@ def

[jira] [Updated] (AIRFLOW-5936) Allow explicit get_pty in SSHOperator

2019-11-14 Thread Zikun Zhu (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zikun Zhu updated AIRFLOW-5936: --- Description: Currently when execution_timeout is reached for an SSHOperator task, the ssh

[GitHub] [airflow] zzk-dev commented on issue #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator

2019-11-14 Thread GitBox
zzk-dev commented on issue #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator URL: https://github.com/apache/airflow/pull/6586#issuecomment-554228929 Just want to make it clearer - by timeout I mean the execution_timeout of the task, not the channel timeout parameter in

[GitHub] [airflow] baolsen commented on issue #6584: Remove duplicate documentation

2019-11-14 Thread GitBox
baolsen commented on issue #6584: Remove duplicate documentation URL: https://github.com/apache/airflow/pull/6584#issuecomment-554222946 Fixed, thanks @potiuk This is an automated message from the Apache Git Service. To

[GitHub] [airflow] whynick1 commented on issue #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset

2019-11-14 Thread GitBox
whynick1 commented on issue #5566: [AIRFLOW-4935] Add method in the bigquery hook to list tables in a dataset URL: https://github.com/apache/airflow/pull/5566#issuecomment-554174357 I am also interested in this PR, any update?  @potiuk @mik-laj @ryanyuan

[GitHub] [airflow] KevinYang21 commented on a change in pull request #6553: [AIRFLOW-5902] avoid unnecessary sleep to maintain local task job heart rate

2019-11-14 Thread GitBox
KevinYang21 commented on a change in pull request #6553: [AIRFLOW-5902] avoid unnecessary sleep to maintain local task job heart rate URL: https://github.com/apache/airflow/pull/6553#discussion_r346625894 ## File path: airflow/jobs/base_job.py ## @@ -171,17 +171,14 @@ def

[GitHub] [airflow] zhongjiajie commented on issue #6582: [AIRFLOW-5928] Hive hooks load_file short circuit

2019-11-14 Thread GitBox
zhongjiajie commented on issue #6582: [AIRFLOW-5928] Hive hooks load_file short circuit URL: https://github.com/apache/airflow/pull/6582#issuecomment-554167510 Thanks, Kevin This is an automated message from the Apache Git

[GitHub] [airflow] KevinYang21 merged pull request #6582: [AIRFLOW-5928] Hive hooks load_file short circuit

2019-11-14 Thread GitBox
KevinYang21 merged pull request #6582: [AIRFLOW-5928] Hive hooks load_file short circuit URL: https://github.com/apache/airflow/pull/6582 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (AIRFLOW-5928) hive hooks load_file short circuit

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974721#comment-16974721 ] ASF GitHub Bot commented on AIRFLOW-5928: - KevinYang21 commented on pull request #6582:

[jira] [Commented] (AIRFLOW-5928) hive hooks load_file short circuit

2019-11-14 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974722#comment-16974722 ] ASF subversion and git services commented on AIRFLOW-5928: -- Commit

[GitHub] [airflow] KevinYang21 merged pull request #6589: [AIRFLOW-XXX] Replace string with str in docstring for consistency

2019-11-14 Thread GitBox
KevinYang21 merged pull request #6589: [AIRFLOW-XXX] Replace string with str in docstring for consistency URL: https://github.com/apache/airflow/pull/6589 This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] codecov-io edited a comment on issue #6589: [AIRFLOW-XXX] Replace string with str in docstring for consistency

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6589: [AIRFLOW-XXX] Replace string with str in docstring for consistency URL: https://github.com/apache/airflow/pull/6589#issuecomment-554128019 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6589?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #6589: [AIRFLOW-XXX] Replace string with str in docstring for consistency

2019-11-14 Thread GitBox
codecov-io commented on issue #6589: [AIRFLOW-XXX] Replace string with str in docstring for consistency URL: https://github.com/apache/airflow/pull/6589#issuecomment-554128019 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6589?src=pr=h1) Report > Merging

[GitHub] [airflow] kaxil opened a new pull request #6589: [AIRFLOW-XXX] Replace string with str in docstring for consistency

2019-11-14 Thread GitBox
kaxil opened a new pull request #6589: [AIRFLOW-XXX] Replace string with str in docstring for consistency URL: https://github.com/apache/airflow/pull/6589 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[jira] [Commented] (AIRFLOW-5781) Migrate AWS Kinesis to /providers/aws [AIP-21]

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974634#comment-16974634 ] ASF GitHub Bot commented on AIRFLOW-5781: - ratb3rt commented on pull request #6588:

[GitHub] [airflow] ratb3rt opened a new pull request #6588: [AIRFLOW-5781] AIP-21 Migrate AWS Kinesis to /providers/amazon/aws

2019-11-14 Thread GitBox
ratb3rt opened a new pull request #6588: [AIRFLOW-5781] AIP-21 Migrate AWS Kinesis to /providers/amazon/aws URL: https://github.com/apache/airflow/pull/6588 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[jira] [Assigned] (AIRFLOW-5781) Migrate AWS Kinesis to /providers/aws [AIP-21]

2019-11-14 Thread Rich Dean (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rich Dean reassigned AIRFLOW-5781: -- Assignee: Rich Dean > Migrate AWS Kinesis to /providers/aws [AIP-21] >

[GitHub] [airflow] ratb3rt commented on a change in pull request #6539: [AIRFLOW-5783] AIP-21 Move aws redshift into providers structure

2019-11-14 Thread GitBox
ratb3rt commented on a change in pull request #6539: [AIRFLOW-5783] AIP-21 Move aws redshift into providers structure URL: https://github.com/apache/airflow/pull/6539#discussion_r346560656 ## File path: scripts/ci/pylint_todo.txt ## @@ -20,7 +20,7 @@

[jira] [Commented] (AIRFLOW-5926) Fix race-condition in TestCliWebServer

2019-11-14 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974600#comment-16974600 ] ASF subversion and git services commented on AIRFLOW-5926: -- Commit

[jira] [Commented] (AIRFLOW-5926) Fix race-condition in TestCliWebServer

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974599#comment-16974599 ] ASF GitHub Bot commented on AIRFLOW-5926: - mik-laj commented on pull request #6579:

[GitHub] [airflow] mik-laj merged pull request #6579: [AIRFLOW-5926] Fix race-condition in TestCliWebServer

2019-11-14 Thread GitBox
mik-laj merged pull request #6579: [AIRFLOW-5926] Fix race-condition in TestCliWebServer URL: https://github.com/apache/airflow/pull/6579 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] codecov-io edited a comment on issue #6568: [AIRFLOW-5919][part of AIRFLOW-5893][depends on AIRFLOW-5926] Group tests for the Users/Roles/Perms commands

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6568: [AIRFLOW-5919][part of AIRFLOW-5893][depends on AIRFLOW-5926] Group tests for the Users/Roles/Perms commands URL: https://github.com/apache/airflow/pull/6568#issuecomment-553615632 #

[GitHub] [airflow] kaxil commented on a change in pull request #6579: [AIRFLOW-5926] Fix race-condition in TestCliWebServer

2019-11-14 Thread GitBox
kaxil commented on a change in pull request #6579: [AIRFLOW-5926] Fix race-condition in TestCliWebServer URL: https://github.com/apache/airflow/pull/6579#discussion_r346508652 ## File path: tests/cli/test_cli.py ## @@ -801,30 +817,24 @@ def

[jira] [Commented] (AIRFLOW-5720) don't call _get_connections_from_db in TestCloudSqlDatabaseHook

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974561#comment-16974561 ] ASF GitHub Bot commented on AIRFLOW-5720: - dstandish commented on pull request #6390:

[GitHub] [airflow] codecov-io edited a comment on issue #6568: [AIRFLOW-5919][part of AIRFLOW-5893][depends on AIRFLOW-5926] Group tests for the Users/Roles/Perms commands

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6568: [AIRFLOW-5919][part of AIRFLOW-5893][depends on AIRFLOW-5926] Group tests for the Users/Roles/Perms commands URL: https://github.com/apache/airflow/pull/6568#issuecomment-553615632 #

[GitHub] [airflow] dstandish commented on issue #6390: [AIRFLOW-5720] don't call _get_connections_from_db in cloud sql test

2019-11-14 Thread GitBox
dstandish commented on issue #6390: [AIRFLOW-5720] don't call _get_connections_from_db in cloud sql test URL: https://github.com/apache/airflow/pull/6390#issuecomment-554045581 closing as this is resolved in #6440 This is an

[GitHub] [airflow] dstandish closed pull request #6390: [AIRFLOW-5720] don't call _get_connections_from_db in cloud sql test

2019-11-14 Thread GitBox
dstandish closed pull request #6390: [AIRFLOW-5720] don't call _get_connections_from_db in cloud sql test URL: https://github.com/apache/airflow/pull/6390 This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-554027473 > WDYT? You are correct. If `local_infile` it will raise warning. if not it will raise error: ``` ERROR 3950

[jira] [Commented] (AIRFLOW-5937) Group tests for the Version/Connection/Db command

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974519#comment-16974519 ] ASF GitHub Bot commented on AIRFLOW-5937: - mik-laj commented on pull request #6587:

[GitHub] [airflow] mik-laj opened a new pull request #6587: [AIRFLOW-5937][part of AIRFLOW-5893][depends on AIRFLOW-5926][AIRFLOW-5919] Group tests for the version/connection/db commands

2019-11-14 Thread GitBox
mik-laj opened a new pull request #6587: [AIRFLOW-5937][part of AIRFLOW-5893][depends on AIRFLOW-5926][AIRFLOW-5919] Group tests for the version/connection/db commands URL: https://github.com/apache/airflow/pull/6587 Make sure you have checked _all_ steps below. ### Jira - [

[jira] [Created] (AIRFLOW-5937) Group tests for the Version/Connection/Db command

2019-11-14 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-5937: -- Summary: Group tests for the Version/Connection/Db command Key: AIRFLOW-5937 URL: https://issues.apache.org/jira/browse/AIRFLOW-5937 Project: Apache Airflow

[GitHub] [airflow] codecov-io commented on issue #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator

2019-11-14 Thread GitBox
codecov-io commented on issue #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator URL: https://github.com/apache/airflow/pull/6586#issuecomment-554015076 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6586?src=pr=h1) Report > Merging

[GitHub] [airflow] ToxaZ commented on a change in pull request #5913: [AIRFLOW-5073] Change SQLSensor to keep poking receving NULL, '0'

2019-11-14 Thread GitBox
ToxaZ commented on a change in pull request #5913: [AIRFLOW-5073] Change SQLSensor to keep poking receving NULL, '0' URL: https://github.com/apache/airflow/pull/5913#discussion_r346462895 ## File path: tests/sensors/test_sql_sensor.py ## @@ -103,6 +103,18 @@ def

[GitHub] [airflow] feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-554003580 But I am not 100 percent sure. If you guys agree I can add something like: ``` .. warning:: According to the mysql docs using

[GitHub] [airflow] feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553997657 I think I wouldn't do it because this can change again and again. It is really special to the mysql client library used. That's why

[GitHub] [airflow] OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553990916 @feluelle do you think it should be mentioned in the notes of the function?

[GitHub] [airflow] zzk-dev opened a new pull request #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator

2019-11-14 Thread GitBox
zzk-dev opened a new pull request #6586: [AIRFLOW-5936] Allow explicit get_pty in SSHOperator URL: https://github.com/apache/airflow/pull/6586 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[jira] [Commented] (AIRFLOW-5936) Allow explicit get_pty in SSHOperator

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974435#comment-16974435 ] ASF GitHub Bot commented on AIRFLOW-5936: - zzk-dev commented on pull request #6586:

[GitHub] [airflow] codecov-io commented on issue #6572: [AIRFLOW-3489] Improve json data handling in PostgresToGcs operator

2019-11-14 Thread GitBox
codecov-io commented on issue #6572: [AIRFLOW-3489] Improve json data handling in PostgresToGcs operator URL: https://github.com/apache/airflow/pull/6572#issuecomment-553979724 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6572?src=pr=h1) Report > Merging

[jira] [Created] (AIRFLOW-5936) Allow explicit get_pty in SSHOperator

2019-11-14 Thread Zikun Zhu (Jira)
Zikun Zhu created AIRFLOW-5936: -- Summary: Allow explicit get_pty in SSHOperator Key: AIRFLOW-5936 URL: https://issues.apache.org/jira/browse/AIRFLOW-5936 Project: Apache Airflow Issue Type:

[GitHub] [airflow] feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553977530 Yes, exactly what you wrote in your jira ticket. This is an

[GitHub] [airflow] feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
feluelle commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553976344 > @feluelle to my understanding bulk load is default to not be supported since MySQL 8 due to security reasons That's true it

[GitHub] [airflow] potiuk commented on issue #6447: [AIRFLOW-5792] Straighten out the migrations

2019-11-14 Thread GitBox
potiuk commented on issue #6447: [AIRFLOW-5792] Straighten out the migrations URL: https://github.com/apache/airflow/pull/6447#issuecomment-553974262 @Fokko :)? This is an automated message from the Apache Git Service. To

[jira] [Commented] (AIRFLOW-5768) Google Cloud SQL - Don't store ephemeral connection object to database

2019-11-14 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974415#comment-16974415 ] ASF subversion and git services commented on AIRFLOW-5768: -- Commit

[jira] [Resolved] (AIRFLOW-5768) Google Cloud SQL - Don't store ephemeral connection object to database

2019-11-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5768. --- Fix Version/s: 2.0.0 Resolution: Fixed > Google Cloud SQL - Don't store ephemeral

[GitHub] [airflow] potiuk merged pull request #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db

2019-11-14 Thread GitBox
potiuk merged pull request #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db URL: https://github.com/apache/airflow/pull/6440 This is an automated message from the Apache Git Service. To respond to

[jira] [Commented] (AIRFLOW-5768) Google Cloud SQL - Don't store ephemeral connection object to database

2019-11-14 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974414#comment-16974414 ] ASF GitHub Bot commented on AIRFLOW-5768: - potiuk commented on pull request #6440:

[GitHub] [airflow] potiuk commented on a change in pull request #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db

2019-11-14 Thread GitBox
potiuk commented on a change in pull request #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db URL: https://github.com/apache/airflow/pull/6440#discussion_r346421931 ## File path: tests/gcp/hooks/test_cloud_sql.py ## @@ -1213,235 +1173,144 @@ def

[jira] [Created] (AIRFLOW-5935) Logs are not sent S3 by S3TaskHandler

2019-11-14 Thread Aidar Mamytov (Jira)
Aidar Mamytov created AIRFLOW-5935: -- Summary: Logs are not sent S3 by S3TaskHandler Key: AIRFLOW-5935 URL: https://issues.apache.org/jira/browse/AIRFLOW-5935 Project: Apache Airflow Issue

[GitHub] [airflow] potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready

2019-11-14 Thread GitBox
potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#discussion_r346408945 ## File path: docs/best-practices.rst ## @@ -0,0 +1,271 @@ + .. Licensed to the Apache Software

[GitHub] [airflow] potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready

2019-11-14 Thread GitBox
potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#discussion_r346409075 ## File path: docs/best-practices.rst ## @@ -0,0 +1,271 @@ + .. Licensed to the Apache Software

[GitHub] [airflow] potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready

2019-11-14 Thread GitBox
potiuk commented on a change in pull request #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#discussion_r346408475 ## File path: docs/best-practices.rst ## @@ -0,0 +1,271 @@ + .. Licensed to the Apache Software

[GitHub] [airflow] potiuk commented on issue #6571: [AIRFLOW-5923] Use absolute paths in GCP system tests

2019-11-14 Thread GitBox
potiuk commented on issue #6571: [AIRFLOW-5923] Use absolute paths in GCP system tests URL: https://github.com/apache/airflow/pull/6571#issuecomment-553950886 Travis is sad :( This is an automated message from the Apache Git

[jira] [Updated] (AIRFLOW-5934) TRAINING ISSUE! Add extra CC: to the emails sent by Airflow

2019-11-14 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-5934: -- Description: {quote}{color:#FF}NOTE! THIS IS A TEST issue that is used in Contributor's

[jira] [Created] (AIRFLOW-5934) TRAINING ISSUE! Add extra CC: to the emails sent by Airflow

2019-11-14 Thread Jarek Potiuk (Jira)
Jarek Potiuk created AIRFLOW-5934: - Summary: TRAINING ISSUE! Add extra CC: to the emails sent by Airflow Key: AIRFLOW-5934 URL: https://issues.apache.org/jira/browse/AIRFLOW-5934 Project: Apache

[jira] [Commented] (AIRFLOW-4789) Add Mysql Exception Handling while pushing data to XCom

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974327#comment-16974327 ] jack commented on AIRFLOW-4789: --- What is the exception? Can you provide example to reproduce ? > Add

[jira] [Commented] (AIRFLOW-4848) MySQL warnings about aborted connections, missing engine disposal

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974325#comment-16974325 ] jack commented on AIRFLOW-4848: --- Duplicate of https://issues.apache.org/jira/browse/AIRFLOW-3544? ? >

[GitHub] [airflow] OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook

2019-11-14 Thread GitBox
OmerJog commented on issue #6575: [AIRFLOW-5921] Add bulk_load_custom to MySqlHook URL: https://github.com/apache/airflow/pull/6575#issuecomment-553918541 @feluelle to my understanding bulk load is default to not be supported since MySQL 8 due to security reasons :

[jira] [Created] (AIRFLOW-5933) Unable to define schema for alembic version table

2019-11-14 Thread Axel Goblet (Jira)
Axel Goblet created AIRFLOW-5933: Summary: Unable to define schema for alembic version table Key: AIRFLOW-5933 URL: https://issues.apache.org/jira/browse/AIRFLOW-5933 Project: Apache Airflow

[jira] [Assigned] (AIRFLOW-5929) Improve Scheduler Performance

2019-11-14 Thread Fokko Driesprong (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong reassigned AIRFLOW-5929: - Assignee: Ash Berlin-Taylor > Improve Scheduler Performance >

[GitHub] [airflow-site] kgabryje opened a new pull request #164: [depends on #151] Feature/share social media

2019-11-14 Thread GitBox
kgabryje opened a new pull request #164: [depends on #151] Feature/share social media URL: https://github.com/apache/airflow-site/pull/164 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (AIRFLOW-5932) Unable to resolve backticks character

2019-11-14 Thread Badri (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974297#comment-16974297 ] Badri commented on AIRFLOW-5932: Apologies for not adding many details. The aim of that command was to

[jira] [Commented] (AIRFLOW-5932) Unable to resolve backticks character

2019-11-14 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974289#comment-16974289 ] Ash Berlin-Taylor commented on AIRFLOW-5932: It's not quite clear what you are asking. What

[jira] [Created] (AIRFLOW-5932) Unable to resolve backticks character

2019-11-14 Thread Badri (Jira)
Badri created AIRFLOW-5932: -- Summary: Unable to resolve backticks character Key: AIRFLOW-5932 URL: https://issues.apache.org/jira/browse/AIRFLOW-5932 Project: Apache Airflow Issue Type: Bug

[GitHub] [airflow] potiuk commented on issue #6585: More GSOD improvements

2019-11-14 Thread GitBox
potiuk commented on issue #6585: More GSOD improvements URL: https://github.com/apache/airflow/pull/6585#issuecomment-553899400 Hey @efedotova -> I made a first quick attempt at separating TESTING.rst from Local/Breeze. I hope it might be a good start for you as discussed :).

[GitHub] [airflow] potiuk opened a new pull request #6585: More GSOD improvements

2019-11-14 Thread GitBox
potiuk opened a new pull request #6585: More GSOD improvements URL: https://github.com/apache/airflow/pull/6585 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and

[jira] [Created] (AIRFLOW-5931) Spawning new python interpreter is slow

2019-11-14 Thread Ash Berlin-Taylor (Jira)
Ash Berlin-Taylor created AIRFLOW-5931: -- Summary: Spawning new python interpreter is slow Key: AIRFLOW-5931 URL: https://issues.apache.org/jira/browse/AIRFLOW-5931 Project: Apache Airflow

[jira] [Assigned] (AIRFLOW-5930) Reduce time spent building SQL strings

2019-11-14 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor reassigned AIRFLOW-5930: -- Assignee: Ash Berlin-Taylor > Reduce time spent building SQL strings >

[jira] [Created] (AIRFLOW-5930) Reduce time spent building SQL strings

2019-11-14 Thread Ash Berlin-Taylor (Jira)
Ash Berlin-Taylor created AIRFLOW-5930: -- Summary: Reduce time spent building SQL strings Key: AIRFLOW-5930 URL: https://issues.apache.org/jira/browse/AIRFLOW-5930 Project: Apache Airflow

[jira] [Created] (AIRFLOW-5929) Improve Scheduler Performance

2019-11-14 Thread Ash Berlin-Taylor (Jira)
Ash Berlin-Taylor created AIRFLOW-5929: -- Summary: Improve Scheduler Performance Key: AIRFLOW-5929 URL: https://issues.apache.org/jira/browse/AIRFLOW-5929 Project: Apache Airflow Issue

[jira] [Commented] (AIRFLOW-5818) Very bad webserver performance when defining many dags with many operators

2019-11-14 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974266#comment-16974266 ] Ash Berlin-Taylor commented on AIRFLOW-5818: [~gary@bidalgo] Ah, when running from a

[GitHub] [airflow] ashb commented on a change in pull request #5913: [AIRFLOW-5073] Change SQLSensor to keep poking receving NULL, '0'

2019-11-14 Thread GitBox
ashb commented on a change in pull request #5913: [AIRFLOW-5073] Change SQLSensor to keep poking receving NULL, '0' URL: https://github.com/apache/airflow/pull/5913#discussion_r346311207 ## File path: tests/sensors/test_sql_sensor.py ## @@ -103,6 +103,18 @@ def

[GitHub] [airflow] potiuk commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE]

2019-11-14 Thread GitBox
potiuk commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE] URL: https://github.com/apache/airflow/pull/5659#issuecomment-553878821 Happy to help @elukey -> please mark my github nickname when you ping them and i will also comment on that!

[GitHub] [airflow] elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE]

2019-11-14 Thread GitBox
elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE] URL: https://github.com/apache/airflow/pull/5659#issuecomment-553870608 Update: my pull request for snakebite-py3 also includes support for SASL encryption for the Hadoop RPC protocol. This means that

[GitHub] [airflow] codecov-io commented on issue #6573: [AIRFLOW-5904] "Trigger DAG" should redirect to the "dag_default_view" page

2019-11-14 Thread GitBox
codecov-io commented on issue #6573: [AIRFLOW-5904] "Trigger DAG" should redirect to the "dag_default_view" page URL: https://github.com/apache/airflow/pull/6573#issuecomment-553864434 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6573?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553564200 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6576?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553564200 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6576?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553564200 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6576?src=pr=h1) Report > Merging

[GitHub] [airflow] ashb commented on a change in pull request #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer URL: https://github.com/apache/airflow/pull/6396#discussion_r346257541 ## File path: tests/operators/test_redshift_to_s3_operator.py ## @@ -51,23 +52,25 @@

[GitHub] [airflow] ashb commented on a change in pull request #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer URL: https://github.com/apache/airflow/pull/6396#discussion_r346257108 ## File path: tests/operators/test_redshift_to_s3_operator.py ## @@ -51,23 +52,25 @@

[jira] [Commented] (AIRFLOW-4066) dag_run failed while one operator is still running

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974156#comment-16974156 ] jack commented on AIRFLOW-4066: --- Which airflow version are you running? > dag_run failed while one

[GitHub] [airflow] ashb commented on issue #5787: [AIRFLOW-5172] Add choice of interval edge scheduling

2019-11-14 Thread GitBox
ashb commented on issue #5787: [AIRFLOW-5172] Add choice of interval edge scheduling URL: https://github.com/apache/airflow/pull/5787#issuecomment-553842789 @aoen That sounds sensible, so long as we clearly "sign-post" the change for users when upgrading somehow. The default could only

[jira] [Commented] (AIRFLOW-5627) transform function should be optional for s3_file_tranformation_operator

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974152#comment-16974152 ] jack commented on AIRFLOW-5627: --- If you don't want to transform then don't use transform operator.

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346247319 ## File path: docs/lineage.rst ## @@ -50,40 +51,43 @@ works. f_final =

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346248047 ## File path: docs/lineage.rst ## @@ -50,40 +51,43 @@ works. f_final =

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346245600 ## File path: airflow/models/baseoperator.py ## @@ -412,25 +412,12 @@ def __init__(

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346245366 ## File path: airflow/models/baseoperator.py ## @@ -316,8 +316,8 @@ def __init__(

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346248832 ## File path: tests/lineage/test_lineage.py ## @@ -85,26 +85,26 @@ def test_lineage(self,

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346246696 ## File path: airflow/models/baseoperator.py ## @@ -316,8 +316,8 @@ def __init__(

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346251187 ## File path: airflow/lineage/__init__.py ## @@ -92,51 +115,43 @@ def prepare_lineage(func):

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346247195 ## File path: airflow/lineage/entity/dataset.py ## @@ -0,0 +1,25 @@ +# -*- coding: utf-8 -*-

[GitHub] [airflow] ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness

2019-11-14 Thread GitBox
ashb commented on a change in pull request #6564: [AIRFLOW-5911] Simplify lineage API and improve robustness URL: https://github.com/apache/airflow/pull/6564#discussion_r346248106 ## File path: docs/lineage.rst ## @@ -50,40 +51,43 @@ works. f_final =

[GitHub] [airflow] OmerJog commented on issue #6549: [AIRFLOW-5880] Enforce unique task ids

2019-11-14 Thread GitBox
OmerJog commented on issue #6549: [AIRFLOW-5880] Enforce unique task ids URL: https://github.com/apache/airflow/pull/6549#issuecomment-553837774 Great. I think we should also enforce unique dag_id / varibales This is an

[GitHub] [airflow] codecov-io edited a comment on issue #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer

2019-11-14 Thread GitBox
codecov-io edited a comment on issue #6396: [AIRFLOW-5726] Delete table as file name in RedshiftToS3Transfer URL: https://github.com/apache/airflow/pull/6396#issuecomment-546875908 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6396?src=pr=h1) Report > Merging

[jira] [Resolved] (AIRFLOW-3235) Add list function in AzureDataLakeHook

2019-11-14 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-3235. Fix Version/s: 1.10.7 Resolution: Fixed > Add list function in

[jira] [Commented] (AIRFLOW-2572) Airflow /admin/configurationview/ Running Configuration not accurate

2019-11-14 Thread jack (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974119#comment-16974119 ] jack commented on AIRFLOW-2572: --- [~joejasinski] can you recheck this with newer version of Airflow? >

[GitHub] [airflow-site] kgabryje opened a new pull request #163: added disclaimer to footer

2019-11-14 Thread GitBox
kgabryje opened a new pull request #163: added disclaimer to footer URL: https://github.com/apache/airflow-site/pull/163 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on issue #6576: [WIP][AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2019-11-14 Thread GitBox
potiuk commented on issue #6576: [WIP][AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-553811419 Yeah @feluelle -> parameterising just one representative tests with all the options (no client,

  1   2   >