[jira] [Commented] (AIRFLOW-5832) pagerduty hook

2019-10-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964629#comment-16964629 ] ASF GitHub Bot commented on AIRFLOW-5832: - houqp commented on pull request #6484:

[GitHub] [airflow] houqp opened a new pull request #6484: [AIRFLOW-5832] add pagerduty hook

2019-10-31 Thread GitBox
houqp opened a new pull request #6484: [AIRFLOW-5832] add pagerduty hook URL: https://github.com/apache/airflow/pull/6484 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/)

[jira] [Created] (AIRFLOW-5832) pagerduty hook

2019-10-31 Thread QP Hou (Jira)
QP Hou created AIRFLOW-5832: --- Summary: pagerduty hook Key: AIRFLOW-5832 URL: https://issues.apache.org/jira/browse/AIRFLOW-5832 Project: Apache Airflow Issue Type: New Feature

[GitHub] [airflow] mingrammer commented on issue #6432: [AIRFLOW-5758] Support the custom cursor classes for the PostgreSQL hook

2019-10-31 Thread GitBox
mingrammer commented on issue #6432: [AIRFLOW-5758] Support the custom cursor classes for the PostgreSQL hook URL: https://github.com/apache/airflow/pull/6432#issuecomment-548674532 Some azure-dependant test cases are failing due to azure library changes.

[GitHub] [airflow] mingrammer opened a new pull request #6483: [AIRFLOW-XXX] Fix typo in mysql_hook docstring

2019-10-31 Thread GitBox
mingrammer opened a new pull request #6483: [AIRFLOW-XXX] Fix typo in mysql_hook docstring URL: https://github.com/apache/airflow/pull/6483 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[GitHub] [airflow] mingrammer commented on issue #6432: [AIRFLOW-5758] Support the custom cursor classes for the PostgreSQL hook

2019-10-31 Thread GitBox
mingrammer commented on issue #6432: [AIRFLOW-5758] Support the custom cursor classes for the PostgreSQL hook URL: https://github.com/apache/airflow/pull/6432#issuecomment-548634354 Done. This is an automated message from

[jira] [Commented] (AIRFLOW-5823) Add a new hook for ingesting data into Apache Pinot

2019-10-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964524#comment-16964524 ] ASF GitHub Bot commented on AIRFLOW-5823: - sekikn commented on pull request #6482:

[GitHub] [airflow] sekikn opened a new pull request #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot

2019-10-31 Thread GitBox
sekikn opened a new pull request #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot URL: https://github.com/apache/airflow/pull/6482 ### Jira - [x] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and

[GitHub] [airflow] ZxMYS edited a comment on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
ZxMYS edited a comment on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548632226 ok, I updated description in default_airflow.cfg and added some simple unit test cases. Let me know if

[GitHub] [airflow] ZxMYS commented on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
ZxMYS commented on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548632226 ok, I updated description and added some simple unit test cases. Let me know if they are enough :)

[GitHub] [airflow] mingrammer commented on issue #6432: [AIRFLOW-5758] Support the custom cursor classes for the PostgreSQL hook

2019-10-31 Thread GitBox
mingrammer commented on issue #6432: [AIRFLOW-5758] Support the custom cursor classes for the PostgreSQL hook URL: https://github.com/apache/airflow/pull/6432#issuecomment-548627525 Sure, I'll do! This is an automated

[jira] [Updated] (AIRFLOW-5704) Docker scripts/image for kind kubernetes tests can be optimised

2019-10-31 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-5704: -- Parent: AIRFLOW-5606 Issue Type: Sub-task (was: Improvement) > Docker scripts/image

[jira] [Created] (AIRFLOW-5827) Breeze's Build pre-commit should be done as late as possible to optimise experience

2019-10-31 Thread Jarek Potiuk (Jira)
Jarek Potiuk created AIRFLOW-5827: - Summary: Breeze's Build pre-commit should be done as late as possible to optimise experience Key: AIRFLOW-5827 URL: https://issues.apache.org/jira/browse/AIRFLOW-5827

[jira] [Created] (AIRFLOW-5831) Add production image support

2019-10-31 Thread Jarek Potiuk (Jira)
Jarek Potiuk created AIRFLOW-5831: - Summary: Add production image support Key: AIRFLOW-5831 URL: https://issues.apache.org/jira/browse/AIRFLOW-5831 Project: Apache Airflow Issue Type:

[jira] [Created] (AIRFLOW-5830) Get rid of slim image

2019-10-31 Thread Jarek Potiuk (Jira)
Jarek Potiuk created AIRFLOW-5830: - Summary: Get rid of slim image Key: AIRFLOW-5830 URL: https://issues.apache.org/jira/browse/AIRFLOW-5830 Project: Apache Airflow Issue Type: Sub-task

[jira] [Updated] (AIRFLOW-5704) Docker image for kind kubernetes tests can be optimised

2019-10-31 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-5704: -- Description: The docker CI image for kind tests can be optimised:   * Kubernetes Version

[jira] [Created] (AIRFLOW-5829) Get rid of checklicence image

2019-10-31 Thread Jarek Potiuk (Jira)
Jarek Potiuk created AIRFLOW-5829: - Summary: Get rid of checklicence image Key: AIRFLOW-5829 URL: https://issues.apache.org/jira/browse/AIRFLOW-5829 Project: Apache Airflow Issue Type:

[jira] [Updated] (AIRFLOW-5828) Build scripts should be simplified

2019-10-31 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-5828: -- Summary: Build scripts should be simplified (was: Build scripts shoudl be simplified) >

[jira] [Created] (AIRFLOW-5828) Build scripts shoudl be simplified

2019-10-31 Thread Jarek Potiuk (Jira)
Jarek Potiuk created AIRFLOW-5828: - Summary: Build scripts shoudl be simplified Key: AIRFLOW-5828 URL: https://issues.apache.org/jira/browse/AIRFLOW-5828 Project: Apache Airflow Issue Type:

[jira] [Created] (AIRFLOW-5826) Breeze output should be auto-generated in BREEZE.rst

2019-10-31 Thread Jarek Potiuk (Jira)
Jarek Potiuk created AIRFLOW-5826: - Summary: Breeze output should be auto-generated in BREEZE.rst Key: AIRFLOW-5826 URL: https://issues.apache.org/jira/browse/AIRFLOW-5826 Project: Apache Airflow

[jira] [Updated] (AIRFLOW-5704) Docker scripts/image for kind kubernetes tests can be optimised

2019-10-31 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-5704: -- Description: The docker CI image for kind tests can be optimised:   * Kubernetes Version

[jira] [Updated] (AIRFLOW-5704) Docker scripts/image for kind kubernetes tests can be optimised

2019-10-31 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-5704: -- Summary: Docker scripts/image for kind kubernetes tests can be optimised (was: Docker image

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-10-31 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r341365857 ## File path: airflow/kubernetes/pod_generator.py ## @@ -62,76 +71,116 @@

[GitHub] [airflow] yegeniy commented on issue #6281: Run batches of (self-terminating) EMR JobFlows [AIRFLOW-5763]

2019-10-31 Thread GitBox
yegeniy commented on issue #6281: Run batches of (self-terminating) EMR JobFlows [AIRFLOW-5763] URL: https://github.com/apache/airflow/pull/6281#issuecomment-548557567 Yeah, this definitely does a lot more than existing EMR operators. I think maybe we should close out the PR, and I can

[GitHub] [airflow] chicochico opened a new pull request #6481: [AIRFLOW-5736] Move gcs_to_gdrive_operator.py to airflow/operators

2019-10-31 Thread GitBox
chicochico opened a new pull request #6481: [AIRFLOW-5736] Move gcs_to_gdrive_operator.py to airflow/operators URL: https://github.com/apache/airflow/pull/6481 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[jira] [Commented] (AIRFLOW-5736) Move airflow.contrib.operators.gcs_to_gdrive_operator to core

2019-10-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964363#comment-16964363 ] ASF GitHub Bot commented on AIRFLOW-5736: - chicochico commented on pull request #6481:

[GitHub] [airflow] mik-laj commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-10-31 Thread GitBox
mik-laj commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r341335644 ## File path: airflow/kubernetes/pod_generator.py ## @@ -62,76 +71,116 @@

[GitHub] [airflow] mik-laj commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-10-31 Thread GitBox
mik-laj commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r341334482 ## File path: airflow/kubernetes/worker_configuration.py ## @@ -361,22

[GitHub] [airflow] mik-laj commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-10-31 Thread GitBox
mik-laj commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r341334337 ## File path: airflow/kubernetes/worker_configuration.py ## @@ -28,7 +28,11

[GitHub] [airflow] aoen commented on issue #5787: [AIRFLOW-5172] Add choice of interval edge scheduling

2019-10-31 Thread GitBox
aoen commented on issue #5787: [AIRFLOW-5172] Add choice of interval edge scheduling URL: https://github.com/apache/airflow/pull/5787#issuecomment-548526225 > > Really like the spirit of this change. My vote would be to change the flag to schedule_at_interval_start and default it to false

[jira] [Commented] (AIRFLOW-5818) Very bad webserver performance when defining many dags with many operators

2019-10-31 Thread Gary Harpaz (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964338#comment-16964338 ] Gary Harpaz commented on AIRFLOW-5818: -- Ok I will check it. > Very bad webserver performance when

[GitHub] [airflow] mik-laj commented on a change in pull request #6472: [AIRFLOW-YYYY][POC] Running tests with pytest

2019-10-31 Thread GitBox
mik-laj commented on a change in pull request #6472: [AIRFLOW-][POC] Running tests with pytest URL: https://github.com/apache/airflow/pull/6472#discussion_r341304726 ## File path: tests/conftest.py ## @@ -0,0 +1,49 @@ +# Licensed to the Apache Software Foundation

[GitHub] [airflow] kaxil commented on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
kaxil commented on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548506434 > Can you add your reply to Kaxil as a comment in the code for posterity? > > I think we may want a test

[jira] [Updated] (AIRFLOW-5825) SageMakerEndpointOperator is not idempotent

2019-10-31 Thread Bas Harenslak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bas Harenslak updated AIRFLOW-5825: --- Description: The SageMakerEndpointOperator currently taken an argument "operation" with

[jira] [Created] (AIRFLOW-5825) SageMakerEndpointOperator is not idempotent

2019-10-31 Thread Bas Harenslak (Jira)
Bas Harenslak created AIRFLOW-5825: -- Summary: SageMakerEndpointOperator is not idempotent Key: AIRFLOW-5825 URL: https://issues.apache.org/jira/browse/AIRFLOW-5825 Project: Apache Airflow

[GitHub] [airflow] aoen edited a comment on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
aoen edited a comment on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548496727 Can you add your reply to Kaxil as a comment in the code for posterity? I think we may want a test

[GitHub] [airflow] aoen commented on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
aoen commented on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548496727 Can you add your reply to Kaxil as a comment in the code for posterity?

[GitHub] [airflow] feluelle commented on issue #6451: [AIRFLOW-5796] Fix case when slot_pool relation is missing

2019-10-31 Thread GitBox
feluelle commented on issue #6451: [AIRFLOW-5796] Fix case when slot_pool relation is missing URL: https://github.com/apache/airflow/pull/6451#issuecomment-548494754 +1 for moving it out of `__init__`. This is an

[GitHub] [airflow] codecov-io edited a comment on issue #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread GitBox
codecov-io edited a comment on issue #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator URL: https://github.com/apache/airflow/pull/6352#issuecomment-548488487 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6352?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread GitBox
codecov-io commented on issue #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator URL: https://github.com/apache/airflow/pull/6352#issuecomment-548488487 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6352?src=pr=h1) Report > Merging

[GitHub] [airflow] davlum commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-10-31 Thread GitBox
davlum commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-548482412 @ashb I've changed the name of the argument to `pod_template_file`. There is a PR open on the [Kubernetes

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-10-31 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r341264000 ## File path: airflow/kubernetes/pod_generator.py ## @@ -62,75 +68,116 @@

[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2019-10-31 Thread GitBox
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r341264000 ## File path: airflow/kubernetes/pod_generator.py ## @@ -62,75 +68,116 @@

[GitHub] [airflow] feluelle commented on a change in pull request #6410: [AIRFLOW-5677] Rename GCPSpeechToTextHook to CloudSpeechToTextHook

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6410: [AIRFLOW-5677] Rename GCPSpeechToTextHook to CloudSpeechToTextHook URL: https://github.com/apache/airflow/pull/6410#discussion_r341260963 ## File path: airflow/contrib/hooks/gcp_speech_to_text_hook.py ## @@ -23,9

[GitHub] [airflow] feluelle commented on a change in pull request #6410: [AIRFLOW-5677] Rename GCPSpeechToTextHook to CloudSpeechToTextHook

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6410: [AIRFLOW-5677] Rename GCPSpeechToTextHook to CloudSpeechToTextHook URL: https://github.com/apache/airflow/pull/6410#discussion_r341261060 ## File path: airflow/contrib/hooks/gcp_speech_to_text_hook.py ## @@ -23,9

[jira] [Work started] (AIRFLOW-5816) S3 to Snowflake operator

2019-10-31 Thread MOHAMMD SHAKEEL SHAIK (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-5816 started by MOHAMMD SHAKEEL SHAIK. -- > S3 to Snowflake operator > > >

[GitHub] [airflow] Sinsin1367 commented on issue #6261: [AIRFLOW-5141] Added optional resource usage logging to KubernetesPod…

2019-10-31 Thread GitBox
Sinsin1367 commented on issue #6261: [AIRFLOW-5141] Added optional resource usage logging to KubernetesPod… URL: https://github.com/apache/airflow/pull/6261#issuecomment-548468858 Hi Ash, Thanks a lot for the new release, I knew you would not have time for this for a while. Do you

[GitHub] [airflow] ZxMYS edited a comment on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
ZxMYS edited a comment on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548467188 @kaxil user can not pass in a dict in connection string, which is needed if we want to set `ssl = {

[GitHub] [airflow] ZxMYS edited a comment on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
ZxMYS edited a comment on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548467188 @kaxil user can not pass in a dict in connection string, which is needed if we want to set `ssl = {

[GitHub] [airflow] ZxMYS edited a comment on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
ZxMYS edited a comment on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548467188 @kaxil user can not pass in a dict in connection string, which is needed if we want to set `ssl = {

[GitHub] [airflow] ZxMYS edited a comment on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
ZxMYS edited a comment on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548467188 @kaxil user can not pass in a dict in connection string, which is needed if we want to set `ssl = {

[GitHub] [airflow] ZxMYS commented on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
ZxMYS commented on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548467188 @kaxil user can not pass in a dict in connection string, which is needed if we want to set `ssl = {

[jira] [Comment Edited] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?

2019-10-31 Thread Nidhi Chourasia (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964175#comment-16964175 ] Nidhi Chourasia edited comment on AIRFLOW-4470 at 10/31/19 4:24 PM:

[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?

2019-10-31 Thread Nidhi Chourasia (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964175#comment-16964175 ] Nidhi Chourasia commented on AIRFLOW-4470: -- [~vinceatbluelabs] Appreciate and kindness and

[GitHub] [airflow] yegeniy commented on issue #6269: Use poke_interval of 0 to make EMR tests run green [AIRFLOW-3897]

2019-10-31 Thread GitBox
yegeniy commented on issue #6269: Use poke_interval of 0 to make EMR tests run green [AIRFLOW-3897] URL: https://github.com/apache/airflow/pull/6269#issuecomment-548453344 This is definitely just a small bit of the full fix for #3897. This does not fix it. I don’t want to change

[jira] [Created] (AIRFLOW-5824) Add a new hook and operator for AWS DataSync

2019-10-31 Thread Bjorn Olsen (Jira)
Bjorn Olsen created AIRFLOW-5824: Summary: Add a new hook and operator for AWS DataSync Key: AIRFLOW-5824 URL: https://issues.apache.org/jira/browse/AIRFLOW-5824 Project: Apache Airflow

[GitHub] [airflow] feluelle commented on issue #6432: [AIRFLOW-5758] Support the custom cursor classes for the PostgreSQL hook

2019-10-31 Thread GitBox
feluelle commented on issue #6432: [AIRFLOW-5758] Support the custom cursor classes for the PostgreSQL hook URL: https://github.com/apache/airflow/pull/6432#issuecomment-548444377 Hey @mingrammer do you also want to add a note to the class docstring about the functionality you added (like

[GitHub] [airflow] feluelle commented on issue #6269: Use poke_interval of 0 to make EMR tests run green [AIRFLOW-3897]

2019-10-31 Thread GitBox
feluelle commented on issue #6269: Use poke_interval of 0 to make EMR tests run green [AIRFLOW-3897] URL: https://github.com/apache/airflow/pull/6269#issuecomment-548430817 @mik-laj does this PR fixes your ticket [AIRFLOW-3897](https://issues.apache.org/jira/browse/AIRFLOW-3897)? Or are

[GitHub] [airflow] codecov-io edited a comment on issue #6473: [AIRFLOW-5819] Update AWSBatchOperator default value

2019-10-31 Thread GitBox
codecov-io edited a comment on issue #6473: [AIRFLOW-5819] Update AWSBatchOperator default value URL: https://github.com/apache/airflow/pull/6473#issuecomment-548057865 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6473?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6473: [AIRFLOW-5819] Update AWSBatchOperator default value

2019-10-31 Thread GitBox
codecov-io edited a comment on issue #6473: [AIRFLOW-5819] Update AWSBatchOperator default value URL: https://github.com/apache/airflow/pull/6473#issuecomment-548057865 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6473?src=pr=h1) Report > Merging

[jira] [Created] (AIRFLOW-5823) Add a new hook for ingesting data into Apache Pinot

2019-10-31 Thread Kengo Seki (Jira)
Kengo Seki created AIRFLOW-5823: --- Summary: Add a new hook for ingesting data into Apache Pinot Key: AIRFLOW-5823 URL: https://issues.apache.org/jira/browse/AIRFLOW-5823 Project: Apache Airflow

[GitHub] [airflow] OmerJog commented on issue #6007: Improved and Fixed Apache Glue hook/operator/sensor from [AIRFLOW-2310]

2019-10-31 Thread GitBox
OmerJog commented on issue #6007: Improved and Fixed Apache Glue hook/operator/sensor from [AIRFLOW-2310] URL: https://github.com/apache/airflow/pull/6007#issuecomment-548424831 @abdulbasitds https://help.github.com/en/github/using-git/about-git-rebase

[GitHub] [airflow] yuqian90 commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG

2019-10-31 Thread GitBox
yuqian90 commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG URL: https://github.com/apache/airflow/pull/6392#issuecomment-548423260 > You could separate them into 2 DAGs, and use TriggerDagRunOperator after task K in DAG 1 to trigger DAG 2 after

[GitHub] [airflow] kaxil merged pull request #6448: [AIRFLOW-XXX] Helpful list of do_xcom - affected operators added.

2019-10-31 Thread GitBox
kaxil merged pull request #6448: [AIRFLOW-XXX] Helpful list of do_xcom - affected operators added. URL: https://github.com/apache/airflow/pull/6448 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] codecov-io commented on issue #6476: WIP [AIRFLOW-5743] Move Google PubSub to providers package

2019-10-31 Thread GitBox
codecov-io commented on issue #6476: WIP [AIRFLOW-5743] Move Google PubSub to providers package URL: https://github.com/apache/airflow/pull/6476#issuecomment-548412103 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6476?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6476: WIP [AIRFLOW-5743] Move Google PubSub to providers package

2019-10-31 Thread GitBox
codecov-io edited a comment on issue #6476: WIP [AIRFLOW-5743] Move Google PubSub to providers package URL: https://github.com/apache/airflow/pull/6476#issuecomment-548412103 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6476?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6476: WIP [AIRFLOW-5743] Move Google PubSub to providers package

2019-10-31 Thread GitBox
codecov-io edited a comment on issue #6476: WIP [AIRFLOW-5743] Move Google PubSub to providers package URL: https://github.com/apache/airflow/pull/6476#issuecomment-548412103 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6476?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?

2019-10-31 Thread Vince (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964094#comment-16964094 ] Vince commented on AIRFLOW-4470: I think we're well off track of what this issue is about, but I

[GitHub] [airflow] kaxil commented on a change in pull request #6460: [AIRFLOW-4691](https://issues.apache.org/jira/browse/AIRFLOW-4691) Make tests/cli Pylint compatible

2019-10-31 Thread GitBox
kaxil commented on a change in pull request #6460: [AIRFLOW-4691](https://issues.apache.org/jira/browse/AIRFLOW-4691) Make tests/cli Pylint compatible URL: https://github.com/apache/airflow/pull/6460#discussion_r341175356 ## File path: tests/cli/test_cli.py ## @@ -79,6

[GitHub] [airflow] kaxil commented on a change in pull request #6460: [AIRFLOW-4691](https://issues.apache.org/jira/browse/AIRFLOW-4691) Make tests/cli Pylint compatible

2019-10-31 Thread GitBox
kaxil commented on a change in pull request #6460: [AIRFLOW-4691](https://issues.apache.org/jira/browse/AIRFLOW-4691) Make tests/cli Pylint compatible URL: https://github.com/apache/airflow/pull/6460#discussion_r341175356 ## File path: tests/cli/test_cli.py ## @@ -79,6

[GitHub] [airflow] kaxil commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG

2019-10-31 Thread GitBox
kaxil commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG URL: https://github.com/apache/airflow/pull/6392#issuecomment-548398662 You could separate them into 2 DAGs, and use TriggerDagRunOperator after task K in DAG 1 to trigger DAG 2 after sensor

[GitHub] [airflow-site] kgabryje opened a new pull request #106: Roadmap

2019-10-31 Thread GitBox
kgabryje opened a new pull request #106: Roadmap URL: https://github.com/apache/airflow-site/pull/106 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] kaxil commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread GitBox
kaxil commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator URL: https://github.com/apache/airflow/pull/6352#discussion_r341156888 ## File path: tests/operators/test_subdag_operator.py ## @@ -237,3 +240,65 @@ def

[GitHub] [airflow] yuqian90 commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG

2019-10-31 Thread GitBox
yuqian90 commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG URL: https://github.com/apache/airflow/pull/6392#issuecomment-548385907 > TBH it feels wrong if you have to clear tasks state on a regular interval! If you have to do that it means you are

[GitHub] [airflow] feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#discussion_r341146539 ## File path: tests/hooks/test_presto_hook.py ## @@ -51,3 +51,39 @@ def test_insert_rows(self,

[GitHub] [airflow] feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#discussion_r341146121 ## File path: tests/hooks/test_presto_hook.py ## @@ -51,3 +51,39 @@ def test_insert_rows(self,

[GitHub] [airflow] feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#discussion_r341146585 ## File path: tests/hooks/test_presto_hook.py ## @@ -51,3 +51,39 @@ def test_insert_rows(self,

[GitHub] [airflow] feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#discussion_r341144676 ## File path: tests/hooks/test_presto_hook.py ## @@ -51,3 +51,39 @@ def test_insert_rows(self,

[GitHub] [airflow] feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6467: [AIRFLOW-5814] Implementing Presto hook tests URL: https://github.com/apache/airflow/pull/6467#discussion_r341145830 ## File path: tests/hooks/test_presto_hook.py ## @@ -51,3 +51,39 @@ def test_insert_rows(self,

[GitHub] [airflow] kimpers commented on a change in pull request #6241: [AIRFLOW-3014] Increase connection password length to match model

2019-10-31 Thread GitBox
kimpers commented on a change in pull request #6241: [AIRFLOW-3014] Increase connection password length to match model URL: https://github.com/apache/airflow/pull/6241#discussion_r341138651 ## File path: airflow/migrations/versions/37c6b55d15a9_merge_migration_heads.py ##

[GitHub] [airflow] codecov-io edited a comment on issue #6241: [AIRFLOW-3014] Increase connection password length to match model

2019-10-31 Thread GitBox
codecov-io edited a comment on issue #6241: [AIRFLOW-3014] Increase connection password length to match model URL: https://github.com/apache/airflow/pull/6241#issuecomment-537896164 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6241?src=pr=h1) Report > Merging

[GitHub] [airflow] Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation

2019-10-31 Thread GitBox
Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation URL: https://github.com/apache/airflow/pull/6461#discussion_r341137279 ## File path: airflow/models/xcom.py ## @@ -124,85 +122,70 @@ def set( @classmethod

[GitHub] [airflow] codecov-io edited a comment on issue #6241: [AIRFLOW-3014] Increase connection password length to match model

2019-10-31 Thread GitBox
codecov-io edited a comment on issue #6241: [AIRFLOW-3014] Increase connection password length to match model URL: https://github.com/apache/airflow/pull/6241#issuecomment-537896164 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6241?src=pr=h1) Report > Merging

[GitHub] [airflow] feluelle commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator URL: https://github.com/apache/airflow/pull/6352#discussion_r341128146 ## File path: tests/operators/test_subdag_operator.py ## @@ -237,3 +240,65 @@ def

[GitHub] [airflow] feluelle commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator URL: https://github.com/apache/airflow/pull/6352#discussion_r341123900 ## File path: tests/operators/test_subdag_operator.py ## @@ -237,3 +240,65 @@ def

[GitHub] [airflow] kaxil commented on issue #6473: [AIRFLOW-5819] Update AWSBatchOperator default value

2019-10-31 Thread GitBox
kaxil commented on issue #6473: [AIRFLOW-5819] Update AWSBatchOperator default value URL: https://github.com/apache/airflow/pull/6473#issuecomment-548361496 Can you fix the failing tests too: ``` == 7) FAIL:

[GitHub] [airflow] Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation

2019-10-31 Thread GitBox
Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation URL: https://github.com/apache/airflow/pull/6461#discussion_r341119840 ## File path: tests/models/test_taskinstance.py ## @@ -869,7 +869,7 @@ def test_xcom_pull(self):

[GitHub] [airflow] Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation

2019-10-31 Thread GitBox
Fokko commented on a change in pull request #6461: [AIRFLOW-5804] Batch the xcom pull operation URL: https://github.com/apache/airflow/pull/6461#discussion_r341119840 ## File path: tests/models/test_taskinstance.py ## @@ -869,7 +869,7 @@ def test_xcom_pull(self):

[GitHub] [airflow] kaxil merged pull request #6475: [AIRFLOW-XXX]Fix typo in "airflow dags list_runs"

2019-10-31 Thread GitBox
kaxil merged pull request #6475: [AIRFLOW-XXX]Fix typo in "airflow dags list_runs" URL: https://github.com/apache/airflow/pull/6475 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] kaxil commented on issue #6478: [AIRFLOW-XXXX] Expose SQLAlchemy's connect_args and make it configurable

2019-10-31 Thread GitBox
kaxil commented on issue #6478: [AIRFLOW-] Expose SQLAlchemy's connect_args and make it configurable URL: https://github.com/apache/airflow/pull/6478#issuecomment-548358845 This can already be done using `db = create_engine('postgresql://scott:tiger@localhost/test?argument1=foo=bar')`

[GitHub] [airflow] kaxil commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG

2019-10-31 Thread GitBox
kaxil commented on issue #6392: [AIRFLOW-5648] Add ClearTaskOperator for clearing tasks in a DAG URL: https://github.com/apache/airflow/pull/6392#issuecomment-548357313 TBH it feels wrong if you have to clear tasks state on a regular interval! If you have to do that it means you are doing

[GitHub] [airflow] elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE]

2019-10-31 Thread GitBox
elukey commented on issue #5659: [AIRFLOW-5033] Switched to snakebite-py3 [DO NOT MERGE] URL: https://github.com/apache/airflow/pull/5659#issuecomment-548348172 > Well, there's barely anything going on: https://github.com/internetarchive/snakebite-py3/commits/master > > We can move

[GitHub] [airflow] kaxil commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread GitBox
kaxil commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator URL: https://github.com/apache/airflow/pull/6352#discussion_r341100793 ## File path: tests/operators/test_subdag_operator.py ## @@ -237,3 +240,65 @@ def

[GitHub] [airflow] feluelle commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread GitBox
feluelle commented on a change in pull request #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator URL: https://github.com/apache/airflow/pull/6352#discussion_r341083815 ## File path: airflow/operators/subdag_operator.py ## @@ -44,47 +56,58 @@ class

[jira] [Updated] (AIRFLOW-5683) Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread Felix Uellendall (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Uellendall updated AIRFLOW-5683: -- Summary: Add propagate_skipped_state to SubDagOperator (was: Add SubDagOperator

[jira] [Updated] (AIRFLOW-5683) Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread Felix Uellendall (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Uellendall updated AIRFLOW-5683: -- Description: Currently there is no way of telling the parent dag of a sub dag that

[jira] [Comment Edited] (AIRFLOW-4719) Scheduler: 'airflow scheduler' fails to make opt directory

2019-10-31 Thread Alan (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963863#comment-16963863 ] Alan edited comment on AIRFLOW-4719 at 10/31/19 10:59 AM: -- I'm currently using

[GitHub] [airflow] feluelle commented on issue #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator

2019-10-31 Thread GitBox
feluelle commented on issue #6352: [AIRFLOW-5683] Add propagate_skipped_state to SubDagOperator URL: https://github.com/apache/airflow/pull/6352#issuecomment-548315998 I updated the code accordingly. PTAL @kaxil @milton0825 (tests should pass - it breezed successfully on my local

[jira] [Comment Edited] (AIRFLOW-4719) Scheduler: 'airflow scheduler' fails to make opt directory

2019-10-31 Thread Alan (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963863#comment-16963863 ] Alan edited comment on AIRFLOW-4719 at 10/31/19 10:57 AM: -- I'm currently using

[jira] [Comment Edited] (AIRFLOW-4719) Scheduler: 'airflow scheduler' fails to make opt directory

2019-10-31 Thread Alan (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963863#comment-16963863 ] Alan edited comment on AIRFLOW-4719 at 10/31/19 10:57 AM: -- I'm currently using

  1   2   >