[GitHub] codecov-io edited a comment on issue #3730: [AIRFLOW-2882] Add import and export for pool cli using JSON

codecov-io edited a comment on issue #3730: [AIRFLOW-2882] Add import and export for pool cli using JSON URL: https://github.com/apache/incubator-airflow/pull/3730#issuecomment-411960977 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3730?src=pr&el=h1) Report > Mergin

[jira] [Created] (AIRFLOW-3011) CLI command to output actual airflow.cfg

Victor created AIRFLOW-3011: --- Summary: CLI command to output actual airflow.cfg Key: AIRFLOW-3011 URL: https://issues.apache.org/jira/browse/AIRFLOW-3011 Project: Apache Airflow Issue Type: New Fea

[jira] [Resolved] (AIRFLOW-2991) Log path to driver output after Dataproc job is completed

[ https://issues.apache.org/jira/browse/AIRFLOW-2991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-2991. --- Resolution: Fixed Fix Version/s: 2.0.0 > Log path to driver output after Dat

[GitHub] ashb commented on a change in pull request #3841: [AIRFLOW-XXX] Fix python3 errors in dev/airflow-jira

ashb commented on a change in pull request #3841: [AIRFLOW-XXX] Fix python3 errors in dev/airflow-jira URL: https://github.com/apache/incubator-airflow/pull/3841#discussion_r215286123 ## File path: dev/airflow-jira ## @@ -134,7 +134,7 @@ def compare(target_version):

[GitHub] Fokko commented on issue #3850: [AIRFLOW-3010] added support for date format to postgres_to_gcs_operator

Fokko commented on issue #3850: [AIRFLOW-3010] added support for date format to postgres_to_gcs_operator URL: https://github.com/apache/incubator-airflow/pull/3850#issuecomment-418744213 Thanks @dgingrich for the PR. Do you think it is possible to test this? Recently we've moved to

[jira] [Commented] (AIRFLOW-3010) Postgres to BigQuery Operator does not handle the Date datatype

[ https://issues.apache.org/jira/browse/AIRFLOW-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604445#comment-16604445 ] ASF GitHub Bot commented on AIRFLOW-3010: - rlkelly opened a new pull request #3

[GitHub] rlkelly opened a new pull request #3850: [AIRFLOW-3010] added support for date format to postgres_to_gcs_operator

rlkelly opened a new pull request #3850: [AIRFLOW-3010] added support for date format to postgres_to_gcs_operator URL: https://github.com/apache/incubator-airflow/pull/3850 Make sure you have checked _all_ steps below. ### Jira - [X ] My PR addresses the following [Airflow Ji

[jira] [Created] (AIRFLOW-3010) Postgres to BigQuery Operator does not handle the Date datatype

robert kelly created AIRFLOW-3010: - Summary: Postgres to BigQuery Operator does not handle the Date datatype Key: AIRFLOW-3010 URL: https://issues.apache.org/jira/browse/AIRFLOW-3010 Project: Apache A

[GitHub] kaxil commented on issue #3848: [AIRFLOW-3008] Move Kubernetes example DAGs to contrib

kaxil commented on issue #3848: [AIRFLOW-3008] Move Kubernetes example DAGs to contrib URL: https://github.com/apache/incubator-airflow/pull/3848#issuecomment-418731918 @Fokko They will need to copy/paste this DAG into their `DAGs` directory similar to how they can use any other example d

[GitHub] kaxil edited a comment on issue #3827: [AIRFLOW-2991] Log path to driver output after Dataproc job is completed

kaxil edited a comment on issue #3827: [AIRFLOW-2991] Log path to driver output after Dataproc job is completed URL: https://github.com/apache/incubator-airflow/pull/3827#issuecomment-418730701 LGTM but can you please Squash commits ? --

[GitHub] kaxil commented on issue #3827: [AIRFLOW-2991] Log path to driver output after Dataproc job is completed

kaxil commented on issue #3827: [AIRFLOW-2991] Log path to driver output after Dataproc job is completed URL: https://github.com/apache/incubator-airflow/pull/3827#issuecomment-418730701 LGTM This is an automated message fro

[GitHub] wmorris75 commented on issue #3828: [AIRFLOW-2993] s3_to_sftp and sftp_to_s3 operators

wmorris75 commented on issue #3828: [AIRFLOW-2993] s3_to_sftp and sftp_to_s3 operators URL: https://github.com/apache/incubator-airflow/pull/3828#issuecomment-418729569 Merge conflicts should now be resolved. On Wed, Sep 5, 2018 at 8:35 AM Fokko Driesprong wrote: > Still

[GitHub] codecov-io commented on issue #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc

codecov-io commented on issue #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc URL: https://github.com/apache/incubator-airflow/pull/3849#issuecomment-418723524 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3849?src=pr&el=h1) Report

[GitHub] codecov-io edited a comment on issue #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc

codecov-io edited a comment on issue #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc URL: https://github.com/apache/incubator-airflow/pull/3849#issuecomment-418723524 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3849?src=pr&el=h1) Re

[GitHub] kaxil commented on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None'

kaxil commented on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None' URL: https://github.com/apache/incubator-airflow/pull/3846#issuecomment-418714547 @fokko @ashb I have just added a note now that we should use `None` and Not `"None"` . Can you review this again please?

[GitHub] Fokko commented on issue #3828: [AIRFLOW-2993] s3_to_sftp and sftp_to_s3 operators

Fokko commented on issue #3828: [AIRFLOW-2993] s3_to_sftp and sftp_to_s3 operators URL: https://github.com/apache/incubator-airflow/pull/3828#issuecomment-418713995 Still some merge conflicts :( @wmorris75 This is an automa

[GitHub] Fokko commented on issue #3848: [AIRFLOW-3008] Move Kubernetes example DAGs to contrib

Fokko commented on issue #3848: [AIRFLOW-3008] Move Kubernetes example DAGs to contrib URL: https://github.com/apache/incubator-airflow/pull/3848#issuecomment-418713667 Yes, I think this is a good change. How can users install the k8s example dags? ---

[GitHub] Fokko commented on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None'

Fokko commented on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None' URL: https://github.com/apache/incubator-airflow/pull/3846#issuecomment-418712659 I agree with @ashb I'd say `'None'` should not be a valid schedule.

[GitHub] Fokko commented on issue #3827: [AIRFLOW-2991] Log path to driver output after Dataproc job is completed

Fokko commented on issue #3827: [AIRFLOW-2991] Log path to driver output after Dataproc job is completed URL: https://github.com/apache/incubator-airflow/pull/3827#issuecomment-418708055 I'm okay with this. @kaxil any thoughts? -

[GitHub] codecov-io commented on issue #3827: [AIRFLOW-2991] Log path to driver output after Dataproc job is completed

codecov-io commented on issue #3827: [AIRFLOW-2991] Log path to driver output after Dataproc job is completed URL: https://github.com/apache/incubator-airflow/pull/3827#issuecomment-418707070 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3827?src=pr&el=h1) Report > M

[GitHub] fralalonde commented on issue #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc

fralalonde commented on issue #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc URL: https://github.com/apache/incubator-airflow/pull/3849#issuecomment-418707012 Good point. Updated to preserve backward compatibility with Python versions [prior to 3.3](ht

[GitHub] ashb commented on issue #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc

ashb commented on issue #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc URL: https://github.com/apache/incubator-airflow/pull/3849#issuecomment-418704669 Lets see/hope this still works on Python2 as well! :) -

[jira] [Commented] (AIRFLOW-3009) Python 3.7 import collections.abc deprecation warning

[ https://issues.apache.org/jira/browse/AIRFLOW-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604318#comment-16604318 ] ASF GitHub Bot commented on AIRFLOW-3009: - fralalonde opened a new pull request

[GitHub] fralalonde opened a new pull request #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc

fralalonde opened a new pull request #3849: [AIRFLOW-3009] Fix Python 3.7 deprecation warning for import collections abc URL: https://github.com/apache/incubator-airflow/pull/3849 This is a fix for https://issues.apache.org/jira/browse/AIRFLOW-3009 After pip-installing Airflow from s

[jira] [Issue Comment Deleted] (AIRFLOW-2797) Add ability to create Google Dataproc cluster with custom image

[ https://issues.apache.org/jira/browse/AIRFLOW-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor updated AIRFLOW-2797: --- Comment: was deleted (was: User 'ashb' has created a pull request for this issue: h

[GitHub] ashb edited a comment on issue #3602: [AIRFLOW-1729] [AIRFLOW-2729] Ignore whole directories in .airflowignore

ashb edited a comment on issue #3602: [AIRFLOW-1729] [AIRFLOW-2729] Ignore whole directories in .airflowignore URL: https://github.com/apache/incubator-airflow/pull/3602#issuecomment-418700397 D'oh. No, no way to correct it anymore! Though I was able to remove the links on the wrong Jira

[GitHub] XD-DENG commented on issue #3834: [AIRFLOW-2965] CLI tool to show the next execution datetime

XD-DENG commented on issue #3834: [AIRFLOW-2965] CLI tool to show the next execution datetime URL: https://github.com/apache/incubator-airflow/pull/3834#issuecomment-418701400 Thank you very much @r39132 for the detailed feedbacks & suggestions! Will re-commit after I address your r

[jira] [Created] (AIRFLOW-3009) Python 3.7 import collections.abc deprecation warning

Francis Lalonde created AIRFLOW-3009: Summary: Python 3.7 import collections.abc deprecation warning Key: AIRFLOW-3009 URL: https://issues.apache.org/jira/browse/AIRFLOW-3009 Project: Apache Airfl

[GitHub] ashb commented on issue #3602: [AIRFLOW-1729] [AIRFLOW-2797] Ignore whole directories in .airflowignore

ashb commented on issue #3602: [AIRFLOW-1729] [AIRFLOW-2797] Ignore whole directories in .airflowignore URL: https://github.com/apache/incubator-airflow/pull/3602#issuecomment-418700397 D'oh. No, no way to correct it anymore! ---

[GitHub] codecov-io edited a comment on issue #3841: [AIRFLOW-XXX] Fix python3 errors in dev/airflow-jira

codecov-io edited a comment on issue #3841: [AIRFLOW-XXX] Fix python3 errors in dev/airflow-jira URL: https://github.com/apache/incubator-airflow/pull/3841#issuecomment-418314621 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3841?src=pr&el=h1) Report > Merging [#384

[GitHub] codecov-io edited a comment on issue #3841: [AIRFLOW-XXX] Fix python3 errors in dev/airflow-jira

codecov-io edited a comment on issue #3841: [AIRFLOW-XXX] Fix python3 errors in dev/airflow-jira URL: https://github.com/apache/incubator-airflow/pull/3841#issuecomment-418314621 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3841?src=pr&el=h1) Report > Merging [#384

[GitHub] ashb commented on a change in pull request #3841: [AIRFLOW-XXX] Fix python3 errors in dev/airflow-jira

ashb commented on a change in pull request #3841: [AIRFLOW-XXX] Fix python3 errors in dev/airflow-jira URL: https://github.com/apache/incubator-airflow/pull/3841#discussion_r215214012 ## File path: dev/airflow-jira ## @@ -134,7 +134,7 @@ def compare(target_version):

[GitHub] ashb commented on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None'

ashb commented on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None' URL: https://github.com/apache/incubator-airflow/pull/3846#issuecomment-418677400 Is it worth issuing a deprecation warning on `"None"`? T

[GitHub] codecov-io commented on issue #3836: [WIP][Airflow-2760] Decouple DAG parsing loop from scheduler loop

codecov-io commented on issue #3836: [WIP][Airflow-2760] Decouple DAG parsing loop from scheduler loop URL: https://github.com/apache/incubator-airflow/pull/3836#issuecomment-418663581 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3836?src=pr&el=h1) Report > Merging

[GitHub] codecov-io edited a comment on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None'

codecov-io edited a comment on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None' URL: https://github.com/apache/incubator-airflow/pull/3846#issuecomment-418556212 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3846?src=pr&el=h1) Report > Merging [#3

[GitHub] codecov-io edited a comment on issue #3823: [AIRFLOW-2985] An operator for S3 object copying/deleting

codecov-io edited a comment on issue #3823: [AIRFLOW-2985] An operator for S3 object copying/deleting URL: https://github.com/apache/incubator-airflow/pull/3823#issuecomment-417296502 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3823?src=pr&el=h1) Report > Merging

[jira] [Resolved] (AIRFLOW-3002) ValueError in dataflow operators when using GCS jar or py_file

[ https://issues.apache.org/jira/browse/AIRFLOW-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-3002. - Resolution: Fixed > ValueError in dataflow operators when using GCS jar or py_file > --

[jira] [Commented] (AIRFLOW-3002) ValueError in dataflow operators when using GCS jar or py_file

[ https://issues.apache.org/jira/browse/AIRFLOW-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604136#comment-16604136 ] Kaxil Naik commented on AIRFLOW-3002: - Resolved by https://github.com/apache/incuba

[jira] [Commented] (AIRFLOW-3002) ValueError in dataflow operators when using GCS jar or py_file

[ https://issues.apache.org/jira/browse/AIRFLOW-3002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604134#comment-16604134 ] ASF GitHub Bot commented on AIRFLOW-3002: - kaxil closed pull request #3843: [AI

[GitHub] kaxil closed pull request #3843: [AIRFLOW-3002] Correctly test os.stat().st_size of local_file in goog…

kaxil closed pull request #3843: [AIRFLOW-3002] Correctly test os.stat().st_size of local_file in goog… URL: https://github.com/apache/incubator-airflow/pull/3843 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake o

[GitHub] ashb commented on a change in pull request #2488: [AIRFLOW-1342] enable S3Hook to use host from connection

ashb commented on a change in pull request #2488: [AIRFLOW-1342] enable S3Hook to use host from connection URL: https://github.com/apache/incubator-airflow/pull/2488#discussion_r215181444 ## File path: airflow/hooks/S3_hook.py ## @@ -105,7 +105,9 @@ def __init__(

[GitHub] kaxil commented on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None'

kaxil commented on issue #3846: [AIRFLOW-3006] Fix issue with schedule_interval='None' URL: https://github.com/apache/incubator-airflow/pull/3846#issuecomment-418648232 @r39132 Updated the docs to add a note and reverted the initial changes. ---

[GitHub] kaxil commented on issue #2488: [AIRFLOW-1342] enable S3Hook to use host from connection

kaxil commented on issue #2488: [AIRFLOW-1342] enable S3Hook to use host from connection URL: https://github.com/apache/incubator-airflow/pull/2488#issuecomment-418643288 @victorddiniz Can you resolve the conflicts? This is

[jira] [Assigned] (AIRFLOW-1555) Backfill job gets killed 1 hour after starting

[ https://issues.apache.org/jira/browse/AIRFLOW-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anonymous reassigned AIRFLOW-1555: -- Assignee: Shreyas Joshi > Backfill job gets killed 1 hour after starting > -

[GitHub] xnuinside edited a comment on issue #2488: [AIRFLOW-1342] enable S3Hook to use host from connection

xnuinside edited a comment on issue #2488: [AIRFLOW-1342] enable S3Hook to use host from connection URL: https://github.com/apache/incubator-airflow/pull/2488#issuecomment-418631984 @victorddiniz, you can re-trigger build by yourself, for example, with "git push --force-with-lease origin

[GitHub] xnuinside edited a comment on issue #2488: [AIRFLOW-1342] enable S3Hook to use host from connection

xnuinside edited a comment on issue #2488: [AIRFLOW-1342] enable S3Hook to use host from connection URL: https://github.com/apache/incubator-airflow/pull/2488#issuecomment-418631984 @victorddiniz, you can re-trigger build by yourself, for example, with "git push --force-with-lease origin

[GitHub] xnuinside commented on issue #2488: [AIRFLOW-1342] enable S3Hook to use host from connection

xnuinside commented on issue #2488: [AIRFLOW-1342] enable S3Hook to use host from connection URL: https://github.com/apache/incubator-airflow/pull/2488#issuecomment-418631984 @victorddiniz, you can re-trigger build by yourself, for example, with "git push --force-with-lease origin feature

[GitHub] Fokko commented on a change in pull request #3767: [AIRFLOW-2524]Add SageMaker Batch Inference

Fokko commented on a change in pull request #3767: [AIRFLOW-2524]Add SageMaker Batch Inference URL: https://github.com/apache/incubator-airflow/pull/3767#discussion_r215155757 ## File path: airflow/contrib/hooks/sagemaker_hook.py ## @@ -219,6 +219,52 @@ def create_tuning_j

[GitHub] Fokko commented on a change in pull request #3767: [AIRFLOW-2524]Add SageMaker Batch Inference

Fokko commented on a change in pull request #3767: [AIRFLOW-2524]Add SageMaker Batch Inference URL: https://github.com/apache/incubator-airflow/pull/3767#discussion_r215155183 ## File path: airflow/contrib/sensors/sagemaker_transform_sensor.py ## @@ -0,0 +1,69 @@ +# -*- co

[GitHub] Fokko commented on a change in pull request #3767: [AIRFLOW-2524]Add SageMaker Batch Inference

Fokko commented on a change in pull request #3767: [AIRFLOW-2524]Add SageMaker Batch Inference URL: https://github.com/apache/incubator-airflow/pull/3767#discussion_r215155209 ## File path: airflow/contrib/sensors/sagemaker_transform_sensor.py ## @@ -0,0 +1,69 @@ +# -*- co

[GitHub] Fokko commented on issue #3730: [AIRFLOW-2882] Add import and export for pool cli using JSON

Fokko commented on issue #3730: [AIRFLOW-2882] Add import and export for pool cli using JSON URL: https://github.com/apache/incubator-airflow/pull/3730#issuecomment-418620984 I'm fine with merging this for now. Would be awesome if we can pick up `[AIRFLOW-2929]` soon. @ChengzhiZha

<    1   2