[GitHub] [airflow] zikun opened a new issue #9033: entrypoint cannot enter bash shell

2020-05-27 Thread GitBox
zikun opened a new issue #9033: URL: https://github.com/apache/airflow/issues/9033 **Apache Airflow version**: 1.10.10 / master **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): **Environment**: Production Docker image *

[GitHub] [airflow] mik-laj commented on pull request #9023: Add snowflake to slack operator

2020-05-27 Thread GitBox
mik-laj commented on pull request #9023: URL: https://github.com/apache/airflow/pull/9023#issuecomment-634600193 @ad-m We have temporary problems with Kubernetes tests running on Travis Ci. This change does not apply to Kubernetes, so it is safe. @dimberman is working on improvements for

[GitHub] [airflow] kaxil commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431046930 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [airflow-site] mik-laj commented on pull request #268: adding a logo for sift use case

2020-05-27 Thread GitBox
mik-laj commented on pull request #268: URL: https://github.com/apache/airflow-site/pull/268#issuecomment-634597352 @mschickensoup Do you have time to look at it? This is an automated message from the Apache Git Service. To

[GitHub] [airflow] stijndehaes commented on issue #8963: SparkSubmitOperator could not get Exit Code after log stream interrupted by k8s old resource version exception

2020-05-27 Thread GitBox
stijndehaes commented on issue #8963: URL: https://github.com/apache/airflow/issues/8963#issuecomment-634596428 @ywan2017 I also have a PR open on airflow to work with spark 3.0 https://github.com/apache/airflow/pull/8730 Th

[GitHub] [airflow] BasPH commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
BasPH commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431044194 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [airflow] BasPH commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
BasPH commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431044194 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[jira] [Commented] (AIRFLOW-5615) BaseJob subclasses shouldn't implement own heartbeat logic

2020-05-27 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117664#comment-17117664 ] ASF subversion and git services commented on AIRFLOW-5615: -- Co

[airflow] branch master updated (30b12a9 -> 8ac90b0)

2020-05-27 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 30b12a9 Filter dags by clicking on tag (#8897) add 8ac90b0 [AIRFLOW-5615] Reduce duplicated logic around job he

[jira] [Commented] (AIRFLOW-5615) BaseJob subclasses shouldn't implement own heartbeat logic

2020-05-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117663#comment-17117663 ] ASF GitHub Bot commented on AIRFLOW-5615: - ashb merged pull request #6311: URL:

[GitHub] [airflow] ashb merged pull request #6311: [AIRFLOW-5615] Reduce duplicated logic around job heartbeating

2020-05-27 Thread GitBox
ashb merged pull request #6311: URL: https://github.com/apache/airflow/pull/6311 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] mik-laj commented on issue #8970: Improve guide for KubernetesPodOperator

2020-05-27 Thread GitBox
mik-laj commented on issue #8970: URL: https://github.com/apache/airflow/issues/8970#issuecomment-634592478 @tanjinP Fantastic! I am waiting for your contribution. This is an automated message from the Apache Git Service. To

[GitHub] [airflow] mik-laj closed issue #8821: Add ability to filter dag list by clicking on specific tag

2020-05-27 Thread GitBox
mik-laj closed issue #8821: URL: https://github.com/apache/airflow/issues/8821 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] mik-laj merged pull request #8897: Filter dags by clicking on tag

2020-05-27 Thread GitBox
mik-laj merged pull request #8897: URL: https://github.com/apache/airflow/pull/8897 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] ad-m commented on pull request #9023: Add snowflake to slack operator

2020-05-27 Thread GitBox
ad-m commented on pull request #9023: URL: https://github.com/apache/airflow/pull/9023#issuecomment-634591770 @potiuk , could you confirm that TravisCI fail is not related to that PR? This is an automated message from the Apa

[airflow] branch master updated (5a7a3d1 -> 30b12a9)

2020-05-27 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 5a7a3d1 Add ADDITIONAL_AIRFLOW_EXTRAS (#9032) add 30b12a9 Filter dags by clicking on tag (#8897) No ne

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8897: Filter dags by clicking on tag

2020-05-27 Thread GitBox
boring-cyborg[bot] commented on pull request #8897: URL: https://github.com/apache/airflow/pull/8897#issuecomment-634591805 Awesome work, congrats on your first merged pull request! This is an automated message from the A

[airflow] branch master updated (6fc555d -> 5a7a3d1)

2020-05-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 6fc555d Add ADDITIONAL_PYTHON_DEPS (#9031) add 5a7a3d1 Add ADDITIONAL_AIRFLOW_EXTRAS (#9032) No new revision

[GitHub] [airflow] potiuk merged pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
potiuk merged pull request #9032: URL: https://github.com/apache/airflow/pull/9032 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [airflow] olchas removed a comment on pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas removed a comment on pull request #9018: URL: https://github.com/apache/airflow/pull/9018#issuecomment-634583902 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [airflow] olchas commented on pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas commented on pull request #9018: URL: https://github.com/apache/airflow/pull/9018#issuecomment-634583902 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Commented] (AIRFLOW-6569) Broken sentry integration

2020-05-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117645#comment-17117645 ] ASF GitHub Bot commented on AIRFLOW-6569: - ad-m commented on pull request #7232

[GitHub] [airflow] ad-m commented on pull request #7232: [AIRFLOW-6569] Flush pending Sentry exceptions before exiting forked process

2020-05-27 Thread GitBox
ad-m commented on pull request #7232: URL: https://github.com/apache/airflow/pull/7232#issuecomment-634583299 @kaxil , thank you for information and add to milestone "Airflow 1.10.11". This is an automated message from the Ap

[jira] [Commented] (AIRFLOW-6569) Broken sentry integration

2020-05-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117639#comment-17117639 ] ASF GitHub Bot commented on AIRFLOW-6569: - kaxil commented on pull request #723

[GitHub] [airflow] mik-laj commented on pull request #8942: #8525 Add SQL Branch Operator

2020-05-27 Thread GitBox
mik-laj commented on pull request #8942: URL: https://github.com/apache/airflow/pull/8942#issuecomment-634582759 @samuelkhtu we can do it in a separate PR. This will allow us to build a better git history. This is an autom

[GitHub] [airflow] kaxil commented on pull request #7232: [AIRFLOW-6569] Flush pending Sentry exceptions before exiting forked process

2020-05-27 Thread GitBox
kaxil commented on pull request #7232: URL: https://github.com/apache/airflow/pull/7232#issuecomment-634580331 Yes can be included in 1.10.11 This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [airflow] turbaszek commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
turbaszek commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r431013369 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :p

[GitHub] [airflow] turbaszek commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
turbaszek commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r431013108 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :p

[airflow] branch master updated (0b0e4f7 -> 6fc555d)

2020-05-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 0b0e4f7 Preparing for RC3 relase of backports (#9026) add 6fc555d Add ADDITIONAL_PYTHON_DEPS (#9031) No new

[GitHub] [airflow] olchas commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r430996595 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :para

[GitHub] [airflow] potiuk commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
potiuk commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634554402 There are a few places - breeze-complete and breeze are the two scripts. You can just search for another flag and do it similarly, or I can add it later after your PRs are merged

[GitHub] [airflow] potiuk merged pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
potiuk merged pull request #9031: URL: https://github.com/apache/airflow/pull/9031 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
boring-cyborg[bot] commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634554520 Awesome work, congrats on your first merged pull request! This is an automated message from the A

[GitHub] [airflow] wittfabian commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
wittfabian commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634552807 @potiuk Must the flag be added here? https://github.com/apache/airflow/blob/master/breeze#L770 This is an

[GitHub] [airflow] olchas commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r430993095 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :para

[GitHub] [airflow] olchas commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r430993095 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :para

[GitHub] [airflow] kaxil commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r430988478 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [airflow] kaxil commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r430985695 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [airflow] potiuk commented on pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
potiuk commented on pull request #9032: URL: https://github.com/apache/airflow/pull/9032#issuecomment-634543457 Nice! As mentioned before - those should also be added as flags in Breeze, but it can be done separately unless you want to do it @wittfabian ! -

[GitHub] [airflow] potiuk commented on a change in pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
potiuk commented on a change in pull request #9032: URL: https://github.com/apache/airflow/pull/9032#discussion_r430981688 ## File path: Dockerfile ## @@ -166,7 +167,8 @@ ARG AIRFLOW_VERSION ENV AIRFLOW_VERSION=${AIRFLOW_VERSION} ARG AIRFLOW_EXTRAS -ENV AIRFLOW_EXTRAS=${AIR

[GitHub] [airflow] potiuk commented on a change in pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
potiuk commented on a change in pull request #9032: URL: https://github.com/apache/airflow/pull/9032#discussion_r430981688 ## File path: Dockerfile ## @@ -166,7 +167,8 @@ ARG AIRFLOW_VERSION ENV AIRFLOW_VERSION=${AIRFLOW_VERSION} ARG AIRFLOW_EXTRAS -ENV AIRFLOW_EXTRAS=${AIR

[GitHub] [airflow] swecooo edited a comment on pull request #9019: Add script_args for S3FileTransformOperator

2020-05-27 Thread GitBox
swecooo edited a comment on pull request #9019: URL: https://github.com/apache/airflow/pull/9019#issuecomment-634493060 Thank you for the review @feluelle, the comment should now be acknowledged. The only remaining problem is the failing build but the same error occurs on `master`.

[GitHub] [airflow] potiuk commented on issue #8866: Additional extras

2020-05-27 Thread GitBox
potiuk commented on issue #8866: URL: https://github.com/apache/airflow/issues/8866#issuecomment-634536103 I used to use AIRFLOW everywher, but ... AIRFLOW is the whole context so we know those are AIRFLOW extras :) This is

[GitHub] [airflow] mik-laj commented on issue #8649: Add support for more than 1 cron exp per DAG

2020-05-27 Thread GitBox
mik-laj commented on issue #8649: URL: https://github.com/apache/airflow/issues/8649#issuecomment-634531019 @mdediana We had long discussions about whether to support multiple scheduler intervals. Many people think that this can affect the presentation and readability of the collected dat

[GitHub] [airflow] albertocalderari commented on issue #8903: BigQueryHook refactor + deterministic BQ Job ID

2020-05-27 Thread GitBox
albertocalderari commented on issue #8903: URL: https://github.com/apache/airflow/issues/8903#issuecomment-634528875 @turbaszek Hey, sorry work was hell this past week and did not find the time or the energy to reply :( I have noticed that inner job does that, and that we'd be building t

[GitHub] [airflow] wittfabian opened a new pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
wittfabian opened a new pull request #9032: URL: https://github.com/apache/airflow/pull/9032 **Description** It would be great to be able to pass "ADDITIONAL_EXTRAS" build arg and add extras to the "base" extras of Production Docker image. **Use case / motivation** If s

[GitHub] [airflow] wittfabian commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
wittfabian commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634526550 I have not yet worked with Breeze, but I would like to see the result ;) I am already working on https://github.com/apache/airflow/issues/8866. ---

[GitHub] [airflow] turbaszek commented on a change in pull request #8922: Add query count test for LocalTaskJob

2020-05-27 Thread GitBox
turbaszek commented on a change in pull request #8922: URL: https://github.com/apache/airflow/pull/8922#discussion_r430958055 ## File path: tests/jobs/test_local_task_job.py ## @@ -406,3 +409,23 @@ def success_callback(context): self.assertTrue(data['called'])

[GitHub] [airflow] wittfabian commented on issue #8866: Additional extras

2020-05-27 Thread GitBox
wittfabian commented on issue #8866: URL: https://github.com/apache/airflow/issues/8866#issuecomment-634517765 Which name is better? "ADDITIONAL_EXTRAS" or "AIRLOW_ADDITIONAL_EXTRAS" This is an automated message from the

[GitHub] [airflow] potiuk commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
potiuk commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634497097 You can mark it as ready for review to merge it @wittfabian - or you might try to extend Breeze with an option to add --additional-python-deps flag if you are brave enough :) -

[GitHub] [airflow] swecooo commented on pull request #9019: Add script_args for S3FileTransformOperator

2020-05-27 Thread GitBox
swecooo commented on pull request #9019: URL: https://github.com/apache/airflow/pull/9019#issuecomment-634493060 Thank you for the review @feluelle, the comment should now be acknowledged. The only problem is now the failing build but the same error occurs on `master`. If it's OK, fe

[GitHub] [airflow] turbaszek commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-05-27 Thread GitBox
turbaszek commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r430910630 ## File path: airflow/providers/google/cloud/hooks/datafusion.py ## @@ -435,20 +435,52 @@ def _get_pipeline( "workflows", "Dat

<    1   2