[GitHub] [airflow] turbaszek commented on a change in pull request #8954: Wait for pipeline state in Data Fusion operators

2020-05-27 Thread GitBox
turbaszek commented on a change in pull request #8954: URL: https://github.com/apache/airflow/pull/8954#discussion_r430910630 ## File path: airflow/providers/google/cloud/hooks/datafusion.py ## @@ -435,20 +435,52 @@ def _get_pipeline( "workflows", "Dat

[GitHub] [airflow] swecooo commented on pull request #9019: Add script_args for S3FileTransformOperator

2020-05-27 Thread GitBox
swecooo commented on pull request #9019: URL: https://github.com/apache/airflow/pull/9019#issuecomment-634493060 Thank you for the review @feluelle, the comment should now be acknowledged. The only problem is now the failing build but the same error occurs on `master`. If it's OK, fe

[GitHub] [airflow] potiuk commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
potiuk commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634497097 You can mark it as ready for review to merge it @wittfabian - or you might try to extend Breeze with an option to add --additional-python-deps flag if you are brave enough :) -

[GitHub] [airflow] wittfabian commented on issue #8866: Additional extras

2020-05-27 Thread GitBox
wittfabian commented on issue #8866: URL: https://github.com/apache/airflow/issues/8866#issuecomment-634517765 Which name is better? "ADDITIONAL_EXTRAS" or "AIRLOW_ADDITIONAL_EXTRAS" This is an automated message from the

[GitHub] [airflow] turbaszek commented on a change in pull request #8922: Add query count test for LocalTaskJob

2020-05-27 Thread GitBox
turbaszek commented on a change in pull request #8922: URL: https://github.com/apache/airflow/pull/8922#discussion_r430958055 ## File path: tests/jobs/test_local_task_job.py ## @@ -406,3 +409,23 @@ def success_callback(context): self.assertTrue(data['called'])

[GitHub] [airflow] wittfabian commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
wittfabian commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634526550 I have not yet worked with Breeze, but I would like to see the result ;) I am already working on https://github.com/apache/airflow/issues/8866. ---

[GitHub] [airflow] wittfabian opened a new pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
wittfabian opened a new pull request #9032: URL: https://github.com/apache/airflow/pull/9032 **Description** It would be great to be able to pass "ADDITIONAL_EXTRAS" build arg and add extras to the "base" extras of Production Docker image. **Use case / motivation** If s

[GitHub] [airflow] albertocalderari commented on issue #8903: BigQueryHook refactor + deterministic BQ Job ID

2020-05-27 Thread GitBox
albertocalderari commented on issue #8903: URL: https://github.com/apache/airflow/issues/8903#issuecomment-634528875 @turbaszek Hey, sorry work was hell this past week and did not find the time or the energy to reply :( I have noticed that inner job does that, and that we'd be building t

[GitHub] [airflow] mik-laj commented on issue #8649: Add support for more than 1 cron exp per DAG

2020-05-27 Thread GitBox
mik-laj commented on issue #8649: URL: https://github.com/apache/airflow/issues/8649#issuecomment-634531019 @mdediana We had long discussions about whether to support multiple scheduler intervals. Many people think that this can affect the presentation and readability of the collected dat

[GitHub] [airflow] potiuk commented on issue #8866: Additional extras

2020-05-27 Thread GitBox
potiuk commented on issue #8866: URL: https://github.com/apache/airflow/issues/8866#issuecomment-634536103 I used to use AIRFLOW everywher, but ... AIRFLOW is the whole context so we know those are AIRFLOW extras :) This is

[GitHub] [airflow] swecooo edited a comment on pull request #9019: Add script_args for S3FileTransformOperator

2020-05-27 Thread GitBox
swecooo edited a comment on pull request #9019: URL: https://github.com/apache/airflow/pull/9019#issuecomment-634493060 Thank you for the review @feluelle, the comment should now be acknowledged. The only remaining problem is the failing build but the same error occurs on `master`.

[GitHub] [airflow] potiuk commented on a change in pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
potiuk commented on a change in pull request #9032: URL: https://github.com/apache/airflow/pull/9032#discussion_r430981688 ## File path: Dockerfile ## @@ -166,7 +167,8 @@ ARG AIRFLOW_VERSION ENV AIRFLOW_VERSION=${AIRFLOW_VERSION} ARG AIRFLOW_EXTRAS -ENV AIRFLOW_EXTRAS=${AIR

[GitHub] [airflow] potiuk commented on a change in pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
potiuk commented on a change in pull request #9032: URL: https://github.com/apache/airflow/pull/9032#discussion_r430981688 ## File path: Dockerfile ## @@ -166,7 +167,8 @@ ARG AIRFLOW_VERSION ENV AIRFLOW_VERSION=${AIRFLOW_VERSION} ARG AIRFLOW_EXTRAS -ENV AIRFLOW_EXTRAS=${AIR

[GitHub] [airflow] potiuk commented on pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
potiuk commented on pull request #9032: URL: https://github.com/apache/airflow/pull/9032#issuecomment-634543457 Nice! As mentioned before - those should also be added as flags in Breeze, but it can be done separately unless you want to do it @wittfabian ! -

[GitHub] [airflow] kaxil commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r430985695 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [airflow] kaxil commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r430988478 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [airflow] olchas commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r430993095 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :para

[GitHub] [airflow] olchas commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r430993095 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :para

[GitHub] [airflow] wittfabian commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
wittfabian commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634552807 @potiuk Must the flag be added here? https://github.com/apache/airflow/blob/master/breeze#L770 This is an

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
boring-cyborg[bot] commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634554520 Awesome work, congrats on your first merged pull request! This is an automated message from the A

[GitHub] [airflow] olchas commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r430996595 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :para

[GitHub] [airflow] potiuk commented on pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
potiuk commented on pull request #9031: URL: https://github.com/apache/airflow/pull/9031#issuecomment-634554402 There are a few places - breeze-complete and breeze are the two scripts. You can just search for another flag and do it similarly, or I can add it later after your PRs are merged

[GitHub] [airflow] potiuk merged pull request #9031: Add ADDITIONAL_PYTHON_DEPS

2020-05-27 Thread GitBox
potiuk merged pull request #9031: URL: https://github.com/apache/airflow/pull/9031 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[airflow] branch master updated (0b0e4f7 -> 6fc555d)

2020-05-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 0b0e4f7 Preparing for RC3 relase of backports (#9026) add 6fc555d Add ADDITIONAL_PYTHON_DEPS (#9031) No new

[GitHub] [airflow] turbaszek commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
turbaszek commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r431013108 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :p

[GitHub] [airflow] turbaszek commented on a change in pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
turbaszek commented on a change in pull request #9018: URL: https://github.com/apache/airflow/pull/9018#discussion_r431013369 ## File path: airflow/jobs/scheduler_job.py ## @@ -1440,94 +1436,89 @@ def _change_state_for_tasks_failed_to_execute(self, session=None): :p

[GitHub] [airflow] kaxil commented on pull request #7232: [AIRFLOW-6569] Flush pending Sentry exceptions before exiting forked process

2020-05-27 Thread GitBox
kaxil commented on pull request #7232: URL: https://github.com/apache/airflow/pull/7232#issuecomment-634580331 Yes can be included in 1.10.11 This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [airflow] mik-laj commented on pull request #8942: #8525 Add SQL Branch Operator

2020-05-27 Thread GitBox
mik-laj commented on pull request #8942: URL: https://github.com/apache/airflow/pull/8942#issuecomment-634582759 @samuelkhtu we can do it in a separate PR. This will allow us to build a better git history. This is an autom

[jira] [Commented] (AIRFLOW-6569) Broken sentry integration

2020-05-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117639#comment-17117639 ] ASF GitHub Bot commented on AIRFLOW-6569: - kaxil commented on pull request #723

[jira] [Commented] (AIRFLOW-6569) Broken sentry integration

2020-05-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117645#comment-17117645 ] ASF GitHub Bot commented on AIRFLOW-6569: - ad-m commented on pull request #7232

[GitHub] [airflow] ad-m commented on pull request #7232: [AIRFLOW-6569] Flush pending Sentry exceptions before exiting forked process

2020-05-27 Thread GitBox
ad-m commented on pull request #7232: URL: https://github.com/apache/airflow/pull/7232#issuecomment-634583299 @kaxil , thank you for information and add to milestone "Airflow 1.10.11". This is an automated message from the Ap

[GitHub] [airflow] olchas commented on pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas commented on pull request #9018: URL: https://github.com/apache/airflow/pull/9018#issuecomment-634583902 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] olchas removed a comment on pull request #9018: Improve SchedulerJob code style

2020-05-27 Thread GitBox
olchas removed a comment on pull request #9018: URL: https://github.com/apache/airflow/pull/9018#issuecomment-634583902 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [airflow] potiuk merged pull request #9032: Add ADDITIONAL_AIRFLOW_EXTRAS

2020-05-27 Thread GitBox
potiuk merged pull request #9032: URL: https://github.com/apache/airflow/pull/9032 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[airflow] branch master updated (6fc555d -> 5a7a3d1)

2020-05-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 6fc555d Add ADDITIONAL_PYTHON_DEPS (#9031) add 5a7a3d1 Add ADDITIONAL_AIRFLOW_EXTRAS (#9032) No new revision

[GitHub] [airflow] mik-laj merged pull request #8897: Filter dags by clicking on tag

2020-05-27 Thread GitBox
mik-laj merged pull request #8897: URL: https://github.com/apache/airflow/pull/8897 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] ad-m commented on pull request #9023: Add snowflake to slack operator

2020-05-27 Thread GitBox
ad-m commented on pull request #9023: URL: https://github.com/apache/airflow/pull/9023#issuecomment-634591770 @potiuk , could you confirm that TravisCI fail is not related to that PR? This is an automated message from the Apa

[airflow] branch master updated (5a7a3d1 -> 30b12a9)

2020-05-27 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 5a7a3d1 Add ADDITIONAL_AIRFLOW_EXTRAS (#9032) add 30b12a9 Filter dags by clicking on tag (#8897) No ne

[GitHub] [airflow] mik-laj closed issue #8821: Add ability to filter dag list by clicking on specific tag

2020-05-27 Thread GitBox
mik-laj closed issue #8821: URL: https://github.com/apache/airflow/issues/8821 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #8897: Filter dags by clicking on tag

2020-05-27 Thread GitBox
boring-cyborg[bot] commented on pull request #8897: URL: https://github.com/apache/airflow/pull/8897#issuecomment-634591805 Awesome work, congrats on your first merged pull request! This is an automated message from the A

[GitHub] [airflow] mik-laj commented on issue #8970: Improve guide for KubernetesPodOperator

2020-05-27 Thread GitBox
mik-laj commented on issue #8970: URL: https://github.com/apache/airflow/issues/8970#issuecomment-634592478 @tanjinP Fantastic! I am waiting for your contribution. This is an automated message from the Apache Git Service. To

[GitHub] [airflow] ashb merged pull request #6311: [AIRFLOW-5615] Reduce duplicated logic around job heartbeating

2020-05-27 Thread GitBox
ashb merged pull request #6311: URL: https://github.com/apache/airflow/pull/6311 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[airflow] branch master updated (30b12a9 -> 8ac90b0)

2020-05-27 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 30b12a9 Filter dags by clicking on tag (#8897) add 8ac90b0 [AIRFLOW-5615] Reduce duplicated logic around job he

[jira] [Commented] (AIRFLOW-5615) BaseJob subclasses shouldn't implement own heartbeat logic

2020-05-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117663#comment-17117663 ] ASF GitHub Bot commented on AIRFLOW-5615: - ashb merged pull request #6311: URL:

[jira] [Commented] (AIRFLOW-5615) BaseJob subclasses shouldn't implement own heartbeat logic

2020-05-27 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117664#comment-17117664 ] ASF subversion and git services commented on AIRFLOW-5615: -- Co

[GitHub] [airflow] BasPH commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
BasPH commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431044194 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [airflow] BasPH commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
BasPH commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431044194 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [airflow] stijndehaes commented on issue #8963: SparkSubmitOperator could not get Exit Code after log stream interrupted by k8s old resource version exception

2020-05-27 Thread GitBox
stijndehaes commented on issue #8963: URL: https://github.com/apache/airflow/issues/8963#issuecomment-634596428 @ywan2017 I also have a PR open on airflow to work with spark 3.0 https://github.com/apache/airflow/pull/8730 Th

[GitHub] [airflow-site] mik-laj commented on pull request #268: adding a logo for sift use case

2020-05-27 Thread GitBox
mik-laj commented on pull request #268: URL: https://github.com/apache/airflow-site/pull/268#issuecomment-634597352 @mschickensoup Do you have time to look at it? This is an automated message from the Apache Git Service. To

[GitHub] [airflow] kaxil commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431046930 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. S

[GitHub] [airflow] mik-laj commented on pull request #9023: Add snowflake to slack operator

2020-05-27 Thread GitBox
mik-laj commented on pull request #9023: URL: https://github.com/apache/airflow/pull/9023#issuecomment-634600193 @ad-m We have temporary problems with Kubernetes tests running on Travis Ci. This change does not apply to Kubernetes, so it is safe. @dimberman is working on improvements for

[GitHub] [airflow] boring-cyborg[bot] commented on issue #9033: entrypoint cannot enter bash shell

2020-05-27 Thread GitBox
boring-cyborg[bot] commented on issue #9033: URL: https://github.com/apache/airflow/issues/9033#issuecomment-634600571 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] zikun commented on issue #9033: entrypoint cannot enter bash shell

2020-05-27 Thread GitBox
zikun commented on issue #9033: URL: https://github.com/apache/airflow/issues/9033#issuecomment-634600902 Before I make a PR to fix it, can someone confirm this is the expected behaviour? This is an automated message from th

[GitHub] [airflow] zikun opened a new issue #9033: entrypoint cannot enter bash shell

2020-05-27 Thread GitBox
zikun opened a new issue #9033: URL: https://github.com/apache/airflow/issues/9033 **Apache Airflow version**: 1.10.10 / master **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): **Environment**: Production Docker image *

[GitHub] [airflow] zikun edited a comment on issue #9033: entrypoint cannot enter bash shell

2020-05-27 Thread GitBox
zikun edited a comment on issue #9033: URL: https://github.com/apache/airflow/issues/9033#issuecomment-634600902 Before I make a PR to fix it, can someone confirm entering bash shell is the expected behaviour? This is an aut

[GitHub] [airflow] turbaszek opened a new issue #9034: Fix GoogleDiscoveryApiHook docs warnings

2020-05-27 Thread GitBox
turbaszek opened a new issue #9034: URL: https://github.com/apache/airflow/issues/9034 **Apache Airflow version**: master branch **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): N/A **Environment**: - breeze **What happened**:

[GitHub] [airflow-site] mschickensoup commented on pull request #268: adding a logo for sift use case

2020-05-27 Thread GitBox
mschickensoup commented on pull request #268: URL: https://github.com/apache/airflow-site/pull/268#issuecomment-634605066 I should be available to take a look at it around Friday, would it work? @mik-laj This is an automate

[GitHub] [airflow] mik-laj commented on issue #9034: Fix GoogleDiscoveryApiHook docs warnings

2020-05-27 Thread GitBox
mik-laj commented on issue #9034: URL: https://github.com/apache/airflow/issues/9034#issuecomment-634606513 Can you remove _api and _build directory? That should solve this problem. This is an automated message from the Apa

[GitHub] [airflow] feluelle commented on a change in pull request #8962: [AIRFLOW-8057] [AIP-31] Add @task decorator

2020-05-27 Thread GitBox
feluelle commented on a change in pull request #8962: URL: https://github.com/apache/airflow/pull/8962#discussion_r431035377 ## File path: airflow/operators/python.py ## @@ -145,6 +147,141 @@ def execute_callable(self): return self.python_callable(*self.op_args, **self

[GitHub] [airflow] zikun commented on issue #8872: Add dependencies on build image

2020-05-27 Thread GitBox
zikun commented on issue #8872: URL: https://github.com/apache/airflow/issues/8872#issuecomment-634616072 This could be a nice feature but looks like there was an effort to remove `ONBUILD` from official images - https://github.com/docker-library/official-images/issues/2076 As the a

[GitHub] [airflow] potiuk commented on issue #9033: entrypoint cannot enter bash shell

2020-05-27 Thread GitBox
potiuk commented on issue #9033: URL: https://github.com/apache/airflow/issues/9033#issuecomment-634630189 For 1.10.10 it was like that but it's been fixed in master and will be cherry-picked for 1.10.11 This is an automated

[GitHub] [airflow] potiuk closed issue #9033: entrypoint cannot enter bash shell

2020-05-27 Thread GitBox
potiuk closed issue #9033: URL: https://github.com/apache/airflow/issues/9033 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [airflow] potiuk commented on issue #8872: Add dependencies on build image

2020-05-27 Thread GitBox
potiuk commented on issue #8872: URL: https://github.com/apache/airflow/issues/8872#issuecomment-634664427 Yeah, I see the point. I think also maybe we should not do onbuild if we can add easily build args to supplement that. ---

[GitHub] [airflow] potiuk commented on pull request #9035: Additional python extras and deps can be set in breeze

2020-05-27 Thread GitBox
potiuk commented on pull request #9035: URL: https://github.com/apache/airflow/pull/9035#issuecomment-634665722 CC @wittfabian This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [airflow] potiuk opened a new pull request #9035: Additional python extras and deps can be set in breeze

2020-05-27 Thread GitBox
potiuk opened a new pull request #9035: URL: https://github.com/apache/airflow/pull/9035 Closes #8604 Closes #8866 --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for changes

[GitHub] [airflow] kaxil commented on a change in pull request #8227: Add run_type to DagRun

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8227: URL: https://github.com/apache/airflow/pull/8227#discussion_r431137238 ## File path: airflow/models/dag.py ## @@ -1468,14 +1472,26 @@ def create_dagrun(self, :param session: database session :type session: sql

[GitHub] [airflow] kaxil commented on a change in pull request #8227: Add run_type to DagRun

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8227: URL: https://github.com/apache/airflow/pull/8227#discussion_r431140537 ## File path: airflow/models/dagrun.py ## @@ -54,25 +54,27 @@ class DagRun(Base, LoggingMixin): _state = Column('state', String(50), default=State.RUNN

[GitHub] [airflow] boring-cyborg[bot] commented on issue #9036: Task instance log_filepath doesn't include try_number

2020-05-27 Thread GitBox
boring-cyborg[bot] commented on issue #9036: URL: https://github.com/apache/airflow/issues/9036#issuecomment-634671177 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] Pymancer opened a new issue #9036: Task instance log_filepath doesn't include try_number

2020-05-27 Thread GitBox
Pymancer opened a new issue #9036: URL: https://github.com/apache/airflow/issues/9036 Is it correct that ti.log_filepath doesn't include try_number? How to get a valid path with it? https://github.com/apache/airflow/blob/8ac90b0c4fca8a89592f3939c76ecb922d93b3da/airflow/models/taskinsta

[GitHub] [airflow] mik-laj commented on a change in pull request #9030: Allow using Airflow with Flask CLI

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #9030: URL: https://github.com/apache/airflow/pull/9030#discussion_r431166489 ## File path: airflow/www/app.py ## @@ -70,6 +73,31 @@ def create_app(config=None, testing=False, app_name="Airflow"): app.json_encoder = AirflowJso

[GitHub] [airflow] joppevos opened a new pull request #9037: Create guide for Dataproc Operators

2020-05-27 Thread GitBox
joppevos opened a new pull request #9037: URL: https://github.com/apache/airflow/pull/9037 A simple guide based on the example_dataproc file. Addressing this [issue](https://github.com/apache/airflow/issues/8203) # qs - @mik-laj All the examples we have of job configurations

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431176549 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431187909 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431176549 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431176549 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] ashb commented on a change in pull request #8227: Add run_type to DagRun

2020-05-27 Thread GitBox
ashb commented on a change in pull request #8227: URL: https://github.com/apache/airflow/pull/8227#discussion_r431190655 ## File path: airflow/models/dagrun.py ## @@ -54,25 +54,27 @@ class DagRun(Base, LoggingMixin): _state = Column('state', String(50), default=State.RUNNI

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431196294 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431205011 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] potiuk merged pull request #9035: Additional python extras and deps can be set in breeze

2020-05-27 Thread GitBox
potiuk merged pull request #9035: URL: https://github.com/apache/airflow/pull/9035 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [airflow] potiuk closed issue #8604: Add ADDITIONAL_PYTHON_DEPS to the image

2020-05-27 Thread GitBox
potiuk closed issue #8604: URL: https://github.com/apache/airflow/issues/8604 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [airflow] potiuk closed issue #8866: Additional extras

2020-05-27 Thread GitBox
potiuk closed issue #8866: URL: https://github.com/apache/airflow/issues/8866 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[airflow] branch master updated (8ac90b0 -> 7386670)

2020-05-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 8ac90b0 [AIRFLOW-5615] Reduce duplicated logic around job heartbeating (#6311) add 7386670 Additional python

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431230679 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431233852 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431234192 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431234581 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431235391 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431234943 ## File path: openapi.yaml ## @@ -0,0 +1,2427 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] mik-laj commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-27 Thread GitBox
mik-laj commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r431235976 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] kaxil commented on a change in pull request #8227: Add run_type to DagRun

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8227: URL: https://github.com/apache/airflow/pull/8227#discussion_r431236903 ## File path: airflow/models/dagrun.py ## @@ -54,25 +54,27 @@ class DagRun(Base, LoggingMixin): _state = Column('state', String(50), default=State.RUNN

[GitHub] [airflow] kaxil commented on a change in pull request #8227: Add run_type to DagRun

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8227: URL: https://github.com/apache/airflow/pull/8227#discussion_r431236903 ## File path: airflow/models/dagrun.py ## @@ -54,25 +54,27 @@ class DagRun(Base, LoggingMixin): _state = Column('state', String(50), default=State.RUNN

[GitHub] [airflow] optimuspaul commented on issue #8281: CLI trigger_dag with json_client enabled raises TypeError: Object of type Pendulum is not JSON serializable

2020-05-27 Thread GitBox
optimuspaul commented on issue #8281: URL: https://github.com/apache/airflow/issues/8281#issuecomment-634746271 https://github.com/apache/airflow/pull/8384 This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] kaxil commented on a change in pull request #8227: Add run_type to DagRun

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8227: URL: https://github.com/apache/airflow/pull/8227#discussion_r431236903 ## File path: airflow/models/dagrun.py ## @@ -54,25 +54,27 @@ class DagRun(Base, LoggingMixin): _state = Column('state', String(50), default=State.RUNN

[GitHub] [airflow] kaxil commented on a change in pull request #8227: Add run_type to DagRun

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8227: URL: https://github.com/apache/airflow/pull/8227#discussion_r431236903 ## File path: airflow/models/dagrun.py ## @@ -54,25 +54,27 @@ class DagRun(Base, LoggingMixin): _state = Column('state', String(50), default=State.RUNN

[GitHub] [airflow] samuelkhtu commented on pull request #8942: #8525 Add SQL Branch Operator

2020-05-27 Thread GitBox
samuelkhtu commented on pull request #8942: URL: https://github.com/apache/airflow/pull/8942#issuecomment-634753579 > @samuelkhtu we can do it in a separate PR. This will allow us to build a better git history. Thanks @mik-laj and @eladkal , can someone help and approve this PR? I am

[GitHub] [airflow] kaxil commented on a change in pull request #8227: Add run_type to DagRun

2020-05-27 Thread GitBox
kaxil commented on a change in pull request #8227: URL: https://github.com/apache/airflow/pull/8227#discussion_r431247683 ## File path: airflow/models/dagrun.py ## @@ -54,25 +54,27 @@ class DagRun(Base, LoggingMixin): _state = Column('state', String(50), default=State.RUNN

[GitHub] [airflow] themantalope commented on issue #8649: Add support for more than 1 cron exp per DAG

2020-05-27 Thread GitBox
themantalope commented on issue #8649: URL: https://github.com/apache/airflow/issues/8649#issuecomment-634766517 @mik-laj I would recommend that the user be allow to supply a list of cron strings or cron strings with comma separation. I would then implement a object that has intern

[GitHub] [airflow] dimberman opened a new pull request #9038: Use production image for k8s tests

2020-05-27 Thread GitBox
dimberman opened a new pull request #9038: URL: https://github.com/apache/airflow/pull/9038 The CI image has become too large to load into KinD, it also only really makes sense to use the production image for integration tests --- Make sure to mark the boxes below before

[GitHub] [airflow] mdediana commented on issue #8649: Add support for more than 1 cron exp per DAG

2020-05-27 Thread GitBox
mdediana commented on issue #8649: URL: https://github.com/apache/airflow/issues/8649#issuecomment-634780827 @mik-laj Sure, I will do that, thanks. This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [airflow] dimberman commented on a change in pull request #9038: Use production image for k8s tests

2020-05-27 Thread GitBox
dimberman commented on a change in pull request #9038: URL: https://github.com/apache/airflow/pull/9038#discussion_r431297732 ## File path: scripts/ci/in_container/kubernetes/docker/rebuild_airflow_image.sh ## @@ -38,46 +38,25 @@ cp /entrypoint.sh scripts/docker/ echo echo "B

  1   2   >