[GitHub] [airflow] khyurri commented on issue #8284: Add support for Microsoft Azure Blob Storage in Google Cloud Storage Transfer Service operators

2020-09-15 Thread GitBox
khyurri commented on issue #8284: URL: https://github.com/apache/airflow/issues/8284#issuecomment-693177910 @turbaszek Sure, https://github.com/apache/airflow/pull/8818 This is an automated message from the Apache Git

[GitHub] [airflow] ephraimbuddy commented on pull request #10951: Add example dag and system test for S3ToGCSOperator

2020-09-15 Thread GitBox
ephraimbuddy commented on pull request #10951: URL: https://github.com/apache/airflow/pull/10951#issuecomment-693171348 Hi @potiuk , I can't seem to figure out this pylint error. I will appreciate your assistance . Thanks

[GitHub] [airflow] yuqian90 closed pull request #10153: [AIP-34] TaskGroup: A UI task grouping concept as an alternative to SubDagOperator

2020-09-15 Thread GitBox
yuqian90 closed pull request #10153: URL: https://github.com/apache/airflow/pull/10153 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] jhtimmins edited a comment on pull request #10594: Add permissions for stable API

2020-09-15 Thread GitBox
jhtimmins edited a comment on pull request #10594: URL: https://github.com/apache/airflow/pull/10594#issuecomment-693154312 Good catch @mik-laj. I've updated the authorization code to properly handle cases where a user has access to some dags but not all. The endpoint is also updated to

[GitHub] [airflow] jhtimmins edited a comment on pull request #10594: Add permissions for stable API

2020-09-15 Thread GitBox
jhtimmins edited a comment on pull request #10594: URL: https://github.com/apache/airflow/pull/10594#issuecomment-693154312 Good catch @mik-laj. I've updated the authorization code to properly handle cases where a user has access to some dags but not all. The endpoint is also updated to

[GitHub] [airflow] jhtimmins commented on pull request #10594: Add permissions for stable API

2020-09-15 Thread GitBox
jhtimmins commented on pull request #10594: URL: https://github.com/apache/airflow/pull/10594#issuecomment-693154312 Good catch @mik-laj. I've updated the authorization code to properly handle cases where a user has access to some dags but not all. The endpoint is also updated to only

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #10970: Add environment variables documentation to cli-ref.rst

2020-09-15 Thread GitBox
boring-cyborg[bot] commented on pull request #10970: URL: https://github.com/apache/airflow/pull/10970#issuecomment-693146665 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] faihegberg opened a new pull request #10970: Add environment variables documentation to cli-ref.rst

2020-09-15 Thread GitBox
faihegberg opened a new pull request #10970: URL: https://github.com/apache/airflow/pull/10970 This PR documents environment variables in cli-ref.rst, along with a brief description of each. This enables the user to refer to these variables from other places in the documentation.

[GitHub] [airflow] yuqian90 commented on a change in pull request #10153: [AIP-34] TaskGroup: A UI task grouping concept as an alternative to SubDagOperator

2020-09-15 Thread GitBox
yuqian90 commented on a change in pull request #10153: URL: https://github.com/apache/airflow/pull/10153#discussion_r489137031 ## File path: tests/utils/test_task_group.py ## @@ -0,0 +1,529 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] dimberman commented on issue #9532: Cannot run KubernetesPodOperator tasks.

2020-09-15 Thread GitBox
dimberman commented on issue #9532: URL: https://github.com/apache/airflow/issues/9532#issuecomment-693137662 @tuan-cs wait, what? The KubernetesExecutor can absolutely work with the KubernetesPodOperator... This is an

[airflow] tag nightly-master updated (87e7ad9 -> 22c6316)

2020-09-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from 87e7ad9 (commit) to 22c6316 (commit) from

[airflow] tag nightly-master updated (87e7ad9 -> 22c6316)

2020-09-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from 87e7ad9 (commit) to 22c6316 (commit) from

[GitHub] [airflow] tuan-cs closed issue #9532: Cannot run KubernetesPodOperator tasks.

2020-09-15 Thread GitBox
tuan-cs closed issue #9532: URL: https://github.com/apache/airflow/issues/9532 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] tuan-cs commented on issue #9532: Cannot run KubernetesPodOperator tasks.

2020-09-15 Thread GitBox
tuan-cs commented on issue #9532: URL: https://github.com/apache/airflow/issues/9532#issuecomment-693123229 I found the reason is KubernetesExecutor cannot work with KubernetesPodOperator. This is an automated message from

[GitHub] [airflow] jaketf opened a new pull request #10969: [WIP] sketch import error alert emails

2020-09-15 Thread GitBox
jaketf opened a new pull request #10969: URL: https://github.com/apache/airflow/pull/10969 This PR is a conversation piece that came from pondering how to support import error alert emails in #10968. This is a feature will be especially useful for admins who use

[GitHub] [airflow] jaketf opened a new issue #10968: import error alert emails

2020-09-15 Thread GitBox
jaketf opened a new issue #10968: URL: https://github.com/apache/airflow/issues/10968 **Description** Send alert emails on import errors. **Use case / motivation** This is a feature will be especially useful for admins who use `ClusterPolicyViolation` or those who have

[GitHub] [airflow] dimberman commented on issue #10035: Security context KubernetesPodOperator

2020-09-15 Thread GitBox
dimberman commented on issue #10035: URL: https://github.com/apache/airflow/issues/10035#issuecomment-693108795 @gardnerdev can you try doing this using a pod_template_file or a pod object and see if that remedies the issue?

[GitHub] [airflow] dimberman commented on issue #8275: Environment variable ordering for refrencing secrets with KubernetesPodOperator

2020-09-15 Thread GitBox
dimberman commented on issue #8275: URL: https://github.com/apache/airflow/issues/8275#issuecomment-693108401 @Rested It's out in 1.10.12, though some of the legacy stuff will be fixed in 1.10.13 :) This is an automated

[GitHub] [airflow] yuqian90 commented on a change in pull request #10930: Introduce TaskMixin

2020-09-15 Thread GitBox
yuqian90 commented on a change in pull request #10930: URL: https://github.com/apache/airflow/pull/10930#discussion_r489093794 ## File path: airflow/models/baseoperator.py ## @@ -1146,27 +1115,22 @@ def add_only_new(self, item_set: Set[str], item: str) -> None: else:

[GitHub] [airflow] jbampton opened a new pull request #10967: Remove redundant parentheses from Python files

2020-09-15 Thread GitBox
jbampton opened a new pull request #10967: URL: https://github.com/apache/airflow/pull/10967 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] kaxil opened a new pull request #10966: Remove redundant parentheses in serialized_objects.py

2020-09-15 Thread GitBox
kaxil opened a new pull request #10966: URL: https://github.com/apache/airflow/pull/10966 While working on another DAG Serialization issue, I found these redundant parentheses `elif isinstance(var, (Timezone)):` -> `elif isinstance(var, Timezone):` --- **^

[airflow] branch master updated (905cdd5 -> 22c6316)

2020-09-15 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 905cdd5 Add a default for DagModel.default_view (#10897) add 22c6316 Fix more docs spellings (#10965) No

[airflow] branch master updated (905cdd5 -> 22c6316)

2020-09-15 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 905cdd5 Add a default for DagModel.default_view (#10897) add 22c6316 Fix more docs spellings (#10965) No

[GitHub] [airflow] kaxil merged pull request #10965: Fix more docs spellings

2020-09-15 Thread GitBox
kaxil merged pull request #10965: URL: https://github.com/apache/airflow/pull/10965 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] kaxil opened a new pull request #10965: Fix more docs spellings

2020-09-15 Thread GitBox
kaxil opened a new pull request #10965: URL: https://github.com/apache/airflow/pull/10965 `desecending` -> `descending` `Retrives` -> `Retrieves` `programatic` -> `programmatic` `subproces` -> `subprocess` --- **^ Add meaningful description above** Read the

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #10951: Add example dag and system test for S3ToGCSOperator

2020-09-15 Thread GitBox
ephraimbuddy commented on a change in pull request #10951: URL: https://github.com/apache/airflow/pull/10951#discussion_r488910813 ## File path: docs/operators-and-hooks-ref.rst ## @@ -934,11 +934,15 @@ These integrations allow you to copy data from/to Google Cloud. -

[airflow] branch master updated (f7da7d9 -> 905cdd5)

2020-09-15 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f7da7d9 Fix ExternalTaskMarker serialized fields (#10924) add 905cdd5 Add a default for

[airflow] branch master updated (f7da7d9 -> 905cdd5)

2020-09-15 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f7da7d9 Fix ExternalTaskMarker serialized fields (#10924) add 905cdd5 Add a default for

[GitHub] [airflow] n-oden closed pull request #10958: Allow customization of navigation bar title text

2020-09-15 Thread GitBox
n-oden closed pull request #10958: URL: https://github.com/apache/airflow/pull/10958 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[airflow] branch master updated: Add a default for DagModel.default_view (#10897)

2020-09-15 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 905cdd5 Add a default for

[GitHub] [airflow] kaxil merged pull request #10897: Add a default for DagModel.default_view

2020-09-15 Thread GitBox
kaxil merged pull request #10897: URL: https://github.com/apache/airflow/pull/10897 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] kaxil closed issue #10283: Airflow Homepage is not able to render due to TypeError: must be str, not NoneType

2020-09-15 Thread GitBox
kaxil closed issue #10283: URL: https://github.com/apache/airflow/issues/10283 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] mik-laj commented on a change in pull request #10897: Add a default for DagModel.default_view

2020-09-15 Thread GitBox
mik-laj commented on a change in pull request #10897: URL: https://github.com/apache/airflow/pull/10897#discussion_r489070805 ## File path: airflow/models/dag.py ## @@ -1621,6 +1621,13 @@ def sync_to_db(self, sync_time=None, session=None): """

[GitHub] [airflow] kaxil commented on a change in pull request #10963: KubernetesPodOperator template fix

2020-09-15 Thread GitBox
kaxil commented on a change in pull request #10963: URL: https://github.com/apache/airflow/pull/10963#discussion_r489057956 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -355,24 +359,12 @@ def _set_name(self, name):

[GitHub] [airflow] pcandoalmeida commented on pull request #10162: Add Airflow UI site_title configuration option

2020-09-15 Thread GitBox
pcandoalmeida commented on pull request #10162: URL: https://github.com/apache/airflow/pull/10162#issuecomment-693019071 Hi @mik-laj I've resolved about 16 failing tests in `www/`. I'm having a strange Github issue when trying to rebase, but will update the PR soon.

[GitHub] [airflow] Rested commented on issue #8275: Environment variable ordering for refrencing secrets with KubernetesPodOperator

2020-09-15 Thread GitBox
Rested commented on issue #8275: URL: https://github.com/apache/airflow/issues/8275#issuecomment-693018138 Has that been added to a 1.x release @dimberman I remember reading https://github.com/apache/airflow/pull/6230#issuecomment-597176715 and assuming it would not? The solution I went

[airflow] branch master updated (ce19657 -> f7da7d9)

2020-09-15 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from ce19657 Fix case of GitHub. (#10955) add f7da7d9 Fix ExternalTaskMarker serialized fields (#10924) No

[airflow] branch master updated: Fix ExternalTaskMarker serialized fields (#10924)

2020-09-15 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new f7da7d9 Fix ExternalTaskMarker serialized

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #10924: Fix operator field update for SerializedBaseOperator

2020-09-15 Thread GitBox
boring-cyborg[bot] commented on pull request #10924: URL: https://github.com/apache/airflow/pull/10924#issuecomment-693016982 Awesome work, congrats on your first merged pull request! This is an automated message from

[GitHub] [airflow] kaxil merged pull request #10924: Fix operator field update for SerializedBaseOperator

2020-09-15 Thread GitBox
kaxil merged pull request #10924: URL: https://github.com/apache/airflow/pull/10924 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] kaxil closed issue #10794: ExternalTaskMarker don't work with store_serialized_dags

2020-09-15 Thread GitBox
kaxil closed issue #10794: URL: https://github.com/apache/airflow/issues/10794 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] dimberman commented on a change in pull request #10963: K8s operator pod template fix

2020-09-15 Thread GitBox
dimberman commented on a change in pull request #10963: URL: https://github.com/apache/airflow/pull/10963#discussion_r489019882 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -240,7 +240,9 @@ def __init__(self, # pylint:

[GitHub] [airflow] kaxil commented on a change in pull request #10963: K8s operator pod template fix

2020-09-15 Thread GitBox
kaxil commented on a change in pull request #10963: URL: https://github.com/apache/airflow/pull/10963#discussion_r489018068 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -240,7 +240,9 @@ def __init__(self, # pylint:

[GitHub] [airflow] dimberman commented on a change in pull request #10963: K8s operator pod template fix

2020-09-15 Thread GitBox
dimberman commented on a change in pull request #10963: URL: https://github.com/apache/airflow/pull/10963#discussion_r489016649 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -240,7 +240,9 @@ def __init__(self, # pylint:

[GitHub] [airflow] kaxil commented on a change in pull request #10897: Add a default for DagModel.default_view

2020-09-15 Thread GitBox
kaxil commented on a change in pull request #10897: URL: https://github.com/apache/airflow/pull/10897#discussion_r489009722 ## File path: airflow/models/dag.py ## @@ -1621,6 +1621,13 @@ def sync_to_db(self, sync_time=None, session=None): """

[GitHub] [airflow] kaxil commented on a change in pull request #10963: K8s operator pod template fix

2020-09-15 Thread GitBox
kaxil commented on a change in pull request #10963: URL: https://github.com/apache/airflow/pull/10963#discussion_r489009282 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -240,7 +240,9 @@ def __init__(self, # pylint:

[GitHub] [airflow] adamo57 opened a new issue #10964: No response from gunicorn master within 120 seconds After Changing Worker Class

2020-09-15 Thread GitBox
adamo57 opened a new issue #10964: URL: https://github.com/apache/airflow/issues/10964 **Apache Airflow version**: 1.10.10 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): ``` Client Version: version.Info{Major:"1", Minor:"18",

[GitHub] [airflow] kaxil commented on pull request #10924: Fix operator field update for SerializedBaseOperator

2020-09-15 Thread GitBox
kaxil commented on pull request #10924: URL: https://github.com/apache/airflow/pull/10924#issuecomment-693001303 I have updated the PR (template_fields were already written and there was a bug in last commit -- hence tests were failing) and added some tests too

[GitHub] [airflow] potiuk commented on pull request #10959: All versions in CI yamls are not hard-coded any more

2020-09-15 Thread GitBox
potiuk commented on pull request #10959: URL: https://github.com/apache/airflow/pull/10959#issuecomment-693001277 All versions in the "CI Build" and "Image Build" are now taken from the scripts - single source of truth that we have for versions: Build Image: ![Screenshot from

[GitHub] [airflow] OmairK commented on issue #10020: Re-Add try_number as identify label on a POD when using KubernetesPodOperator

2020-09-15 Thread GitBox
OmairK commented on issue #10020: URL: https://github.com/apache/airflow/issues/10020#issuecomment-692998441 > Hi @kaxil @OmairK > > I don't think this is the right way to solve this issue. > > In the case of pod failures, we WANT to reattach the first time so the failure can

[GitHub] [airflow] mik-laj commented on a change in pull request #10963: K8s operator pod template fix

2020-09-15 Thread GitBox
mik-laj commented on a change in pull request #10963: URL: https://github.com/apache/airflow/pull/10963#discussion_r488973557 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -410,12 +403,32 @@ def create_new_pod_for_operator(self, labels,

[GitHub] [airflow] mik-laj commented on a change in pull request #10963: K8s operator pod template fix

2020-09-15 Thread GitBox
mik-laj commented on a change in pull request #10963: URL: https://github.com/apache/airflow/pull/10963#discussion_r488972693 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -355,24 +359,13 @@ def _set_name(self, name):

[GitHub] [airflow] dimberman commented on issue #8275: Environment variable ordering for refrencing secrets with KubernetesPodOperator

2020-09-15 Thread GitBox
dimberman commented on issue #8275: URL: https://github.com/apache/airflow/issues/8275#issuecomment-692977054 @Rested you should be able to get around this by using the `pod` or `pod_template_file` parameters. This is an

[GitHub] [airflow] dimberman commented on issue #9532: Cannot run KubernetesPodOperator tasks.

2020-09-15 Thread GitBox
dimberman commented on issue #9532: URL: https://github.com/apache/airflow/issues/9532#issuecomment-692975590 @tuan-cs your volume and volume_mount don't match (your volume is named `intput`). Could this be the root of your problem ? can you please post task logs?

[GitHub] [airflow] pcandoalmeida commented on pull request #10162: Add Airflow UI site_title configuration option

2020-09-15 Thread GitBox
pcandoalmeida commented on pull request #10162: URL: https://github.com/apache/airflow/pull/10162#issuecomment-692975873 Sure @mik-laj doing that now. This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] pcandoalmeida commented on pull request #10958: Allow customization of navigation bar title text

2020-09-15 Thread GitBox
pcandoalmeida commented on pull request #10958: URL: https://github.com/apache/airflow/pull/10958#issuecomment-692975108 No worries at all @n-oden I'm currently reviewing the latest updates by @mik-laj so I'm hoping to get this functionality reviewed and merged in soon, I'll give you a

[GitHub] [airflow] mik-laj commented on pull request #10162: Add Airflow UI site_title configuration option

2020-09-15 Thread GitBox
mik-laj commented on pull request #10162: URL: https://github.com/apache/airflow/pull/10162#issuecomment-692973693 @pcandoalmeida I support this change, but it is helpful if you rebase your branch to the newest master and fix all failed tests.

[GitHub] [airflow] n-oden commented on pull request #10958: Allow customization of navigation bar title text

2020-09-15 Thread GitBox
n-oden commented on pull request #10958: URL: https://github.com/apache/airflow/pull/10958#issuecomment-692972732 oh, sorry, didn't realize that there was another change in-flight with the same goal. my bad for not scrutinizing the open PRs more closely! @pcandoalmeida I'm happy to

[GitHub] [airflow] pcandoalmeida commented on pull request #10162: Add Airflow UI site_title configuration option

2020-09-15 Thread GitBox
pcandoalmeida commented on pull request #10162: URL: https://github.com/apache/airflow/pull/10162#issuecomment-692970956 Thanks @mik-laj I'm reviewing your updates now; let me know how you want to proceed. Can close this issue if you want to go with the other update.

[GitHub] [airflow] dimberman commented on pull request #10062: [WIP] Try_number as identifying label on a POD when using KubernetesPodOperator

2020-09-15 Thread GitBox
dimberman commented on pull request #10062: URL: https://github.com/apache/airflow/pull/10062#issuecomment-692969705 Hi @kaxil @OmairK I don't think this is the right way to solve this issue. In the case of pod failures, we WANT to reattach the first time so the failure can

[GitHub] [airflow] dimberman commented on issue #10020: Re-Add try_number as identify label on a POD when using KubernetesPodOperator

2020-09-15 Thread GitBox
dimberman commented on issue #10020: URL: https://github.com/apache/airflow/issues/10020#issuecomment-692969606 Hi @kaxil @OmairK I don't think this is the right way to solve this issue. In the case of pod failures, we WANT to reattach the first time so the failure can be

[GitHub] [airflow] dimberman commented on issue #10037: KubernetesPodOperator namespace argument conflict when using pod_template_file

2020-09-15 Thread GitBox
dimberman commented on issue #10037: URL: https://github.com/apache/airflow/issues/10037#issuecomment-692967644 Also @SilvrDuck heads up that template won't work. The K8sPodOperator only takes a pod as a template (it would be bad to have two levels of retry logic so we explicitly do not

[GitHub] [airflow] dimberman commented on issue #10037: KubernetesPodOperator namespace argument conflict when using pod_template_file

2020-09-15 Thread GitBox
dimberman commented on issue #10037: URL: https://github.com/apache/airflow/issues/10037#issuecomment-692966901 https://github.com/apache/airflow/pull/10963 PR for a fix. This should go out in 1.10.13. This is an automated

[GitHub] [airflow] dimberman opened a new pull request #10963: K8s operator pod template fix

2020-09-15 Thread GitBox
dimberman opened a new pull request #10963: URL: https://github.com/apache/airflow/pull/10963 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for

[GitHub] [airflow] mik-laj commented on pull request #10958: Allow customization of navigation bar title text

2020-09-15 Thread GitBox
mik-laj commented on pull request #10958: URL: https://github.com/apache/airflow/pull/10958#issuecomment-692964406 Similar: https://github.com/apache/airflow/pull/10162 CC: @pcandoalmeida This is an automated message

[GitHub] [airflow] ephraimbuddy opened a new pull request #10962: Fix doc preview error in editor for google operators doc

2020-09-15 Thread GitBox
ephraimbuddy opened a new pull request #10962: URL: https://github.com/apache/airflow/pull/10962 When I open the documentation for google operators in my editor, I receive an error instead of a preview. Here's the error: ``` No preview available. Error output: :34:

[GitHub] [airflow] iseessel opened a new issue #10961: airflow initdb: undefined symbol: Py_GetArgcArgv

2020-09-15 Thread GitBox
iseessel opened a new issue #10961: URL: https://github.com/apache/airflow/issues/10961 **Apache Airflow version**: 1.10.12 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): N/A **Environment**: - **Cloud provider or hardware

[GitHub] [airflow] boring-cyborg[bot] commented on issue #10961: airflow initdb: undefined symbol: Py_GetArgcArgv

2020-09-15 Thread GitBox
boring-cyborg[bot] commented on issue #10961: URL: https://github.com/apache/airflow/issues/10961#issuecomment-692960941 Thanks for opening your first issue here! Be sure to follow the issue template! This is an

[GitHub] [airflow] jbampton opened a new pull request #10960: Fix case of Python.

2020-09-15 Thread GitBox
jbampton opened a new pull request #10960: URL: https://github.com/apache/airflow/pull/10960 Changed `python` to `Python`. This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] nasir-bhanpuri commented on issue #10435: Negsignal.SIGKILL error on macOS

2020-09-15 Thread GitBox
nasir-bhanpuri commented on issue #10435: URL: https://github.com/apache/airflow/issues/10435#issuecomment-692949804 I am running into a similar issue with Airflow 1.10.10 using Docker on macOS. This is an automated message

[GitHub] [airflow] XD-DENG commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
XD-DENG commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488939428 ## File path: airflow/migrations/versions/e1a11ece99cc_add_external_executor_id_to_ti.py ## @@ -0,0 +1,46 @@ +# +# Licensed to the Apache Software

[GitHub] [airflow] ashb commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
ashb commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488937065 ## File path: airflow/executors/celery_executor.py ## @@ -274,6 +330,61 @@ def execute_async(self, def terminate(self): pass +def

[GitHub] [airflow] ashb commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
ashb commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488937450 ## File path: airflow/migrations/versions/e1a11ece99cc_add_external_executor_id_to_ti.py ## @@ -0,0 +1,46 @@ +# +# Licensed to the Apache Software

[GitHub] [airflow] potiuk opened a new pull request #10959: All versions in CI scripts are not hard-coded any more

2020-09-15 Thread GitBox
potiuk opened a new pull request #10959: URL: https://github.com/apache/airflow/pull/10959 GitHub Actions allow to use `fromJson` method to read arrays or even more complex json objects into the CI workflow yaml files. This, connected with set::output commands, allows to read the

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #10951: Add example dag and system test for S3ToGCSOperator

2020-09-15 Thread GitBox
ephraimbuddy commented on a change in pull request #10951: URL: https://github.com/apache/airflow/pull/10951#discussion_r488935277 ## File path: docs/operators-and-hooks-ref.rst ## @@ -934,11 +934,15 @@ These integrations allow you to copy data from/to Google Cloud. -

[GitHub] [airflow] XD-DENG commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
XD-DENG commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488925717 ## File path: airflow/executors/celery_executor.py ## @@ -274,6 +330,61 @@ def execute_async(self, def terminate(self): pass +def

[GitHub] [airflow] XD-DENG commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
XD-DENG commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488925717 ## File path: airflow/executors/celery_executor.py ## @@ -274,6 +330,61 @@ def execute_async(self, def terminate(self): pass +def

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #10951: Add example dag and system test for S3ToGCSOperator

2020-09-15 Thread GitBox
ephraimbuddy commented on a change in pull request #10951: URL: https://github.com/apache/airflow/pull/10951#discussion_r488933951 ## File path: docs/operators-and-hooks-ref.rst ## @@ -934,11 +934,15 @@ These integrations allow you to copy data from/to Google Cloud. -

[GitHub] [airflow] XD-DENG commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
XD-DENG commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488925717 ## File path: airflow/executors/celery_executor.py ## @@ -274,6 +330,61 @@ def execute_async(self, def terminate(self): pass +def

[GitHub] [airflow] XD-DENG commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
XD-DENG commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488932380 ## File path: airflow/migrations/versions/e1a11ece99cc_add_external_executor_id_to_ti.py ## @@ -0,0 +1,46 @@ +# +# Licensed to the Apache Software

[GitHub] [airflow] XD-DENG commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
XD-DENG commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488929900 ## File path: airflow/executors/celery_executor.py ## @@ -274,6 +330,61 @@ def execute_async(self, def terminate(self): pass +def

[GitHub] [airflow] XD-DENG commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
XD-DENG commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488928231 ## File path: airflow/executors/celery_executor.py ## @@ -274,6 +330,61 @@ def execute_async(self, def terminate(self): pass +def

[GitHub] [airflow] XD-DENG commented on a change in pull request #10949: Allow CeleryExecutor to "adopt" an orphaned queued or running task

2020-09-15 Thread GitBox
XD-DENG commented on a change in pull request #10949: URL: https://github.com/apache/airflow/pull/10949#discussion_r488925717 ## File path: airflow/executors/celery_executor.py ## @@ -274,6 +330,61 @@ def execute_async(self, def terminate(self): pass +def

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #10951: Add example dag and system test for S3ToGCSOperator

2020-09-15 Thread GitBox
ephraimbuddy commented on a change in pull request #10951: URL: https://github.com/apache/airflow/pull/10951#discussion_r488910813 ## File path: docs/operators-and-hooks-ref.rst ## @@ -934,11 +934,15 @@ These integrations allow you to copy data from/to Google Cloud. -

[GitHub] [airflow] kaxil edited a comment on pull request #10924: fix operator field update for SerializedBaseOperator

2020-09-15 Thread GitBox
kaxil edited a comment on pull request #10924: URL: https://github.com/apache/airflow/pull/10924#issuecomment-692922474 I have rebased to the Master, do you need any help @xOnelinx This is an automated message from the

[GitHub] [airflow] kaxil commented on pull request #10924: fix operator field update for SerializedBaseOperator

2020-09-15 Thread GitBox
kaxil commented on pull request #10924: URL: https://github.com/apache/airflow/pull/10924#issuecomment-692922474 I have rebased to the Master This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] turbaszek commented on pull request #10958: Allow customization of navigation bar title text

2020-09-15 Thread GitBox
turbaszek commented on pull request #10958: URL: https://github.com/apache/airflow/pull/10958#issuecomment-692921275 @n-oden can you provide a screen shot? This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #10958: Allow customization of navigation bar title text

2020-09-15 Thread GitBox
boring-cyborg[bot] commented on pull request #10958: URL: https://github.com/apache/airflow/pull/10958#issuecomment-692920697 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] n-oden opened a new pull request #10958: Allow customization of navigation bar title text

2020-09-15 Thread GitBox
n-oden opened a new pull request #10958: URL: https://github.com/apache/airflow/pull/10958 For organizations who may run more than one airflow instance, this will allow for quick visual distinguishing between for example a "production" and "test" airflow instance.

[GitHub] [airflow] turbaszek commented on a change in pull request #10956: Officially support running more than one scheduler concurrently.

2020-09-15 Thread GitBox
turbaszek commented on a change in pull request #10956: URL: https://github.com/apache/airflow/pull/10956#discussion_r488904165 ## File path: airflow/jobs/scheduler_job.py ## @@ -1707,53 +1343,119 @@ def _run_scheduler_loop(self) -> None: )

[GitHub] [airflow] SalAlba commented on pull request #10864: Increase typing coverage for postgres provider

2020-09-15 Thread GitBox
SalAlba commented on pull request #10864: URL: https://github.com/apache/airflow/pull/10864#issuecomment-692919402 > Please rebase to latest master - we had some breaking change a commit ago and the PR will need to be rebased. @potiuk I will do that according to this tips [How to

[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #10947: WIP Add transfer operator amazon gcs

2020-09-15 Thread GitBox
michalslowikowski00 commented on a change in pull request #10947: URL: https://github.com/apache/airflow/pull/10947#discussion_r488902837 ## File path: tests/providers/amazon/aws/hooks/test_glacier.py ## @@ -0,0 +1,180 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #10947: WIP Add transfer operator amazon gcs

2020-09-15 Thread GitBox
michalslowikowski00 commented on a change in pull request #10947: URL: https://github.com/apache/airflow/pull/10947#discussion_r488901592 ## File path: tests/providers/amazon/aws/hooks/test_glacier.py ## @@ -0,0 +1,180 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] turbaszek commented on a change in pull request #10956: Officially support running more than one scheduler concurrently.

2020-09-15 Thread GitBox
turbaszek commented on a change in pull request #10956: URL: https://github.com/apache/airflow/pull/10956#discussion_r488900780 ## File path: airflow/jobs/scheduler_job.py ## @@ -303,10 +303,10 @@ def done(self) -> bool: return False @property -def

[GitHub] [airflow] turbaszek commented on a change in pull request #10956: Officially support running more than one scheduler concurrently.

2020-09-15 Thread GitBox
turbaszek commented on a change in pull request #10956: URL: https://github.com/apache/airflow/pull/10956#discussion_r488899697 ## File path: airflow/executors/base_executor.py ## @@ -69,16 +70,16 @@ def start(self): # pragma: no cover """ def

[GitHub] [airflow] kaxil commented on a change in pull request #10897: Add a default for DagModel.default_view

2020-09-15 Thread GitBox
kaxil commented on a change in pull request #10897: URL: https://github.com/apache/airflow/pull/10897#discussion_r488895516 ## File path: airflow/models/dag.py ## @@ -1621,6 +1621,13 @@ def sync_to_db(self, sync_time=None, session=None): """

[GitHub] [airflow] kaxil commented on a change in pull request #10897: Add a default for DagModel.default_view

2020-09-15 Thread GitBox
kaxil commented on a change in pull request #10897: URL: https://github.com/apache/airflow/pull/10897#discussion_r488895516 ## File path: airflow/models/dag.py ## @@ -1621,6 +1621,13 @@ def sync_to_db(self, sync_time=None, session=None): """

[GitHub] [airflow] turbaszek commented on a change in pull request #10947: WIP Add transfer operator amazon gcs

2020-09-15 Thread GitBox
turbaszek commented on a change in pull request #10947: URL: https://github.com/apache/airflow/pull/10947#discussion_r488895252 ## File path: tests/providers/amazon/aws/hooks/test_glacier.py ## @@ -0,0 +1,180 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [airflow] ryw edited a comment on pull request #10956: Officially support running more than one scheduler concurrently.

2020-09-15 Thread GitBox
ryw edited a comment on pull request #10956: URL: https://github.com/apache/airflow/pull/10956#issuecomment-692908582 We're working on a short doc that will provide an easy way to test all the components/PRs of AIP-15 together, and some preliminary benchmarks for performance improvement,

[GitHub] [airflow] ryw edited a comment on pull request #10956: Officially support running more than one scheduler concurrently.

2020-09-15 Thread GitBox
ryw edited a comment on pull request #10956: URL: https://github.com/apache/airflow/pull/10956#issuecomment-692908582 We're working on a short doc that will provide an easy way to test the components/PRs of AIP-15 together, and some preliminary benchmarks for performance improvement,

  1   2   3   >