[GitHub] [airflow] potiuk edited a comment on pull request #10708: Make breeeze-complete Google Shell Guide compatible

2020-09-07 Thread GitBox
potiuk edited a comment on pull request #10708: URL: https://github.com/apache/airflow/pull/10708#issuecomment-688633019 And yeah. The bats tests work just fine and pre-commit fails when any of the tests fail if that's what you asked for (and with the bats-assert extension it's output is

[GitHub] [airflow] potiuk edited a comment on pull request #10708: Make breeeze-complete Google Shell Guide compatible

2020-09-07 Thread GitBox
potiuk edited a comment on pull request #10708: URL: https://github.com/apache/airflow/pull/10708#issuecomment-688633019 And yeah. The bats tests work just fine and pre-commit fails when any of the tests fail if that's what you asked for (and with the bats-assert extension it's output is

[GitHub] [airflow] potiuk edited a comment on pull request #10708: Make breeeze-complete Google Shell Guide compatible

2020-09-07 Thread GitBox
potiuk edited a comment on pull request #10708: URL: https://github.com/apache/airflow/pull/10708#issuecomment-688633019 And yeah. The bats tests work just fine and pre-commit fails when any of the tests fail if that's what you asked for (and with the bats-assert extension it's output is

[GitHub] [airflow] potiuk commented on pull request #10708: Make breeeze-complete Google Shell Guide compatible

2020-09-07 Thread GitBox
potiuk commented on pull request #10708: URL: https://github.com/apache/airflow/pull/10708#issuecomment-688633019 And yeah. The bats tests work just fine and pre-commit fails when any of the tests fail if that's what you asked for (and with the bats-assert extension it's output is really

[GitHub] [airflow] potiuk commented on pull request #10708: Make breeeze-complete Google Shell Guide compatible

2020-09-07 Thread GitBox
potiuk commented on pull request #10708: URL: https://github.com/apache/airflow/pull/10708#issuecomment-688630908 > Look good to me, however, would you mind testing pre-commit just to check that error information will be propagated? We all depend on this in our development flow Not

[airflow] branch master updated (3a4f517 -> b746f33)

2020-09-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 3a4f517 fix task instance modal open performance issue (#10764) add b746f33 Removes stable tests from

[GitHub] [airflow] potiuk merged pull request #10768: Removes stable tests from quarantine

2020-09-07 Thread GitBox
potiuk merged pull request #10768: URL: https://github.com/apache/airflow/pull/10768 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk commented on issue #10783: MsSqlOperator backport can't use ODBC

2020-09-07 Thread GitBox
potiuk commented on issue #10783: URL: https://github.com/apache/airflow/issues/10783#issuecomment-688627635 Good Point. I will think about it. Watch at the devlist the next few days. I will remember to link the PR to this issue so that you see it.

[GitHub] [airflow] KimchaC commented on issue #10783: MsSqlOperator backport can't use ODBC

2020-09-07 Thread GitBox
KimchaC commented on issue #10783: URL: https://github.com/apache/airflow/issues/10783#issuecomment-688624618 Sweet, thank you. I hope the general solution is going to allow to add custom connection types. I have many custom plugins and end up selecting "HTTP" as the type very

[GitHub] [airflow] potiuk commented on pull request #10153: [AIP-34] TaskGroup: A UI task grouping concept as an alternative to SubDagOperator

2020-09-07 Thread GitBox
potiuk commented on pull request #10153: URL: https://github.com/apache/airflow/pull/10153#issuecomment-688622957 > I love this change! This DAG representation is nice and clean. I'm just wondering if this will work with AIP-31 approach This would be awesome. Actually I

[GitHub] [airflow] potiuk commented on issue #10730: Dockerfile - Enable gpg to receive keys behind a proxy

2020-09-07 Thread GitBox
potiuk commented on issue #10730: URL: https://github.com/apache/airflow/issues/10730#issuecomment-688616549 Awesome! Let me know if I can help :) This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on issue #10783: MsSqlOperator backport can't use ODBC

2020-09-07 Thread GitBox
potiuk commented on issue #10783: URL: https://github.com/apache/airflow/issues/10783#issuecomment-688615843 Update: We have discussed yesterday at Airflow 2.0 dev call a "general" solution to this problem. I am going to implement a proof of concept for 2.0 of solving that for all

[GitHub] [airflow] Adil-Ibragimov commented on a change in pull request #10315: Add retry_only_on_pod_launching_failure and log_container_statuses_on…

2020-09-07 Thread GitBox
Adil-Ibragimov commented on a change in pull request #10315: URL: https://github.com/apache/airflow/pull/10315#discussion_r484642940 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -138,6 +138,9 @@ class KubernetesPodOperator(BaseOperator): #

[GitHub] [airflow] Adil-Ibragimov commented on a change in pull request #10315: Add retry_only_on_pod_launching_failure and log_container_statuses_on…

2020-09-07 Thread GitBox
Adil-Ibragimov commented on a change in pull request #10315: URL: https://github.com/apache/airflow/pull/10315#discussion_r484642741 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -294,6 +299,10 @@ def execute(self, context) -> Optional[str]:

[GitHub] [airflow] Adil-Ibragimov commented on a change in pull request #10315: Add retry_only_on_pod_launching_failure and log_container_statuses_on…

2020-09-07 Thread GitBox
Adil-Ibragimov commented on a change in pull request #10315: URL: https://github.com/apache/airflow/pull/10315#discussion_r484639009 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -408,17 +430,18 @@ def create_new_pod_for_operator(self,

[GitHub] [airflow] Adil-Ibragimov commented on a change in pull request #10315: Add retry_only_on_pod_launching_failure and log_container_statuses_on…

2020-09-07 Thread GitBox
Adil-Ibragimov commented on a change in pull request #10315: URL: https://github.com/apache/airflow/pull/10315#discussion_r484638836 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -350,6 +359,19 @@ def _set_name(self, name):

[GitHub] [airflow] mik-laj commented on a change in pull request #10756: Add pod_override setting for KubernetesExecutor

2020-09-07 Thread GitBox
mik-laj commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484608088 ## File path: docs/concepts.rst ## @@ -158,6 +158,96 @@ Example DAG with functional abstraction .. _concepts:dagruns: +executor_config

[GitHub] [airflow] neil90 commented on pull request #10787: Add kwarg jar_params for DatabricksRunNowOperator and updated docs

2020-09-07 Thread GitBox
neil90 commented on pull request #10787: URL: https://github.com/apache/airflow/pull/10787#issuecomment-688582665 Updated the unit test to account for jar_params This is an automated message from the Apache Git Service. To

[GitHub] [airflow] dimberman commented on a change in pull request #10756: Add pod_override setting for KubernetesExecutor

2020-09-07 Thread GitBox
dimberman commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484612897 ## File path: docs/concepts.rst ## @@ -158,6 +158,96 @@ Example DAG with functional abstraction .. _concepts:dagruns: +executor_config

[GitHub] [airflow] mik-laj commented on pull request #10261: 10166 : Add capability to specify gunicorn access log format for airflow

2020-09-07 Thread GitBox
mik-laj commented on pull request #10261: URL: https://github.com/apache/airflow/pull/10261#issuecomment-688574641 We have a problem with CI. I restarted it. If that doesn't help, please rebase and png me again. This is an

[airflow] tag nightly-master updated (ddee0aa -> 3a4f517)

2020-09-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from ddee0aa (commit) to 3a4f517 (commit) from

[airflow] tag nightly-master updated (ddee0aa -> 3a4f517)

2020-09-07 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from ddee0aa (commit) to 3a4f517 (commit) from

[GitHub] [airflow] mik-laj commented on a change in pull request #10756: Add pod_override setting for KubernetesExecutor

2020-09-07 Thread GitBox
mik-laj commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484608088 ## File path: docs/concepts.rst ## @@ -158,6 +158,96 @@ Example DAG with functional abstraction .. _concepts:dagruns: +executor_config

[GitHub] [airflow] mik-laj commented on a change in pull request #10756: Add pod_override setting for KubernetesExecutor

2020-09-07 Thread GitBox
mik-laj commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484607175 ## File path: docs/concepts.rst ## @@ -158,6 +158,96 @@ Example DAG with functional abstraction .. _concepts:dagruns: +executor_config

[GitHub] [airflow] mik-laj commented on a change in pull request #10756: Add pod_override setting for KubernetesExecutor

2020-09-07 Thread GitBox
mik-laj commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484606726 ## File path: docs/concepts.rst ## @@ -158,6 +158,96 @@ Example DAG with functional abstraction .. _concepts:dagruns: +executor_config Review

[GitHub] [airflow] neil90 opened a new pull request #10787: Add kwarg jar_params for DatabricksRunNowOperator and updated docs

2020-09-07 Thread GitBox
neil90 opened a new pull request #10787: URL: https://github.com/apache/airflow/pull/10787 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] neil90 opened a new issue #10786: DatabricksRunNowOperator missing jar_params as a kwarg

2020-09-07 Thread GitBox
neil90 opened a new issue #10786: URL: https://github.com/apache/airflow/issues/10786 **Description** DatabricksRunNowOperator is missing the option to take in key word arguement _jar_params_ , it already can take the other ones

[GitHub] [airflow] boring-cyborg[bot] commented on issue #10786: DatabricksRunNowOperator missing jar_params as a kwarg

2020-09-07 Thread GitBox
boring-cyborg[bot] commented on issue #10786: URL: https://github.com/apache/airflow/issues/10786#issuecomment-688551745 Thanks for opening your first issue here! Be sure to follow the issue template! This is an

[airflow] branch master updated: fix task instance modal open performance issue (#10764)

2020-09-07 Thread houqp
This is an automated email from the ASF dual-hosted git repository. houqp pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 3a4f517 fix task instance modal open

[GitHub] [airflow] houqp merged pull request #10764: fix task instance modal open performance issue

2020-09-07 Thread GitBox
houqp merged pull request #10764: URL: https://github.com/apache/airflow/pull/10764 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] houqp commented on pull request #10621: Fix slow task instance modal with extra link

2020-09-07 Thread GitBox
houqp commented on pull request #10621: URL: https://github.com/apache/airflow/pull/10621#issuecomment-688523719 @dstandish removing the tooltip call definitely prevents the link not ready tooltip from showing up for me. Adding sleep is not the best way to confirm it since it's subject to

[GitHub] [airflow] potiuk commented on issue #10783: MsSqlOperator backport can't use ODBC

2020-09-07 Thread GitBox
potiuk commented on issue #10783: URL: https://github.com/apache/airflow/issues/10783#issuecomment-688504997 Cool. Start from "master". We have the https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst but it might be a bit overwhelming for first-time contributors, so I am happy

[airflow] branch master updated (f14f379 -> ef0d639)

2020-09-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f14f379 [AIRFLOW-10672] Refactor BigQueryToGCSOperator to use new method (#10773) add ef0d639 Fixes

[airflow] branch master updated (f14f379 -> ef0d639)

2020-09-07 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f14f379 [AIRFLOW-10672] Refactor BigQueryToGCSOperator to use new method (#10773) add ef0d639 Fixes

[GitHub] [airflow] potiuk merged pull request #10785: Fixes pre-commit failing on build step

2020-09-07 Thread GitBox
potiuk merged pull request #10785: URL: https://github.com/apache/airflow/pull/10785 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk opened a new pull request #10785: Fixes pre-commit failing on build step

2020-09-07 Thread GitBox
potiuk opened a new pull request #10785: URL: https://github.com/apache/airflow/pull/10785 When rebuildig the image during commit, kill command failed to find the spinner job to kill (this is just preventive measure) and failed the rebuild step in pre-commit. This is now fixed.

[GitHub] [airflow] potiuk opened a new pull request #10784: Requirements might get upgraded without setup.py change

2020-09-07 Thread GitBox
potiuk opened a new pull request #10784: URL: https://github.com/apache/airflow/pull/10784 I noticed that when there is no setup.py changes, the constraints are not upgraded automatically. This is because of the docker caching strategy used - it simply does not even know that the

[GitHub] [airflow] zoomzoomTnT commented on a change in pull request #10606: Feature/airflow 10605

2020-09-07 Thread GitBox
zoomzoomTnT commented on a change in pull request #10606: URL: https://github.com/apache/airflow/pull/10606#discussion_r484540189 ## File path: airflow/kubernetes/pod_generator.py ## @@ -292,6 +299,15 @@ def __init__( # pylint: disable=too-many-arguments,too-many-locals

[GitHub] [airflow] dimberman commented on a change in pull request #10393: Simplify the K8sExecutor and K8sPodOperator

2020-09-07 Thread GitBox
dimberman commented on a change in pull request #10393: URL: https://github.com/apache/airflow/pull/10393#discussion_r484537691 ## File path: airflow/executors/kubernetes_executor.py ## @@ -483,17 +326,23 @@ def run_next(self, next_job: KubernetesJobType) -> None: if

[GitHub] [airflow] KimchaC commented on issue #10783: MsSqlOperator backport can't use ODBC

2020-09-07 Thread GitBox
KimchaC commented on issue #10783: URL: https://github.com/apache/airflow/issues/10783#issuecomment-688446321 Yes, I would be happy to do this. But I have never contributed a PR. What branch or tag should I branch from before I add my commit?

[GitHub] [airflow] kaxil commented on a change in pull request #10393: Simplify the K8sExecutor and K8sPodOperator

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10393: URL: https://github.com/apache/airflow/pull/10393#discussion_r484527717 ## File path: airflow/executors/kubernetes_executor.py ## @@ -483,17 +326,23 @@ def run_next(self, next_job: KubernetesJobType) -> None: if

[GitHub] [airflow] kaxil commented on a change in pull request #10393: Simplify the K8sExecutor and K8sPodOperator

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10393: URL: https://github.com/apache/airflow/pull/10393#discussion_r484527338 ## File path: airflow/example_dags/example_kubernetes_executor_config.py ## @@ -52,59 +53,68 @@ def test_volume_mount(): start_task =

[GitHub] [airflow] kaxil commented on a change in pull request #10393: Simplify the K8sExecutor and K8sPodOperator

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10393: URL: https://github.com/apache/airflow/pull/10393#discussion_r484527163 ## File path: airflow/example_dags/example_kubernetes_executor_config.py ## @@ -52,59 +53,68 @@ def test_volume_mount(): start_task =

[GitHub] [airflow] jmcarp commented on pull request #10774: Always return a list from s3 list methods.

2020-09-07 Thread GitBox
jmcarp commented on pull request #10774: URL: https://github.com/apache/airflow/pull/10774#issuecomment-688443684 Thanks, I updated `UPDATING.md`. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] rootcss commented on pull request #10770: Add Plugins View in web UI

2020-09-07 Thread GitBox
rootcss commented on pull request #10770: URL: https://github.com/apache/airflow/pull/10770#issuecomment-688441882 @mik-laj currently while loading the plugins from either of the sources, we're not keeping the information about the source, path etc. To get this information, we'd need to

[GitHub] [airflow] potiuk commented on issue #10783: MsSqlOperator backport can't use ODBC

2020-09-07 Thread GitBox
potiuk commented on issue #10783: URL: https://github.com/apache/airflow/issues/10783#issuecomment-688441001 Could you please make a PR with this ? I think it's a great candidate fo 1.10.13 This is an automated message from

[GitHub] [airflow] KimchaC opened a new issue #10783: MsSqlOperator backport can't use ODBC

2020-09-07 Thread GitBox
KimchaC opened a new issue #10783: URL: https://github.com/apache/airflow/issues/10783 The MsSqlOperator included in 1.10.* is using MsSqlHook, which is using pymssql. But pymssql has been discontinued and has some nasty, unfixed bugs. Airflow 2.0 has deprecated MsSqlHook in favor

[GitHub] [airflow] potiuk commented on pull request #10768: Removes stable tests from quarantine

2020-09-07 Thread GitBox
potiuk commented on pull request #10768: URL: https://github.com/apache/airflow/pull/10768#issuecomment-688438930 > @potiuk +1 for heisentest Yeah. I also like the term. It's a slight variation of the https://en.wikipedia.org/wiki/Heisenbug - which is worth reading on its own.

[GitHub] [airflow] boring-cyborg[bot] commented on issue #10783: MsSqlOperator backport can't use ODBC

2020-09-07 Thread GitBox
boring-cyborg[bot] commented on issue #10783: URL: https://github.com/apache/airflow/issues/10783#issuecomment-688438936 Thanks for opening your first issue here! Be sure to follow the issue template! This is an

[GitHub] [airflow] kaxil commented on a change in pull request #10756: Add pod_override setting for KubernetesExecutor

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484521631 ## File path: airflow/kubernetes/pod_generator_deprecated.py ## @@ -0,0 +1,652 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review

[GitHub] [airflow] kaxil commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484520899 ## File path: airflow/kubernetes/pod_generator_deprecated.py ## @@ -0,0 +1,652 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review

[GitHub] [airflow] kaxil commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484521631 ## File path: airflow/kubernetes/pod_generator_deprecated.py ## @@ -0,0 +1,652 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review

[GitHub] [airflow] kaxil commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484520899 ## File path: airflow/kubernetes/pod_generator_deprecated.py ## @@ -0,0 +1,652 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review

[GitHub] [airflow] dimberman commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
dimberman commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484520242 ## File path: airflow/kubernetes/pod_generator_deprecated.py ## @@ -0,0 +1,652 @@ +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [airflow] kaxil commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484519932 ## File path: airflow/kubernetes/pod_generator_deprecated.py ## @@ -0,0 +1,652 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review

[GitHub] [airflow] mik-laj edited a comment on pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
mik-laj edited a comment on pull request #10756: URL: https://github.com/apache/airflow/pull/10756#issuecomment-688424585 Can you add more docs about this feature? Code cleaning is also a good time to fill gaps in the documentation. ;-) Reference docs is not enougth for beginner. Small

[GitHub] [airflow] kaxil commented on a change in pull request #10729: Detect orphaned task instances by SchedulerJob id and heartbeat

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10729: URL: https://github.com/apache/airflow/pull/10729#discussion_r483853854 ## File path: airflow/migrations/versions/b247b1e3d1ed_add_queued_by_job_id_to_ti.py ## @@ -0,0 +1,46 @@ +# +# Licensed to the Apache Software

[GitHub] [airflow] mik-laj commented on pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
mik-laj commented on pull request #10756: URL: https://github.com/apache/airflow/pull/10756#issuecomment-688424585 Can you add more docs about this feature? Code cleaning is also a good time to fill gaps in the documentation. ;-)

[GitHub] [airflow] ashb edited a comment on pull request #10729: Detect orphaned task instances by SchedulerJob id and heartbeat

2020-09-07 Thread GitBox
ashb edited a comment on pull request #10729: URL: https://github.com/apache/airflow/pull/10729#issuecomment-688423376 > @ashb lemme know if you'd like me to look into the SQLite issue as well Please. I suspect we will have to have an `if sqlite:` and not do it as a single query for

[GitHub] [airflow] ashb commented on pull request #10729: Detect orphaned task instances by SchedulerJob id and heartbeat

2020-09-07 Thread GitBox
ashb commented on pull request #10729: URL: https://github.com/apache/airflow/pull/10729#issuecomment-688423376 > @ashb lemme know if you'd like me to look into the SQLite issue as well Please. I suspect we will have to have an `if sqlite:` and not do it as a single query for that

[GitHub] [airflow] dimberman commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
dimberman commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484513877 ## File path: airflow/kubernetes/pod_generator.py ## @@ -324,47 +325,36 @@ def add_sidecar(pod: k8s.V1Pod) -> k8s.V1Pod: @staticmethod def

[GitHub] [airflow] kaxil commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484513319 ## File path: airflow/kubernetes/pod_generator.py ## @@ -324,47 +325,36 @@ def add_sidecar(pod: k8s.V1Pod) -> k8s.V1Pod: @staticmethod def

[GitHub] [airflow] dimberman commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
dimberman commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484512936 ## File path: airflow/kubernetes/pod_generator.py ## @@ -324,47 +325,36 @@ def add_sidecar(pod: k8s.V1Pod) -> k8s.V1Pod: @staticmethod def

[GitHub] [airflow] dimberman commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
dimberman commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484512639 ## File path: airflow/kubernetes/pod_generator.py ## @@ -324,47 +325,36 @@ def add_sidecar(pod: k8s.V1Pod) -> k8s.V1Pod: @staticmethod def

[GitHub] [airflow] kaxil commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484512148 ## File path: airflow/kubernetes/pod_generator.py ## @@ -324,47 +325,36 @@ def add_sidecar(pod: k8s.V1Pod) -> k8s.V1Pod: @staticmethod def

[GitHub] [airflow] kaxil commented on a change in pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10756: URL: https://github.com/apache/airflow/pull/10756#discussion_r484510031 ## File path: airflow/kubernetes/pod_generator.py ## @@ -324,47 +325,36 @@ def add_sidecar(pod: k8s.V1Pod) -> k8s.V1Pod: @staticmethod def

[GitHub] [airflow] dimberman commented on pull request #10729: Detect orphaned task instances by SchedulerJob id and heartbeat

2020-09-07 Thread GitBox
dimberman commented on pull request #10729: URL: https://github.com/apache/airflow/pull/10729#issuecomment-688405729 @ashb lemme know if you'd like me to look into the SQLite issue as well This is an automated message from

[GitHub] [airflow] kaxil commented on a change in pull request #10768: Removes stable tests from quarantine

2020-09-07 Thread GitBox
kaxil commented on a change in pull request #10768: URL: https://github.com/apache/airflow/pull/10768#discussion_r484501712 ## File path: TESTING.rst ## @@ -324,6 +324,18 @@ Those tests are marked with ``@pytest.mark.quarantined`` annotation. Those tests are skipped by

[GitHub] [airflow] dimberman commented on pull request #10768: Removes stable tests from quarantine

2020-09-07 Thread GitBox
dimberman commented on pull request #10768: URL: https://github.com/apache/airflow/pull/10768#issuecomment-688403242 @potiuk +1 for heisentest  This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] zbstof opened a new issue #10782: TaskInstance tooltip in DAG tree view shows incorrect Operator

2020-09-07 Thread GitBox
zbstof opened a new issue #10782: URL: https://github.com/apache/airflow/issues/10782 **Apache Airflow version:** 2.0.0.dev0, built from HEAD master branch **Environment:** kubernetes **What happened:** 1. Create task **A** with operator **First** 2. Run the dag

[GitHub] [airflow] dimberman commented on pull request #10393: Simplify the K8sExecutor and K8sPodOperator

2020-09-07 Thread GitBox
dimberman commented on pull request #10393: URL: https://github.com/apache/airflow/pull/10393#issuecomment-688398119 @mik-laj fixed those tests, PTAL :) This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] dimberman commented on pull request #10756: Add podOverride setting for KubernetesExecutor

2020-09-07 Thread GitBox
dimberman commented on pull request #10756: URL: https://github.com/apache/airflow/pull/10756#issuecomment-688366172 @potiuk @kaxil @turbaszek can I get a review on this? :) This is an automated message from the Apache Git

[GitHub] [airflow] stale[bot] closed pull request #8918: Impala hook implention

2020-09-07 Thread GitBox
stale[bot] closed pull request #8918: URL: https://github.com/apache/airflow/pull/8918 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[airflow] branch master updated (c8ee455 -> f14f379)

2020-09-07 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from c8ee455 Refactor DataprocCreateCluster operator to use simpler interface (#10403) add f14f379

[airflow] branch master updated (c8ee455 -> f14f379)

2020-09-07 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from c8ee455 Refactor DataprocCreateCluster operator to use simpler interface (#10403) add f14f379

[GitHub] [airflow] MateuszKukiela commented on pull request #10773: [AIRFLOW-10672] Refactor BigQueryToGCSOperator to use new method

2020-09-07 Thread GitBox
MateuszKukiela commented on pull request #10773: URL: https://github.com/apache/airflow/pull/10773#issuecomment-688356284 > Looks good to me! I've run the system tests and everything seems to be working as expected Good to hear that :+1:

[GitHub] [airflow] turbaszek closed issue #10672: Refactor BigQueryToGCSOperator to use new method

2020-09-07 Thread GitBox
turbaszek closed issue #10672: URL: https://github.com/apache/airflow/issues/10672 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #10773: [AIRFLOW-10672] Refactor BigQueryToGCSOperator to use new method

2020-09-07 Thread GitBox
boring-cyborg[bot] commented on pull request #10773: URL: https://github.com/apache/airflow/pull/10773#issuecomment-688355414 Awesome work, congrats on your first merged pull request! This is an automated message from

[GitHub] [airflow] turbaszek merged pull request #10773: [AIRFLOW-10672] Refactor BigQueryToGCSOperator to use new method

2020-09-07 Thread GitBox
turbaszek merged pull request #10773: URL: https://github.com/apache/airflow/pull/10773 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] enriqueayala commented on issue #10730: Dockerfile - Enable gpg to receive keys behind a proxy

2020-09-07 Thread GitBox
enriqueayala commented on issue #10730: URL: https://github.com/apache/airflow/issues/10730#issuecomment-688339812 @potiuk Sounds good :) I will work on it This is an automated message from the Apache Git Service. To

[GitHub] [airflow] aranjanthakur commented on pull request #10261: 10166 : Add capability to specify gunicorn access log format for airflow

2020-09-07 Thread GitBox
aranjanthakur commented on pull request #10261: URL: https://github.com/apache/airflow/pull/10261#issuecomment-688339311 @mik-laj please approve if the changes look good This is an automated message from the Apache Git

[GitHub] [airflow] turbaszek commented on issue #10780: Backported BigQueryExecuteQueryOperator throws 409 Already Exists in job with multiple operator instances

2020-09-07 Thread GitBox
turbaszek commented on issue #10780: URL: https://github.com/apache/airflow/issues/10780#issuecomment-688327167 @rbkn have you tried using `BigQueryInsertJobOperator` from backport packages? It improves the idempotency as discussed #9590 and will replace `BigQueryExecuteQueryOperator`

[GitHub] [airflow] MateuszKukiela commented on a change in pull request #10773: [AIRFLOW-10672] Refactor BigQueryToGCSOperator to use new method

2020-09-07 Thread GitBox
MateuszKukiela commented on a change in pull request #10773: URL: https://github.com/apache/airflow/pull/10773#discussion_r484430769 ## File path: airflow/providers/google/cloud/transfers/bigquery_to_gcs.py ## @@ -139,14 +141,32 @@ def execute(self, context):

[GitHub] [airflow] turbaszek commented on a change in pull request #10773: [AIRFLOW-10672] Refactor BigQueryToGCSOperator to use new method

2020-09-07 Thread GitBox
turbaszek commented on a change in pull request #10773: URL: https://github.com/apache/airflow/pull/10773#discussion_r484429681 ## File path: airflow/providers/google/cloud/transfers/bigquery_to_gcs.py ## @@ -139,14 +141,32 @@ def execute(self, context):

[GitHub] [airflow] mik-laj commented on pull request #10081: Add CLI for connections export (#9856)

2020-09-07 Thread GitBox
mik-laj commented on pull request #10081: URL: https://github.com/apache/airflow/pull/10081#issuecomment-688322482 @JeffryMAC Yes. It is planned, but we are slowly developing step by step. This is an automated message from

[GitHub] [airflow] mik-laj commented on a change in pull request #10664: Validate values.yaml

2020-09-07 Thread GitBox
mik-laj commented on a change in pull request #10664: URL: https://github.com/apache/airflow/pull/10664#discussion_r484427991 ## File path: chart/values.schema.json ## @@ -0,0 +1,1250 @@ +{ +"$schema": "http://json-schema.org/draft-07/schema;, +"description": "Default

[GitHub] [airflow] mickaelgervais commented on a change in pull request #10606: Feature/airflow 10605

2020-09-07 Thread GitBox
mickaelgervais commented on a change in pull request #10606: URL: https://github.com/apache/airflow/pull/10606#discussion_r484425706 ## File path: airflow/kubernetes/pod_generator.py ## @@ -224,6 +228,8 @@ def __init__( # pylint: disable=too-many-arguments,too-many-locals

[GitHub] [airflow] mickaelgervais commented on a change in pull request #10606: Feature/airflow 10605

2020-09-07 Thread GitBox
mickaelgervais commented on a change in pull request #10606: URL: https://github.com/apache/airflow/pull/10606#discussion_r484425495 ## File path: airflow/kubernetes/pod_generator.py ## @@ -292,6 +299,15 @@ def __init__( # pylint: disable=too-many-arguments,too-many-locals

[GitHub] [airflow] mickaelgervais commented on a change in pull request #10606: Feature/airflow 10605

2020-09-07 Thread GitBox
mickaelgervais commented on a change in pull request #10606: URL: https://github.com/apache/airflow/pull/10606#discussion_r484424833 ## File path: airflow/kubernetes/pod_generator.py ## @@ -237,7 +243,8 @@ def __init__( # pylint: disable=too-many-arguments,too-many-locals

[GitHub] [airflow] boring-cyborg[bot] commented on issue #10781: Setting cookie_samesite to None in airflow.cfg throws an error

2020-09-07 Thread GitBox
boring-cyborg[bot] commented on issue #10781: URL: https://github.com/apache/airflow/issues/10781#issuecomment-688292459 Thanks for opening your first issue here! Be sure to follow the issue template! This is an

[GitHub] [airflow] pbaisla opened a new issue #10781: Setting cookie_samesite to None in airflow.cfg throws an error

2020-09-07 Thread GitBox
pbaisla opened a new issue #10781: URL: https://github.com/apache/airflow/issues/10781 **Apache Airflow version**: 1.10.10 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): N/A **Environment**: Airflow v1.10.10, Python v3.6.11 on Ubuntu 16.04

[GitHub] [airflow] flvndh commented on a change in pull request #10664: Validate values.yaml

2020-09-07 Thread GitBox
flvndh commented on a change in pull request #10664: URL: https://github.com/apache/airflow/pull/10664#discussion_r484372296 ## File path: chart/values.schema.json ## @@ -0,0 +1,1250 @@ +{ +"$schema": "http://json-schema.org/draft-07/schema;, +"description": "Default

[GitHub] [airflow] boring-cyborg[bot] commented on issue #10780: Backported BigQueryExecuteQueryOperator throws 409 Already Exists in job with multiple operator instances

2020-09-07 Thread GitBox
boring-cyborg[bot] commented on issue #10780: URL: https://github.com/apache/airflow/issues/10780#issuecomment-688263038 Thanks for opening your first issue here! Be sure to follow the issue template! This is an

[GitHub] [airflow] rbkn opened a new issue #10780: Backported BigQueryExecuteQueryOperator throws 409 Already Exists in job with multiple operator instances

2020-09-07 Thread GitBox
rbkn opened a new issue #10780: URL: https://github.com/apache/airflow/issues/10780 **Airflow Version: v1.10.10** **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): `Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0",

[GitHub] [airflow] XD-DENG commented on pull request #10774: Always return a list from s3 list methods.

2020-09-07 Thread GitBox
XD-DENG commented on pull request #10774: URL: https://github.com/apache/airflow/pull/10774#issuecomment-688262170 To correct myself, rather than CHANGELOG, I meant `UPDATING.md` (https://github.com/apache/airflow/blob/master/UPDATING.md)

[GitHub] [airflow] flvndh commented on pull request #10664: Validate values.yaml

2020-09-07 Thread GitBox
flvndh commented on pull request #10664: URL: https://github.com/apache/airflow/pull/10664#issuecomment-688251630 > It would be fantastic if we could generate documentation for README.md based on this, but we should do it in a separate PR. It has a lot of potential. Indeed, this is

[GitHub] [airflow] flvndh commented on a change in pull request #10664: Validate values.yaml

2020-09-07 Thread GitBox
flvndh commented on a change in pull request #10664: URL: https://github.com/apache/airflow/pull/10664#discussion_r484350265 ## File path: chart/values.schema.json ## @@ -0,0 +1,1250 @@ +{ +"$schema": "http://json-schema.org/draft-07/schema;, +"description": "Default

[GitHub] [airflow] turbaszek commented on a change in pull request #9467: Add airflow upgrade-check command (MVP) (v1-10-test)

2020-09-07 Thread GitBox
turbaszek commented on a change in pull request #9467: URL: https://github.com/apache/airflow/pull/9467#discussion_r484342592 ## File path: airflow/upgrade/formatters.py ## @@ -0,0 +1,114 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] JeffryMAC commented on pull request #10081: Add CLI for connections export (#9856)

2020-09-07 Thread GitBox
JeffryMAC commented on pull request #10081: URL: https://github.com/apache/airflow/pull/10081#issuecomment-688230597 @rootcss can this export be supported also from the UI? This is an automated message from the Apache Git

[airflow] branch master updated: Refactor DataprocCreateCluster operator to use simpler interface (#10403)

2020-09-07 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new c8ee455 Refactor DataprocCreateCluster

  1   2   >