[GitHub] [airflow] pingzh opened a new pull request #10654: Add CeleryKubernetesExecutor

2020-08-30 Thread GitBox
pingzh opened a new pull request #10654: URL: https://github.com/apache/airflow/pull/10654 it consists of CeleryExecutor and KubernetesExecutor, which allows users to route their tasks to either Kubernetes or Celery based on the queue defined on a task --- **^ Add

[GitHub] [airflow] amithmathew commented on issue #8803: Impersonate service accounts while running GCP Operators without key material (if airflow is running on GCP)

2020-08-30 Thread GitBox
amithmathew commented on issue #8803: URL: https://github.com/apache/airflow/issues/8803#issuecomment-683527681 Took a look and did some quick and dirty tests with the `impersonation_chain` parameter. Looks good to me. I would love to see this implemented for Dataflow as well, will

[GitHub] [airflow] scottrigby commented on issue #10523: Host Airflow-managed Helm repo

2020-08-30 Thread GitBox
scottrigby commented on issue #10523: URL: https://github.com/apache/airflow/issues/10523#issuecomment-683526788 @dimberman I see  Thanks for clearing that up. One important thing I'd suggest is bringing in git history from the stable repo chart source (there have been a number of

[GitHub] [airflow] dimberman commented on issue #10523: Host Airflow-managed Helm repo

2020-08-30 Thread GitBox
dimberman commented on issue #10523: URL: https://github.com/apache/airflow/issues/10523#issuecomment-683521046 Ohhh kay I think there's a bit of confusion so lemme try to patch this up. @ryw @schnie @mik-laj It appears that Scott works on the helm project and is asking us

[GitHub] [airflow] scottrigby commented on issue #10523: Host Airflow-managed Helm repo

2020-08-30 Thread GitBox
scottrigby commented on issue #10523: URL: https://github.com/apache/airflow/issues/10523#issuecomment-683511997 @mik-laj I'm not sure I understand fully. Are you saying the chart at https://github.com/apache/airflow/tree/master/chart installs an enterprise edition of Airflow – or are you

[GitHub] [airflow] ohbarye commented on pull request #10647: Fix typo: duplicated "the"

2020-08-30 Thread GitBox
ohbarye commented on pull request #10647: URL: https://github.com/apache/airflow/pull/10647#issuecomment-683506884 Thanks for you quick review too! This is an automated message from the Apache Git Service. To respond to the

[airflow] tag nightly-master updated (9b2efc6 -> 4476a09)

2020-08-30 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from 9b2efc6 (commit) to 4476a09 (commit) from

[airflow] tag nightly-master updated (9b2efc6 -> 4476a09)

2020-08-30 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to tag nightly-master in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag nightly-master was modified! *** from 9b2efc6 (commit) to 4476a09 (commit) from

[GitHub] [airflow] potiuk edited a comment on pull request #10651: Implement better shell conventions for breeze

2020-08-30 Thread GitBox
potiuk edited a comment on pull request #10651: URL: https://github.com/apache/airflow/pull/10651#issuecomment-683495471 This is the first of a series of changes that brigs us much closer to a "good" convention for the bash scripts we have (mainly CI and Breeze). It's been asked several

[GitHub] [airflow] houqp commented on issue #10552: Airflow CLI client based on Stable API

2020-08-30 Thread GitBox
houqp commented on issue #10552: URL: https://github.com/apache/airflow/issues/10552#issuecomment-683497248 @mik-laj @ad-m should we create a public repo to kick off the experiment? i am interested in giving the cli a try as well.

[GitHub] [airflow] potiuk edited a comment on pull request #10651: Implement better shell conventions for breeze

2020-08-30 Thread GitBox
potiuk edited a comment on pull request #10651: URL: https://github.com/apache/airflow/pull/10651#issuecomment-683495471 This is the first of a series of changes that brigs us much closer to a "good" convention for the bash scripts we have (mainly CI and Breeze). It's been asked several

[GitHub] [airflow] houqp commented on a change in pull request #10594: WIP: Add permissions for stable API

2020-08-30 Thread GitBox
houqp commented on a change in pull request #10594: URL: https://github.com/apache/airflow/pull/10594#discussion_r479838304 ## File path: airflow/www/security.py ## @@ -521,6 +524,18 @@ def sync_roles(self): self.update_admin_perm_view() self.clean_perms()

[GitHub] [airflow] potiuk commented on pull request #10651: Implement better shell conventions for breeze

2020-08-30 Thread GitBox
potiuk commented on pull request #10651: URL: https://github.com/apache/airflow/pull/10651#issuecomment-683495471 This is the first of a series of changes that brigs us much closer to a "good" convention for the bash scripts we have (mainly CI and Breeze). It's been asked several times in

[GitHub] [airflow] potiuk edited a comment on pull request #10651: Implement better shell conventions for breeze

2020-08-30 Thread GitBox
potiuk edited a comment on pull request #10651: URL: https://github.com/apache/airflow/pull/10651#issuecomment-683495471 This is the first of a series of changes that brigs us much closer to a "good" convention for the bash scripts we have (mainly CI and Breeze). It's been asked several

[GitHub] [airflow-client-go] houqp opened a new pull request #5: add test & doc for authentication

2020-08-30 Thread GitBox
houqp opened a new pull request #5: URL: https://github.com/apache/airflow-client-go/pull/5 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] stale[bot] commented on pull request #9844: Replace deprecated SQL operators

2020-08-30 Thread GitBox
stale[bot] commented on pull request #9844: URL: https://github.com/apache/airflow/pull/9844#issuecomment-683489433 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] andreyvital commented on issue #8467: ImportError: cannot import name 'bytes_type' from 'oauthlib.common'

2020-08-30 Thread GitBox
andreyvital commented on issue #8467: URL: https://github.com/apache/airflow/issues/8467#issuecomment-683481336 Also struggling with this. Right now we're using Pomerium as a proxy in front of our internal Airflow: https://github.com/pomerium/pomerium

[GitHub] [airflow] yesemsanthoshkumar commented on a change in pull request #10531: Move scheduler basics documentation to Airflow docs

2020-08-30 Thread GitBox
yesemsanthoshkumar commented on a change in pull request #10531: URL: https://github.com/apache/airflow/pull/10531#discussion_r479822281 ## File path: docs/scheduler.rst ## @@ -18,11 +18,41 @@ Scheduler == -The Airflow scheduler monitors all tasks and DAGs, then

[GitHub] [airflow] yesemsanthoshkumar commented on a change in pull request #10531: Move scheduler basics documentation to Airflow docs

2020-08-30 Thread GitBox
yesemsanthoshkumar commented on a change in pull request #10531: URL: https://github.com/apache/airflow/pull/10531#discussion_r479822281 ## File path: docs/scheduler.rst ## @@ -18,11 +18,41 @@ Scheduler == -The Airflow scheduler monitors all tasks and DAGs, then

[GitHub] [airflow] yesemsanthoshkumar commented on a change in pull request #10531: Move scheduler basics documentation to Airflow docs

2020-08-30 Thread GitBox
yesemsanthoshkumar commented on a change in pull request #10531: URL: https://github.com/apache/airflow/pull/10531#discussion_r479822072 ## File path: docs/scheduler.rst ## @@ -18,11 +18,41 @@ Scheduler == -The Airflow scheduler monitors all tasks and DAGs, then

[GitHub] [airflow] yesemsanthoshkumar commented on a change in pull request #10343: Add extra links for google dataproc

2020-08-30 Thread GitBox
yesemsanthoshkumar commented on a change in pull request #10343: URL: https://github.com/apache/airflow/pull/10343#discussion_r479820695 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -627,6 +681,11 @@ def execute(self, context): cluster

[GitHub] [airflow] dimberman commented on pull request #10393: Simplify the K8sExecutor and K8sPodOperator

2020-08-30 Thread GitBox
dimberman commented on pull request #10393: URL: https://github.com/apache/airflow/pull/10393#issuecomment-683468309 @potiuk @kaxil @ashb @turbaszek all tests passing and added documentation. PTAL! This is an automated

[GitHub] [airflow] dimberman commented on a change in pull request #10393: Simplify the K8sExecutor and K8sPodOperator

2020-08-30 Thread GitBox
dimberman commented on a change in pull request #10393: URL: https://github.com/apache/airflow/pull/10393#discussion_r479813400 ## File path: airflow/kubernetes/pod_generator_deprecated.py ## @@ -0,0 +1,620 @@ +# Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [airflow] houqp opened a new pull request #10653: fix type hints for s3 hook read_key method

2020-08-30 Thread GitBox
houqp opened a new pull request #10653: URL: https://github.com/apache/airflow/pull/10653 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] houqp commented on pull request #10652: add securitySchemes in openapi spec

2020-08-30 Thread GitBox
houqp commented on pull request #10652: URL: https://github.com/apache/airflow/pull/10652#issuecomment-683462016 This only affects documentation for go clients, but for some other languages like typescript and java, it is used a feature gate to generate the actual client code. The

[GitHub] [airflow] houqp opened a new pull request #10652: add securitySchemes in openapi spec

2020-08-30 Thread GitBox
houqp opened a new pull request #10652: URL: https://github.com/apache/airflow/pull/10652 openapi-generator relies on this component to generate auth code in some of the clients. --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] potiuk opened a new pull request #10651: Implement better shell conventions for breeze

2020-08-30 Thread GitBox
potiuk opened a new pull request #10651: URL: https://github.com/apache/airflow/pull/10651 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #10650: Add missing imports to app.py

2020-08-30 Thread GitBox
boring-cyborg[bot] commented on pull request #10650: URL: https://github.com/apache/airflow/pull/10650#issuecomment-683455344 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] kunz07 opened a new pull request #10650: Add missing imports to app.py

2020-08-30 Thread GitBox
kunz07 opened a new pull request #10650: URL: https://github.com/apache/airflow/pull/10650 Add missing imports to app.py to fix exception thrown when force_log_out_after config setting is set to values other than 0 closes: #10053

[airflow] branch master updated (11c00bc -> 4476a09)

2020-08-30 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 11c00bc Fix typos: duplicated "the" (#10647) add 4476a09 Update INTHEWILD.md (#10649) No new revisions were

[airflow] branch master updated (11c00bc -> 4476a09)

2020-08-30 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 11c00bc Fix typos: duplicated "the" (#10647) add 4476a09 Update INTHEWILD.md (#10649) No new revisions were

[GitHub] [airflow] potiuk merged pull request #10649: Update INTHEWILD.md

2020-08-30 Thread GitBox
potiuk merged pull request #10649: URL: https://github.com/apache/airflow/pull/10649 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] lopesdiego12 opened a new pull request #10649: Update INTHEWILD.md

2020-08-30 Thread GitBox
lopesdiego12 opened a new pull request #10649: URL: https://github.com/apache/airflow/pull/10649 Added Datasprints as INTHEWILD markdown section --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] boring-cyborg[bot] commented on issue #10648: dbapi_hook target_fields not working

2020-08-30 Thread GitBox
boring-cyborg[bot] commented on issue #10648: URL: https://github.com/apache/airflow/issues/10648#issuecomment-683433932 Thanks for opening your first issue here! Be sure to follow the issue template! This is an

[GitHub] [airflow] galzak opened a new issue #10648: dbapi_hook target_fields not working

2020-08-30 Thread GitBox
galzak opened a new issue #10648: URL: https://github.com/apache/airflow/issues/10648 I have a table in Postgres with the below columns: re_ads_snapshot_id serial NOT NULL, ad_id int8 NOT NULL, origin_table_id int4 NULL My ETL suppose to insert data into this

[GitHub] [airflow] dimberman closed pull request #8926: Refactor k8s

2020-08-30 Thread GitBox
dimberman closed pull request #8926: URL: https://github.com/apache/airflow/pull/8926 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] OmairK commented on issue #10576: Adapt better variable naming conventions in our bash scripts

2020-08-30 Thread GitBox
OmairK commented on issue #10576: URL: https://github.com/apache/airflow/issues/10576#issuecomment-683429226 > I am about to submit a big Breeze change for that, so you can take a look how I 've done that and review :) @OmairK Yes that would be very helpful thanks :smile:

[GitHub] [airflow] potiuk commented on issue #10576: Adapt better variable naming conventions in our bash scripts

2020-08-30 Thread GitBox
potiuk commented on issue #10576: URL: https://github.com/apache/airflow/issues/10576#issuecomment-683428985 I am about to submit a big Breeze change for that, so you can take a look how I 've done that and review :) @OmairK

[GitHub] [airflow] aranjanthakur commented on issue #10166: Add capability to specify gunicorn access log format for airflow webserver

2020-08-30 Thread GitBox
aranjanthakur commented on issue #10166: URL: https://github.com/apache/airflow/issues/10166#issuecomment-683415872 @mik-laj I have added the test cases, could you please review the changes https://github.com/apache/airflow/pull/10261

[GitHub] [airflow] aranjanthakur commented on pull request #10261: 10166 : Add capability to specify gunicorn access log format for airflow

2020-08-30 Thread GitBox
aranjanthakur commented on pull request #10261: URL: https://github.com/apache/airflow/pull/10261#issuecomment-683413302 @mik-laj could you please review the changes? This is an automated message from the Apache Git Service.

[GitHub] [airflow] turbaszek commented on a change in pull request #10343: Add extra links for google dataproc

2020-08-30 Thread GitBox
turbaszek commented on a change in pull request #10343: URL: https://github.com/apache/airflow/pull/10343#discussion_r479758381 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -627,6 +681,11 @@ def execute(self, context): cluster =

[GitHub] [airflow] yesemsanthoshkumar commented on a change in pull request #10343: Add extra links for google dataproc

2020-08-30 Thread GitBox
yesemsanthoshkumar commented on a change in pull request #10343: URL: https://github.com/apache/airflow/pull/10343#discussion_r479757412 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -1051,17 +1051,17 @@ def execute(self, context):

[GitHub] [airflow] yesemsanthoshkumar commented on a change in pull request #10343: Add extra links for google dataproc

2020-08-30 Thread GitBox
yesemsanthoshkumar commented on a change in pull request #10343: URL: https://github.com/apache/airflow/pull/10343#discussion_r479757447 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -627,6 +681,11 @@ def execute(self, context): cluster

[airflow-site] branch asf-site updated: Update asf-site to output generated at 529754a

2020-08-30 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 1cb5c5b Update asf-site to

[airflow-site] branch master updated: Updated date for Omair's post

2020-08-30 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/master by this push: new 529754a Updated date for Omair's post

[airflow-site] branch asf-site updated: Update asf-site to output generated at ef70b5c

2020-08-30 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/asf-site by this push: new a7b Update asf-site to

[airflow-site] branch master updated: Remove draft from Omair's post

2020-08-30 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/master by this push: new ef70b5c Remove draft from Omair's post

[GitHub] [airflow] potiuk commented on pull request #10327: Move roles to CONTRIBUTING.rst

2020-08-30 Thread GitBox
potiuk commented on pull request #10327: URL: https://github.com/apache/airflow/pull/10327#issuecomment-683398788 Thanks @jpmarques66 ! One step closer to clean docs ! Looking forward to next contributions! This is an

[GitHub] [airflow] potiuk closed issue #10274: Add Ecosystem page to Airflow Website

2020-08-30 Thread GitBox
potiuk closed issue #10274: URL: https://github.com/apache/airflow/issues/10274 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on issue #10180: Review and move Common Pitfalls Documentation to Airflow Docs

2020-08-30 Thread GitBox
potiuk commented on issue #10180: URL: https://github.com/apache/airflow/issues/10180#issuecomment-683397925 I've assigned it to you This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] potiuk commented on issue #10180: Review and move Common Pitfalls Documentation to Airflow Docs

2020-08-30 Thread GitBox
potiuk commented on issue #10180: URL: https://github.com/apache/airflow/issues/10180#issuecomment-683397890 Hello - just see the message now. Sorry I missed it :( The context - we had a lot of documentation which was split into several different sources - Github docs, Cwiki,

[airflow-site] branch asf-site updated: Update asf-site to output generated at c6fa911

2020-08-30 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 8747e59 Update asf-site to

[airflow-site] branch master updated: [Blog entry] Journey with Airflow as an Outreachy intern (#280)

2020-08-30 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/master by this push: new c6fa911 [Blog entry] Journey with

[GitHub] [airflow-site] potiuk commented on pull request #280: [Blog entry] Journey with Airflow as an Outreachy intern

2020-08-30 Thread GitBox
potiuk commented on pull request #280: URL: https://github.com/apache/airflow-site/pull/280#issuecomment-683396612 Love it. Merging ! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow-site] potiuk merged pull request #280: [Blog entry] Journey with Airflow as an Outreachy intern

2020-08-30 Thread GitBox
potiuk merged pull request #280: URL: https://github.com/apache/airflow-site/pull/280 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow-site] potiuk commented on a change in pull request #280: [Blog entry] Journey with Airflow as an Outreachy intern

2020-08-30 Thread GitBox
potiuk commented on a change in pull request #280: URL: https://github.com/apache/airflow-site/pull/280#discussion_r479744452 ## File path: landing-pages/site/content/en/blog/experience-with-airflow-as-an-outreachy-intern.md ## @@ -0,0 +1,60 @@ +--- +title: "Journey with

[GitHub] [airflow] potiuk commented on issue #10576: Adapt better variable naming conventions in our bash scripts

2020-08-30 Thread GitBox
potiuk commented on issue #10576: URL: https://github.com/apache/airflow/issues/10576#issuecomment-683396349 Sure @OmairK. Feel free :) Ideally one-by-one :) This is an automated message from the Apache Git Service. To

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #10647: Fix typo: duplicated "the"

2020-08-30 Thread GitBox
boring-cyborg[bot] commented on pull request #10647: URL: https://github.com/apache/airflow/pull/10647#issuecomment-683389382 Awesome work, congrats on your first merged pull request! This is an automated message from

[airflow] branch master updated: Fix typos: duplicated "the" (#10647)

2020-08-30 Thread xddeng
This is an automated email from the ASF dual-hosted git repository. xddeng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 11c00bc Fix typos: duplicated "the" (#10647)

[GitHub] [airflow] XD-DENG merged pull request #10647: Fix typo: duplicated "the"

2020-08-30 Thread GitBox
XD-DENG merged pull request #10647: URL: https://github.com/apache/airflow/pull/10647 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #10647: Fix typo: duplicated "the"

2020-08-30 Thread GitBox
boring-cyborg[bot] commented on pull request #10647: URL: https://github.com/apache/airflow/pull/10647#issuecomment-683386293 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] ohbarye opened a new pull request #10647: Fix typo: duplicated "the"

2020-08-30 Thread GitBox
ohbarye opened a new pull request #10647: URL: https://github.com/apache/airflow/pull/10647 Just fixed typo: duplicated "the" --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] aimran-adroll opened a new issue #10646: Kubernetes config dags_volume_subpath breaks PVC in helm chart

2020-08-30 Thread GitBox
aimran-adroll opened a new issue #10646: URL: https://github.com/apache/airflow/issues/10646 **Apache Airflow version**: 1.10.12, master **Kubernetes version: v1.17.9-eks-4c6976 (server)/ v.1.18.6 (client) **Environment**: - **Cloud provider or hardware