[GitHub] [airflow] jhtimmins commented on issue #10469: Make permissions consistent between UI and API

2020-08-21 Thread GitBox
jhtimmins commented on issue #10469: URL: https://github.com/apache/airflow/issues/10469#issuecomment-678597419 @mik-laj @kaxil @ashb @potiuk Would love to hear your thoughts on renaming permissions to make them consistent across the UI and API.

[GitHub] [airflow] jhtimmins opened a new issue #10469: Make permissions consistent between UI and API

2020-08-21 Thread GitBox
jhtimmins opened a new issue #10469: URL: https://github.com/apache/airflow/issues/10469 ## Rationale Airflow currently has 185 permissions (where 1 permission is a combo of Class + Action). The design of the permissions is inconsistent, making them difficult to modify or organize into

[airflow] branch constraints-master updated: Updating constraints. Build id:219186797

2020-08-21 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-master by this push: new a8d9dc7 Updating

[airflow] branch constraints-master updated: Updating constraints. Build id:219186797

2020-08-21 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-master by this push: new a8d9dc7 Updating

[GitHub] [airflow] kaxil opened a new pull request #10468: [PEP 577] Replace assigment with Augmented assignment

2020-08-21 Thread GitBox
kaxil opened a new pull request #10468: URL: https://github.com/apache/airflow/pull/10468 Use PEP 577: https://www.python.org/dev/peps/pep-0577/ --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] kaxil opened a new pull request #10467: Fix identation in executor_config example

2020-08-21 Thread GitBox
kaxil opened a new pull request #10467: URL: https://github.com/apache/airflow/pull/10467 Few tabs weren't indented properly. This will now make it easy to copy paste --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] kaxil opened a new pull request #10466: Add Type Annotations & Docstrings to airflow/models/dagrun.py

2020-08-21 Thread GitBox
kaxil opened a new pull request #10466: URL: https://github.com/apache/airflow/pull/10466 Add "Type Annotations" & Docstrings to `airflow/models/dagrun.py` --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] kaxil opened a new pull request #10465: Test exact match of Executor name

2020-08-21 Thread GitBox
kaxil opened a new pull request #10465: URL: https://github.com/apache/airflow/pull/10465 Use `self.assertEqual` instead of `self.assertIn` to do an exact match of string name instead of partial match --- **^ Add meaningful description above** Read the **[Pull

[GitHub] [airflow] kaxil opened a new pull request #10464: Remove redudandant checks in test_views.py

2020-08-21 Thread GitBox
kaxil opened a new pull request #10464: URL: https://github.com/apache/airflow/pull/10464 - `self.check_content_in_response` already checks that response code is 200 - `self.assertEqual(None, ...)` -> `self.assertIsNone(...)` - Fix typo: "succcess" -> `success` ---

[GitHub] [airflow] kaxil opened a new pull request #10463: Fix typos in scripts/perf/scheduler_dag_execution_timing.py

2020-08-21 Thread GitBox
kaxil opened a new pull request #10463: URL: https://github.com/apache/airflow/pull/10463 `MockExcutor` -> `MockExecutor` `ShortCircutExecutor` -> `ShortCircuitExecutor` --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] syudb opened a new pull request #10462: Updated REST API call so GET requests pass payload in query string instead of request body

2020-08-21 Thread GitBox
syudb opened a new pull request #10462: URL: https://github.com/apache/airflow/pull/10462 This PR addresses an issue with the way GET requests were being built/sent. In the original implementation, the request payload is sent in the body of the request and not in the query string

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #10461: Updated REST API call so GET requests pass payload in query string instead of request body

2020-08-21 Thread GitBox
boring-cyborg[bot] commented on pull request #10461: URL: https://github.com/apache/airflow/pull/10461#issuecomment-678556493 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] syudb opened a new pull request #10461: Updated REST API call so GET requests pass payload in query string instead of request body

2020-08-21 Thread GitBox
syudb opened a new pull request #10461: URL: https://github.com/apache/airflow/pull/10461 This PR addresses an issue with the way GET requests were being built/sent. In the original implementation, the request payload is sent in the body of the request and not in the query string

[GitHub] [airflow] kaxil opened a new pull request #10460: Remove mentions of Airflow Gitter

2020-08-21 Thread GitBox
kaxil opened a new pull request #10460: URL: https://github.com/apache/airflow/pull/10460 Remove mentions of Airflow Gitter as it isn't that active. --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] kaxil opened a new pull request #10459: Fix typo in timed_out

2020-08-21 Thread GitBox
kaxil opened a new pull request #10459: URL: https://github.com/apache/airflow/pull/10459 `timeouted` -> `timed_out` --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] kaxil opened a new pull request #10458: Fix typo in "wait_for_done"

2020-08-21 Thread GitBox
kaxil opened a new pull request #10458: URL: https://github.com/apache/airflow/pull/10458 `waitfordone` -> `wait_for_done` --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] kaxil opened a new pull request #10457: Use assertEqual instead of assertEqual

2020-08-21 Thread GitBox
kaxil opened a new pull request #10457: URL: https://github.com/apache/airflow/pull/10457 `assertEqual` will show show the proper diff instead of just "False is not True" error --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] atsalolikhin-spokeo commented on issue #10389: Documentation is missing instructions for preparing database for Airflow (during installation)

2020-08-21 Thread GitBox
atsalolikhin-spokeo commented on issue #10389: URL: https://github.com/apache/airflow/issues/10389#issuecomment-678534509 Yes, I did. I use GNU Screen, so I closed that window, and started another window (which starts a fresh bash process). On Fri, Aug 21, 2020 at 1:41 PM Jarek

[GitHub] [airflow] dimon222 opened a new issue #10456: PodRuntimeInfoEnv is not working in 1.10.11

2020-08-21 Thread GitBox
dimon222 opened a new issue #10456: URL: https://github.com/apache/airflow/issues/10456

[GitHub] [airflow] kaxil commented on issue #10455: Stale documentation for Kubernetes Executor

2020-08-21 Thread GitBox
kaxil commented on issue #10455: URL: https://github.com/apache/airflow/issues/10455#issuecomment-678484327 aah nice, yes I have marked it for 1.10.13 :) This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] potiuk commented on issue #10389: Documentation is missing instructions for preparing database for Airflow (during installation)

2020-08-21 Thread GitBox
potiuk commented on issue #10389: URL: https://github.com/apache/airflow/issues/10389#issuecomment-678484017 Did you log out/log in as suggested by the message ? Bash cashes the binaries it uses so even if your PATH is updated you might still use the old getopt. I believe there is a

[GitHub] [airflow] rgrizzell edited a comment on issue #10024: Writing Logs to Elasticsearch: asctime field in JSON log output set to null

2020-08-21 Thread GitBox
rgrizzell edited a comment on issue #10024: URL: https://github.com/apache/airflow/issues/10024#issuecomment-678469071 A further deep dive into this issue, I believe this is the result of default behavior in the Python logging module. When utilizing the [PercentStyle

[GitHub] [airflow] rgrizzell edited a comment on issue #10024: Writing Logs to Elasticsearch: asctime field in JSON log output set to null

2020-08-21 Thread GitBox
rgrizzell edited a comment on issue #10024: URL: https://github.com/apache/airflow/issues/10024#issuecomment-678469071 A further deep dive into this issue, I believe this is the result of default behavior in the Python logging module. When utilizing the [PercentStyle

[GitHub] [airflow] ephraimbuddy commented on pull request #10333: Unify error messages and complete type field in response

2020-08-21 Thread GitBox
ephraimbuddy commented on pull request #10333: URL: https://github.com/apache/airflow/pull/10333#issuecomment-678475229 I don't really know what caused the earlier breeze issues廊. I had to delete the branch and recreate it at an earlier commit. Then after rebasing, it worked. However, I

[GitHub] [airflow] gmontanola commented on pull request #10436: Fixes S3ToRedshift COPY query

2020-08-21 Thread GitBox
gmontanola commented on pull request #10436: URL: https://github.com/apache/airflow/pull/10436#issuecomment-678473322 Haha, the emojis are a courtesy of https://github.com/commitizen/cz-cli. When you say if it is backwards compatible you mean if it will work with older versions?

[GitHub] [airflow] XD-DENG commented on issue #10455: Stale documentation for Kubernetes Executor

2020-08-21 Thread GitBox
XD-DENG commented on issue #10455: URL: https://github.com/apache/airflow/issues/10455#issuecomment-678470644 A kind ping to our K8S subject expert @dimberman ;-) @kaxil we may want to address this doc change in our next release as well ? If I missed or misunderstood anything,

[GitHub] [airflow] XD-DENG opened a new issue #10455: Stale documentation for Kubernetes Executor

2020-08-21 Thread GitBox
XD-DENG opened a new issue #10455: URL: https://github.com/apache/airflow/issues/10455 **What happened**: In documentation about Kubenetes Executor (https://airflow.apache.org/docs/stable/executor/kubernetes.html or

[GitHub] [airflow] rgrizzell commented on issue #10024: Writing Logs to Elasticsearch: asctime field in JSON log output set to null

2020-08-21 Thread GitBox
rgrizzell commented on issue #10024: URL: https://github.com/apache/airflow/issues/10024#issuecomment-678469071 A further deep dive into this issue, I believe this is the result of default behavior in the Python logging module. When utilizing the [PercentStyle

[GitHub] [airflow] atsalolikhin-spokeo commented on issue #10389: Documentation is missing instructions for preparing database for Airflow (during installation)

2020-08-21 Thread GitBox
atsalolikhin-spokeo commented on issue #10389: URL: https://github.com/apache/airflow/issues/10389#issuecomment-678464759 I am using bash. On Fri, Aug 21, 2020 at 12:48 PM Jarek Potiuk wrote: > Are you using zsh ? If so can you run the bash command suggested ? I think >

[airflow] branch master updated (7dd6e73 -> c979de7)

2020-08-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 7dd6e73 Mount gcloud kubeconifg to breeze (#10439) add c979de7 Fixes problem with Python image not ready to

[airflow] branch master updated (7dd6e73 -> c979de7)

2020-08-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 7dd6e73 Mount gcloud kubeconifg to breeze (#10439) add c979de7 Fixes problem with Python image not ready to

[GitHub] [airflow] potiuk merged pull request #10430: Fixes problem with Python image not ready to be pushed

2020-08-21 Thread GitBox
potiuk merged pull request #10430: URL: https://github.com/apache/airflow/pull/10430 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk commented on issue #10389: Documentation is missing instructions for preparing database for Airflow (during installation)

2020-08-21 Thread GitBox
potiuk commented on issue #10389: URL: https://github.com/apache/airflow/issues/10389#issuecomment-678462371 Are you using zsh ? If so can you run the bash command suggested ? I think inside the pre-commits we always use bash so the message is actually misleading - I can correct it if we

[airflow] branch master updated (3357d8d -> 7dd6e73)

2020-08-21 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 3357d8d Fix port number in webserver for kind setup (#10452) add 7dd6e73 Mount gcloud kubeconifg to

[airflow] branch master updated (3357d8d -> 7dd6e73)

2020-08-21 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 3357d8d Fix port number in webserver for kind setup (#10452) add 7dd6e73 Mount gcloud kubeconifg to

[GitHub] [airflow] turbaszek merged pull request #10439: Mount gcloud kubeconifg to breeze

2020-08-21 Thread GitBox
turbaszek merged pull request #10439: URL: https://github.com/apache/airflow/pull/10439 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] turbaszek commented on pull request #10453: Allow to specify path to kubeconfig in KubernetesHook

2020-08-21 Thread GitBox
turbaszek commented on pull request #10453: URL: https://github.com/apache/airflow/pull/10453#issuecomment-678451945 CC: @roitvt This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] turbaszek commented on a change in pull request #10447: Add connection caching to KubernetesHook

2020-08-21 Thread GitBox
turbaszek commented on a change in pull request #10447: URL: https://github.com/apache/airflow/pull/10447#discussion_r474886643 ## File path: airflow/providers/cncf/kubernetes/hooks/kubernetes.py ## @@ -46,11 +46,14 @@ def __init__( ): super().__init__()

[GitHub] [airflow] atsalolikhin-spokeo commented on issue #10389: Documentation is missing instructions for preparing database for Airflow (during installation)

2020-08-21 Thread GitBox
atsalolikhin-spokeo commented on issue #10389: URL: https://github.com/apache/airflow/issues/10389#issuecomment-678445952 ``` [~/git/airflow] fix/10389 ± pre-commit run build --verbose Check if image build is needed...Failed - hook id: build

[GitHub] [airflow] potiuk commented on pull request #10445: Building backport packages generates README files

2020-08-21 Thread GitBox
potiuk commented on pull request #10445: URL: https://github.com/apache/airflow/pull/10445#issuecomment-678437459 Interesting - I will take a look if this was not some teething problem. There is a potential problem I was anticipating might happen - which could have similar behaviour.

[GitHub] [airflow] potiuk commented on pull request #10446: Add kubernetes connection type

2020-08-21 Thread GitBox
potiuk commented on pull request #10446: URL: https://github.com/apache/airflow/pull/10446#issuecomment-678435531 My questions was answered beautifully. Thanks @kaxil @turbaszek ! This is an automated message from the Apache

[GitHub] [airflow] potiuk edited a comment on pull request #10446: Add kubernetes connection type

2020-08-21 Thread GitBox
potiuk edited a comment on pull request #10446: URL: https://github.com/apache/airflow/pull/10446#issuecomment-678435531 My question was answered beautifully. Thanks @kaxil @turbaszek ! This is an automated message from the

[GitHub] [airflow] potiuk commented on pull request #10439: Mount gcloud kubeconifg to breeze

2020-08-21 Thread GitBox
potiuk commented on pull request #10439: URL: https://github.com/apache/airflow/pull/10439#issuecomment-678434145 Nice! This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on a change in pull request #10441: Fix docs build uploading artifacts

2020-08-21 Thread GitBox
potiuk commented on a change in pull request #10441: URL: https://github.com/apache/airflow/pull/10441#discussion_r474863556 ## File path: tests/conftest.py ## @@ -38,11 +38,6 @@ sys.path.append(perf_directory) -from perf_kit.sqlalchemy import ( # noqa: E402

[GitHub] [airflow] potiuk commented on a change in pull request #10441: Fix docs build uploading artifacts

2020-08-21 Thread GitBox
potiuk commented on a change in pull request #10441: URL: https://github.com/apache/airflow/pull/10441#discussion_r474863556 ## File path: tests/conftest.py ## @@ -38,11 +38,6 @@ sys.path.append(perf_directory) -from perf_kit.sqlalchemy import ( # noqa: E402

[airflow] branch master updated (f0727b3 -> 3357d8d)

2020-08-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f0727b3 Change provider configuration keys for OAuth (#9759) add 3357d8d Fix port number in webserver for

[airflow] branch master updated (f0727b3 -> 3357d8d)

2020-08-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from f0727b3 Change provider configuration keys for OAuth (#9759) add 3357d8d Fix port number in webserver for

[GitHub] [airflow] potiuk merged pull request #10452: Fix port number in webserver for kind setup

2020-08-21 Thread GitBox
potiuk merged pull request #10452: URL: https://github.com/apache/airflow/pull/10452 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk commented on issue #10389: Documentation is missing instructions for preparing database for Airflow (during installation)

2020-08-21 Thread GitBox
potiuk commented on issue #10389: URL: https://github.com/apache/airflow/issues/10389#issuecomment-678425401 Hey @atsalolikhin-spokeo - please rebase to latest master. I fixed this morning a Mac related problem that would look very much like this: #10440

[GitHub] [airflow] potiuk commented on issue #10454: Add Terraform Hook

2020-08-21 Thread GitBox
potiuk commented on issue #10454: URL: https://github.com/apache/airflow/issues/10454#issuecomment-678424669 Looking forward to it! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] mik-laj commented on pull request #10052: Add impersonation to google operators

2020-08-21 Thread GitBox
mik-laj commented on pull request #10052: URL: https://github.com/apache/airflow/pull/10052#issuecomment-678412038 Are you planning to add impersonations for DataflowCreatePythonJobOperator and DataflowCreateJavaJobOperator, GKEStartPodOperator and BigQueryTablePartitionExistenceSensor?

[GitHub] [airflow] mik-laj commented on pull request #10052: Add impersonation to google operators

2020-08-21 Thread GitBox
mik-laj commented on pull request #10052: URL: https://github.com/apache/airflow/pull/10052#issuecomment-678406046 I would like to do a review. Please wait a moment. This is an automated message from the Apache Git Service.

[GitHub] [airflow] jaketf opened a new issue #10454: Add Terraform Hook

2020-08-21 Thread GitBox
jaketf opened a new issue #10454: URL: https://github.com/apache/airflow/issues/10454 **Description** Create a terraform integration for apache airflow. **Use case / motivation** Use terraform to manage ephemeral infrastructure used in airflow DAGs taking advantage

[GitHub] [airflow] atsalolikhin-spokeo commented on issue #10389: Documentation is missing instructions for preparing database for Airflow (during installation)

2020-08-21 Thread GitBox
atsalolikhin-spokeo commented on issue #10389: URL: https://github.com/apache/airflow/issues/10389#issuecomment-678398482 ``` [~/git/airflow] fix/10389 ± git commit --verify No-tabs

[GitHub] [airflow] turbaszek commented on pull request #10052: Add impersonation to google operators

2020-08-21 Thread GitBox
turbaszek commented on pull request #10052: URL: https://github.com/apache/airflow/pull/10052#issuecomment-678396131 That's huge. The docs look good and I hope that the auto refactors worked as expected. Regarding the example, I think we should add it but we can create an issue for that

[GitHub] [airflow] atsalolikhin-spokeo commented on issue #10389: Documentation is missing instructions for preparing database for Airflow (during installation)

2020-08-21 Thread GitBox
atsalolikhin-spokeo commented on issue #10389: URL: https://github.com/apache/airflow/issues/10389#issuecomment-678395214 ``` [~/git/airflow] fix/10389 ± pre-commit run build --verbose Check if image build is needed...Failed - hook id: build

[GitHub] [airflow] mik-laj commented on a change in pull request #10447: Add connection caching to KubernetesHook

2020-08-21 Thread GitBox
mik-laj commented on a change in pull request #10447: URL: https://github.com/apache/airflow/pull/10447#discussion_r474820447 ## File path: airflow/providers/cncf/kubernetes/hooks/kubernetes.py ## @@ -46,11 +46,14 @@ def __init__( ): super().__init__()

[airflow] branch master updated (1cf1af6 -> f0727b3)

2020-08-21 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 1cf1af6 Do not override in_container scripts when building the image (#10442) add f0727b3 Change

[GitHub] [airflow] turbaszek commented on a change in pull request #10439: Mount gcloud kubeconifg to breeze

2020-08-21 Thread GitBox
turbaszek commented on a change in pull request #10439: URL: https://github.com/apache/airflow/pull/10439#discussion_r474811192 ## File path: scripts/ci/docker-compose/forward-credentials.yml ## @@ -29,3 +29,4 @@ services: - ${HOME}/.azure:/root/.azure:cached -

[airflow] branch master updated (1cf1af6 -> f0727b3)

2020-08-21 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 1cf1af6 Do not override in_container scripts when building the image (#10442) add f0727b3 Change

[GitHub] [airflow] smishra commented on issue #10451: Airflow tries running Jinja on any property name ending with .json

2020-08-21 Thread GitBox
smishra commented on issue #10451: URL: https://github.com/apache/airflow/issues/10451#issuecomment-678385795 In case you want to answer it on StackOverflow (for others to find):

[airflow] branch master updated: Change provider configuration keys for OAuth (#9759)

2020-08-21 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new f0727b3 Change provider configuration

[GitHub] [airflow] mik-laj closed issue #9662: Replace flask_oauthlib with Authlib

2020-08-21 Thread GitBox
mik-laj closed issue #9662: URL: https://github.com/apache/airflow/issues/9662 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] mik-laj merged pull request #9759: Change provider configuration keys for OAuth

2020-08-21 Thread GitBox
mik-laj merged pull request #9759: URL: https://github.com/apache/airflow/pull/9759 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] turbaszek opened a new pull request #10453: Allow to specify path to kubeconfig in KubernetesHook

2020-08-21 Thread GitBox
turbaszek opened a new pull request #10453: URL: https://github.com/apache/airflow/pull/10453 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] turbaszek commented on a change in pull request #10304: WIP Added operator, hook, docs and tests for Dataprep

2020-08-21 Thread GitBox
turbaszek commented on a change in pull request #10304: URL: https://github.com/apache/airflow/pull/10304#discussion_r474802652 ## File path: docs/howto/operator/google/cloud/dataprep.rst ## @@ -17,7 +17,25 @@ Google Dataprep Operators = -`Google

[GitHub] [airflow] turbaszek commented on a change in pull request #10304: WIP Added operator, hook, docs and tests for Dataprep

2020-08-21 Thread GitBox
turbaszek commented on a change in pull request #10304: URL: https://github.com/apache/airflow/pull/10304#discussion_r474802652 ## File path: docs/howto/operator/google/cloud/dataprep.rst ## @@ -17,7 +17,25 @@ Google Dataprep Operators = -`Google

[GitHub] [airflow] ltken123 removed a comment on issue #8212: Can't read S3 remote logs using Airflow 1.10.9

2020-08-21 Thread GitBox
ltken123 removed a comment on issue #8212: URL: https://github.com/apache/airflow/issues/8212#issuecomment-678015944 > Having gotten remote logging working on 1.10.10 I've noticed there seems to be a difference in how the workers handle the log upload versus the webserver component. The

[GitHub] [airflow] ephraimbuddy commented on pull request #10333: Unify error messages and complete type field in response

2020-08-21 Thread GitBox
ephraimbuddy commented on pull request #10333: URL: https://github.com/apache/airflow/pull/10333#issuecomment-678372479 Nice, I will do that. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] OmairK commented on pull request #10333: Unify error messages and complete type field in response

2020-08-21 Thread GitBox
OmairK commented on pull request #10333: URL: https://github.com/apache/airflow/pull/10333#issuecomment-678365011 @ephraimbuddy can you rebase to the latest master, #9740 is merged can you update the error message for `patch_dag` endpoint, thanks in advance. :smile_cat:

[airflow] branch master updated (fdd68ec -> 1cf1af6)

2020-08-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from fdd68ec Make system test work with 1.10 (#10444) add 1cf1af6 Do not override in_container scripts when

[airflow] branch master updated (fdd68ec -> 1cf1af6)

2020-08-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from fdd68ec Make system test work with 1.10 (#10444) add 1cf1af6 Do not override in_container scripts when

[GitHub] [airflow] potiuk merged pull request #10442: Do not override in_container scripts when building the image

2020-08-21 Thread GitBox
potiuk merged pull request #10442: URL: https://github.com/apache/airflow/pull/10442 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk commented on pull request #10442: Do not override in_container scripts when building the image

2020-08-21 Thread GitBox
potiuk commented on pull request #10442: URL: https://github.com/apache/airflow/pull/10442#issuecomment-678348127 Looks good: https://github.com/potiuk/airflow/runs/1012859548?check_suite_focus=true pushing it. I had to do another thing - disable the 6000 "skipped" tests in integration

[GitHub] [airflow] smishra opened a new issue #10451: Airflow tries running Jinja on any property name ending with .json

2020-08-21 Thread GitBox
smishra opened a new issue #10451: URL: https://github.com/apache/airflow/issues/10451 Looks like there is a bug in Airflow templated operator argument when it has any string that ends with _.json._ Following is my DAG - please notice **"--files", "s3://dummy/spark/application.json"** in

[GitHub] [airflow] boring-cyborg[bot] commented on issue #10451: Airflow tries running Jinja on any property name ending with .json

2020-08-21 Thread GitBox
boring-cyborg[bot] commented on issue #10451: URL: https://github.com/apache/airflow/issues/10451#issuecomment-678345450 Thanks for opening your first issue here! Be sure to follow the issue template! This is an

[GitHub] [airflow] kaxil commented on pull request #10446: Add kubernetes connection type

2020-08-21 Thread GitBox
kaxil commented on pull request #10446: URL: https://github.com/apache/airflow/pull/10446#issuecomment-678335857 > Using cli - right. Using webui - no. You can't create "custom conn type" due to input validation. No strong opinion here, I just thought that having this conn type may help

[airflow] branch master updated: Make system test work with 1.10 (#10444)

2020-08-21 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new fdd68ec Make system test work with 1.10

[airflow] branch master updated: Make system test work with 1.10 (#10444)

2020-08-21 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new fdd68ec Make system test work with 1.10

[GitHub] [airflow] turbaszek merged pull request #10444: Make system test work with 1.10

2020-08-21 Thread GitBox
turbaszek merged pull request #10444: URL: https://github.com/apache/airflow/pull/10444 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] turbaszek edited a comment on pull request #10446: Add kubernetes connection type

2020-08-21 Thread GitBox
turbaszek edited a comment on pull request #10446: URL: https://github.com/apache/airflow/pull/10446#issuecomment-678327477 Using cli - right. Using webui - no. You can't create "custom conn type" due to input validation. No strong opinion here, I just thought that having this conn type

[GitHub] [airflow] turbaszek commented on pull request #10446: Add kubernetes connection type

2020-08-21 Thread GitBox
turbaszek commented on pull request #10446: URL: https://github.com/apache/airflow/pull/10446#issuecomment-678327477 Using cli - right. Using webui - no. You can't create "custom conn type". This is an automated message

[airflow] branch master updated (95f4768 -> 5bf47e3)

2020-08-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 95f4768 Building backport packages generates README files (#10445) add 5bf47e3 Be nice to fork repositories

[GitHub] [airflow] potiuk commented on pull request #10448: Stop running Scheduled Workflows in forks

2020-08-21 Thread GitBox
potiuk commented on pull request #10448: URL: https://github.com/apache/airflow/pull/10448#issuecomment-678316064 Ah .. too fast .. .it took the old name from the commit :( . Sorry @kaxil .. This is an automated message from

[GitHub] [airflow] potiuk merged pull request #10448: Stop running Scheduled Workflows in forks

2020-08-21 Thread GitBox
potiuk merged pull request #10448: URL: https://github.com/apache/airflow/pull/10448 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk commented on issue #10450: Implement retention for our CI images

2020-08-21 Thread GitBox
potiuk commented on issue #10450: URL: https://github.com/apache/airflow/issues/10450#issuecomment-678313424 cc: @ad-m @turbaszek This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] potiuk opened a new issue #10450: Implement retention for our CI images

2020-08-21 Thread GitBox
potiuk opened a new issue #10450: URL: https://github.com/apache/airflow/issues/10450 **Description** We started pushing quite a number of images to GitHub Registry we should clean them up. **Use case / motivation** We should run a cron action that should clean-up the

[GitHub] [airflow] frosk1 opened a new issue #10449: Dynamic Schedule Intervals

2020-08-21 Thread GitBox
frosk1 opened a new issue #10449: URL: https://github.com/apache/airflow/issues/10449 **Description** There are several question (issues) on stack overflow, asking for the need of a dynamic schedule interval. This means, the ability to change the schedule interval after DAG

[GitHub] [airflow] boring-cyborg[bot] commented on issue #10449: Dynamic Schedule Intervals

2020-08-21 Thread GitBox
boring-cyborg[bot] commented on issue #10449: URL: https://github.com/apache/airflow/issues/10449#issuecomment-678312262 Thanks for opening your first issue here! Be sure to follow the issue template! This is an

[GitHub] [airflow] kaxil commented on pull request #10446: Add kubernetes connection type

2020-08-21 Thread GitBox
kaxil commented on pull request #10446: URL: https://github.com/apache/airflow/pull/10446#issuecomment-678311136 > @kaxil but, don't we have user input validation on conn type in WebUI? >

[GitHub] [airflow] kaxil edited a comment on pull request #10446: Add kubernetes connection type

2020-08-21 Thread GitBox
kaxil edited a comment on pull request #10446: URL: https://github.com/apache/airflow/pull/10446#issuecomment-678311136 > @kaxil but, don't we have user input validation on conn type in WebUI? >

[airflow] branch master updated (1e37186 -> 95f4768)

2020-08-21 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 1e37186 Add update endpoint for DAG (#9101) (#9740) add 95f4768 Building backport packages generates

[airflow] branch master updated (1e37186 -> 95f4768)

2020-08-21 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 1e37186 Add update endpoint for DAG (#9101) (#9740) add 95f4768 Building backport packages generates

[GitHub] [airflow] turbaszek merged pull request #10445: Building backport packages generates README files

2020-08-21 Thread GitBox
turbaszek merged pull request #10445: URL: https://github.com/apache/airflow/pull/10445 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] potiuk commented on pull request #10380: Updated Documentation for the CI with mermaid sequence diagrams

2020-08-21 Thread GitBox
potiuk commented on pull request #10380: URL: https://github.com/apache/airflow/pull/10380#issuecomment-678301385 Udpated the first review by @feluelle . This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] turbaszek commented on pull request #10446: Add kubernetes connection type

2020-08-21 Thread GitBox
turbaszek commented on pull request #10446: URL: https://github.com/apache/airflow/pull/10446#issuecomment-678300185 And we do enforce: ``` The following args are required to add a connection: ['conn_uri or conn_type'] ```

[GitHub] [airflow] potiuk commented on a change in pull request #10380: Updated Documentation for the CI with mermaid sequence diagrams

2020-08-21 Thread GitBox
potiuk commented on a change in pull request #10380: URL: https://github.com/apache/airflow/pull/10380#discussion_r474708503 ## File path: CI.rst ## @@ -17,6 +17,10 @@ .. contents:: :local: +NOTE! This is the target architecture we are aspiring to. We have a

[GitHub] [airflow] potiuk opened a new pull request #10448: Be nice to fork repositories when it comes to scheduled events

2020-08-21 Thread GitBox
potiuk opened a new pull request #10448: URL: https://github.com/apache/airflow/pull/10448 Only runs scheduled CI runs in the 'apache/airflow' forks --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] turbaszek commented on pull request #10446: Add kubernetes connection type

2020-08-21 Thread GitBox
turbaszek commented on pull request #10446: URL: https://github.com/apache/airflow/pull/10446#issuecomment-678296209 @kaxil but, don't we have user input validation on conn type in WebUI?

[GitHub] [airflow] turbaszek commented on pull request #10445: Building backport packages generates README files

2020-08-21 Thread GitBox
turbaszek commented on pull request #10445: URL: https://github.com/apache/airflow/pull/10445#issuecomment-678295574 Not sure what's wrong here but the errors are definitely unrelated: ``` mypy: can't read file 'airflow/providers/amazon/aws/hooks/ses.py': No such file or directory

  1   2   >