[GitHub] [airflow] turbaszek commented on a change in pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18197: URL: https://github.com/apache/airflow/pull/18197#discussion_r707051981 ## File path: airflow/providers/google/cloud/example_dags/example_dataproc.py ## @@ -45,6 +45,8 @@ PYSPARK_URI = f"gs://{BUCKET}/{PYSPARK_MAIN}" SPA

[GitHub] [airflow] turbaszek commented on a change in pull request #17100: Advanced Params using json-schema

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r707050988 ## File path: airflow/models/param.py ## @@ -0,0 +1,208 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licens

[GitHub] [airflow] AnweshaSen edited a comment on issue #220: Global variable not being accessed in pipline

2021-09-12 Thread GitBox
AnweshaSen edited a comment on issue #220: URL: https://github.com/apache/airflow/issues/220#issuecomment-917669059 Hi all, I want to have a dynamic list with value from a previously executed step on every time run, upon that list I will be iterating and generating dynamic steps. I hav

[GitHub] [airflow-client-python] houqp commented on pull request #30: Issue #27 : Fixed dag details to accept None type for dag_run_timeout

2021-09-12 Thread GitBox
houqp commented on pull request #30: URL: https://github.com/apache/airflow-client-python/pull/30#issuecomment-917888496 Hi @er1shivam these code are auto generated using openapi-generator. So you need to fix it in the openapi spec or openapi-generator code base instead, then come back to

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #17207: Fix external_executor_id not being set for manually run jobs.

2021-09-12 Thread GitBox
ephraimbuddy commented on a change in pull request #17207: URL: https://github.com/apache/airflow/pull/17207#discussion_r707038035 ## File path: tests/executors/test_celery_executor.py ## @@ -288,6 +291,9 @@ def test_command_validation(self, command, expected_exception):

[GitHub] [airflow] laserpedro edited a comment on issue #17507: Task processes killed with WARNING - Recorded pid does not match the current pid

2021-09-12 Thread GitBox
laserpedro edited a comment on issue #17507: URL: https://github.com/apache/airflow/issues/17507#issuecomment-917879561 I will revert also to lastest 2.0 version available: however I do not know what impact it will have on the metadata db (need a downgrade or sth like that) ... Does

[GitHub] [airflow] laserpedro commented on issue #17507: Task processes killed with WARNING - Recorded pid does not match the current pid

2021-09-12 Thread GitBox
laserpedro commented on issue #17507: URL: https://github.com/apache/airflow/issues/17507#issuecomment-917879561 I will revert also to lastest 2.0 version available: however I do not know what impact it will have on the metadata db (need a downgrade or sth like that) ... -- This is an a

[GitHub] [airflow] Jorricks commented on a change in pull request #17207: Fix external_executor_id not being set for manually run jobs.

2021-09-12 Thread GitBox
Jorricks commented on a change in pull request #17207: URL: https://github.com/apache/airflow/pull/17207#discussion_r707032501 ## File path: tests/executors/test_celery_executor.py ## @@ -288,6 +291,9 @@ def test_command_validation(self, command, expected_exception):

[GitHub] [airflow] Jorricks commented on a change in pull request #17207: Fix external_executor_id not being set for manually run jobs.

2021-09-12 Thread GitBox
Jorricks commented on a change in pull request #17207: URL: https://github.com/apache/airflow/pull/17207#discussion_r707032501 ## File path: tests/executors/test_celery_executor.py ## @@ -288,6 +291,9 @@ def test_command_validation(self, command, expected_exception):

[GitHub] [airflow] Jorricks commented on a change in pull request #17207: Fix external_executor_id not being set for manually run jobs.

2021-09-12 Thread GitBox
Jorricks commented on a change in pull request #17207: URL: https://github.com/apache/airflow/pull/17207#discussion_r707031769 ## File path: tests/cli/commands/test_task_command.py ## @@ -106,7 +106,41 @@ def test_test_with_existing_dag_run(self): task_command.task

[GitHub] [airflow] uranusjr commented on a change in pull request #17552: AIP 39: Documentation

2021-09-12 Thread GitBox
uranusjr commented on a change in pull request #17552: URL: https://github.com/apache/airflow/pull/17552#discussion_r707030238 ## File path: airflow/example_dags/example_workday_timetable.py ## @@ -0,0 +1,92 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

[airflow] branch aip-39-docs updated (159b186 -> 137b599)

2021-09-12 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a change to branch aip-39-docs in repository https://gitbox.apache.org/repos/asf/airflow.git. from 159b186 Rewrite example with new last_automated_dagrun add 137b599 Remove execution date reference in

[GitHub] [airflow] casra-developers commented on pull request #16110: Added windows extensions

2021-09-12 Thread GitBox
casra-developers commented on pull request #16110: URL: https://github.com/apache/airflow/pull/16110#issuecomment-917861965 Yes, the plan is to continue. With that said I don't think we can continue until we have figured out why this crash of the scheduler happens. It did not do that befor

[GitHub] [airflow] msumit commented on a change in pull request #17100: Advanced Params using json-schema

2021-09-12 Thread GitBox
msumit commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r707016819 ## File path: airflow/models/param.py ## @@ -0,0 +1,208 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license a

[GitHub] [airflow] msumit commented on a change in pull request #17100: Advanced Params using json-schema

2021-09-12 Thread GitBox
msumit commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r707015703 ## File path: airflow/example_dags/example_complex_params.py ## @@ -0,0 +1,46 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or m

[GitHub] [airflow] msumit commented on a change in pull request #17100: Advanced Params using json-schema

2021-09-12 Thread GitBox
msumit commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r707015599 ## File path: airflow/example_dags/example_complex_params.py ## @@ -0,0 +1,49 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or m

[GitHub] [airflow] dimberman commented on a change in pull request #15330: Add a Docker Taskflow decorator

2021-09-12 Thread GitBox
dimberman commented on a change in pull request #15330: URL: https://github.com/apache/airflow/pull/15330#discussion_r706998932 ## File path: airflow/decorators/__init__.py ## @@ -15,34 +15,58 @@ # specific language governing permissions and limitations # under the License.

[GitHub] [airflow] dimberman commented on a change in pull request #15330: Add a Docker Taskflow decorator

2021-09-12 Thread GitBox
dimberman commented on a change in pull request #15330: URL: https://github.com/apache/airflow/pull/15330#discussion_r706998645 ## File path: airflow/providers_manager.py ## @@ -278,6 +309,51 @@ def _get_attr(obj: Any, attr_name: str): return None return g

[GitHub] [airflow] goodhamgupta commented on issue #16728: SparkKubernetesOperator to support arguments

2021-09-12 Thread GitBox
goodhamgupta commented on issue #16728: URL: https://github.com/apache/airflow/issues/16728#issuecomment-917810716 @potiuk, @uranusjr Sorry for the noise, but I would appreciate any hints you might be able to provide šŸ˜„ -- This is an automated message from the Apache Git Service. To res

[airflow] branch aip-39-docs updated (331694b -> 159b186)

2021-09-12 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a change to branch aip-39-docs in repository https://gitbox.apache.org/repos/asf/airflow.git. discard 331694b Fix imports in example DAG discard 2ffbf30 Typos discard 190bb46 Document timetable registration

[GitHub] [airflow] tnyz commented on pull request #17998: Fix error when create external table using table resource

2021-09-12 Thread GitBox
tnyz commented on pull request #17998: URL: https://github.com/apache/airflow/pull/17998#issuecomment-917772057 > Static checks are failing test has been fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [airflow] zhongjiajie commented on pull request #8231: Dag bulk_sync_to_db dag_tag only remove not exists

2021-09-12 Thread GitBox
zhongjiajie commented on pull request #8231: URL: https://github.com/apache/airflow/pull/8231#issuecomment-917768543 Hello? Does someone have time to review this PR @potiuk @turbaszek @mik-laj -- This is an automated message from the Apache Git Service. To respond to the message, please

[airflow] branch constraints-main updated: Updating constraints. Build id:1227697754

2021-09-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 8f65711 Updating const

[GitHub] [airflow] github-actions[bot] commented on pull request #8231: Dag bulk_sync_to_db dag_tag only remove not exists

2021-09-12 Thread GitBox
github-actions[bot] commented on pull request #8231: URL: https://github.com/apache/airflow/pull/8231#issuecomment-917738731 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you

[airflow] branch main updated (81ebd78 -> d119ae8)

2021-09-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 81ebd78 Added upsert method on S3ToRedshift operator (#18027) add d119ae8 Rename LocalToAzureDataLakeStorageOpe

[GitHub] [airflow] potiuk merged pull request #18168: Rename LocalToAzureDataLakeStorageOperator to LocalFilesystemToADLSOperator

2021-09-12 Thread GitBox
potiuk merged pull request #18168: URL: https://github.com/apache/airflow/pull/18168 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow] github-actions[bot] commented on pull request #18168: Rename LocalToAzureDataLakeStorageOperator to LocalFilesystemToADLSOperator

2021-09-12 Thread GitBox
github-actions[bot] commented on pull request #18168: URL: https://github.com/apache/airflow/pull/18168#issuecomment-917712057 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] iostreamdoth edited a comment on pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-12 Thread GitBox
iostreamdoth edited a comment on pull request #18197: URL: https://github.com/apache/airflow/pull/18197#issuecomment-917711447 https://github.com/apache/airflow/issues/17089 this is the issue discussion for reference -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] iostreamdoth commented on pull request #18197: Add gcs yml config to create dataproc cluster

2021-09-12 Thread GitBox
iostreamdoth commented on pull request #18197: URL: https://github.com/apache/airflow/pull/18197#issuecomment-917711447 https://github.com/apache/airflow/issues/17089 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [airflow] iostreamdoth opened a new pull request #18197: Issue-17089: add gcs yml config to create dataproc cluster

2021-09-12 Thread GitBox
iostreamdoth opened a new pull request #18197: URL: https://github.com/apache/airflow/pull/18197 --- **^ Add meaningful description above** - Add functionality for creating dataproc cluster and workflow config from yml config given on a gcs path. - Add documentation for cr

[GitHub] [airflow] potiuk commented on pull request #17998: Fix error when create external table using table resource

2021-09-12 Thread GitBox
potiuk commented on pull request #17998: URL: https://github.com/apache/airflow/pull/17998#issuecomment-917709060 Static checks are failing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [airflow] potiuk closed issue #18193: /admin/metrics/ fails if using MySQL < 8 as ROW_NUMBER() is not available

2021-09-12 Thread GitBox
potiuk closed issue #18193: URL: https://github.com/apache/airflow/issues/18193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[GitHub] [airflow] potiuk commented on issue #18193: /admin/metrics/ fails if using MySQL < 8 as ROW_NUMBER() is not available

2021-09-12 Thread GitBox
potiuk commented on issue #18193: URL: https://github.com/apache/airflow/issues/18193#issuecomment-917708919 Just out of the curiosity - where did you take the "/admin/metrics" endpoint from ? I guess it is something you have in your deployment that your admins wrote, because I cannot find

[GitHub] [airflow] github-actions[bot] commented on pull request #18027: Added upsert method on S3ToRedshift operator

2021-09-12 Thread GitBox
github-actions[bot] commented on pull request #18027: URL: https://github.com/apache/airflow/pull/18027#issuecomment-917706450 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[airflow] branch main updated: Added upsert method on S3ToRedshift operator (#18027)

2021-09-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 81ebd78 Added upsert method on S3ToRedshift operat

[GitHub] [airflow] potiuk merged pull request #18027: Added upsert method on S3ToRedshift operator

2021-09-12 Thread GitBox
potiuk merged pull request #18027: URL: https://github.com/apache/airflow/pull/18027 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[airflow] branch main updated: Doc: Improve installing from sources (#18194)

2021-09-12 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ec79da0 Doc: Improve installing from sources (#

[GitHub] [airflow] kaxil merged pull request #18194: Doc: Improve installing from sources

2021-09-12 Thread GitBox
kaxil merged pull request #18194: URL: https://github.com/apache/airflow/pull/18194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [airflow] github-actions[bot] commented on pull request #17998: Fix error when create external table using table resource

2021-09-12 Thread GitBox
github-actions[bot] commented on pull request #17998: URL: https://github.com/apache/airflow/pull/17998#issuecomment-917705721 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] potiuk commented on a change in pull request #17998: Fix error when create external table using table resource

2021-09-12 Thread GitBox
potiuk commented on a change in pull request #17998: URL: https://github.com/apache/airflow/pull/17998#discussion_r706898814 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -1195,36 +1200,24 @@ def execute(self, context) -> None: else:

[GitHub] [airflow] potiuk closed issue #18125: Create pytest plugin for prepare test environment

2021-09-12 Thread GitBox
potiuk closed issue #18125: URL: https://github.com/apache/airflow/issues/18125 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[airflow] branch main updated: Adding Variable.update method and improving detection of variable key collisions (#18159)

2021-09-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d6e7c45 Adding Variable.update method and improvin

[GitHub] [airflow] potiuk merged pull request #18159: Adding Variable.update method and improving detection of variable key collisions

2021-09-12 Thread GitBox
potiuk merged pull request #18159: URL: https://github.com/apache/airflow/pull/18159 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow] github-actions[bot] commented on pull request #18159: Adding Variable.update method and improving detection of variable key collisions

2021-09-12 Thread GitBox
github-actions[bot] commented on pull request #18159: URL: https://github.com/apache/airflow/pull/18159#issuecomment-917705021 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] potiuk commented on pull request #18092: Graceful scheduler shutdown on error

2021-09-12 Thread GitBox
potiuk commented on pull request #18092: URL: https://github.com/apache/airflow/pull/18092#issuecomment-917704090 Would you be able to refactor the one small thing a bit @spoutnickgp ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] github-actions[bot] commented on pull request #18092: Graceful scheduler shutdown on error

2021-09-12 Thread GitBox
github-actions[bot] commented on pull request #18092: URL: https://github.com/apache/airflow/pull/18092#issuecomment-917703924 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not m

[GitHub] [airflow] github-actions[bot] commented on pull request #18194: Doc: Improve installing from sources

2021-09-12 Thread GitBox
github-actions[bot] commented on pull request #18194: URL: https://github.com/apache/airflow/pull/18194#issuecomment-917702248 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide

[airflow] branch main updated: Fix typo in decorator test (#18191)

2021-09-12 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1fdde76 Fix typo in decorator test (#18191) 1fd

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #18191: Fix typo in decorator test

2021-09-12 Thread GitBox
boring-cyborg[bot] commented on pull request #18191: URL: https://github.com/apache/airflow/pull/18191#issuecomment-917701896 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] kaxil merged pull request #18191: Fix typo in decorator test

2021-09-12 Thread GitBox
kaxil merged pull request #18191: URL: https://github.com/apache/airflow/pull/18191 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubs

[GitHub] [airflow] kaxil opened a new pull request #18194: Doc: Improve installing from sources

2021-09-12 Thread GitBox
kaxil opened a new pull request #18194: URL: https://github.com/apache/airflow/pull/18194 Based on https://github.com/apache/airflow/pull/18187/#discussion_r706861500 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/

[airflow] 01/01: Doc: Improve installing from sources

2021-09-12 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch improve-rel-guide in repository https://gitbox.apache.org/repos/asf/airflow.git commit e212977111d09962ba3d182b03b05487219e7365 Author: Kaxil Naik AuthorDate: Sun Sep 12 21:06:24 2021 +0100

[airflow] branch improve-rel-guide created (now e212977)

2021-09-12 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch improve-rel-guide in repository https://gitbox.apache.org/repos/asf/airflow.git. at e212977 Doc: Improve installing from sources This branch includes the following new commits: n

[airflow] branch main updated: Add official download page for providers (#18187)

2021-09-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1cb456c Add official download page for providers (

[GitHub] [airflow] potiuk merged pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
potiuk merged pull request #18187: URL: https://github.com/apache/airflow/pull/18187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow] github-actions[bot] commented on pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
github-actions[bot] commented on pull request #18187: URL: https://github.com/apache/airflow/pull/18187#issuecomment-917698023 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] potiuk commented on a change in pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
potiuk commented on a change in pull request #18187: URL: https://github.com/apache/airflow/pull/18187#discussion_r706883392 ## File path: docs/installing-providers-from-sources.rst ## @@ -0,0 +1,116 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or mor

[GitHub] [airflow] potiuk commented on a change in pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
potiuk commented on a change in pull request #18187: URL: https://github.com/apache/airflow/pull/18187#discussion_r706883179 ## File path: docs/installing-providers-from-sources.rst ## @@ -0,0 +1,116 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or mor

[GitHub] [airflow] potiuk commented on a change in pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
potiuk commented on a change in pull request #18187: URL: https://github.com/apache/airflow/pull/18187#discussion_r706883028 ## File path: docs/installing-providers-from-sources.rst ## @@ -0,0 +1,116 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or mor

[GitHub] [airflow] ramsesrm opened a new issue #18193: /admin/metrics/ fails if using MySQL < 8 as ROW_NUMBER() is not available

2021-09-12 Thread GitBox
ramsesrm opened a new issue #18193: URL: https://github.com/apache/airflow/issues/18193 ### Apache Airflow version 2.1.3 (latest released) ### Operating System Ubuntu 20.10 ### Versions of Apache Airflow Providers apache-airflow-providers-amazon==2.1.0 a

[GitHub] [airflow] potiuk commented on pull request #18048: Declared readonly variable and added double quote in variables containing strings

2021-09-12 Thread GitBox
potiuk commented on pull request #18048: URL: https://github.com/apache/airflow/pull/18048#issuecomment-917692141 I think you need to rebase. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] turbaszek commented on a change in pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18187: URL: https://github.com/apache/airflow/pull/18187#discussion_r706878198 ## File path: docs/installing-providers-from-sources.rst ## @@ -0,0 +1,116 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or

[GitHub] [airflow] turbaszek commented on a change in pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18187: URL: https://github.com/apache/airflow/pull/18187#discussion_r706878198 ## File path: docs/installing-providers-from-sources.rst ## @@ -0,0 +1,116 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or

[airflow] branch main updated: Automatically create section when migrating config (#16814)

2021-09-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0c43e68 Automatically create section when migratin

[GitHub] [airflow] potiuk merged pull request #16814: Automatically create section when migrating config

2021-09-12 Thread GitBox
potiuk merged pull request #16814: URL: https://github.com/apache/airflow/pull/16814 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[airflow] branch main updated: sets encoding to utf-8 by default while reading task logs (#17965)

2021-09-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2f1ed34 sets encoding to utf-8 by default while re

[GitHub] [airflow] potiuk merged pull request #17965: sets encoding to utf-8 by default while reading task logs

2021-09-12 Thread GitBox
potiuk merged pull request #17965: URL: https://github.com/apache/airflow/pull/17965 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow] potiuk closed issue #16834: Airflow dashboard cannot load logs containing emoji

2021-09-12 Thread GitBox
potiuk closed issue #16834: URL: https://github.com/apache/airflow/issues/16834 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[airflow] branch main updated: Deprecate default pod name in EKSPodOperator (#18036)

2021-09-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0239776 Deprecate default pod name in EKSPodOperat

[GitHub] [airflow] potiuk merged pull request #18036: Deprecate default pod name in EKSPodOperator

2021-09-12 Thread GitBox
potiuk merged pull request #18036: URL: https://github.com/apache/airflow/pull/18036 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsub

[GitHub] [airflow] potiuk commented on issue #17956: Celery fails with: "Failed to execute task maximum recursion depth exceeded"

2021-09-12 Thread GitBox
potiuk commented on issue #17956: URL: https://github.com/apache/airflow/issues/17956#issuecomment-917679469 Unfortunately, it's not easy to reproduce it, Do you have more details on this issue? We need full stack trace, not only end of it. Do you have a some reproduction path we ca

[GitHub] [airflow] potiuk closed issue #17956: Celery fails with: "Failed to execute task maximum recursion depth exceeded"

2021-09-12 Thread GitBox
potiuk closed issue #17956: URL: https://github.com/apache/airflow/issues/17956 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[GitHub] [airflow-site] potiuk closed issue #415: Links not updated on page footer

2021-09-12 Thread GitBox
potiuk closed issue #415: URL: https://github.com/apache/airflow-site/issues/415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr.

[GitHub] [airflow] potiuk commented on issue #16737: Scheduler memory creeping and plateauing

2021-09-12 Thread GitBox
potiuk commented on issue #16737: URL: https://github.com/apache/airflow/issues/16737#issuecomment-917677256 Duplicate of ##14924. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [airflow] potiuk commented on issue #16737: Scheduler memory creeping and plateauing

2021-09-12 Thread GitBox
potiuk commented on issue #16737: URL: https://github.com/apache/airflow/issues/16737#issuecomment-917677177 The problem is most likely connected with the use PageCache - see extensive discussion in #14924. The growing memory observed was most likely just growing cache, which is no

[GitHub] [airflow] potiuk closed issue #16737: Scheduler memory creeping and plateauing

2021-09-12 Thread GitBox
potiuk closed issue #16737: URL: https://github.com/apache/airflow/issues/16737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr..

[GitHub] [airflow] potiuk commented on pull request #16110: Added windows extensions

2021-09-12 Thread GitBox
potiuk commented on pull request #16110: URL: https://github.com/apache/airflow/pull/16110#issuecomment-917674049 Hey @casra-developers - are you going to continue with this? I think it would be pretty useful! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on a change in pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
potiuk commented on a change in pull request #18187: URL: https://github.com/apache/airflow/pull/18187#discussion_r706864358 ## File path: docs/installing-providers-from-sources.rst ## @@ -0,0 +1,116 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or mor

[GitHub] [airflow] potiuk commented on issue #17381: Bug in the logic of Celery executor for checking stalled adopted tasks?

2021-09-12 Thread GitBox
potiuk commented on issue #17381: URL: https://github.com/apache/airflow/issues/17381#issuecomment-917669925 CC: @ephraimbuddy - maybe somethig worth looking at for 2.2 Seems like quite easy to reproduce. -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] AnweshaSen commented on issue #220: Global variable not being accessed in pipline

2021-09-12 Thread GitBox
AnweshaSen commented on issue #220: URL: https://github.com/apache/airflow/issues/220#issuecomment-917669059 Hi all, I want to have a dynamic list with value from a previously executed step on every time run, upon that list I will be iterating and generating dynamic steps. I have tried

[GitHub] [airflow] turbaszek commented on a change in pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18187: URL: https://github.com/apache/airflow/pull/18187#discussion_r706861500 ## File path: docs/installing-providers-from-sources.rst ## @@ -0,0 +1,116 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or

[GitHub] [airflow] turbaszek commented on a change in pull request #18189: Allow using default celery commands for custom Celery executors subclassed from existing

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18189: URL: https://github.com/apache/airflow/pull/18189#discussion_r706860841 ## File path: airflow/executors/executor_loader.py ## @@ -77,14 +77,36 @@ def load_executor(cls, executor_name: str) -> BaseExecutor: """

[GitHub] [airflow] turbaszek commented on a change in pull request #18189: Allow using default celery commands for custom Celery executors subclassed from existing

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18189: URL: https://github.com/apache/airflow/pull/18189#discussion_r706860841 ## File path: airflow/executors/executor_loader.py ## @@ -77,14 +77,36 @@ def load_executor(cls, executor_name: str) -> BaseExecutor: """

[GitHub] [airflow] turbaszek commented on a change in pull request #18189: Allow using default celery commands for custom Celery executors subclassed from existing

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18189: URL: https://github.com/apache/airflow/pull/18189#discussion_r706860315 ## File path: airflow/executors/executor_loader.py ## @@ -77,14 +77,36 @@ def load_executor(cls, executor_name: str) -> BaseExecutor: """

[GitHub] [airflow] turbaszek commented on a change in pull request #18189: Allow using default celery commands for custom Celery executors subclassed from existing

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18189: URL: https://github.com/apache/airflow/pull/18189#discussion_r706860315 ## File path: airflow/executors/executor_loader.py ## @@ -77,14 +77,36 @@ def load_executor(cls, executor_name: str) -> BaseExecutor: """

[GitHub] [airflow] github-actions[bot] commented on pull request #18191: Fix typo in decorator test

2021-09-12 Thread GitBox
github-actions[bot] commented on pull request #18191: URL: https://github.com/apache/airflow/pull/18191#issuecomment-917665015 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and tak

[GitHub] [airflow] potiuk commented on issue #18190: Docker image - Migrate to 3.x-slim-bullseye from 3.x-slim-buster

2021-09-12 Thread GitBox
potiuk commented on issue #18190: URL: https://github.com/apache/airflow/issues/18190#issuecomment-917664362 Good point. Assigned you to it. It should be started with changing base image in both ci and prod image but you should make sure to a) update documentation b) update all the script

[GitHub] [airflow] uranusjr commented on issue #18050: Duplicating the same connection twice gives "Integrity error, probably unique constraint"

2021-09-12 Thread GitBox
uranusjr commented on issue #18050: URL: https://github.com/apache/airflow/issues/18050#issuecomment-917664028 The problem with a partial match is that `LIKE` queries (which is what I imagine `startswith()` would produce) can easily end up producing a table scan, so Iā€™d be extremely wary t

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #18191: Fix typo in decorator test

2021-09-12 Thread GitBox
boring-cyborg[bot] commented on pull request #18191: URL: https://github.com/apache/airflow/pull/18191#issuecomment-917662601 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Con

[GitHub] [airflow] harikaduyu opened a new pull request #18191: Fix typo in decorator test

2021-09-12 Thread GitBox
harikaduyu opened a new pull request #18191: URL: https://github.com/apache/airflow/pull/18191 `unknow` --> `unknown` --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-reques

[GitHub] [airflow] turbaszek commented on a change in pull request #18184: Migrate Google Cloud Build from Discovery API to Python SDK

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18184: URL: https://github.com/apache/airflow/pull/18184#discussion_r706853992 ## File path: airflow/providers/google/cloud/operators/cloud_build.py ## @@ -15,121 +15,933 @@ # KIND, either express or implied. See the License fo

[GitHub] [airflow] turbaszek commented on a change in pull request #18184: Migrate Google Cloud Build from Discovery API to Python SDK

2021-09-12 Thread GitBox
turbaszek commented on a change in pull request #18184: URL: https://github.com/apache/airflow/pull/18184#discussion_r706853939 ## File path: airflow/providers/google/cloud/operators/cloud_build.py ## @@ -15,121 +15,933 @@ # KIND, either express or implied. See the License fo

[GitHub] [airflow] vmalakhin opened a new issue #18190: Docker image - Migrate to 3.x-slim-bullseye from 3.x-slim-buster

2021-09-12 Thread GitBox
vmalakhin opened a new issue #18190: URL: https://github.com/apache/airflow/issues/18190 ### Description Currently docker images are produced based on top of 3.x-slim-buster python and as a result output inherits related debian vulnerabilities. ### Use case/motivation

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18190: Docker image - Migrate to 3.x-slim-bullseye from 3.x-slim-buster

2021-09-12 Thread GitBox
boring-cyborg[bot] commented on issue #18190: URL: https://github.com/apache/airflow/issues/18190#issuecomment-917656871 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] georborodin commented on pull request #18189: Allow using default celery commands for custom Celery executors subclassed from existing

2021-09-12 Thread GitBox
georborodin commented on pull request #18189: URL: https://github.com/apache/airflow/pull/18189#issuecomment-917640702 @potiuk unfortunately due to health issues I had to take a pause from working on this issue, and had to re-open PR since it now included more than 250 commits and, apparen

[GitHub] [airflow] georborodin opened a new pull request #18189: Allow using default celery commands for custom Celery executors subclassed from existing

2021-09-12 Thread GitBox
georborodin opened a new pull request #18189: URL: https://github.com/apache/airflow/pull/18189 Currently Airflow CLI does not allow using Celery commands for custom executors subclassed from CeleryExecutor or CeleryKubernetesExecutor. With this PR, additional check is run to determine whe

[GitHub] [airflow] georborodin closed pull request #17426: Allow using default celery commands for custom Celery executors subclassed from existing

2021-09-12 Thread GitBox
georborodin closed pull request #17426: URL: https://github.com/apache/airflow/pull/17426 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-

[GitHub] [airflow] xingdao closed pull request #17258: Fix serialize_operator task_type

2021-09-12 Thread GitBox
xingdao closed pull request #17258: URL: https://github.com/apache/airflow/pull/17258 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsu

[GitHub] [airflow] eladkal commented on a change in pull request #18184: Migrate Google Cloud Build from Discovery API to Python SDK

2021-09-12 Thread GitBox
eladkal commented on a change in pull request #18184: URL: https://github.com/apache/airflow/pull/18184#discussion_r706834594 ## File path: UPDATING.md ## @@ -1777,6 +1777,16 @@ previous one was (project_id, dataset_id, ...) (breaking change) - `get_tabledata` returns list of

[GitHub] [airflow] potiuk commented on pull request #18187: Add official download page for providers

2021-09-12 Thread GitBox
potiuk commented on pull request #18187: URL: https://github.com/apache/airflow/pull/18187#issuecomment-917630521 ![Screenshot 2021-09-12 13 25 33](https://user-images.githubusercontent.com/595491/132988267-7d036167-62b6-49c4-ac2a-540a0412fcd5.png) -- This is an automated message fro

  1   2   >