[GitHub] [airflow] dstandish opened a new pull request, #28454: Fix bad pods pickled in executor_config

2022-12-18 Thread GitBox
dstandish opened a new pull request, #28454: URL: https://github.com/apache/airflow/pull/28454 We used to pickle raw pods objects but found that when unpickling across k8s lib versions we would get missing attr errors. We can fix this by roundtripping to json in a more tolerant fashion tha

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28453: Update dates.py

2022-12-18 Thread GitBox
boring-cyborg[bot] commented on PR #28453: URL: https://github.com/apache/airflow/pull/28453#issuecomment-1357193944 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] xyc-pz opened a new pull request, #28453: Update dates.py

2022-12-18 Thread GitBox
xyc-pz opened a new pull request, #28453: URL: https://github.com/apache/airflow/pull/28453 The example import sentence in line 51 is incorrect --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/mai

[GitHub] [airflow] Soonmok commented on a diff in pull request #28400: Rewrite timezone assign codeline

2022-12-18 Thread GitBox
Soonmok commented on code in PR #28400: URL: https://github.com/apache/airflow/pull/28400#discussion_r1051854266 ## airflow/models/dag.py: ## @@ -462,8 +462,10 @@ def __init__( # set timezone from start_date tz = None if start_date and start_date.tzinf

[GitHub] [airflow] Soonmok closed pull request #28400: Rewrite timezone assign codeline

2022-12-18 Thread GitBox
Soonmok closed pull request #28400: Rewrite timezone assign codeline URL: https://github.com/apache/airflow/pull/28400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

[airflow] branch main updated: Add `ensure_ascii=False` in trigger dag run API (#28451)

2022-12-18 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c3eee43725 Add `ensure_ascii=False` in tri

[GitHub] [airflow] ephraimbuddy merged pull request #28451: Add `ensure_ascii=False` in trigger dag run API

2022-12-18 Thread GitBox
ephraimbuddy merged PR #28451: URL: https://github.com/apache/airflow/pull/28451 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

[GitHub] [airflow] dstandish commented on a diff in pull request #28436: Log FileTaskHandler to work with multi_namespace_mode when running using KubernetesExecutor

2022-12-18 Thread GitBox
dstandish commented on code in PR #28436: URL: https://github.com/apache/airflow/pull/28436#discussion_r1051830952 ## airflow/utils/log/file_task_handler.py: ## @@ -191,16 +191,20 @@ def _read(self, ti: TaskInstance, try_number: int, metadata: dict[str, Any] | No

[GitHub] [airflow] bharanidharan14 commented on a diff in pull request #28262: Hook for managing directories and files in Azure Data Lake Storage Gen2

2022-12-18 Thread GitBox
bharanidharan14 commented on code in PR #28262: URL: https://github.com/apache/airflow/pull/28262#discussion_r1051823169 ## airflow/providers/microsoft/azure/hooks/adls.py: ## @@ -0,0 +1,221 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] bharanidharan14 commented on a diff in pull request #28262: Hook for managing directories and files in Azure Data Lake Storage Gen2

2022-12-18 Thread GitBox
bharanidharan14 commented on code in PR #28262: URL: https://github.com/apache/airflow/pull/28262#discussion_r1051823169 ## airflow/providers/microsoft/azure/hooks/adls.py: ## @@ -0,0 +1,221 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] bharanidharan14 commented on a diff in pull request #28262: Hook for managing directories and files in Azure Data Lake Storage Gen2

2022-12-18 Thread GitBox
bharanidharan14 commented on code in PR #28262: URL: https://github.com/apache/airflow/pull/28262#discussion_r1051820566 ## airflow/providers/microsoft/azure/hooks/adls.py: ## @@ -0,0 +1,264 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] XD-DENG commented on a diff in pull request #28436: Log FileTaskHandler to work with multi_namespace_mode when running using KubernetesExecutor

2022-12-18 Thread GitBox
XD-DENG commented on code in PR #28436: URL: https://github.com/apache/airflow/pull/28436#discussion_r1051814838 ## airflow/utils/log/file_task_handler.py: ## @@ -191,16 +191,20 @@ def _read(self, ti: TaskInstance, try_number: int, metadata: dict[str, Any] | No

[GitHub] [airflow] uranusjr commented on a diff in pull request #28400: Rewrite timezone assign codeline

2022-12-18 Thread GitBox
uranusjr commented on code in PR #28400: URL: https://github.com/apache/airflow/pull/28400#discussion_r1051813105 ## airflow/models/dag.py: ## @@ -462,8 +462,10 @@ def __init__( # set timezone from start_date tz = None if start_date and start_date.tzin

[GitHub] [airflow] uranusjr commented on pull request #28451: Add `ensure_ascii=False` in trigger dag run API

2022-12-18 Thread GitBox
uranusjr commented on PR #28451: URL: https://github.com/apache/airflow/pull/28451#issuecomment-1357108752 Please fix the static check error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] dstandish commented on issue #28227: Scheduler error: 'V1PodSpec' object has no attribute '_ephemeral_containers'

2022-12-18 Thread GitBox
dstandish commented on issue #28227: URL: https://github.com/apache/airflow/issues/28227#issuecomment-1357099778 The problem is that when you take a kubernetes object created from one lib version, and pickle it, then when you unpickle within a python env with a diff kubernetes lib version,

[airflow] branch constraints-main updated: Updating constraints. Build id:

2022-12-18 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 44b2632f0d Updating con

[GitHub] [airflow] dstandish commented on a diff in pull request #28436: Log FileTaskHandler to work with multi_namespace_mode when running using KubernetesExecutor

2022-12-18 Thread GitBox
dstandish commented on code in PR #28436: URL: https://github.com/apache/airflow/pull/28436#discussion_r1051781510 ## airflow/utils/log/file_task_handler.py: ## @@ -191,16 +191,20 @@ def _read(self, ti: TaskInstance, try_number: int, metadata: dict[str, Any] | No

[GitHub] [airflow] EugeneChung commented on issue #20468: Need to set `tags=[]` to remove all tags from a DAG.

2022-12-18 Thread GitBox
EugeneChung commented on issue #20468: URL: https://github.com/apache/airflow/issues/20468#issuecomment-1357051721 You can remove tags from the `dag_tag` table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [airflow] boring-cyborg[bot] commented on issue #28452: TaskInstances do not succeed when using enable_logging=True option in DockerSwarmOperator

2022-12-18 Thread GitBox
boring-cyborg[bot] commented on issue #28452: URL: https://github.com/apache/airflow/issues/28452#issuecomment-1357049661 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] chriscmorgan opened a new issue, #28452: TaskInstances do not succeed when using enable_logging=True option in DockerSwarmOperator

2022-12-18 Thread GitBox
chriscmorgan opened a new issue, #28452: URL: https://github.com/apache/airflow/issues/28452 ### Apache Airflow Provider(s) docker ### Versions of Apache Airflow Providers apache-airflow-providers-celery==3.1.0 apache-airflow-providers-docker==3.3.0 ### Apache A

[GitHub] [airflow] akakakakakaa opened a new pull request, #28451: Add `ensure_ascii=False` in trigger dag run API

2022-12-18 Thread GitBox
akakakakakaa opened a new pull request, #28451: URL: https://github.com/apache/airflow/pull/28451 In Trigger DAG Run UI, non-unicode character broken. This change only affect to UI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[airflow] branch main updated: Log FileTaskHandler to work with KubernetesExecutor's multi_namespace_mode (#28436)

2022-12-18 Thread xddeng
This is an automated email from the ASF dual-hosted git repository. xddeng pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 497c2c243d Log FileTaskHandler to work with Kuberne

[GitHub] [airflow] XD-DENG merged pull request #28436: Log FileTaskHandler to work with multi_namespace_mode when running using KubernetesExecutor

2022-12-18 Thread GitBox
XD-DENG merged PR #28436: URL: https://github.com/apache/airflow/pull/28436 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

[GitHub] [airflow] XD-DENG commented on a diff in pull request #28416: Update index.rst

2022-12-18 Thread GitBox
XD-DENG commented on code in PR #28416: URL: https://github.com/apache/airflow/pull/28416#discussion_r1051753542 ## docs/helm-chart/index.rst: ## @@ -122,10 +122,10 @@ The command removes all the Kubernetes components associated with the chart and .. note:: Some kubernetes

[GitHub] [airflow] XD-DENG commented on a diff in pull request #28436: Log FileTaskHandler to work with multi_namespace_mode when running using KubernetesExecutor

2022-12-18 Thread GitBox
XD-DENG commented on code in PR #28436: URL: https://github.com/apache/airflow/pull/28436#discussion_r1051751431 ## airflow/utils/log/file_task_handler.py: ## @@ -191,16 +191,20 @@ def _read(self, ti: TaskInstance, try_number: int, metadata: dict[str, Any] | No

[GitHub] [airflow] XD-DENG commented on a diff in pull request #28436: Log FileTaskHandler to work with multi_namespace_mode when running using KubernetesExecutor

2022-12-18 Thread GitBox
XD-DENG commented on code in PR #28436: URL: https://github.com/apache/airflow/pull/28436#discussion_r1051751431 ## airflow/utils/log/file_task_handler.py: ## @@ -191,16 +191,20 @@ def _read(self, ti: TaskInstance, try_number: int, metadata: dict[str, Any] | No

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28450: Remove extra H1 & improve formatting of Listeners docs page

2022-12-18 Thread GitBox
boring-cyborg[bot] commented on PR #28450: URL: https://github.com/apache/airflow/pull/28450#issuecomment-1357002489 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] nathan-contino opened a new pull request, #28450: Remove extra H1 & improve formatting of Listeners docs page

2022-12-18 Thread GitBox
nathan-contino opened a new pull request, #28450: URL: https://github.com/apache/airflow/pull/28450 I noticed that the documentation has an unclickable "Usage" page in the TOC. A little digging later, I discovered that this page contains an extra H1, and since this page is in the top level

[airflow] branch constraints-main updated: Updating constraints. Build id:

2022-12-18 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 9bc809b07e Updating con

[GitHub] [airflow] vchiapaikeo commented on issue #28393: Webserver reports "ImportError: Module "airflow.providers.google.cloud.operators.mlengine" does not define a "AIPlatformConsoleLink" attribute

2022-12-18 Thread GitBox
vchiapaikeo commented on issue #28393: URL: https://github.com/apache/airflow/issues/28393#issuecomment-1356961632 Looks like the additions were properly added here: https://github.com/apache/airflow/pull/26836/files#diff-b8d2d329a113cf05d9bbd18b125843e86a3ce5c3c39704d29ad58bb83bb07861R1035-

[GitHub] [airflow] vchiapaikeo opened a new pull request, #28449: Remove deprecated AIPlatformConsoleLinkk from google/provider.yaml

2022-12-18 Thread GitBox
vchiapaikeo opened a new pull request, #28449: URL: https://github.com/apache/airflow/pull/28449 Exception is raised on webserver because a deprecated extra link was not removed from google/provider.yaml. This was removed as part of this PR: https://github.com/apache/airflow/pull/26836/file

[GitHub] [airflow] uranusjr commented on issue #28227: Scheduler error: 'V1PodSpec' object has no attribute '_ephemeral_containers'

2022-12-18 Thread GitBox
uranusjr commented on issue #28227: URL: https://github.com/apache/airflow/issues/28227#issuecomment-1356946147 This looks like an issue in the Kubernetes package to me, from what I can tell there’s nothing out of the ordinary from Airflow’s perspective. I suspect the upgrade is not done co

[airflow] branch main updated: Maintain manual scroll position in task logs (#28386)

2022-12-18 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5c80d985a3 Maintain manual scroll position in tas

[GitHub] [airflow] bbovenzi closed issue #28143: Logs tab is automatically scrolling to the bottom while user is reading logs

2022-12-18 Thread GitBox
bbovenzi closed issue #28143: Logs tab is automatically scrolling to the bottom while user is reading logs URL: https://github.com/apache/airflow/issues/28143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] bbovenzi merged pull request #28386: Maintain manual scroll position in task logs

2022-12-18 Thread GitBox
bbovenzi merged PR #28386: URL: https://github.com/apache/airflow/pull/28386 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[GitHub] [airflow] uranusjr commented on issue #28393: Webserver reports "ImportError: Module "airflow.providers.google.cloud.operators.mlengine" does not define a "AIPlatformConsoleLink" attribute/cl

2022-12-18 Thread GitBox
uranusjr commented on issue #28393: URL: https://github.com/apache/airflow/issues/28393#issuecomment-1356935480 You can submit a PR to do the removal (and additions—see #26836 for what hooks were added) and the pre-commit check can be dealt separately later. -- This is an automated messag

[GitHub] [airflow] github-actions[bot] commented on issue #12644: Network instabilities are able to freeze KubernetesJobWatcher

2022-12-18 Thread GitBox
github-actions[bot] commented on issue #12644: URL: https://github.com/apache/airflow/issues/12644#issuecomment-1356911790 This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further

[GitHub] [airflow] github-actions[bot] commented on issue #13818: Default user is not working with AUTH_LDAP - Airflow 2.0

2022-12-18 Thread GitBox
github-actions[bot] commented on issue #13818: URL: https://github.com/apache/airflow/issues/13818#issuecomment-1356911780 This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further

[GitHub] [airflow] potiuk commented on pull request #28182: Better error message for bad stat name

2022-12-18 Thread GitBox
potiuk commented on PR #28182: URL: https://github.com/apache/airflow/pull/28182#issuecomment-1356907529 And just to explain a bit - I realised closing the issue could have sounded a bit rude. But it was not really and in my mind it was pretty clear at that time (and this is how I interpre

[airflow] branch constraints-main updated: Updating constraints. Build id:

2022-12-18 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new b620792452 Updating con

[GitHub] [airflow] potiuk commented on pull request #28283: Trigger gevent monkeypatching via environment variable

2022-12-18 Thread GitBox
potiuk commented on PR #28283: URL: https://github.com/apache/airflow/pull/28283#issuecomment-1356898026 Seems we have confirmation that at least this part of the gevent configuration (monkeypatching) works with this approach. -- This is an automated message from the Apache Git Service. T

[GitHub] [airflow] RachitSharma2001 commented on a diff in pull request #28318: Add FTPSFileTransmitOperator

2022-12-18 Thread GitBox
RachitSharma2001 commented on code in PR #28318: URL: https://github.com/apache/airflow/pull/28318#discussion_r1051680027 ## tests/providers/ftp/operators/test_ftp.py: ## @@ -184,3 +180,108 @@ def test_unequal_local_remote_file_paths(self): local_filepath=["/tmp

[GitHub] [airflow] RachitSharma2001 commented on a diff in pull request #28318: Add FTPSFileTransmitOperator

2022-12-18 Thread GitBox
RachitSharma2001 commented on code in PR #28318: URL: https://github.com/apache/airflow/pull/28318#discussion_r1051680027 ## tests/providers/ftp/operators/test_ftp.py: ## @@ -184,3 +180,108 @@ def test_unequal_local_remote_file_paths(self): local_filepath=["/tmp

[GitHub] [airflow] victorjourne commented on pull request #28283: Trigger gevent monkeypatching via environment variable

2022-12-18 Thread GitBox
victorjourne commented on PR #28283: URL: https://github.com/apache/airflow/pull/28283#issuecomment-1356892685 I can confirm that this fix solves my issue #28380 and #8164 warnings have disappeared. -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [airflow] victorjourne commented on issue #28380: Celery green threads incompatibility

2022-12-18 Thread GitBox
victorjourne commented on issue #28380: URL: https://github.com/apache/airflow/issues/28380#issuecomment-1356891596 After testing many configurations about celery backend, the solution I found is the combination of : - Force celery **not to store the task return** by passing the env varia

[GitHub] [airflow] potiuk closed issue #28232: Allow changing the Task comments via a task

2022-12-18 Thread GitBox
potiuk closed issue #28232: Allow changing the Task comments via a task URL: https://github.com/apache/airflow/issues/28232 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [airflow] potiuk commented on issue #28232: Allow changing the Task comments via a task

2022-12-18 Thread GitBox
potiuk commented on issue #28232: URL: https://github.com/apache/airflow/issues/28232#issuecomment-1356890786 IMHO - This should be done via API not via decorator/task. There is no "internal" mechanism to change some tasks from other tasks and stable REST API is the best way of doing it. Le

[airflow] branch main updated (21f0600f69 -> 5b54e8d21b)

2022-12-18 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 21f0600f69 Correctly template Glue Jobs `create_job_kwargs` arg (#28403) add 5b54e8d21b fix: very small typo erro

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28447: fix: very small typo error in readme

2022-12-18 Thread GitBox
boring-cyborg[bot] commented on PR #28447: URL: https://github.com/apache/airflow/pull/28447#issuecomment-1356889896 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] potiuk merged pull request #28447: fix: very small typo error in readme

2022-12-18 Thread GitBox
potiuk merged PR #28447: URL: https://github.com/apache/airflow/pull/28447 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] 0xflotus opened a new pull request, #28447: fix: very small typo error in readme

2022-12-18 Thread GitBox
0xflotus opened a new pull request, #28447: URL: https://github.com/apache/airflow/pull/28447 I only fixed a small typo errror in the readme file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28447: fix: very small typo error in readme

2022-12-18 Thread GitBox
boring-cyborg[bot] commented on PR #28447: URL: https://github.com/apache/airflow/pull/28447#issuecomment-1356888244 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] potiuk commented on pull request #26993: Fix ti._try_number for deferred and up_for_reschedule tasks

2022-12-18 Thread GitBox
potiuk commented on PR #26993: URL: https://github.com/apache/airflow/pull/26993#issuecomment-1356887634 some errors @bbovenzi :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [airflow] potiuk commented on issue #28227: Scheduler error: 'V1PodSpec' object has no attribute '_ephemeral_containers'

2022-12-18 Thread GitBox
potiuk commented on issue #28227: URL: https://github.com/apache/airflow/issues/28227#issuecomment-1356887060 Also @KulykDmytro - could you please explain the exact deplouyment you have - which versionsm, what kind of chart or any other deployment you have, how exactly did you upgrade (did

[GitHub] [airflow] potiuk commented on issue #28227: Scheduler error: 'V1PodSpec' object has no attribute '_ephemeral_containers'

2022-12-18 Thread GitBox
potiuk commented on issue #28227: URL: https://github.com/apache/airflow/issues/28227#issuecomment-1356886524 Ah I see you seeem to do run it. @dstandish @jedcunningham - does it ring a bell ? Did you have anything specfic in your upgrade also around the k8s version change during mi

[GitHub] [airflow] potiuk commented on issue #28227: Scheduler error: 'V1PodSpec' object has no attribute '_ephemeral_containers'

2022-12-18 Thread GitBox
potiuk commented on issue #28227: URL: https://github.com/apache/airflow/issues/28227#issuecomment-1356885469 Did you try `airflow dags reserialize` command @KulykDmytro after the upgrade? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[airflow] branch constraints-main updated: Updating constraints. Build id:

2022-12-18 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 913fdd3401 Updating con

[GitHub] [airflow] potiuk closed issue #28201: Tasks get terminated with SIGTERM

2022-12-18 Thread GitBox
potiuk closed issue #28201: Tasks get terminated with SIGTERM URL: https://github.com/apache/airflow/issues/28201 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe

[GitHub] [airflow] potiuk commented on issue #28201: Tasks get terminated with SIGTERM

2022-12-18 Thread GitBox
potiuk commented on issue #28201: URL: https://github.com/apache/airflow/issues/28201#issuecomment-1356881079 Converting into discussion as it seems it is just misconfiguration. We can always re-open if we have evidences it's not. -- This is an automated message from the Apache Git Servic

[GitHub] [airflow] potiuk commented on issue #28201: Tasks get terminated with SIGTERM

2022-12-18 Thread GitBox
potiuk commented on issue #28201: URL: https://github.com/apache/airflow/issues/28201#issuecomment-1356880839 Looks like the problem is connected somehow with bad configuration of your S3 logs. It seems to fail when logging fails with 403 - forbidden error. Fix the problem (by configuring t

[GitHub] [airflow] potiuk commented on pull request #28182: Better error message for bad stat name

2022-12-18 Thread GitBox
potiuk commented on PR #28182: URL: https://github.com/apache/airflow/pull/28182#issuecomment-1356876053 > you don't think that, given that the work has already been done, it's more positive than negative to remove this constant, which ultimately is not really needed and is somewhat vague?

[GitHub] [airflow] potiuk closed issue #28059: Massive slowdown of DAG File processor due to JSON schema upgrade in Airflow's core.

2022-12-18 Thread GitBox
potiuk closed issue #28059: Massive slowdown of DAG File processor due to JSON schema upgrade in Airflow's core. URL: https://github.com/apache/airflow/issues/28059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] potiuk commented on issue #28059: Massive slowdown of DAG File processor due to JSON schema upgrade in Airflow's core.

2022-12-18 Thread GitBox
potiuk commented on issue #28059: URL: https://github.com/apache/airflow/issues/28059#issuecomment-1356873332 Since we have not heard from the user - converting it into a discussion. Shall there be more information/data provided we can consider what to do with it. -- This is an automated

[airflow] branch main updated (bea91b9018 -> 21f0600f69)

2022-12-18 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from bea91b9018 Re-enable Plyvel on ARM as it now builds cleanly (#28443) add 21f0600f69 Correctly template Glue Jobs `

[GitHub] [airflow] potiuk merged pull request #28403: Correctly template Glue Jobs `create_job_kwargs` arg

2022-12-18 Thread GitBox
potiuk merged PR #28403: URL: https://github.com/apache/airflow/pull/28403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[airflow] branch main updated (2a78f50b36 -> bea91b9018)

2022-12-18 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 2a78f50b36 Fix discoverability of tests for ARM in Breeze (#28432) add bea91b9018 Re-enable Plyvel on ARM as it no

[GitHub] [airflow] potiuk merged pull request #28443: Re-enable Plyvel on ARM as it now builds cleanly

2022-12-18 Thread GitBox
potiuk merged PR #28443: URL: https://github.com/apache/airflow/pull/28443 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #28034: Added custom environment variables to statsd

2022-12-18 Thread GitBox
potiuk commented on PR #28034: URL: https://github.com/apache/airflow/pull/28034#issuecomment-1356864111 Are you still working on it @RagingPuppies ? If so - it needs rebase/conflict resolving -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] potiuk commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly

2022-12-18 Thread GitBox
potiuk commented on PR #28442: URL: https://github.com/apache/airflow/pull/28442#issuecomment-1356858609 > Oh, I just repeat step which probably most of user do first, just try to install it by `pip` into the regular image. I see what you mean. But the same error might happen for a nu

[GitHub] [airflow] IAL32 commented on a diff in pull request #28403: Correctly template Glue Jobs `create_job_kwargs` arg

2022-12-18 Thread GitBox
IAL32 commented on code in PR #28403: URL: https://github.com/apache/airflow/pull/28403#discussion_r1051651346 ## tests/providers/amazon/aws/operators/test_glue.py: ## @@ -16,20 +16,62 @@ # under the License. from __future__ import annotations +import copy from unittest imp

[airflow] branch constraints-main updated: Updating constraints. Build id:

2022-12-18 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 06a0afc94c Updating con

[GitHub] [airflow] vchiapaikeo commented on issue #28393: Webserver reports "ImportError: Module "airflow.providers.google.cloud.operators.mlengine" does not define a "AIPlatformConsoleLink" attribute

2022-12-18 Thread GitBox
vchiapaikeo commented on issue #28393: URL: https://github.com/apache/airflow/issues/28393#issuecomment-1356852316 Hey @uranusjr , I'm happy to remove this from the config but modifying the pre-commit hook is a bit outside my wheelhouse. Is it okay if we scope the pre-commit hook edits out

[GitHub] [airflow] Taragolis commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly

2022-12-18 Thread GitBox
Taragolis commented on PR #28442: URL: https://github.com/apache/airflow/pull/28442#issuecomment-1356851154 Oh, I just repeat step which probably most of user do first, just try to install it by `pip` into the regular image. Some of them after would read documentation and found this i

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28444: Fix GCSToBigQueryOperator not respecting schema_obj

2022-12-18 Thread GitBox
boring-cyborg[bot] commented on PR #28444: URL: https://github.com/apache/airflow/pull/28444#issuecomment-1356850980 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] vchiapaikeo opened a new pull request, #28444: Fix GCSToBigQueryOperator not respecting schema_obj

2022-12-18 Thread GitBox
vchiapaikeo opened a new pull request, #28444: URL: https://github.com/apache/airflow/pull/28444 GCSToBigQueryOperator allows multiple ways to specify schema of the BigQuery table: 1. Setting autodetect == True 1. Setting schema_fields directly with autodetect == False 1

[GitHub] [airflow] vchiapaikeo commented on issue #28441: GCSToBigQueryOperator fails when schema_object is specified without schema_fields

2022-12-18 Thread GitBox
vchiapaikeo commented on issue #28441: URL: https://github.com/apache/airflow/issues/28441#issuecomment-1356851027 Sure @eladkal - this is here: https://github.com/apache/airflow/pull/28444 -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [airflow] potiuk opened a new pull request, #28443: Re-enable Plyvel on ARM as it now builds cleanly

2022-12-18 Thread GitBox
potiuk opened a new pull request, #28443: URL: https://github.com/apache/airflow/pull/28443 Previously, Plyvel had to be disabled in order to get ARM compatibility (it did not have binary wheels released and it failed to compile cleanly on debian). But the last problem is fixed now, so w

[GitHub] [airflow] potiuk commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly

2022-12-18 Thread GitBox
potiuk commented on PR #28442: URL: https://github.com/apache/airflow/pull/28442#issuecomment-1356844403 > Hmmm.. I've unable to install it into the regular airflow image We have a number of requirements that require customising image rather than extending it. We have good instruction

[GitHub] [airflow] Taragolis commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly

2022-12-18 Thread GitBox
Taragolis commented on PR #28442: URL: https://github.com/apache/airflow/pull/28442#issuecomment-1356842242 Hmmm.. I've unable to install it into the regular airflow image ```console ❯ docker run -it --rm apache/airflow:2.5.0-python3.9 bash airflow@7e0bc7525889:/opt/airflow$

[GitHub] [airflow] potiuk commented on pull request #28442: Re-enable azure service bus on ARM as it now builds cleanly

2022-12-18 Thread GitBox
potiuk commented on PR #28442: URL: https://github.com/apache/airflow/pull/28442#issuecomment-1356840195 Actually - when I tried it on ARM now, the azure-service-bus package built cleanly on ARM without any setup and special libraries. I think it is quite ok to add it - we have quite a few

[GitHub] [airflow] potiuk opened a new pull request, #28442: Re-enable azure service bus on ARM as it now builds cleanly

2022-12-18 Thread GitBox
potiuk opened a new pull request, #28442: URL: https://github.com/apache/airflow/pull/28442 Previously, the Azure Service Bus had to be disabled in order to get ARM compatibility (it failed to build cleanly as uampq did not have binary wheels released and they failed to compile cleanly on d

[GitHub] [airflow] eladkal commented on issue #28441: GCSToBigQueryOperator fails when schema_object is specified without schema_fields

2022-12-18 Thread GitBox
eladkal commented on issue #28441: URL: https://github.com/apache/airflow/issues/28441#issuecomment-1356838736 The previous issue is 2 years old so I'm not sure if the description there is out of date? In any case the issue is assigned to you. waiting for your PR :) -- This is an autom

[GitHub] [airflow] vchiapaikeo commented on issue #28441: GCSToBigQueryOperator fails when schema_object is specified without schema_fields

2022-12-18 Thread GitBox
vchiapaikeo commented on issue #28441: URL: https://github.com/apache/airflow/issues/28441#issuecomment-1356838361 @eladkal , I don't think this is a duplicate. This issue seems like a bug because we are not setting the instance variable, self.schema_fields and are instead setting the local

[airflow] branch main updated: Fix discoverability of tests for ARM in Breeze (#28432)

2022-12-18 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2a78f50b36 Fix discoverability of tests for ARM in

[GitHub] [airflow] potiuk merged pull request #28432: Fix discoverability of tests for ARM in Breeze

2022-12-18 Thread GitBox
potiuk merged PR #28432: URL: https://github.com/apache/airflow/pull/28432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #28432: Fix discoverability of tests for ARM in Breeze

2022-12-18 Thread GitBox
potiuk commented on PR #28432: URL: https://github.com/apache/airflow/pull/28432#issuecomment-1356837794 All looks good. Merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [airflow] potiuk commented on pull request #28432: Fix discoverability of tests for ARM in Breeze

2022-12-18 Thread GitBox
potiuk commented on PR #28432: URL: https://github.com/apache/airflow/pull/28432#issuecomment-1356837188 > Actually this is a "grey zone": uamqp still do not have wheel packages for [aarch64](https://pypi.org/project/uamqp/#files) - open issue in Github: https://github.com/Azure/azure-uamqp

[GitHub] [airflow] Taragolis commented on a diff in pull request #28432: Fix discoverability of tests for ARM in Breeze

2022-12-18 Thread GitBox
Taragolis commented on code in PR #28432: URL: https://github.com/apache/airflow/pull/28432#discussion_r1051632929 ## tests/providers/microsoft/azure/hooks/test_asb.py: ## @@ -19,8 +19,12 @@ from unittest import mock import pytest -from azure.servicebus import ServiceBusClie

[airflow] branch main updated (f115b207bc -> efaac217e9)

2022-12-18 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from f115b207bc fIx isort problems introduced by recent isort release (#28434) add efaac217e9 Fix output buffering for

[GitHub] [airflow] potiuk merged pull request #28433: Fix output buffering for `breeze testing test`

2022-12-18 Thread GitBox
potiuk merged PR #28433: URL: https://github.com/apache/airflow/pull/28433 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #28433: Fix output buffering for `breeze testing test`

2022-12-18 Thread GitBox
potiuk commented on PR #28433: URL: https://github.com/apache/airflow/pull/28433#issuecomment-1356833772 Static checks passed. Merging/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [airflow] potiuk commented on a diff in pull request #28432: Fix discoverability of tests for ARM in Breeze

2022-12-18 Thread GitBox
potiuk commented on code in PR #28432: URL: https://github.com/apache/airflow/pull/28432#discussion_r1051630643 ## tests/providers/microsoft/azure/hooks/test_asb.py: ## @@ -19,8 +19,12 @@ from unittest import mock import pytest -from azure.servicebus import ServiceBusClient,

[GitHub] [airflow] potiuk commented on a diff in pull request #28432: Fix discoverability of tests for ARM in Breeze

2022-12-18 Thread GitBox
potiuk commented on code in PR #28432: URL: https://github.com/apache/airflow/pull/28432#discussion_r1051630531 ## tests/system/providers/google/leveldb/example_leveldb.py: ## @@ -23,8 +23,16 @@ import os from datetime import datetime +import pytest + from airflow import mo

[GitHub] [airflow] potiuk commented on pull request #28300: Add Public Interface description to Airflow documentation

2022-12-18 Thread GitBox
potiuk commented on PR #28300: URL: https://github.com/apache/airflow/pull/28300#issuecomment-1356830770 Rebased to apply static check fixes with isort. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] potiuk commented on pull request #28433: Fix output buffering for `breeze testing test`

2022-12-18 Thread GitBox
potiuk commented on PR #28433: URL: https://github.com/apache/airflow/pull/28433#issuecomment-1356830183 Rebased to account for static check fixes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch main updated: fIx isort problems introduced by recent isort release (#28434)

2022-12-18 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f115b207bc fIx isort problems introduced by recent

[GitHub] [airflow] potiuk merged pull request #28434: Fix isort problems introduced by recent isort release

2022-12-18 Thread GitBox
potiuk merged PR #28434: URL: https://github.com/apache/airflow/pull/28434 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #28434: Fix isort problems introduced by recent isort release

2022-12-18 Thread GitBox
potiuk commented on PR #28434: URL: https://github.com/apache/airflow/pull/28434#issuecomment-1356828189 Just flaky test failed in MSSQL. Merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] eladkal commented on issue #28441: GCSToBigQueryOperator fails when schema_object is specified without schema_fields

2022-12-18 Thread GitBox
eladkal commented on issue #28441: URL: https://github.com/apache/airflow/issues/28441#issuecomment-1356827403 Duplicate of https://github.com/apache/airflow/issues/12329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] potiuk commented on pull request #28436: Log FileTaskHandler to work with multi_namespace_mode when running using KubernetesExecutor

2022-12-18 Thread GitBox
potiuk commented on PR #28436: URL: https://github.com/apache/airflow/pull/28436#issuecomment-1356818337 > The static check error is due to `isort`, and is being resolved by #28434 It took surprisingly long to sort that one out but I **think** I sorted it out (pun intended). -- Th

  1   2   >