Re: [PR] Add helper function for CRUD operations on weaviate's schema and class objects [airflow]

2023-11-29 Thread via GitHub
pankajkoti commented on code in PR #35919: URL: https://github.com/apache/airflow/pull/35919#discussion_r1410245744 ## airflow/providers/weaviate/hooks/weaviate.py: ## @@ -112,22 +117,170 @@ def test_connection(self) -> tuple[bool, str]: self.log.error("Error testin

Re: [PR] Pass conn ID to ObjectStoragePath via URI [airflow]

2023-11-29 Thread via GitHub
bolkedebruin commented on PR #35913: URL: https://github.com/apache/airflow/pull/35913#issuecomment-1833200830 Can you add this to the documentation of object storage? Then lgtm. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Move `duckdb` & `pandas` import in tutorial DAG into task [airflow]

2023-11-29 Thread via GitHub
bolkedebruin commented on PR #35964: URL: https://github.com/apache/airflow/pull/35964#issuecomment-1833199752 Makes sense! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

(airflow) branch main updated: Move `duckdb` & `pandas` import in tutorial DAG into task (#35964)

2023-11-29 Thread bolke
This is an automated email from the ASF dual-hosted git repository. bolke pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f0ba2dced9 Move `duckdb` & `pandas` import in tutori

Re: [PR] Move `duckdb` & `pandas` import in tutorial DAG into task [airflow]

2023-11-29 Thread via GitHub
bolkedebruin merged PR #35964: URL: https://github.com/apache/airflow/pull/35964 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

(airflow) branch main updated: Rename `Connection.to_json_dict` to `Connection.to_dict` (#35894)

2023-11-29 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7594b7a8ee Rename `Connection.to_json_dict` to `Co

Re: [PR] Rename `Connection.to_json_dict` to `Connection.to_dict` [airflow]

2023-11-29 Thread via GitHub
eladkal merged PR #35894: URL: https://github.com/apache/airflow/pull/35894 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

Re: [PR] Note unsupported ExternalTaskSensor deferrable modes [airflow]

2023-11-29 Thread via GitHub
eladkal commented on code in PR #35854: URL: https://github.com/apache/airflow/pull/35854#discussion_r1410219173 ## airflow/sensors/external_task.py: ## @@ -131,6 +131,12 @@ class ExternalTaskSensor(BaseSensorOperator): or DAG does not exist (default value: False).

[PR] New breeze command to clean up previous provider artifacts [airflow]

2023-11-29 Thread via GitHub
amoghrajesh opened a new pull request, #35970: URL: https://github.com/apache/airflow/pull/35970 Currently during every release of a provider, cleanup of the older providers was done using a script. It would much better for the release manager if this were a breeze command.

Re: [PR] Introduce @task.bash TaskFlow decorator [airflow]

2023-11-29 Thread via GitHub
josh-fell commented on PR #30176: URL: https://github.com/apache/airflow/pull/30176#issuecomment-1833047011 One question I do have is should the `bash_command` attribute (aka the return value from the callable) be templated? Clearly `BashOperator` has this now, but with TaskFlow historicall

[I] Filter Mappable XCOM for Dynamic Task Mapping [airflow]

2023-11-29 Thread via GitHub
ketozhang opened a new issue, #35969: URL: https://github.com/apache/airflow/issues/35969 ### Description Allow filtering mappable XCOMs such that inputs to dynamic task mapping run on a subset of the XCOM. ```python @task def get_integers(): return [-1, 0, 1]

Re: [PR] Add multiselect to run state in grid view [airflow]

2023-11-29 Thread via GitHub
vchiapaikeo commented on code in PR #35403: URL: https://github.com/apache/airflow/pull/35403#discussion_r1410026756 ## airflow/www/static/js/api/useGridData.ts: ## @@ -105,6 +105,7 @@ const useGridData = () => { }; const response = await axios.get(gridDataUrl, {

Re: [PR] Handle AzureBatch task result correctly [airflow]

2023-11-29 Thread via GitHub
github-actions[bot] commented on PR #31878: URL: https://github.com/apache/airflow/pull/31878#issuecomment-1832897688 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [I] Airflow use more swap of ubuntu [airflow]

2023-11-29 Thread via GitHub
github-actions[bot] closed issue #35501: Airflow use more swap of ubuntu URL: https://github.com/apache/airflow/issues/35501 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] Airflow use more swap of ubuntu [airflow]

2023-11-29 Thread via GitHub
github-actions[bot] commented on issue #35501: URL: https://github.com/apache/airflow/issues/35501#issuecomment-1832897593 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

[PR] Have `BaseOperator` get DAG from current task group if necessary [airflow]

2023-11-29 Thread via GitHub
sean-rose opened a new pull request, #35968: URL: https://github.com/apache/airflow/pull/35968 If there isn't an active DAG context but there is an active task group context where the task group has an associated DAG, then currently operators won't get the DAG from the task

(airflow) 01/01: Add feature to build "chicken-egg" packages from sources

2023-11-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch build-common-io-from-sources in repository https://gitbox.apache.org/repos/asf/airflow.git commit acec7a4a24db0a63ac259b0d07fc9c4f575f9555 Author: Jarek Potiuk AuthorDate: Mon Nov 27 15:48:43 202

(airflow) branch build-common-io-from-sources updated (e53bb0d69c -> acec7a4a24)

2023-11-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch build-common-io-from-sources in repository https://gitbox.apache.org/repos/asf/airflow.git omit e53bb0d69c Add feature to build "chicken-egg" packages from sources add 92cc2ffd86 Add sec

Re: [PR] Mark daskexecutor provider as removed [airflow]

2023-11-29 Thread via GitHub
potiuk commented on code in PR #35965: URL: https://github.com/apache/airflow/pull/35965#discussion_r1409980893 ## airflow/providers/daskexecutor/CHANGELOG.rst: ## @@ -27,6 +27,12 @@ Changelog - +1.1.1 +. + +* Mark daskececutor provider as removed Review Comment

Re: [PR] Create FAB provider and move FAB auth manager in it [airflow]

2023-11-29 Thread via GitHub
ferruzzi commented on code in PR #35926: URL: https://github.com/apache/airflow/pull/35926#discussion_r1409879281 ## .pre-commit-config.yaml: ## @@ -525,6 +524,7 @@ repos: ^airflow/providers/apache/spark/hooks/| ^airflow/providers/apache/spark/operators/|

Re: [PR] Add multiselect to run state in grid view [airflow]

2023-11-29 Thread via GitHub
pierrejeambrun commented on code in PR #35403: URL: https://github.com/apache/airflow/pull/35403#discussion_r1409952218 ## airflow/www/static/js/api/useGridData.ts: ## @@ -105,6 +105,7 @@ const useGridData = () => { }; const response = await axios.get(gridDataUrl,

[PR] [WIP] Add type hints for AWS provider [airflow]

2023-11-29 Thread via GitHub
hussein-awala opened a new pull request, #35966: URL: https://github.com/apache/airflow/pull/35966 closes: #11297 I will split the PR into small PRs, especially the refactoring/fixes changes, but I need to make it work first. -- This is an automated message from the Apache Git Serv

Re: [PR] Support not/and/or search filter for tags [airflow]

2023-11-29 Thread via GitHub
raghukpara commented on PR #33490: URL: https://github.com/apache/airflow/pull/33490#issuecomment-1832731583 Hi, is this available yet? I'm trying to work with the operators but it doesn't seem to work. Please confirm if this is released. Thank you. -- This is an automated message from th

(airflow) branch constraints-main updated: Updating constraints. Github run id:7037941331

2023-11-29 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 9e9c932cb1 Updating con

Re: [PR] Mark daskexecutor provider as removed [airflow]

2023-11-29 Thread via GitHub
eladkal commented on code in PR #35965: URL: https://github.com/apache/airflow/pull/35965#discussion_r1409830150 ## airflow/providers/daskexecutor/CHANGELOG.rst: ## @@ -27,6 +27,12 @@ Changelog - +1.1.1 +. + +* Mark daskececutor provider as removed Review Commen

Re: [I] GlueJobSensor stuck in Running state with Verbose set to True [airflow]

2023-11-29 Thread via GitHub
vincbeck commented on issue #35431: URL: https://github.com/apache/airflow/issues/35431#issuecomment-1832646569 I cannot reproduce the issue. Moreover, our system test `example_glue` is running the operator **GlueJobSensor** with `verbose` set to `True` (see [here](https://github.com/apache

(airflow) branch main updated: AWS auth manager: implement all `is_authorized_*` methods (but `is_authorized_dag`) (#35928)

2023-11-29 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 985d0589af AWS auth manager: implement all `is_au

Re: [PR] AWS auth manager: implement all `is_authorized_*` methods (but `is_authorized_dag`) [airflow]

2023-11-29 Thread via GitHub
vincbeck merged PR #35928: URL: https://github.com/apache/airflow/pull/35928 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [I] Triggering DAG from UI Creates 2 DAG runs if the DAG was paused [airflow]

2023-11-29 Thread via GitHub
RNHTTR commented on issue #24534: URL: https://github.com/apache/airflow/issues/24534#issuecomment-1832613820 I no longer think this is actually a bug for DAG runs that are within a DAG's schedule interval. It makes sense that when a DAG is unpaused, the run for the current DAG interval is

[PR] Mark daskexecutor provider as removed [airflow]

2023-11-29 Thread via GitHub
potiuk opened a new pull request, #35965: URL: https://github.com/apache/airflow/pull/35965 Following the discussion in airflow devlist we mark daskexecutor as removed. See: https://lists.apache.org/thread/fxv44cqqljrrhll3fdpdgc9h9fz5ghcy --- **^ Add

Re: [PR] Move `duckdb` & `pandas` import in tutorial DAG into task [airflow]

2023-11-29 Thread via GitHub
potiuk commented on code in PR #35964: URL: https://github.com/apache/airflow/pull/35964#discussion_r1409750177 ## airflow/example_dags/tutorial_objectstorage.py: ## @@ -62,9 +62,6 @@ def tutorial_objectstorage(): located [here](https://airflow.apache.org/docs/apache-

(airflow-site) branch gh-pages updated (b9f3e96d5c -> 8e64b68dd9)

2023-11-29 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard b9f3e96d5c Rewritten history to remove past gh-pages deployments new 8e64b68dd9 Rewritten history to

Re: [PR] Add explanation about cleaning logs with helm [airflow]

2023-11-29 Thread via GitHub
hussein-awala commented on code in PR #35783: URL: https://github.com/apache/airflow/pull/35783#discussion_r1409741997 ## airflow/config_templates/config.yml: ## @@ -658,6 +658,8 @@ logging: description: | Whether the local log files for GCS, S3, WASB and OSS rem

(airflow) branch main updated: Update providers metadata 2023-11-29 (#35943)

2023-11-29 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 3a084b70c2 Update providers metadata 2023-11-29 (#

Re: [PR] Update providers metadata 2023-11-29 [airflow]

2023-11-29 Thread via GitHub
eladkal merged PR #35943: URL: https://github.com/apache/airflow/pull/35943 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

(airflow-site) branch main updated: Update _index.md (#900)

2023-11-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 519002bbf1 Update _index.md (#900) 519002bbf1

Re: [PR] Update _index.md to include the Domino project [airflow-site]

2023-11-29 Thread via GitHub
potiuk merged PR #900: URL: https://github.com/apache/airflow-site/pull/900 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.

(airflow) branch main updated: Revert "Prevent assignment of non JSON serializable values to DagRun.conf dict (#35096)" (#35959)

2023-11-29 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4a7c7460bf Revert "Prevent assignment of n

Re: [I] Scheduler getting crashed when downgrading from 2.8.0b1 to 2.7.3 [airflow]

2023-11-29 Thread via GitHub
ephraimbuddy closed issue #35914: Scheduler getting crashed when downgrading from 2.8.0b1 to 2.7.3 URL: https://github.com/apache/airflow/issues/35914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Revert "Prevent assignment of non JSON serializable values to DagRun.conf dict (#35096)" [airflow]

2023-11-29 Thread via GitHub
ephraimbuddy merged PR #35959: URL: https://github.com/apache/airflow/pull/35959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@air

Re: [PR] Move `duckdb` & `pandas` import in tutorial DAG into task [airflow]

2023-11-29 Thread via GitHub
ephraimbuddy commented on code in PR #35964: URL: https://github.com/apache/airflow/pull/35964#discussion_r1409709511 ## airflow/example_dags/tutorial_objectstorage.py: ## @@ -62,9 +62,6 @@ def tutorial_objectstorage(): located [here](https://airflow.apache.org/docs/a

Re: [I] Refactor `S3KeysUnchangedSensor` system test [airflow]

2023-11-29 Thread via GitHub
o-nikolas closed issue #26725: Refactor `S3KeysUnchangedSensor` system test URL: https://github.com/apache/airflow/issues/26725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] Move `duckdb` & `pandas` import in tutorial DAG into task [airflow]

2023-11-29 Thread via GitHub
ephraimbuddy opened a new pull request, #35964: URL: https://github.com/apache/airflow/pull/35964 This improves the code as per best practices and avoids import error if duckdb is not installed -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] AWS auth manager: implement all `is_authorized_*` methods (but `is_authorized_dag`) [airflow]

2023-11-29 Thread via GitHub
o-nikolas commented on code in PR #35928: URL: https://github.com/apache/airflow/pull/35928#discussion_r1409690314 ## airflow/providers/amazon/aws/auth_manager/aws_auth_manager.py: ## @@ -91,7 +91,13 @@ def is_authorized_configuration( details: ConfigurationDetails | No

[PR] Feature/trino provider timezone [airflow]

2023-11-29 Thread via GitHub
romsharon98 opened a new pull request, #35963: URL: https://github.com/apache/airflow/pull/35963 This PR continue this PR: #35147 With cherry picking @duyet --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://githu

Re: [I] Shared taskflow dags and tasks aren't counted in dag_bag stats [airflow]

2023-11-29 Thread via GitHub
getaaron commented on issue #35962: URL: https://github.com/apache/airflow/issues/35962#issuecomment-1832415747 Never mind, this was user error. šŸ˜… -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Shared taskflow dags and tasks aren't counted in dag_bag stats [airflow]

2023-11-29 Thread via GitHub
getaaron closed issue #35962: Shared taskflow dags and tasks aren't counted in dag_bag stats URL: https://github.com/apache/airflow/issues/35962 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[I] Shared taskflow dags and tasks aren't counted in dag_bag stats [airflow]

2023-11-29 Thread via GitHub
getaaron opened a new issue, #35962: URL: https://github.com/apache/airflow/issues/35962 ### Apache Airflow version 2.7.3 ### What happened The [taskflow docs](https://github.com/apache/airflow/blob/main/docs/apache-airflow/tutorial/taskflow.rst?plain=1#L176-L213) say t

Re: [PR] Base date for fetching dag grid view must include selected run_id [airflow]

2023-11-29 Thread via GitHub
bbovenzi commented on PR #34887: URL: https://github.com/apache/airflow/pull/34887#issuecomment-1832384351 > Is it possible to take the base date changes separately as an improvement? This bug makes the most recent versions of airflow completely unusable for us, would be great to have some

Re: [PR] confirmation dialog box for DAG run actions [airflow]

2023-11-29 Thread via GitHub
bbovenzi commented on code in PR #35393: URL: https://github.com/apache/airflow/pull/35393#discussion_r1409633670 ## airflow/www/static/js/dag/details/dagRun/ActionModal.tsx: ## @@ -0,0 +1,99 @@ +/*! + * Licensed to the Apache Software Foundation (ASF) under one + * or more cont

Re: [PR] confirmation dialog box for DAG run actions [airflow]

2023-11-29 Thread via GitHub
bbovenzi commented on code in PR #35393: URL: https://github.com/apache/airflow/pull/35393#discussion_r1409632000 ## airflow/www/static/js/dag/details/dagRun/ClearRun.tsx: ## @@ -59,31 +60,67 @@ const ClearRun = ({ runId, ...otherProps }: Props) => { onQueue({ confirmed: tr

Re: [PR] confirmation dialog box for DAG run actions [airflow]

2023-11-29 Thread via GitHub
bbovenzi commented on code in PR #35393: URL: https://github.com/apache/airflow/pull/35393#discussion_r1409630534 ## airflow/www/static/js/dag/details/dagRun/MarkRunAs.tsx: ## @@ -58,42 +65,115 @@ const MarkRunAs = ({ runId, state, ...otherProps }: Props) => { markSuccess(

Re: [PR] confirmation dialog box for DAG run actions [airflow]

2023-11-29 Thread via GitHub
bbovenzi commented on code in PR #35393: URL: https://github.com/apache/airflow/pull/35393#discussion_r1409626735 ## airflow/www/static/js/dag/details/dagRun/ClearRun.tsx: ## @@ -59,31 +60,67 @@ const ClearRun = ({ runId, ...otherProps }: Props) => { onQueue({ confirmed: tr

Re: [I] Cannot download packages for 2.7.3 [airflow]

2023-11-29 Thread via GitHub
potiuk closed issue #35953: Cannot download packages for 2.7.3 URL: https://github.com/apache/airflow/issues/35953 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscrib

Re: [I] Cannot download packages for 2.7.3 [airflow]

2023-11-29 Thread via GitHub
potiuk commented on issue #35953: URL: https://github.com/apache/airflow/issues/35953#issuecomment-1832353897 All is only a development dependency - you should not use it to download the released version. You need to individually specify all packages you want to install: https://ai

Re: [PR] Fix EC2Hook get_instance for client_type api [airflow]

2023-11-29 Thread via GitHub
hussein-awala merged PR #35960: URL: https://github.com/apache/airflow/pull/35960 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [PR] Fix EC2Hook get_instance for client_type api [airflow]

2023-11-29 Thread via GitHub
hussein-awala commented on PR #35960: URL: https://github.com/apache/airflow/pull/35960#issuecomment-1832344840 Thanks to all of you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

(airflow) branch main updated: Fix EC2Hook get_instance for client_type api (#35960)

2023-11-29 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new ab835c20b2 Fix EC2Hook get_instance for clien

Re: [I] Airflow users/list/ is very slow [airflow]

2023-11-29 Thread via GitHub
tirkarthi commented on issue #31340: URL: https://github.com/apache/airflow/issues/31340#issuecomment-1832332100 @potiuk The issue exists in main branch too. We have this fixed internally and we don't use columns like created, created_by, changed_by etc. The issue is that fields like create

Re: [I] Airflow SSO integration issue with github [airflow]

2023-11-29 Thread via GitHub
eladkal closed issue #35896: Airflow SSO integration issue with github URL: https://github.com/apache/airflow/issues/35896 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [I] Airflow SSO integration issue with github [airflow]

2023-11-29 Thread via GitHub
eladkal commented on issue #35896: URL: https://github.com/apache/airflow/issues/35896#issuecomment-1832312563 For the moment this doesnt seem to be Airflow bug. Its a custom logic you wrote which doesnt work as expected thus I am converting this report into github discussion where the co

Re: [PR] Fix EC2Hook get_instance for client_type api [airflow]

2023-11-29 Thread via GitHub
eladkal commented on code in PR #35960: URL: https://github.com/apache/airflow/pull/35960#discussion_r1409578764 ## airflow/providers/amazon/aws/hooks/ec2.py: ## @@ -85,7 +85,7 @@ def get_instance(self, instance_id: str, filters: list | None = None): :return: Instance

Re: [PR] Fix EC2Hook get_instance for client_type api [airflow]

2023-11-29 Thread via GitHub
pankajkoti commented on code in PR #35960: URL: https://github.com/apache/airflow/pull/35960#discussion_r1409553054 ## airflow/providers/amazon/aws/hooks/ec2.py: ## @@ -85,7 +85,7 @@ def get_instance(self, instance_id: str, filters: list | None = None): :return: Instan

Re: [I] Scheduler getting crashed when downgrading from 2.8.0b1 to 2.7.3 [airflow]

2023-11-29 Thread via GitHub
ephraimbuddy commented on issue #35914: URL: https://github.com/apache/airflow/issues/35914#issuecomment-1832231345 cc @PashkPashk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] AWS auth manager: implement all `is_authorized_*` methods (but `is_authorized_dag`) [airflow]

2023-11-29 Thread via GitHub
vincbeck commented on code in PR #35928: URL: https://github.com/apache/airflow/pull/35928#discussion_r1409510501 ## airflow/providers/amazon/aws/auth_manager/aws_auth_manager.py: ## @@ -91,7 +91,13 @@ def is_authorized_configuration( details: ConfigurationDetails | Non

Re: [PR] AWS auth manager: implement all `is_authorized_*` methods (but `is_authorized_dag`) [airflow]

2023-11-29 Thread via GitHub
vincbeck commented on code in PR #35928: URL: https://github.com/apache/airflow/pull/35928#discussion_r1409505876 ## airflow/providers/amazon/aws/auth_manager/aws_auth_manager.py: ## @@ -91,7 +91,13 @@ def is_authorized_configuration( details: ConfigurationDetails | Non

Re: [PR] AWS auth manager: implement all `is_authorized_*` methods (but `is_authorized_dag`) [airflow]

2023-11-29 Thread via GitHub
vincbeck commented on code in PR #35928: URL: https://github.com/apache/airflow/pull/35928#discussion_r1409501281 ## airflow/providers/amazon/aws/auth_manager/aws_auth_manager.py: ## @@ -91,7 +91,13 @@ def is_authorized_configuration( details: ConfigurationDetails | Non

Re: [I] triggerer block_watchdog is crying "wolf!" [airflow]

2023-11-29 Thread via GitHub
dlesco commented on issue #35806: URL: https://github.com/apache/airflow/issues/35806#issuecomment-1832176058 OK, I'm working on setting up my local environment so that pre-commit, etc, is enabled, before submitting. -- This is an automated message from the Apache Git Service. To respond

Re: [PR] AWS auth manager: implement all `is_authorized_*` methods (but `is_authorized_dag`) [airflow]

2023-11-29 Thread via GitHub
vincbeck commented on code in PR #35928: URL: https://github.com/apache/airflow/pull/35928#discussion_r1409492309 ## airflow/providers/amazon/aws/auth_manager/aws_auth_manager.py: ## @@ -142,7 +166,12 @@ def is_authorized_view( access_view: AccessView, user: Ba

Re: [PR] Fix EC2Hook get_instance for client_type api [airflow]

2023-11-29 Thread via GitHub
hussein-awala commented on code in PR #35960: URL: https://github.com/apache/airflow/pull/35960#discussion_r1409490870 ## airflow/providers/amazon/aws/hooks/ec2.py: ## @@ -85,7 +85,7 @@ def get_instance(self, instance_id: str, filters: list | None = None): :return: Ins

Re: [PR] Fix EC2Hook get_instance for client_type api [airflow]

2023-11-29 Thread via GitHub
vincbeck commented on code in PR #35960: URL: https://github.com/apache/airflow/pull/35960#discussion_r1409488348 ## airflow/providers/amazon/aws/hooks/ec2.py: ## @@ -85,7 +85,7 @@ def get_instance(self, instance_id: str, filters: list | None = None): :return: Instance

Re: [I] Triggering DAG from UI Creates 2 DAG runs if the DAG was paused [airflow]

2023-11-29 Thread via GitHub
vaibhavnsingh commented on issue #24534: URL: https://github.com/apache/airflow/issues/24534#issuecomment-1832149901 **Schedule Created time - 5:52 PM (frequency ā€“ 4 mins, starting 6:00 PM)** 1st trigger - 6:04 PM 2nd trigger - 6:08 PM **Disabled - 6:09 PM** **Enabled - 6:

[PR] Fix EC2Hook get_instance for client_type api [airflow]

2023-11-29 Thread via GitHub
hussein-awala opened a new pull request, #35960: URL: https://github.com/apache/airflow/pull/35960 The method `get_instance` should return a single instance and not a list. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] Add multiselect to run state in grid view [airflow]

2023-11-29 Thread via GitHub
vchiapaikeo commented on PR #35403: URL: https://github.com/apache/airflow/pull/35403#issuecomment-1832092127 Latest demo: https://github.com/apache/airflow/assets/9200263/b315b53a-6f2e-47cc-971c-bcada1ca0a3f -- This is an automated message from the Apache Git Service. To r

[PR] Revert "Prevent assignment of non JSON serializable values to DagRun.conf dict (#35096)" [airflow]

2023-11-29 Thread via GitHub
ephraimbuddy opened a new pull request, #35959: URL: https://github.com/apache/airflow/pull/35959 This reverts commit 84c40a7877e5ea9dbee03b707065cb590f872111. See discussions at #35914 closes: #35914 -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Unable to filter SQS messages using jsonpath [airflow]

2023-11-29 Thread via GitHub
cliebBS commented on issue #35932: URL: https://github.com/apache/airflow/issues/35932#issuecomment-1832045361 Turns out this isn't an issue. While I was writing a test for my fix, I discovered that I was actually matching against a different part of the message than I thought I was, meani

Re: [I] Unable to filter SQS messages using jsonpath [airflow]

2023-11-29 Thread via GitHub
cliebBS closed issue #35932: Unable to filter SQS messages using jsonpath URL: https://github.com/apache/airflow/issues/35932 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Add multiselect to run state in grid view [airflow]

2023-11-29 Thread via GitHub
vchiapaikeo commented on code in PR #35403: URL: https://github.com/apache/airflow/pull/35403#discussion_r1409402526 ## airflow/www/static/js/api/useGridData.ts: ## @@ -105,6 +105,7 @@ const useGridData = () => { }; const response = await axios.get(gridDataUrl, {

Re: [PR] Fix Properly parse out the value from a jsonpath match in SQS [airflow]

2023-11-29 Thread via GitHub
cliebBS closed pull request #35933: Fix Properly parse out the value from a jsonpath match in SQS URL: https://github.com/apache/airflow/pull/35933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

(airflow) branch constraints-main updated: Updating constraints. Github run id:7033355714

2023-11-29 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 20734551ab Updating con

Re: [I] Connection for S3KeySensor [airflow]

2023-11-29 Thread via GitHub
Taragolis closed issue #35955: Connection for S3KeySensor URL: https://github.com/apache/airflow/issues/35955 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-

Re: [I] Volume is missing for sshKeySecret when dag persistence is enabled. [airflow]

2023-11-29 Thread via GitHub
potiuk commented on issue #27476: URL: https://github.com/apache/airflow/issues/27476#issuecomment-1831965673 > So if it's not expected that we will use git-sync with shared volumes, it would be deleted from the documentation and stop misleading people? We alredy heavily discourage it

Re: [I] Scheduler getting crashed when downgrading from 2.8.0b1 to 2.7.3 [airflow]

2023-11-29 Thread via GitHub
ephraimbuddy commented on issue #35914: URL: https://github.com/apache/airflow/issues/35914#issuecomment-1831944415 Iā€™m getting a different error than the reported error in https://github.com/apache/airflow/issues/35095. The error was reported in 2.5.3. When I revert the change, main cu

(airflow) branch main updated: Remove setting a non-existing object param and use local var instead in S3Hook (#35950)

2023-11-29 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9ab343714a Remove setting a non-existing obje

Re: [PR] Remove setting a non-existing object param and use local var instead in S3Hook [airflow]

2023-11-29 Thread via GitHub
hussein-awala merged PR #35950: URL: https://github.com/apache/airflow/pull/35950 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[PR] Add processor_subdir to import_error table to handle multiple dag processors [airflow]

2023-11-29 Thread via GitHub
tirkarthi opened a new pull request, #35956: URL: https://github.com/apache/airflow/pull/35956 Add processor_subdir to import_error table to update the value while creating ImportError entry. Use the processor_subdir to filter for ImportError entries that belong to the current dag processor

[I] Connection for S3KeySensor [airflow]

2023-11-29 Thread via GitHub
M-Waleed-Khan opened a new issue, #35955: URL: https://github.com/apache/airflow/issues/35955 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Hi. I am trying to use S3KeySensor in my dag to trigger a task inside the same

Re: [PR] Fix S3Hook get_key method by removing load extra args [airflow]

2023-11-29 Thread via GitHub
Taragolis commented on PR #35951: URL: https://github.com/apache/airflow/pull/35951#issuecomment-1831890335 Long term solution would be, get rid of (deprecate) `boto3.resource` usage in favor of `boto3.client`: 1. Looks like this objects frozen in a time and do not support all of the fea

Re: [PR] Fix S3Hook get_key method by removing load extra args [airflow]

2023-11-29 Thread via GitHub
hussein-awala commented on PR #35951: URL: https://github.com/apache/airflow/pull/35951#issuecomment-1831881518 > Actually it does accept, it just not well documented, maybe because `boto3.resoure` abandoned, at least how it look from my perspective as a regular user of `boto3`. > >

Re: [PR] Bump min s3fs version [airflow]

2023-11-29 Thread via GitHub
potiuk commented on PR #35947: URL: https://github.com/apache/airflow/pull/35947#issuecomment-1831873245 Closing in favour of https://github.com/apache/airflow/pull/35952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Bump min s3fs version [airflow]

2023-11-29 Thread via GitHub
potiuk closed pull request #35947: Bump min s3fs version URL: https://github.com/apache/airflow/pull/35947 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [PR] Limit aibotocore to help `pip` with backtracking issue for s3fs [airflow]

2023-11-29 Thread via GitHub
potiuk commented on PR #35952: URL: https://github.com/apache/airflow/pull/35952#issuecomment-1831871888 I will merge quickly - tested it locally and it **should** work (at least the CI image in main should work). -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Relax aibotocore requirements for eager upgrade [airflow]

2023-11-29 Thread via GitHub
potiuk commented on PR #35945: URL: https://github.com/apache/airflow/pull/35945#issuecomment-1831872806 Closing in favour of https://github.com/apache/airflow/pull/35952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Relax aibotocore requirements for eager upgrade [airflow]

2023-11-29 Thread via GitHub
potiuk closed pull request #35945: Relax aibotocore requirements for eager upgrade URL: https://github.com/apache/airflow/pull/35945 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

(airflow) branch main updated: Limit aibotocore to help `pip` with backtracking issue for s3fs (#35952)

2023-11-29 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 98aa84b6c0 Limit aibotocore to help `pip` with back

Re: [PR] Limit aibotocore to help `pip` with backtracking issue for s3fs [airflow]

2023-11-29 Thread via GitHub
potiuk merged PR #35952: URL: https://github.com/apache/airflow/pull/35952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[I] Cannot download packages for 2.7.3 [airflow]

2023-11-29 Thread via GitHub
eugeny-stoyka opened a new issue, #35953: URL: https://github.com/apache/airflow/issues/35953 ### What do you see as an issue? Hello. Cannot download all packages by **Installing from PyPI** section in **README.md** `pip install 'apache-airflow[all]==2.7.3' --constraint "https

[PR] Limit aibotocore to help `pip` with backtracking issue for s3fs [airflow]

2023-11-29 Thread via GitHub
potiuk opened a new pull request, #35952: URL: https://github.com/apache/airflow/pull/35952 Recent release of aiobotocore, boto3, botocore conflicts with s3fs and confuses `pip` enough to go into conflict or long backtracking. --- **^ Add meaningful desc

Re: [I] Cannot download packages for 2.7.3 [airflow]

2023-11-29 Thread via GitHub
boring-cyborg[bot] commented on issue #35953: URL: https://github.com/apache/airflow/issues/35953#issuecomment-1831860280 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Fix S3Hook get_key method by removing load extra args [airflow]

2023-11-29 Thread via GitHub
Taragolis commented on PR #35951: URL: https://github.com/apache/airflow/pull/35951#issuecomment-1831843230 Actually it does accept, it just not well documented, maybe because `boto3.resoure` abandoned, at least how it look from my perspective as a regular user of `boto3`. And exampl

  1   2   >