Re: [PR] feat: enable HTTPRoute for Gateway API [airflow]

2023-10-11 Thread via GitHub
jedcunningham commented on code in PR #34716: URL: https://github.com/apache/airflow/pull/34716#discussion_r1356131694 ## helm_tests/webserver/test_httproute_flower.py: ## @@ -0,0 +1,97 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licen

Re: [PR] Let auth managers provide their own API endpoints [airflow]

2023-10-11 Thread via GitHub
jedcunningham commented on code in PR #34349: URL: https://github.com/apache/airflow/pull/34349#discussion_r1356129378 ## airflow/www/extensions/init_views.py: ## @@ -308,5 +318,16 @@ def init_api_experimental(app): "The authenticated user has full access.", Re

Re: [PR] feat: enable HTTPRoute for Gateway API [airflow]

2023-10-11 Thread via GitHub
Kyrremann commented on code in PR #34716: URL: https://github.com/apache/airflow/pull/34716#discussion_r1356126098 ## helm_tests/webserver/test_httproute_flower.py: ## @@ -0,0 +1,97 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license a

Re: [PR] feat: enable HTTPRoute for Gateway API [airflow]

2023-10-11 Thread via GitHub
Kyrremann commented on code in PR #34716: URL: https://github.com/apache/airflow/pull/34716#discussion_r1356119603 ## chart/templates/flower/flower-httproute.yaml: ## @@ -0,0 +1,82 @@ +{{/* + Licensed to the Apache Software Foundation (ASF) under one + or more contributor licens

[I] Scheduler is spending most of its time in clear_not_launched_queued_tasks function [airflow]

2023-10-11 Thread via GitHub
dirrao opened a new issue, #34877: URL: https://github.com/apache/airflow/issues/34877 ### Apache Airflow version 2.7.1 ### What happened Airflow running the clear_not_launched_queued_tasks function on a certain frequency (default 30 seconds). When we run the airflow on

[airflow] branch main updated: Add back decorator `has_access` (#34786)

2023-10-11 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 84941f86d7 Add back decorator `has_access` (#34786)

Re: [PR] Add back decorator `has_access` [airflow]

2023-10-11 Thread via GitHub
potiuk merged PR #34786: URL: https://github.com/apache/airflow/pull/34786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] Git Sync 4 support [airflow]

2023-10-11 Thread via GitHub
potiuk commented on issue #32335: URL: https://github.com/apache/airflow/issues/32335#issuecomment-1758912683 > https://github.com/kubernetes/git-sync/issues/827 Interesting. I see you proposed a fix (cool!) -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Fix _SECRET and _CMD broker configuration [airflow]

2023-10-11 Thread via GitHub
potiuk commented on PR #34782: URL: https://github.com/apache/airflow/pull/34782#issuecomment-1758908356 BTW. I still do not understand how "worker" could be affected but at least this fix might fix the liveness probe issue mentioned in https://github.com/apache/airflow/issues/34612#issueco

Re: [I] BROKER_URL_SECRET Not working as of Airflow 2.7 [airflow]

2023-10-11 Thread via GitHub
potiuk commented on issue #34612: URL: https://github.com/apache/airflow/issues/34612#issuecomment-1758907339 While I understand why the liveness probe mentioned by @Aakcht (because it runs the app directly) in Helm chart could fail in this case, I do not think the "worker" should have thi

Re: [PR] Fix _SECRET and _CMD broker configuration [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34782: URL: https://github.com/apache/airflow/pull/34782#discussion_r1356050515 ## airflow/providers/celery/executors/celery_executor_utils.py: ## @@ -60,16 +60,26 @@ TaskInstanceInCelery = Tuple[TaskInstanceKey, CommandType, Optional[str],

Re: [PR] Fix _SECRET and _CMD broker configuration [airflow]

2023-10-11 Thread via GitHub
potiuk commented on PR #34782: URL: https://github.com/apache/airflow/pull/34782#issuecomment-1758902407 > I wonder if we should put the entire `app` initialise logic into a function and have `app = _get_celery_app()` instead. Yes. This is better idea. Also `@provider_configuration_lo

Re: [I] Adding property files option in the Spark Submit command [airflow]

2023-10-11 Thread via GitHub
Bowrna commented on issue #34838: URL: https://github.com/apache/airflow/issues/34838#issuecomment-1758885562 sure @ghostp13409 , ping here if you need any help -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] DatabricksHook ClusterState [airflow]

2023-10-11 Thread via GitHub
Seokyun-Ha commented on PR #34643: URL: https://github.com/apache/airflow/pull/34643#issuecomment-1758863918 Thanks you all guys to review this PR ❤️ I will request to schedule PR merge on Airflow slack. cc. @alexott @Lee-W @utkarsharma2 -- This is an automated message from the A

[airflow] branch main updated: fix(TimeSensorAsync): use DAG timezone (#33406)

2023-10-11 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6c50ef59cc fix(TimeSensorAsync): use DAG timezone (

Re: [PR] fix(TimeSensorAsync): use DAG timezone [airflow]

2023-10-11 Thread via GitHub
boring-cyborg[bot] commented on PR #33406: URL: https://github.com/apache/airflow/pull/33406#issuecomment-1758854997 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.

Re: [PR] fix(TimeSensorAsync): use DAG timezone [airflow]

2023-10-11 Thread via GitHub
potiuk merged PR #33406: URL: https://github.com/apache/airflow/pull/33406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [I] TimeSensorAsync does not use DAG timezone to convert naive time input [airflow]

2023-10-11 Thread via GitHub
potiuk closed issue #33256: TimeSensorAsync does not use DAG timezone to convert naive time input URL: https://github.com/apache/airflow/issues/33256 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Let auth managers provide their own API endpoints [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34349: URL: https://github.com/apache/airflow/pull/34349#discussion_r1356013126 ## airflow/www/extensions/init_views.py: ## @@ -308,5 +318,16 @@ def init_api_experimental(app): "The authenticated user has full access.", RemovedIn

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1356003454 ## airflow/www/security_manager.py: ## @@ -738,24 +633,9 @@ def create_perm_vm_for_all_dag(self) -> None: def check_authorization( self, perms:

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355994191 ## airflow/www/security_manager.py: ## @@ -269,125 +272,47 @@ def get_user_roles(user=None): user = g.user return user.roles -def get_reada

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355994191 ## airflow/www/security_manager.py: ## @@ -269,125 +272,47 @@ def get_user_roles(user=None): user = g.user return user.roles -def get_reada

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355994191 ## airflow/www/security_manager.py: ## @@ -269,125 +272,47 @@ def get_user_roles(user=None): user = g.user return user.roles -def get_reada

Re: [PR] Add general links about TCP keepalive into AWS Lambda docs [airflow]

2023-10-11 Thread via GitHub
shubham22 commented on PR #34870: URL: https://github.com/apache/airflow/pull/34870#issuecomment-1758820613 Thanks for adding this @Taragolis. I haven't had time to do this myself due to travel for [Community over Code](https://communityovercode.org/) -- This is an automated message fro

Re: [I] Adding property files option in the Spark Submit command [airflow]

2023-10-11 Thread via GitHub
ghostp13409 commented on issue #34838: URL: https://github.com/apache/airflow/issues/34838#issuecomment-1758817607 Hi @Bowrna, I'm a first time contributor, and i feel like I could give this a try. Can I work on it? -- This is an automated message from the Apache Git Service. To respon

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355927810 ## airflow/api_connexion/security.py: ## @@ -48,10 +61,194 @@ def requires_access_decorator(func: T): @wraps(func) def decorated(*args, **kwargs):

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355974367 ## airflow/api_connexion/security.py: ## @@ -48,10 +61,194 @@ def requires_access_decorator(func: T): @wraps(func) def decorated(*args, **kwargs):

Re: [I] Add on_kill method to DataprocInstantiateWorkflowTemplateOperator [airflow]

2023-10-11 Thread via GitHub
michalsosn commented on issue #10385: URL: https://github.com/apache/airflow/issues/10385#issuecomment-1758793381 [this](https://github.com/apache/airflow/compare/main...michalsosn:airflow:10385-DataprocInstantiateWorkflowTemplateOperator-on_kill) change looks reasonable. Now I need to thor

Re: [PR] Add description to pools view [airflow]

2023-10-11 Thread via GitHub
vchiapaikeo commented on PR #34862: URL: https://github.com/apache/airflow/pull/34862#issuecomment-1758778346 Good call. What about some additional styling? https://github.com/apache/airflow/assets/9200263/0c9bcb63-342d-4e70-b726-a52de7388283";> https://github.com/apache/airflow

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355941755 ## airflow/www/extensions/init_jinja_globals.py: ## @@ -69,10 +70,13 @@ def prepare_jinja_globals(): "git_version": git_version, "k8s_or_k8sc

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355941755 ## airflow/www/extensions/init_jinja_globals.py: ## @@ -69,10 +70,13 @@ def prepare_jinja_globals(): "git_version": git_version, "k8s_or_k8sc

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355941755 ## airflow/www/extensions/init_jinja_globals.py: ## @@ -69,10 +70,13 @@ def prepare_jinja_globals(): "git_version": git_version, "k8s_or_k8sc

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355927810 ## airflow/api_connexion/security.py: ## @@ -48,10 +61,194 @@ def requires_access_decorator(func: T): @wraps(func) def decorated(*args, **kwargs):

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355925044 ## airflow/api_connexion/endpoints/xcom_endpoint.py: ## @@ -39,14 +39,7 @@ from airflow.api_connexion.types import APIResponse -@security.requires_access( -

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355923325 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -(permissions.ACTI

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355894015 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -(permissions.ACTI

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355917769 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -(permissions.ACTI

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355917769 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -(permissions.ACTI

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355917769 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -(permissions.ACTI

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355917769 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -(permissions.ACTI

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355907725 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -646,13 +606,8 @@ def patch_task_instance( return task_instance_reference_schema.dump(ti)

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355907009 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -527,13 +497,8 @@ def post_clear_task_instances(*, dag_id: str, session: Session = NEW_SESSION) ->

Re: [I] Postgres conn type should normalize to postgresql (instead of the inverse) [airflow]

2023-10-11 Thread via GitHub
Taragolis closed issue #34876: Postgres conn type should normalize to postgresql (instead of the inverse) URL: https://github.com/apache/airflow/issues/34876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] Postgres conn type should normalize to postgresql (instead of the inverse) [airflow]

2023-10-11 Thread via GitHub
Taragolis commented on issue #34876: URL: https://github.com/apache/airflow/issues/34876#issuecomment-1758725309 Airflow Connection URL it is representation (serialisation) of Airflow Connection, not psycopg2 URL nor SQLAlchemy URL > Call Connection.get_uri() on a connection with conn

Re: [PR] Add SparkKubernetesOperator crd implementation [airflow]

2023-10-11 Thread via GitHub
github-actions[bot] commented on PR #22253: URL: https://github.com/apache/airflow/pull/22253#issuecomment-1758720195 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355902071 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -461,13 +436,8 @@ def get_task_instances_batch(session: Session = NEW_SESSION) -> APIResponse:

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355894015 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -(permissions.ACTI

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355894015 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -(permissions.ACTI

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
potiuk commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355894015 ## airflow/api_connexion/endpoints/task_instance_endpoint.py: ## @@ -61,13 +61,8 @@ T = TypeVar("T") -@security.requires_access( -[ -(permissions.ACTI

Re: [I] Git Sync 4 support [airflow]

2023-10-11 Thread via GitHub
bakome commented on issue #32335: URL: https://github.com/apache/airflow/issues/32335#issuecomment-1758683759 I was able to implement git-sync v4 but faced interesting problem there. I also open a PR on git-sync because I think airflow instances can be heavily impacted by this weird behavio

[I] Postgres conn type should normalize to postgresql (instead of the inverse) [airflow]

2023-10-11 Thread via GitHub
dhuang opened a new issue, #34876: URL: https://github.com/apache/airflow/issues/34876 ### Apache Airflow version 2.7.1 ### What happened Currently Airflow normalizes conn types of `postgresql` to `postgres`, which means calling`Connection.get_uri()` always gets a `postg

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-11 Thread via GitHub
ferruzzi merged PR #34784: URL: https://github.com/apache/airflow/pull/34784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[airflow] branch main updated (27671fa53d -> 84a3daed86)

2023-10-11 Thread ferruzzi
This is an automated email from the ASF dual-hosted git repository. ferruzzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 27671fa53d Add session configuration to deployment manager's responsibilities. (#34866) add 84a3daed86 Implemen

Re: [I] Invalid stat name using opentelemetry [airflow]

2023-10-11 Thread via GitHub
ferruzzi commented on issue #34845: URL: https://github.com/apache/airflow/issues/34845#issuecomment-1758654459 I just double-checked and this fix should be in Airflow 2.7.2 which is currently being voted on and should be out Very Soon :tm: ((next week I think, but don't hold me to that))

Re: [PR] Glue DataBrew operator [airflow]

2023-10-11 Thread via GitHub
Taragolis commented on code in PR #34807: URL: https://github.com/apache/airflow/pull/34807#discussion_r1355852578 ## airflow/providers/amazon/aws/waiters/databrew.json: ## @@ -0,0 +1,54 @@ +{ +"version": 2, +"waiters": { +"job_complete": { +"operatio

Re: [PR] Glue DataBrew operator [airflow]

2023-10-11 Thread via GitHub
Taragolis commented on code in PR #34807: URL: https://github.com/apache/airflow/pull/34807#discussion_r1355837567 ## airflow/providers/amazon/aws/waiters/databrew.json: ## @@ -0,0 +1,54 @@ +{ +"version": 2, +"waiters": { +"job_complete": { +"operatio

Re: [I] Status of testing of Apache Airflow 2.7.2rc1 [airflow]

2023-10-11 Thread via GitHub
jens-scheffler-bosch commented on issue #34830: URL: https://github.com/apache/airflow/issues/34830#issuecomment-1758623517 Checked for #34248 and can confirm that (1) code is correctly merged and (2) fix is working! Thanks for making the release! -- This is an automated message from the

Re: [PR] Added "overrides" parameter for CloudRunExecuteJobOperator [airflow]

2023-10-11 Thread via GitHub
Taragolis commented on code in PR #34874: URL: https://github.com/apache/airflow/pull/34874#discussion_r1355831314 ## airflow/providers/google/cloud/hooks/cloud_run.py: ## @@ -113,9 +113,11 @@ def update_job( @GoogleBaseHook.fallback_to_default_project_id def execute

[PR] Added "overrides" parameter for CloudRunExecuteJobOperator [airflow]

2023-10-11 Thread via GitHub
ChloeSheasby opened a new pull request, #34874: URL: https://github.com/apache/airflow/pull/34874 --- This PR aims to add the functionality of the "overrides" variable for the CloudRunExecuteJobOperator as provided by the google.cloud.run.v2 package. - https://cloud.google.com/ru

Re: [PR] Added "overrides" parameter for CloudRunExecuteJobOperator [airflow]

2023-10-11 Thread via GitHub
boring-cyborg[bot] commented on PR #34874: URL: https://github.com/apache/airflow/pull/34874#issuecomment-1758609418 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

Re: [I] Dataset view shows nothing when DAGs inlet and outlet are the same [airflow]

2023-10-11 Thread via GitHub
jens-scheffler-bosch commented on issue #34867: URL: https://github.com/apache/airflow/issues/34867#issuecomment-1758601396 Okay, having a circular reference in DAG triggers might be generating effects unforseen. Can you tell about the use case? Otherwise I rather believe that we might need

Re: [I] Logs using @task.virtualenv in Kubernetes [airflow]

2023-10-11 Thread via GitHub
jens-scheffler-bosch commented on issue #34865: URL: https://github.com/apache/airflow/issues/34865#issuecomment-1758596926 Just to double-check, are you able to make a regression on Airflow 2.7.1? -- This is an automated message from the Apache Git Service. To respond to the message, plea

Re: [I] Add on_kill method to DataprocInstantiateWorkflowTemplateOperator [airflow]

2023-10-11 Thread via GitHub
michalsosn commented on issue #10385: URL: https://github.com/apache/airflow/issues/10385#issuecomment-1758577018 ### operators in dataproc module ``` not killable, don't start a process: class DataprocCreateClusterOperator(GoogleCloudBaseOperator) class DataprocScaleC

Re: [PR] Corrected the user creation command CONTRIBUTORS_QUICK_START.rst [airflow]

2023-10-11 Thread via GitHub
jens-scheffler-bosch commented on PR #34842: URL: https://github.com/apache/airflow/pull/34842#issuecomment-1758556457 FYI Pipeline fails but unfortunately (at least for me) the logs do not show details, error that I can see in raw logs are: 2023-10-11T15:03:03.3370615Z Remove trailin

Re: [I] Scheduler crashes with OverflowError: date value out of range when retries are added to ShortCircuitOperator [airflow]

2023-10-11 Thread via GitHub
Taragolis commented on issue #34869: URL: https://github.com/apache/airflow/issues/34869#issuecomment-1758539398 There are only option to overflow date it try to get it greater than greater `-12-31T23:59:59` in this method: - So it possible if delay (provided or backoff) greater than

Re: [I] Status of testing of Apache Airflow 2.7.2rc1 [airflow]

2023-10-11 Thread via GitHub
ephraimbuddy commented on issue #34830: URL: https://github.com/apache/airflow/issues/34830#issuecomment-1758528533 > > > #34072 and #34136 work as expected. > > > I also tested a few other PRs: > > > > > > * Works as expected: [Add missing audit logs for default flask action: add,

Re: [PR] Glue DataBrew operator [airflow]

2023-10-11 Thread via GitHub
vincbeck commented on code in PR #34807: URL: https://github.com/apache/airflow/pull/34807#discussion_r1355765277 ## airflow/providers/amazon/aws/operators/glue.py: ## @@ -230,3 +230,99 @@ def on_kill(self): ) if not response["SuccessfulSubmissions"]:

[airflow] branch main updated: Add session configuration to deployment manager's responsibilities. (#34866)

2023-10-11 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 27671fa53d Add session configuration to depl

Re: [PR] Add session configuration to deployment manager's responsibilities. [airflow]

2023-10-11 Thread via GitHub
jedcunningham merged PR #34866: URL: https://github.com/apache/airflow/pull/34866 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

Re: [I] Status of testing of Apache Airflow 2.7.2rc1 [airflow]

2023-10-11 Thread via GitHub
hussein-awala commented on issue #34830: URL: https://github.com/apache/airflow/issues/34830#issuecomment-1758490902 I tested my changes, they are present in the RC. I tested only the bug fixes, and all looks good, except #34120 as discussed above. -- This is an automated message from the

[airflow] branch main updated: Doc: Add info on getting variables and config in custom secrets backend (#34834)

2023-10-11 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 8a85fb4161 Doc: Add info on getting variables an

Re: [PR] Doc: Add info on getting variables and config in custom secrets backend [airflow]

2023-10-11 Thread via GitHub
Taragolis merged PR #34834: URL: https://github.com/apache/airflow/pull/34834 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [I] Status of testing of Apache Airflow 2.7.2rc1 [airflow]

2023-10-11 Thread via GitHub
hussein-awala commented on issue #34830: URL: https://github.com/apache/airflow/issues/34830#issuecomment-1758489492 > > #34072 and #34136 work as expected. > > I also tested a few other PRs: > > > > * Works as expected: [Add missing audit logs for default flask action: add, edit a

Re: [I] Support split_statements AthenaOperator [Amazon provider] [airflow]

2023-10-11 Thread via GitHub
flolas commented on issue #34583: URL: https://github.com/apache/airflow/issues/34583#issuecomment-1758489883 @vincbeck yes, i hope to send a PR in the weekend. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[PR] Docstring correction for `SlackAPIOperator` [airflow]

2023-10-11 Thread via GitHub
Taragolis opened a new pull request, #34871: URL: https://github.com/apache/airflow/pull/34871 Deprecated `token` parameter was removed in 8.0.0 of provider, the only available option for provide token is use Slack API Connection --- **^ Add meaningful descripti

[PR] Add general links about TCP keepalive into AWS Lambda docs [airflow]

2023-10-11 Thread via GitHub
Taragolis opened a new pull request, #34870: URL: https://github.com/apache/airflow/pull/34870 This info might want to be useful for someone who intent to invoke AWS Lambda synchronously 🙄 with timeout more than 5-6 minutes cc: @shubham22 --- **^ Add mea

Re: [I] Scheduler crashes with OverflowError: date value out of range when retries are added to ShortCircuitOperator [airflow]

2023-10-11 Thread via GitHub
sahar1454 commented on issue #34869: URL: https://github.com/apache/airflow/issues/34869#issuecomment-1758460080 @Taragolis I don't believe the fix you linked is related to the issue I reported. That one solves the problem for this issue: https://github.com/apache/airflow/issues/28171 Bu

Re: [PR] Glue DataBrew operator [airflow]

2023-10-11 Thread via GitHub
ellisms commented on code in PR #34807: URL: https://github.com/apache/airflow/pull/34807#discussion_r1355681377 ## airflow/providers/amazon/aws/operators/glue.py: ## @@ -230,3 +230,99 @@ def on_kill(self): ) if not response["SuccessfulSubmissions"]:

Re: [PR] Glue DataBrew operator [airflow]

2023-10-11 Thread via GitHub
ellisms commented on code in PR #34807: URL: https://github.com/apache/airflow/pull/34807#discussion_r1355680039 ## airflow/providers/amazon/aws/operators/glue.py: ## @@ -230,3 +230,99 @@ def on_kill(self): ) if not response["SuccessfulSubmissions"]:

Re: [PR] Glue DataBrew operator [airflow]

2023-10-11 Thread via GitHub
ferruzzi commented on code in PR #34807: URL: https://github.com/apache/airflow/pull/34807#discussion_r1355676198 ## tests/providers/amazon/aws/waiters/test_custom_waiters.py: ## @@ -424,3 +425,42 @@ def test_steps_failed(self, mock_list_steps): StepIds=[self.ST

Re: [PR] Add helpers to better handle session in tests [airflow]

2023-10-11 Thread via GitHub
uranusjr closed pull request #33307: Add helpers to better handle session in tests URL: https://github.com/apache/airflow/pull/33307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] Make it possible to change the default cron timetable [airflow]

2023-10-11 Thread via GitHub
uranusjr commented on PR #34851: URL: https://github.com/apache/airflow/pull/34851#issuecomment-1758284518 Because conf is not called conf in this file https://github.com/apache/airflow/blob/03a305f6600b822d4f4ab74ed5ae8c0f8905fbc2/airflow/models/dag.py#L80 -- This is an automated

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355579389 ## airflow/www/extensions/init_jinja_globals.py: ## @@ -69,10 +70,13 @@ def prepare_jinja_globals(): "git_version": git_version, "k8s_or_k8

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355579727 ## airflow/api_connexion/endpoints/task_endpoint.py: ## @@ -22,21 +22,16 @@ from airflow.api_connexion import security from airflow.api_connexion.exceptions import

Re: [I] Scheduler crashes with OverflowError: date value out of range when retries are added to ShortCircuitOperator [airflow]

2023-10-11 Thread via GitHub
Taragolis commented on issue #34869: URL: https://github.com/apache/airflow/issues/34869#issuecomment-1758259679 Should be fixed as part of Airflow 2.6.0 - https://github.com/apache/airflow/pull/28172 -- This is an automated message from the Apache Git Service. To respond to the message

Re: [I] Scheduler crashes with OverflowError: date value out of range when retries are added to ShortCircuitOperator [airflow]

2023-10-11 Thread via GitHub
Taragolis closed issue #34869: Scheduler crashes with OverflowError: date value out of range when retries are added to ShortCircuitOperator URL: https://github.com/apache/airflow/issues/34869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
vincbeck commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r1355567508 ## airflow/api_connexion/endpoints/task_endpoint.py: ## @@ -22,21 +22,16 @@ from airflow.api_connexion import security from airflow.api_connexion.exceptions import

Re: [I] Invalid stat name using opentelemetry [airflow]

2023-10-11 Thread via GitHub
ferruzzi commented on issue #34845: URL: https://github.com/apache/airflow/issues/34845#issuecomment-1758253375 I know this isn't really an answer, but the root cause id that when you combine a long `dag_id` and a long `task_id`, the total length of the metric name exceeds OTel's max name l

Re: [PR] Use auth manager `is_authorized_` APIs to check user permissions in Rest API [airflow]

2023-10-11 Thread via GitHub
uranusjr commented on code in PR #34317: URL: https://github.com/apache/airflow/pull/34317#discussion_r138587 ## airflow/api_connexion/endpoints/task_endpoint.py: ## @@ -22,21 +22,16 @@ from airflow.api_connexion import security from airflow.api_connexion.exceptions import

Re: [I] Invalid stat name using opentelemetry [airflow]

2023-10-11 Thread via GitHub
ferruzzi closed issue #34845: Invalid stat name using opentelemetry URL: https://github.com/apache/airflow/issues/34845 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsub

Re: [PR] Make it possible to change the default cron timetable [airflow]

2023-10-11 Thread via GitHub
uranusjr commented on code in PR #34851: URL: https://github.com/apache/airflow/pull/34851#discussion_r1355543632 ## airflow/config_templates/config.yml: ## @@ -2237,6 +2237,16 @@ scheduler: type: string example: ~ default: "15" +default_cron_timetable:

Re: [I] Invalid stat name using opentelemetry [airflow]

2023-10-11 Thread via GitHub
ferruzzi commented on issue #34845: URL: https://github.com/apache/airflow/issues/34845#issuecomment-1758214925 > should we add them to the exemption list? Must have been some awkward timing, adding new non-compliant names should have been prevented by the unit tests Yeah, I guess

Re: [PR] Upgrade watchtower to 3.0.1 (#25019) [airflow]

2023-10-11 Thread via GitHub
o-nikolas commented on PR #34747: URL: https://github.com/apache/airflow/pull/34747#issuecomment-1758208475 Nice work @cBiscuitSurprise, I'm glad to see this one finally closed out. Smart idea to make the new behaviour configurable and off by default, it limits the blast radius of doing thi

Re: [I] Add a command line option to start a triggerer process with breeze [airflow]

2023-10-11 Thread via GitHub
o-nikolas commented on issue #31826: URL: https://github.com/apache/airflow/issues/31826#issuecomment-1758161057 Fantastic! Thanks @Bowrna, I've assigned the issue to you :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Scheduler crashes with OverflowError: date value out of range when retries are added to ShortCircuitOperator [airflow]

2023-10-11 Thread via GitHub
boring-cyborg[bot] commented on issue #34869: URL: https://github.com/apache/airflow/issues/34869#issuecomment-1758153133 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] Scheduler crashes with OverflowError: date value out of range when retries are added to ShortCircuitOperator [airflow]

2023-10-11 Thread via GitHub
sahar1454 opened a new issue, #34869: URL: https://github.com/apache/airflow/issues/34869 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened In airflow version `2.5.1`, adding retries on `ShortCircuitOperator` causes the Scheduler

Re: [I] Grid View: Task order rearranges constantly with auto-refresh on [airflow]

2023-10-11 Thread via GitHub
tagoad commented on issue #23542: URL: https://github.com/apache/airflow/issues/23542#issuecomment-1758100903 It looks like this issue has been addressed, @rajrao you should update to the latest Airflow version #27523 -- This is an automated message from the Apache Git Service. To res

Re: [I] Grid View: Task order rearranges constantly with auto-refresh on [airflow]

2023-10-11 Thread via GitHub
tagoad closed issue #23542: Grid View: Task order rearranges constantly with auto-refresh on URL: https://github.com/apache/airflow/issues/23542 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[PR] Enhancement: Added Code from Image as Text [airflow]

2023-10-11 Thread via GitHub
theaadya opened a new pull request, #34868: URL: https://github.com/apache/airflow/pull/34868 I've written out the code that was previously only available as an image in `CONTRIBUTORS_QUICK_START_VSCODE.rst`, making it more convenient for users as they don't have to manually transcribe it f

Re: [I] Dataset view shows nothing when DAGs inlet and outlet are the same [airflow]

2023-10-11 Thread via GitHub
boring-cyborg[bot] commented on issue #34867: URL: https://github.com/apache/airflow/issues/34867#issuecomment-1758071871 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

  1   2   >