Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
amoghrajesh commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1778563839 > What if we didn't have "mode", but a "one-time" flag instead. I doubt we will have another mode down the road and sidesteps the "normal/daemon/standard" naming problem.

Re: [PR] AWS ECS Executor [airflow]

2023-10-24 Thread via GitHub
o-nikolas commented on PR #34381: URL: https://github.com/apache/airflow/pull/34381#issuecomment-1778467807 @jedcunningham @Taragolis @eladkal, We've address the feedback and gotten the build green once again. If you folks have time to have another look and perhaps approve if all

Re: [PR] Disable celery `task_acks_late` [airflow]

2023-10-24 Thread via GitHub
thesuperzapper commented on PR #31829: URL: https://github.com/apache/airflow/pull/31829#issuecomment-1778329665 Bumping for bot. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] fix(helm): set worker safeToEvict properly [airflow]

2023-10-24 Thread via GitHub
hakuno commented on PR #35130: URL: https://github.com/apache/airflow/pull/35130#issuecomment-1778325712 Ps.¹ Other Airflow components also has `safeToEvict` values. Ps.² At another moment, I see I can contribute with other PRs. Because `workers.podAnnotations` and

Re: [PR] fix(helm): set worker safeToEvict properly [airflow]

2023-10-24 Thread via GitHub
hakuno commented on PR #35130: URL: https://github.com/apache/airflow/pull/35130#issuecomment-1778309120 @jedcunningham @hussein-awala @potiuk I got it. ``` pytest helm_tests/airflow_core/test_worker.py::TestWorker::test_safetoevict_annotations

Re: [PR] Update GCS hook to get crc32c hash for CMEK-protected objects [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on PR #34982: URL: https://github.com/apache/airflow/pull/34982#issuecomment-1778304223 Are you worry about that you can't test internal static method `_prepare_sync_plan`? I don't see any problem here because it is internal part of Google Provider so you could also

Re: [PR] Yandex dataproc deduce default service account [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on code in PR #35059: URL: https://github.com/apache/airflow/pull/35059#discussion_r1370969011 ## airflow/providers/yandex/provider.yaml: ## @@ -70,3 +70,15 @@ hooks: connection-types: - hook-class-name:

Re: [PR] Add operator to invoke Azure-Synapse pipeline [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on code in PR #35091: URL: https://github.com/apache/airflow/pull/35091#discussion_r1370965788 ## CONTRIBUTING.rst: ## Review Comment: Seems like you use Windows. As I could see there is LF (common posix line ending character) changes to CRLF

Re: [PR] Fix service account impersonation with computeEngineSSHHook (google provider) [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on PR #35136: URL: https://github.com/apache/airflow/pull/35136#issuecomment-1778263226 The simplest way have a look how it done in other Google Provider test, for example

Re: [PR] Disable celery `task_acks_late` [airflow]

2023-10-24 Thread via GitHub
github-actions[bot] commented on PR #31829: URL: https://github.com/apache/airflow/pull/31829#issuecomment-1778260584 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] Replace print by logger.info in db_cleanup util [airflow]

2023-10-24 Thread via GitHub
github-actions[bot] commented on PR #33862: URL: https://github.com/apache/airflow/pull/33862#issuecomment-1778260495 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [I] psycopg2 deadlock detected after task run completed [airflow]

2023-10-24 Thread via GitHub
github-actions[bot] closed issue #34357: psycopg2 deadlock detected after task run completed URL: https://github.com/apache/airflow/issues/34357 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Add bandit to pre-commit to detect common security issues [airflow]

2023-10-24 Thread via GitHub
github-actions[bot] commented on PR #34247: URL: https://github.com/apache/airflow/pull/34247#issuecomment-1778260448 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [I] psycopg2 deadlock detected after task run completed [airflow]

2023-10-24 Thread via GitHub
github-actions[bot] commented on issue #34357: URL: https://github.com/apache/airflow/issues/34357#issuecomment-1778260414 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Replace blocking IO with async IO in AsyncKubernetesHook [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on PR #35162: URL: https://github.com/apache/airflow/pull/35162#issuecomment-1778249170 > I'm not sure if it's caused by my change? Any help is appreciated! I guess this related to changes in https://github.com/apache/airflow/pull/35099 For some reason

Re: [PR] fix: set dry_run to be optional. [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on code in PR #35167: URL: https://github.com/apache/airflow/pull/35167#discussion_r1370941702 ## tests/api_connexion/schemas/test_task_instance_schema.py: ## @@ -236,6 +236,22 @@ def test_success(self): } assert expected_result == result

Re: [PR] Added "overrides" parameter for CloudRunExecuteJobOperator [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on code in PR #34874: URL: https://github.com/apache/airflow/pull/34874#discussion_r1370938336 ## airflow/providers/google/cloud/operators/cloud_run.py: ## @@ -270,6 +271,7 @@ def __init__( project_id: str, region: str, job_name:

Re: [PR] Update GCS hook to get crc32c hash for CMEK-protected objects [airflow]

2023-10-24 Thread via GitHub
dmedora commented on PR #34982: URL: https://github.com/apache/airflow/pull/34982#issuecomment-1778213394 Thanks, still figuring out how to write the test correctly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Added "overrides" parameter for CloudRunExecuteJobOperator [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on PR #34874: URL: https://github.com/apache/airflow/pull/34874#issuecomment-1778212684 Need to fix remaining [Static Checks](https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#id5) For quick fix would be enough to install pre-commit and run `ruff`

Re: [PR] Replace blocking IO with async IO in AsyncKubernetesHook [airflow]

2023-10-24 Thread via GitHub
functicons commented on PR #35162: URL: https://github.com/apache/airflow/pull/35162#issuecomment-1778201782 Some checks failed with the following error: ``` RuntimeError: By default one of Airflow's dependencies installs a GPL dependency (unidecode). To avoid this dependency set

Re: [PR] Update GCS hook to get crc32c hash for CMEK-protected objects [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on PR #34982: URL: https://github.com/apache/airflow/pull/34982#issuecomment-1778200695 Need to fix [static checks](https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#id5) and failed test -- This is an automated message from the Apache Git Service. To

[airflow] branch main updated: Consolidate stacklevel in ssh operator warning (#35151)

2023-10-24 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 543db7004e Consolidate stacklevel in ssh

Re: [I] Consolidate stacklevel in ssh operator warning [airflow]

2023-10-24 Thread via GitHub
Taragolis closed issue #34421: Consolidate stacklevel in ssh operator warning URL: https://github.com/apache/airflow/issues/34421 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Consolidate stacklevel in ssh operator warning [airflow]

2023-10-24 Thread via GitHub
Taragolis merged PR #35151: URL: https://github.com/apache/airflow/pull/35151 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: feat: add hostAliases to pod spec in KubernetesPodOperator (#35063)

2023-10-24 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4767f48a3b feat: add hostAliases to pod spec in

Re: [PR] feat: add hostAliases to pod spec in KubernetesPodOperator [airflow]

2023-10-24 Thread via GitHub
Taragolis merged PR #35063: URL: https://github.com/apache/airflow/pull/35063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] feat: add hostAliases to pod spec in KubernetesPodOperator [airflow]

2023-10-24 Thread via GitHub
boring-cyborg[bot] commented on PR #35063: URL: https://github.com/apache/airflow/pull/35063#issuecomment-1778195994 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional

Re: [PR] openlineage, aws: Add OpenLineage support for AthenaOperator. [airflow]

2023-10-24 Thread via GitHub
JDarDagran commented on code in PR #35090: URL: https://github.com/apache/airflow/pull/35090#discussion_r1370914455 ## airflow/providers/amazon/aws/operators/athena.py: ## @@ -163,3 +167,132 @@ def on_kill(self) -> None: "Polling Athena for query with

Re: [PR] openlineage, aws: Add OpenLineage support for AthenaOperator. [airflow]

2023-10-24 Thread via GitHub
JDarDagran commented on code in PR #35090: URL: https://github.com/apache/airflow/pull/35090#discussion_r1370913555 ## airflow/providers/amazon/aws/operators/athena.py: ## @@ -163,3 +167,132 @@ def on_kill(self) -> None: "Polling Athena for query with

Re: [PR] Fix error handling in EksCreateClusterOperator in deferrable mode [airflow]

2023-10-24 Thread via GitHub
syedahsn commented on PR #34966: URL: https://github.com/apache/airflow/pull/34966#issuecomment-1778192202 Looking at this, it looks like a few things were missing. 1. The first is that the `EksCreateClusterTrigger` does not set a status as `failed`, which means

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-24 Thread via GitHub
dacort commented on code in PR #34225: URL: https://github.com/apache/airflow/pull/34225#discussion_r1370895671 ## airflow/providers/amazon/aws/links/emr.py: ## @@ -66,3 +81,100 @@ def get_log_uri( return None log_uri = S3Hook.parse_s3_url(cluster_info["LogUri"])

Re: [PR] Prevent assignment of non JSON serializable values to DagRun.conf dict [airflow]

2023-10-24 Thread via GitHub
PashkPashk commented on PR #35096: URL: https://github.com/apache/airflow/pull/35096#issuecomment-1778140569 > This looks good. Thanks for the contribution. > > I was thinging (but have no strong opinion) as whether it makes sense to extract this to a utility, but keeping it here

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-24 Thread via GitHub
Joffreybvn commented on code in PR #34964: URL: https://github.com/apache/airflow/pull/34964#discussion_r1370818602 ## docs/apache-airflow/administration-and-deployment/logging-monitoring/advanced-logging-configuration.rst: ## @@ -89,3 +94,63 @@ See :doc:`../modules_management`

Re: [I] Confusing log for long running tasks: "dependency 'Task Instance Not Running' FAILED: Task is in the running state" [airflow]

2023-10-24 Thread via GitHub
richard-iovanisci commented on issue #16163: URL: https://github.com/apache/airflow/issues/16163#issuecomment-1778045512 I am seeing this as well with Airflow v2.5.1 (not using kubernetes) w/Celery Executor. We also utilize s3 remote logging, and observe regular log for the first hour,

Re: [PR] Replace blocking IO with async IO in AsyncKubernetesHook [airflow]

2023-10-24 Thread via GitHub
functicons commented on PR #35162: URL: https://github.com/apache/airflow/pull/35162#issuecomment-1778027470 > I agree that this needs to be improved, I tried to implement a similar solution in the past but I had a problem with an older version of aiofiles, could you add a unit test fo

Re: [I] logging/remote_logging - option to disable creation of cloudwatch log groups [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on issue #35164: URL: https://github.com/apache/airflow/issues/35164#issuecomment-1778018702 > This is unnecessary if the log group was already created. Yeah, I think better to create configuration option in AWS Provider which could control this behaviour.

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-24 Thread via GitHub
Joffreybvn commented on code in PR #34964: URL: https://github.com/apache/airflow/pull/34964#discussion_r1370793124 ## docs/apache-airflow/administration-and-deployment/logging-monitoring/advanced-logging-configuration.rst: ## @@ -89,3 +94,63 @@ See :doc:`../modules_management`

Re: [PR] Chart: configurable mountPath for DAGs volume [airflow]

2023-10-24 Thread via GitHub
jedcunningham commented on code in PR #35083: URL: https://github.com/apache/airflow/pull/35083#discussion_r1370790531 ## chart/values.schema.json: ## @@ -6931,6 +6931,14 @@ "x-docsSection": "Airflow", "additionalProperties": false,

Re: [PR] Disallow create/extend thick-wrapped hooks in Amazon Provider [airflow]

2023-10-24 Thread via GitHub
o-nikolas commented on PR #35163: URL: https://github.com/apache/airflow/pull/35163#issuecomment-1778007125 > I have a look in detail on listed hook and some of them created on Airflow 1.10.0, some other migrated from old `boto` to `boto3` in the period of airflow 1.8 - 1.10 (s3/dynamodb)

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-24 Thread via GitHub
Joffreybvn commented on code in PR #34964: URL: https://github.com/apache/airflow/pull/34964#discussion_r1370789066 ## tests/providers/docker/hooks/test_docker.py: ## @@ -36,7 +36,7 @@ TEST_CONN = {"host": "some.docker.registry.com", "login": "some_user", "password":

Re: [PR] Add LiteralValue wrapper rendered without templating [airflow]

2023-10-24 Thread via GitHub
michalsosn commented on code in PR #35017: URL: https://github.com/apache/airflow/pull/35017#discussion_r1370785630 ## tests/template/test_templater.py: ## @@ -60,3 +60,23 @@ def test_render_template(self): templater.template_ext = [".txt"] rendered_content =

Re: [PR] Disallow create/extend thick-wrapped hooks in Amazon Provider [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on PR #35163: URL: https://github.com/apache/airflow/pull/35163#issuecomment-1777990166 I have a look in detail on listed hook and some of them created on Airflow 1.10.0, some other migrated from old `boto` to `boto3` in the period of airflow 1.8 - 1.10 (s3/dynamodb)

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-24 Thread via GitHub
Joffreybvn commented on code in PR #34964: URL: https://github.com/apache/airflow/pull/34964#discussion_r1370774789 ## airflow/models/baseoperator.py: ## @@ -1218,15 +1225,17 @@ def __deepcopy__(self, memo): result.__instantiated = self.__instantiated return

Re: [PR] Add pagination to `SimpleHttpOperator` and make it more modular [airflow]

2023-10-24 Thread via GitHub
Joffreybvn commented on PR #34669: URL: https://github.com/apache/airflow/pull/34669#issuecomment-1777959720 I implemented back everything into the SimpleHttpOperator, and changed the title. This PR is not about an extra operator anymore, but extending the SimpleHttpOperator. Test should

Re: [PR] fix: set dry_run to be optional. [airflow]

2023-10-24 Thread via GitHub
boring-cyborg[bot] commented on PR #35167: URL: https://github.com/apache/airflow/pull/35167#issuecomment-1777934235 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[PR] fix: set dry_run to be optional. [airflow]

2023-10-24 Thread via GitHub
austinau opened a new pull request, #35167: URL: https://github.com/apache/airflow/pull/35167 Fixes #35166. Duplicates the solution in #34568. --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Add pagination to `SimpleHttpOperator` and make it more modular [airflow]

2023-10-24 Thread via GitHub
Joffreybvn commented on code in PR #34669: URL: https://github.com/apache/airflow/pull/34669#discussion_r1370745878 ## docs/apache-airflow-providers-http/operators.rst: ## @@ -118,3 +118,24 @@ Here we pass form data to a ``POST`` operation which is equal to a usual form su

Re: [I] dry_run not optional in api for set task instances state [airflow]

2023-10-24 Thread via GitHub
boring-cyborg[bot] commented on issue #35166: URL: https://github.com/apache/airflow/issues/35166#issuecomment-1777914542 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] dry_run not optional in api for set task instances state [airflow]

2023-10-24 Thread via GitHub
austinau opened a new issue, #35166: URL: https://github.com/apache/airflow/issues/35166 ### Apache Airflow version 2.7.2 ### What happened Sent request without dry_run parameter to dags/{dag_id}/updateTaskInstancesState and got a 500 error. ### What you think

Re: [PR] Disallow create/extend thick-wrapped hooks in Amazon Provider [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on PR #35163: URL: https://github.com/apache/airflow/pull/35163#issuecomment-1777884728 If you need create boto3 client then is already exists `AwsBaseHook(aws_conn_id="foo", client_type="one-of-355-aws-service").conn` maybe this behaviour not well documented ---

Re: [PR] AIP-58: Add Airflow ObjectStore (AFS) [airflow]

2023-10-24 Thread via GitHub
jens-scheffler-bosch commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1777859553 > I can consider adding it to the higher level module and export it _also_ from there so you could do `from airflow.io import ObjectStoragePath`. Note that we got rid of

Re: [PR] Prevent assignment of non JSON serializable values to DagRun.conf dict [airflow]

2023-10-24 Thread via GitHub
PashkPashk commented on code in PR #35096: URL: https://github.com/apache/airflow/pull/35096#discussion_r1370693317 ## airflow/models/dagrun.py: ## @@ -126,7 +150,7 @@ class DagRun(Base, LoggingMixin): creating_job_id = Column(Integer) external_trigger =

Re: [PR] Prevent assignment of non JSON serializable values to DagRun.conf dict [airflow]

2023-10-24 Thread via GitHub
PashkPashk commented on code in PR #35096: URL: https://github.com/apache/airflow/pull/35096#discussion_r1370692991 ## airflow/models/dagrun.py: ## @@ -210,7 +234,7 @@ def __init__( execution_date: datetime | None = None, start_date: datetime | None = None,

Re: [PR] Added example for defaults in conn.extras [airflow]

2023-10-24 Thread via GitHub
boring-cyborg[bot] commented on PR #35165: URL: https://github.com/apache/airflow/pull/35165#issuecomment-1777829057 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[PR] Added example for defaults in conn.extras [airflow]

2023-10-24 Thread via GitHub
vedrankolka opened a new pull request, #35165: URL: https://github.com/apache/airflow/pull/35165 Added example for providing a default value when reading from extras from connections using `extra_dejson.get`. --- **^ Add meaningful description above**

Re: [PR] Disallow create/extend thick-wrapped hooks in Amazon Provider [airflow]

2023-10-24 Thread via GitHub
o-nikolas commented on code in PR #35163: URL: https://github.com/apache/airflow/pull/35163#discussion_r1370676442 ## tests/providers/amazon/aws/hooks/test_hooks_signature.py: ## Review Comment: I think keeping the constructor args clean could be nice, but I'm not sure we

Re: [PR] Feature: Implement pagination in new `PaginatedHttpOperator` + refactor `SimpleHttpOperator` [airflow]

2023-10-24 Thread via GitHub
jens-scheffler-bosch commented on code in PR #34669: URL: https://github.com/apache/airflow/pull/34669#discussion_r1370673376 ## docs/apache-airflow-providers-http/operators.rst: ## @@ -118,3 +118,24 @@ Here we pass form data to a ``POST`` operation which is equal to a usual

Re: [I] logging/remote_logging - option to disable creation of cloudwatch log groups [airflow]

2023-10-24 Thread via GitHub
boring-cyborg[bot] commented on issue #35164: URL: https://github.com/apache/airflow/issues/35164#issuecomment-1777810733 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] logging/remote_logging - option to disable creation of cloudwatch log groups [airflow]

2023-10-24 Thread via GitHub
nwsparks opened a new issue, #35164: URL: https://github.com/apache/airflow/issues/35164 ### Description Please add an option to disable attempting to create the cloudwatch log group when remote logging is enabled or just disable the behavior and require the the group already

Re: [PR] Disallow create/extend thick-wrapped hooks in Amazon Provider [airflow]

2023-10-24 Thread via GitHub
vincbeck commented on PR #35163: URL: https://github.com/apache/airflow/pull/35163#issuecomment-1777804841 Interesting! I am divided on this one to be honest. 1. I like the fact to try to normalize all hooks within the Amazon provider package. I agree that contributors, who might

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
potiuk commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-157089 > What if we didn't have "mode", but a "one-time" flag instead. I doubt we will have another mode down the road and sidesteps the "normal/daemon/standard" naming problem. :heart:

Re: [PR] Disallow create/extend thick-wrapped hooks in Amazon Provider [airflow]

2023-10-24 Thread via GitHub
ferruzzi commented on PR #35163: URL: https://github.com/apache/airflow/pull/35163#issuecomment-151491 I would argue the exact opposite. If a hook is just a thin wrapper around boto, then it might as well be a taskflow boto call. I'd argue that if we were going to make a restriction,

Re: [I] BigQueryValueCheckOperator does not accept non-default project_id, which lead to jobs with impersonation_chain to fail. [airflow]

2023-10-24 Thread via GitHub
eladkal closed issue #32870: BigQueryValueCheckOperator does not accept non-default project_id, which lead to jobs with impersonation_chain to fail. URL: https://github.com/apache/airflow/issues/32870 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
jedcunningham commented on code in PR #35146: URL: https://github.com/apache/airflow/pull/35146#discussion_r1370605608 ## airflow/security/kerberos.py: ## @@ -176,18 +189,25 @@ def detect_conf_var() -> bool: return b"X-CACHECONF:" in file.read() -def run(principal:

[PR] Disallow create/extend thick-wrapped hooks in Amazon Provider [airflow]

2023-10-24 Thread via GitHub
Taragolis opened a new pull request, #35163: URL: https://github.com/apache/airflow/pull/35163 Add simple test which are validate that hooks based on AwsGenericHook / AwsBaseHook allow only common AWS Hooks parameters, such as: `aws_conn_id`, `region_name`, `verify`,

[PR] Replace blocking IO with async IO in AsyncKubernetesHook [airflow]

2023-10-24 Thread via GitHub
functicons opened a new pull request, #35162: URL: https://github.com/apache/airflow/pull/35162 Currently there are blocking IO operations in AsyncKubernetesHook, which are causing triggerer failures or high CPU usage. This change replaces them with async IO. ---

Re: [I] Dag Dependency page not showing anything [airflow]

2023-10-24 Thread via GitHub
leozitor commented on issue #16610: URL: https://github.com/apache/airflow/issues/16610#issuecomment-1777677946 Hey guys is anyone facing issues regarding this, I notices in my code that if i have a TriggerDagRunOperator inside task_group like this, it shows the dag dependency on viewer

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
potiuk commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1777669677 That's why best to start with main always. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
potiuk commented on code in PR #35146: URL: https://github.com/apache/airflow/pull/35146#discussion_r1370550640 ## airflow/cli/cli_config.py: ## @@ -561,6 +561,7 @@ def string_lower_type(val): # kerberos ARG_PRINCIPAL = Arg(("principal",), help="kerberos principal",

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
amoghrajesh commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1777666123 Oh the main branch seems to have diverged quite a bit. I will rebase it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
amoghrajesh commented on code in PR #35146: URL: https://github.com/apache/airflow/pull/35146#discussion_r1370548258 ## airflow/cli/cli_config.py: ## @@ -561,6 +561,7 @@ def string_lower_type(val): # kerberos ARG_PRINCIPAL = Arg(("principal",), help="kerberos principal",

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
amoghrajesh commented on code in PR #35146: URL: https://github.com/apache/airflow/pull/35146#discussion_r1370547760 ## airflow/cli/cli_config.py: ## @@ -561,6 +561,7 @@ def string_lower_type(val): # kerberos ARG_PRINCIPAL = Arg(("principal",), help="kerberos principal",

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
amoghrajesh commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1777660457 > (BTW. I think you should rebase as you have conflicts to solve :) Just saw the merge conflict. Sure, will rebase now -- This is an automated message from the Apache Git

[airflow] branch main updated: Correct MySQL backend version in Breeze docs (#35161)

2023-10-24 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new fcd6160f7e Correct MySQL backend version in

Re: [PR] Correct MySQL backend version in Breeze docs [airflow]

2023-10-24 Thread via GitHub
Taragolis merged PR #35161: URL: https://github.com/apache/airflow/pull/35161 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
potiuk commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1777619818 (BTW. I think you should rebase :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
potiuk commented on code in PR #35146: URL: https://github.com/apache/airflow/pull/35146#discussion_r1370512003 ## airflow/cli/cli_config.py: ## @@ -561,6 +561,7 @@ def string_lower_type(val): # kerberos ARG_PRINCIPAL = Arg(("principal",), help="kerberos principal",

Re: [I] MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade. [airflow]

2023-10-24 Thread via GitHub
Taragolis commented on issue #32708: URL: https://github.com/apache/airflow/issues/32708#issuecomment-1777580166 > Guys, is there a workaround to suppress this warning at all? No. The messages printed directly to stdout/stderr by `libmysqlclient`, until it doesn't changed in upstream

[PR] fix typo [airflow]

2023-10-24 Thread via GitHub
simonprydden opened a new pull request, #35161: URL: https://github.com/apache/airflow/pull/35161 Correct the version number in example code blocks ``` root@d20033cda4df:/opt/airflow# breeze --python 3.8 --backend mysql --mysql-version 8 Provided outdated value 8 for

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
amoghrajesh commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1777561228 @potiuk @eladkal can you take a look at this when you have some time? I have added screenshots regarding the command too in the PR description. -- This is an automated message

Re: [PR] Permitting airflow kerberos to run in different modes [airflow]

2023-10-24 Thread via GitHub
amoghrajesh commented on code in PR #35146: URL: https://github.com/apache/airflow/pull/35146#discussion_r1370461767 ## airflow/cli/commands/kerberos_command.py: ## @@ -33,6 +34,17 @@ def kerberos(args): """Start a kerberos ticket renewer.""" print(settings.HEADER)

Re: [PR] Add support for pure unit tests [airflow]

2023-10-24 Thread via GitHub
potiuk commented on PR #35160: URL: https://github.com/apache/airflow/pull/35160#issuecomment-1777559691 Devlist proposal coming based on that one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] Add support for pure unit tests [airflow]

2023-10-24 Thread via GitHub
potiuk opened a new pull request, #35160: URL: https://github.com/apache/airflow/pull/35160 --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Allowing airflow kerberos mode to be configurable [airflow]

2023-10-24 Thread via GitHub
amoghrajesh commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1777525306 > > How do I test the CLI here? Will my env pick up MY airflow kerberos if I try to run the command? > > Yes. You will have it in breeze Sure, working on adding it!

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-24 Thread via GitHub
dacort commented on PR #34225: URL: https://github.com/apache/airflow/pull/34225#issuecomment-1777496523 @eladkal Ah yes, that's right, I was trying to figure out a better way to pass the operator in, but I think I'll just pass in the `aws_conn_id` as a string instead. -- This is an

Re: [I] Cannot install Helm chart via Argo CD [airflow]

2023-10-24 Thread via GitHub
boring-cyborg[bot] commented on issue #35159: URL: https://github.com/apache/airflow/issues/35159#issuecomment-1777492741 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] Cannot install Helm chart via Argo CD [airflow]

2023-10-24 Thread via GitHub
jmilldotdev opened a new issue, #35159: URL: https://github.com/apache/airflow/issues/35159 ### Official Helm Chart version 1.11.0 (latest released) ### Apache Airflow version 2.7.2 ### Kubernetes Version 1.23.x ### Helm Chart configuration

[airflow] branch main updated: Add missing tests for the DAG endpoint (#35158)

2023-10-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9645786ad3 Add missing tests for the DAG endpoint

Re: [PR] Add missing tests for the DAG endpoint [airflow]

2023-10-24 Thread via GitHub
potiuk merged PR #35158: URL: https://github.com/apache/airflow/pull/35158 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Use base aws classes in Amazon S3 Glacier Operators/Sensors (#35108)

2023-10-24 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 3721c9a441 Use base aws classes in Amazon S3

Re: [PR] Use base aws classes in Amazon S3 Glacier Operators/Sensors [airflow]

2023-10-24 Thread via GitHub
vincbeck merged PR #35108: URL: https://github.com/apache/airflow/pull/35108 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (b18ce3e1ac -> c3fbefa07c)

2023-10-24 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from b18ce3e1ac Drop MySQL 5.7 support in dev/ci (#35152) add c3fbefa07c Move authentication related methods to FAB

Re: [PR] Move authentication related methods to FAB auth manager [airflow]

2023-10-24 Thread via GitHub
vincbeck commented on code in PR #35038: URL: https://github.com/apache/airflow/pull/35038#discussion_r1370347338 ## airflow/auth/managers/fab/security_manager/override.py: ## @@ -1650,3 +1950,82 @@ def _ldap_calculate_user_roles(self, user_attributes: dict[str, list[bytes]])

Re: [PR] Move authentication related methods to FAB auth manager [airflow]

2023-10-24 Thread via GitHub
vincbeck merged PR #35038: URL: https://github.com/apache/airflow/pull/35038 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add extra operator links for EMR Serverless [airflow]

2023-10-24 Thread via GitHub
dacort commented on PR #34225: URL: https://github.com/apache/airflow/pull/34225#issuecomment-1777400410 @eladkal Yes, let me check out the failing tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade. [airflow]

2023-10-24 Thread via GitHub
bmarquesplanet commented on issue #32708: URL: https://github.com/apache/airflow/issues/32708#issuecomment-1777386989 Guys, is there a workaround to suppress this warning at all? Tried some of the suggestions here

Re: [PR] Allowing airflow kerberos mode to be configurable [airflow]

2023-10-24 Thread via GitHub
potiuk commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1777372831 > @potiuk I am open to assigning #35154 to others but would like to lead this enter effort to completion. But, rest is up to you. :) Chart change can be follow-up but we should

Re: [PR] Allowing airflow kerberos mode to be configurable [airflow]

2023-10-24 Thread via GitHub
potiuk commented on PR #35146: URL: https://github.com/apache/airflow/pull/35146#issuecomment-1777371203 > How do I test the CLI here? Will my env pick up MY airflow kerberos if I try to run the command? Yes. You will have it in breeze -- This is an automated message from the

[airflow] branch main updated: Drop MySQL 5.7 support in dev/ci (#35152)

2023-10-24 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new b18ce3e1ac Drop MySQL 5.7 support in dev/ci

Re: [PR] Drop MySQL 5.7 support in dev/ci [airflow]

2023-10-24 Thread via GitHub
potiuk merged PR #35152: URL: https://github.com/apache/airflow/pull/35152 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

  1   2   3   >