Re: [I] There is a bug in connecting to EKS using the airflow.providers.amazon.aws.operators.eks library in China. [airflow]
o-nikolas closed issue #45368: There is a bug in connecting to EKS using the airflow.providers.amazon.aws.operators.eks library in China. URL: https://github.com/apache/airflow/issues/45368 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] There is a bug in connecting to EKS using the airflow.providers.amazon.aws.operators.eks library in China. [airflow]
vincbeck commented on issue #45368: URL: https://github.com/apache/airflow/issues/45368#issuecomment-2581196535 Re-opening it because we are reverting the fix (because it introduced another bug, more harmful). Revert: #45526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] There is a bug in connecting to EKS using the airflow.providers.amazon.aws.operators.eks library in China. [airflow]
vincbeck closed issue #45368: There is a bug in connecting to EKS using the airflow.providers.amazon.aws.operators.eks library in China. URL: https://github.com/apache/airflow/issues/45368 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] There is a bug in connecting to EKS using the airflow.providers.amazon.aws.operators.eks library in China. [airflow]
QiaoLiar commented on issue #45368: URL: https://github.com/apache/airflow/issues/45368#issuecomment-2575004573 > Hi there, is there any easy way to replicate this issue without holding an AWS China account? No, this bug is only triggered when using AWS credentials based in China. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] There is a bug in connecting to EKS using the airflow.providers.amazon.aws.operators.eks library in China. [airflow]
willdanckwerts commented on issue #45368: URL: https://github.com/apache/airflow/issues/45368#issuecomment-2574998409 Hi there, is there any easy way to replicate this issue without holding an AWS China account? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] There is a bug in connecting to EKS using the airflow.providers.amazon.aws.operators.eks library in China. [airflow]
boring-cyborg[bot] commented on issue #45368: URL: https://github.com/apache/airflow/issues/45368#issuecomment-2568670706 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] There is a bug in connecting to EKS using the airflow.providers.amazon.aws.operators.eks library in China. [airflow]
QiaoLiar opened a new issue, #45368:
URL: https://github.com/apache/airflow/issues/45368
### Apache Airflow Provider(s)
amazon
### Versions of Apache Airflow Providers
9.2.0
Even though I'm using the latest version, I think this bug exists in all
historical versions.
### Apache Airflow version
2.10.1
### Operating System
Amazon Linux 2023
### Deployment
Amazon (AWS) MWAA
### Deployment details
You can reproduce this bug stably without any customization.
### What happened
Their dag script to connect to EKS cluster using [from
airflow.providers.amazon.aws.operators.eks import EksPodOperator] returned 401 .
script：
```
from airflow import DAG
from airflow.operators.dummy import DummyOperator
from airflow.providers.amazon.aws.operators.eks import EksPodOperator
from kubernetes.client import models as k8s
from datetime import datetime
DEFAULT_ARGS = {
'owner': 'XXX',
}
with DAG(
'test2_eks_pod_operator_poc',
default_args=DEFAULT_ARGS,
schedule_interval=None, # trigger manually for now
start_date=datetime(2024, 4, 28),
catchup=False,
tags=['examples']
) as dag:
start = DummyOperator(task_id='start', retries=2)
end = DummyOperator(task_id='end', retries=2)
test2_eks_pod_operator = EksPodOperator(
task_id='test2_eks_pod_operator',
region='cn-north-1',
cluster_name='eks2-sas_cnnorth1',
namespace='mwaa',
service_account_name='default',
pod_name='eks_pod_operator_poc',
image='amazon/aws-cli:latest',
image_pull_policy='IfNotPresent',
node_selector={
'type': 'app'
},
tolerations=[
k8s.V1Toleration(
effect='NoSchedule',
key='type',
operator='Equal',
value='app'
)
],
cmds=['/bin/bash', '-c'],
arguments=['echo "hello world"'],
is_delete_operator_pod=True,
)
start >> test2_eks_pod_operator >> end
```
error log:
```
*** Reading remote log from Cloudwatch log_group:
airflow-mwaa-sas-cnnorth1-Task log_stream:
dag_id=test2_eks_pod_operator_poc/run_id=manual__2024-12-31T08_47_12.225701+00_00/task_id=test2_eks_pod_operator/attempt=1.log.
[2024-12-31, 08:47:15 UTC] {local_task_job_runner.py:123} ▶ Pre task
execution logs
[2024-12-31, 08:47:16 UTC] {base.py:84} INFO - Retrieving connection
'aws_default'
[2024-12-31, 08:47:16 UTC] {baseoperator.py:405} WARNING -
EksPodOperator.execute cannot be called outside TaskInstance!
[2024-12-31, 08:47:16 UTC] {pod.py:1133} INFO - Building pod
eks-pod-operator-poc-ylyc6uh3 with labels: {'dag_id':
'test2_eks_pod_operator_poc', 'task_id': 'test2_eks_pod_operator', 'run_id':
'manual__2024-12-31T084712.225701-a8af56277', 'kubernetes_pod_operator':
'True', 'try_number': '1'}
[2024-12-31, 08:47:16 UTC] {base.py:84} INFO - Retrieving connection
'kubernetes_default'
[2024-12-31, 08:47:19 UTC] {taskinstance.py:3310} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py",
line 767, in _execute_task
result = _execute_callable(context=context, **execute_callable_kwargs)
^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py",
line 733, in _execute_callable
return ExecutionCallableRunner(
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/utils/operator_helpers.py",
line 252, in run
return self.func(*args, **kwargs)
^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/models/baseoperator.py",
line 406, in wrapper
return func(self, *args, **kwargs)
^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/operators/eks.py",
line 1103, in execute
return super().execute(context)
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/models/baseoperator.py",
line 406, in wrapper
return func(self, *args, **kwargs)
^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kubernetes/operators/pod.py",
line 593, in execute
return self.execute_sync(context)
^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/providers/cncf/kub
