Scope0910 opened a new issue #21638:
URL: https://github.com/apache/airflow/issues/21638
### Official Helm Chart version
1.2.0
### Apache Airflow version
2.1.4
### Kubernetes Version
v1.21.6+bb8d50a
### Helm Chart configuration
I defined a new Connection String for AIRFLOW_CONN_SPARK_DEFAULT in
values.yaml like the following section (base64 encoded or with correct string
(spark://k8s://100.68.0.1:443?deploy-mode=cluster) :
extraSecrets:
'{{ .Release.Name }}-airflow-connections':
data: |
AIRFLOW_CONN_SPARK_DEFAULT:
'c3Bhcms6Ly9rOHM6Ly8xMDAuNjguMC4xOjQ0Mz9kZXBsb3ktbW9kZT1jbHVzdGVy'
in Section extraEnvFrom i defined the following:
extraEnvFrom: |
- secretRef:
name: '{{ .Release.Name }}-airflow-connections'
### Docker Image customisations
added apache-airflow-providers-apache-spark to base Image
### What happened
Airflow Connection mapped wrong because of the k8s:// within the url. if i
ask for the connection with cmd "airflow connections get spark_default" then
host=k8s and schema=/100.60.0.1:443 which is wrong
### What you expected to happen
the spark Connection based on k8s
(spark://k8s://100.68.0.1:443?deploy-mode=cluster) should be parsed correctly
### How to reproduce
define in values.yaml
extraSecrets:
'{{ .Release.Name }}-airflow-connections':
data: |
AIRFLOW_CONN_SPARK_DEFAULT:
'c3Bhcms6Ly9rOHM6Ly8xMDAuNjguMC4xOjQ0Mz9kZXBsb3ktbW9kZT1jbHVzdGVy'
extraEnvFrom: |
- secretRef:
name: '{{ .Release.Name }}-airflow-connections'
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
