[jira] [Updated] (FLINK-31966) Flink Kubernetes operator lacks TLS support
[ https://issues.apache.org/jira/browse/FLINK-31966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Vasiliu updated FLINK-31966: --- Attachment: (was: image-2024-04-16-16-33-39-644.png) > Flink Kubernetes operator lacks TLS support > > > Key: FLINK-31966 > URL: https://issues.apache.org/jira/browse/FLINK-31966 > Project: Flink > Issue Type: New Feature > Components: Kubernetes Operator >Affects Versions: kubernetes-operator-1.4.0 >Reporter: Adrian Vasiliu >Assignee: Tony Garrard >Priority: Major > Fix For: kubernetes-operator-1.8.0 > > > *Summary* > The Flink Kubernetes operator lacks support inside the FlinkDeployment > operand for configuring Flink with TLS (both one-way and mutual) for the > internal communication between jobmanagers and taskmanagers, and for the > external REST endpoint. Although a workaround exists to configure the job and > task managers, this breaks the operator and renders it unable to reconcile. > *Additional information* > * The Apache Flink operator supports passing through custom flink > configuration to be applied to job and task managers. > * If you supply SSL-based properties, the operator can no longer speak to > the deployed job manager. The operator is reading the flink conf and using it > to create a connection to the job manager REST endpoint, but it uses the > truststore file paths within flink-conf.yaml, which are unresolvable from the > operator. This leaves the operator hanging in a pending state as it cannot > complete a reconcile. > *Proposal* > Our proposal is to make changes to the operator code. A simple change exists > that would be enough to enable anonymous SSL at the REST endpoint, but more > invasive changes would be required to enable full mTLS throughout. > The simple change to enable anonymous SSL would be for the operator to parse > flink-conf and podTemplate to identify the Kubernetes resource that contains > the certificate from the job manager keystore and use it inside the > operator’s trust store. > In the case of mutual TLS, further changes are required: the operator would > need to generate a certificate signed by the same issuing authority as the > job manager’s certificates and then use it in a keystore when challenged by > that job manager. We propose that the operator becomes responsible for making > CertificateSigningRequests to generate certificates for job manager, task > manager and operator. The operator can then coordinate deploying the job and > task managers with the correct flink-conf and volume mounts. This would also > work for anonymous SSL. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-31966) Flink Kubernetes operator lacks TLS support
[ https://issues.apache.org/jira/browse/FLINK-31966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Vasiliu updated FLINK-31966: --- Attachment: image-2024-04-16-16-33-39-644.png > Flink Kubernetes operator lacks TLS support > > > Key: FLINK-31966 > URL: https://issues.apache.org/jira/browse/FLINK-31966 > Project: Flink > Issue Type: New Feature > Components: Kubernetes Operator >Affects Versions: kubernetes-operator-1.4.0 >Reporter: Adrian Vasiliu >Assignee: Tony Garrard >Priority: Major > Fix For: kubernetes-operator-1.8.0 > > Attachments: image-2024-04-16-16-33-39-644.png > > > *Summary* > The Flink Kubernetes operator lacks support inside the FlinkDeployment > operand for configuring Flink with TLS (both one-way and mutual) for the > internal communication between jobmanagers and taskmanagers, and for the > external REST endpoint. Although a workaround exists to configure the job and > task managers, this breaks the operator and renders it unable to reconcile. > *Additional information* > * The Apache Flink operator supports passing through custom flink > configuration to be applied to job and task managers. > * If you supply SSL-based properties, the operator can no longer speak to > the deployed job manager. The operator is reading the flink conf and using it > to create a connection to the job manager REST endpoint, but it uses the > truststore file paths within flink-conf.yaml, which are unresolvable from the > operator. This leaves the operator hanging in a pending state as it cannot > complete a reconcile. > *Proposal* > Our proposal is to make changes to the operator code. A simple change exists > that would be enough to enable anonymous SSL at the REST endpoint, but more > invasive changes would be required to enable full mTLS throughout. > The simple change to enable anonymous SSL would be for the operator to parse > flink-conf and podTemplate to identify the Kubernetes resource that contains > the certificate from the job manager keystore and use it inside the > operator’s trust store. > In the case of mutual TLS, further changes are required: the operator would > need to generate a certificate signed by the same issuing authority as the > job manager’s certificates and then use it in a keystore when challenged by > that job manager. We propose that the operator becomes responsible for making > CertificateSigningRequests to generate certificates for job manager, task > manager and operator. The operator can then coordinate deploying the job and > task managers with the correct flink-conf and volume mounts. This would also > work for anonymous SSL. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-31966) Flink Kubernetes operator lacks TLS support
[ https://issues.apache.org/jira/browse/FLINK-31966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-31966: --- Priority: Major (was: Critical) > Flink Kubernetes operator lacks TLS support > > > Key: FLINK-31966 > URL: https://issues.apache.org/jira/browse/FLINK-31966 > Project: Flink > Issue Type: New Feature > Components: Kubernetes Operator >Affects Versions: kubernetes-operator-1.4.0 >Reporter: Adrian Vasiliu >Priority: Major > > *Summary* > The Flink Kubernetes operator lacks support inside the FlinkDeployment > operand for configuring Flink with TLS (both one-way and mutual) for the > internal communication between jobmanagers and taskmanagers, and for the > external REST endpoint. Although a workaround exists to configure the job and > task managers, this breaks the operator and renders it unable to reconcile. > *Additional information* > * The Apache Flink operator supports passing through custom flink > configuration to be applied to job and task managers. > * If you supply SSL-based properties, the operator can no longer speak to > the deployed job manager. The operator is reading the flink conf and using it > to create a connection to the job manager REST endpoint, but it uses the > truststore file paths within flink-conf.yaml, which are unresolvable from the > operator. This leaves the operator hanging in a pending state as it cannot > complete a reconcile. > *Proposal* > Our proposal is to make changes to the operator code. A simple change exists > that would be enough to enable anonymous SSL at the REST endpoint, but more > invasive changes would be required to enable full mTLS throughout. > The simple change to enable anonymous SSL would be for the operator to parse > flink-conf and podTemplate to identify the Kubernetes resource that contains > the certificate from the job manager keystore and use it inside the > operator’s trust store. > In the case of mutual TLS, further changes are required: the operator would > need to generate a certificate signed by the same issuing authority as the > job manager’s certificates and then use it in a keystore when challenged by > that job manager. We propose that the operator becomes responsible for making > CertificateSigningRequests to generate certificates for job manager, task > manager and operator. The operator can then coordinate deploying the job and > task managers with the correct flink-conf and volume mounts. This would also > work for anonymous SSL. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-31966) Flink Kubernetes operator lacks TLS support
[ https://issues.apache.org/jira/browse/FLINK-31966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martijn Visser updated FLINK-31966: --- Issue Type: New Feature (was: Bug) > Flink Kubernetes operator lacks TLS support > > > Key: FLINK-31966 > URL: https://issues.apache.org/jira/browse/FLINK-31966 > Project: Flink > Issue Type: New Feature > Components: Kubernetes Operator >Affects Versions: kubernetes-operator-1.4.0 >Reporter: Adrian Vasiliu >Priority: Critical > > *Summary* > The Flink Kubernetes operator lacks support inside the FlinkDeployment > operand for configuring Flink with TLS (both one-way and mutual) for the > internal communication between jobmanagers and taskmanagers, and for the > external REST endpoint. Although a workaround exists to configure the job and > task managers, this breaks the operator and renders it unable to reconcile. > *Additional information* > * The Apache Flink operator supports passing through custom flink > configuration to be applied to job and task managers. > * If you supply SSL-based properties, the operator can no longer speak to > the deployed job manager. The operator is reading the flink conf and using it > to create a connection to the job manager REST endpoint, but it uses the > truststore file paths within flink-conf.yaml, which are unresolvable from the > operator. This leaves the operator hanging in a pending state as it cannot > complete a reconcile. > *Proposal* > Our proposal is to make changes to the operator code. A simple change exists > that would be enough to enable anonymous SSL at the REST endpoint, but more > invasive changes would be required to enable full mTLS throughout. > The simple change to enable anonymous SSL would be for the operator to parse > flink-conf and podTemplate to identify the Kubernetes resource that contains > the certificate from the job manager keystore and use it inside the > operator’s trust store. > In the case of mutual TLS, further changes are required: the operator would > need to generate a certificate signed by the same issuing authority as the > job manager’s certificates and then use it in a keystore when challenged by > that job manager. We propose that the operator becomes responsible for making > CertificateSigningRequests to generate certificates for job manager, task > manager and operator. The operator can then coordinate deploying the job and > task managers with the correct flink-conf and volume mounts. This would also > work for anonymous SSL. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-31966) Flink Kubernetes operator lacks TLS support
[ https://issues.apache.org/jira/browse/FLINK-31966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Vasiliu updated FLINK-31966: --- Description: *Summary* The Flink Kubernetes operator lacks support inside the FlinkDeployment operand for configuring Flink with TLS (both one-way and mutual) for the internal communication between jobmanagers and taskmanagers, and for the external REST endpoint. Although a workaround exists to configure the job and task managers, this breaks the operator and renders it unable to reconcile. *Additional information* * The Apache Flink operator supports passing through custom flink configuration to be applied to job and task managers. * If you supply SSL-based properties, the operator can no longer speak to the deployed job manager. The operator is reading the flink conf and using it to create a connection to the job manager REST endpoint, but it uses the truststore file paths within flink-conf.yaml, which are unresolvable from the operator. This leaves the operator hanging in a pending state as it cannot complete a reconcile. *Proposal* Our proposal is to make changes to the operator code. A simple change exists that would be enough to enable anonymous SSL at the REST endpoint, but more invasive changes would be required to enable full mTLS throughout. The simple change to enable anonymous SSL would be for the operator to parse flink-conf and podTemplate to identify the Kubernetes resource that contains the certificate from the job manager keystore and use it inside the operator’s trust store. In the case of mutual TLS, further changes are required: the operator would need to generate a certificate signed by the same issuing authority as the job manager’s certificates and then use it in a keystore when challenged by that job manager. We propose that the operator becomes responsible for making CertificateSigningRequests to generate certificates for job manager, task manager and operator. The operator can then coordinate deploying the job and task managers with the correct flink-conf and volume mounts. This would also work for anonymous SSL. was: *Summary* The Flink Kubernetes operator lacks support inside the FlinkDeployment operand for configuring Flink with TLS (both one-way and mutual) for the internal communication between jobmanagers and taskmanagers, and for the external REST endpoint. Although a workaround exists to configure the job and task managers, this breaks the operator and renders it unable to reconcile. *Additional information* * The Apache Flink operator supports passing through custom flink configuration to be applied to job and task managers. * If you supply SSL-based properties, the operator can no longer speak to the deployed job manager. The operator is reading the flink conf and using it to create a connection to the job manager REST endpoint, but it uses the truststore file paths within flink-conf.yaml, which are unresolvable from the operator. This leaves the operator hanging in a pending state as it cannot complete a reconcile. *Proposal* Our proposal is to make changes to the operator code. A simple change exists that would be enough to enable anonymous SSL at the REST endpoint, but more invasive changes would be required to enable full mTLS throughout. The simple change to enable anonymous SSL would be for the operator to parse flink-conf and podTemplate to identify the Kubernetes resource that contains the certificate from the job manager keystore and use it inside the operator’s trust store. In the case of mutual TLS, further changes are required: the operator would need to generate a certificate signed by the same issuing authority as the job manager’s certificates and then use it in a keystore when challenged by that job manager. We propose that the operator becomes responsible for making CertificateSigningRequests to generate certificates for job manager, task manager and operator. The operator can then coordinate deploying the job and task managers with the correct flink-conf and volume mounts. This would also work for anonymous SSL. > Flink Kubernetes operator lacks TLS support > > > Key: FLINK-31966 > URL: https://issues.apache.org/jira/browse/FLINK-31966 > Project: Flink > Issue Type: Bug > Components: Kubernetes Operator >Affects Versions: kubernetes-operator-1.4.0 >Reporter: Adrian Vasiliu >Priority: Critical > > *Summary* > The Flink Kubernetes operator lacks support inside the FlinkDeployment > operand for configuring Flink with TLS (both one-way and mutual) for the > internal communication between jobmanagers and taskmanagers, and for the > external REST endpoint. Although a workaround exists to configure the job and > task managers, this breaks the
