Hi Flink team,
I'm trying to configure a Flink on YARN with SSL enabled. I've followed the
documentation's instruction [1] to generate a Keystore and Truststore locally,
and added a the properties to my flink-conf.yaml.
security.ssl.rest.keystore: /home/user/ssl/deploy-keys/rest.keystore
security.ssl.rest.truststore: /home/user/ssl/deploy-keys/rest.truststore
I've also added the yarnship option so that the keystore and truststore are
deployed as suggested in [1].
-m yarn-cluster --class <class> [...] -yt /home/user/ssl/deploy-keys/
However, starting the Flink cluster results in a NoSuchFileException,
Caused by: java.nio.file.NoSuchFileException:
/home/user/ssl/deploy-keys/rest.keystore
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at
sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at
java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
at java.nio.file.Files.newInputStream(Files.java:152)
at
org.apache.flink.runtime.net.SSLUtils.getKeyManagerFactory(SSLUtils.java:266)
at
org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUtils.java:392)
at
org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUtils.java:365)
at
org.apache.flink.runtime.net.SSLUtils.createRestServerSSLEngineFactory(SSLUtils.java:163)
at
org.apache.flink.runtime.rest.RestServerEndpointConfiguration.fromConfiguration(RestServerEndpointConfiguration.java:160)
I'm able to see in launch_container.sh that the shipped directory was able to
be created successfully:
mkdir -p deploy-keys
ln -sf
"/fs/htmp/yarn/local/usercache/delp/appcache/application_1618711298408_2664/filecache/16/rest.truststore"
"deploy-keys/rest.truststore"
mkdir -p deploy-keys
ln -sf
"/fs/htmp/yarn/local/usercache/delp/appcache/application_1618711298408_2664/filecache/13/rest.keystore"
"deploy-keys/rest.keystore"
So given the above logs, I tried editing flink-conf.yaml to reflect what I saw:
security.ssl.rest.keystore: deploy-keys/rest.keystore
security.ssl.rest.truststore: deploy-keys/rest.truststore
But that didn't seem to work, either:
Caused by: java.nio.file.NoSuchFileException: deploy-keys/rest.truststore
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at
sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at
java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
at java.nio.file.Files.newInputStream(Files.java:152)
at
org.apache.flink.runtime.net.SSLUtils.getTrustManagerFactory(SSLUtils.java:233)
at
org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUtils.java:397)
at
org.apache.flink.runtime.net.SSLUtils.createRestNettySSLContext(SSLUtils.java:365)
at
org.apache.flink.runtime.net.SSLUtils.createRestClientSSLEngineFactory(SSLUtils.java:181)
at
org.apache.flink.runtime.rest.RestClientConfiguration.fromConfiguration(RestClientConfiguration.java:106)
What needs to be done to get the YARN application to point to the right
keystore and truststore?
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/security-ssl.html#tips-for-yarn--mesos-deployment
____________
Andreas Hailu
Data Lake Engineering | Goldman Sachs & Co.
________________________________
Your Personal Data: We may collect and process information about you that may
be subject to data protection laws. For more information about how we use and
disclose your personal data, how we protect your information, our legal basis
to use your information, your rights and who you can contact, please refer to:
www.gs.com/privacy-notices<http://www.gs.com/privacy-notices>
