[
https://issues.apache.org/jira/browse/BEAM-9627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072966#comment-17072966
]
Alexey Romanenko commented on BEAM-9627:
----------------------------------------
Actually, this hook already exists. You can try to use
{{KafkaIO.Read.withConsumerFactoryFn()}} and implement custom consumer
function, which will be called to create Kafka consumer on worker. In this
function you can copy your files to local temp directories from whatever place
you need.
Some more concrete examples about that approach:
https://stackoverflow.com/questions/42726011/truststore-and-google-cloud-dataflow/53549757
https://stackoverflow.com/questions/54337653/connect-to-kafka-with-ssl-using-kafkaio-on-google-dataflow
> KafkaIO needs better support for SSL
> ------------------------------------
>
> Key: BEAM-9627
> URL: https://issues.apache.org/jira/browse/BEAM-9627
> Project: Beam
> Issue Type: Improvement
> Components: io-java-kafka
> Reporter: Daniel Mills
> Priority: Minor
>
> Configuring SSL for kafka requires pointing an option at local files
> containing keys and roots of trust as described here:
> [https://www.confluent.io/blog/apache-kafka-security-authorization-authentication-encryption/]
> Currently, it is somewhat tricky to ensure that these files are written
> before KafkaIO starts reading from the source; one potential option would be
> to add an init hook where the user could download keys from the keystore of
> their choice and write them to local files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
