[jira] [Commented] (BEAM-7519) Improve work with SSL/TLS certificates in SDK IO, for Google Dataflow

2020-05-24 Thread Yu Watanabe (Jira)


[ 
https://issues.apache.org/jira/browse/BEAM-7519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115600#comment-17115600
 ] 

Yu Watanabe commented on BEAM-7519:
---

In general, elasticsearch provides its own self-signed certificate and is not 
required to set mutual authentication .

[https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#tls-http]

Depending on strictness of securing the connection, I thought disabling clinet 
side certificate might be an option.

Reading the [Basic code 
requirements|[https://beam.apache.org/documentation/io/developing-io-java/]] , 
it does not state about the strctness of SSL connection so I thought above 
might be a option.

> Improve work with SSL/TLS certificates in SDK IO, for Google Dataflow
> -
>
> Key: BEAM-7519
> URL: https://issues.apache.org/jira/browse/BEAM-7519
> Project: Beam
>  Issue Type: Wish
>  Components: io-ideas, io-java-elasticsearch, io-java-gcp, 
> io-java-kafka
>Reporter: Oleksandr Priadko
>Priority: P2
>
> Sinks like Kafka or Elasticsearch have worked with SSL/TLS. But all of these 
> can read only certificates from a local machine (use 
> java.io.File#File(java.lang.String). 
> In the case of Google Dataflow, we can not easily copy files to a worker, I 
> see two possible variants to read certificates from the worker:
> 1) somehow copy certificates to a worker from GC Storage 
> 2) in Kafka IO or Elasticsearch IO read certificates directly from GC Storage 
> Any thoughts about how I can solve this issue easily or mb other variants 
> will be helpful.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (BEAM-7519) Improve work with SSL/TLS certificates in SDK IO, for Google Dataflow

2020-04-02 Thread Alexey Romanenko (Jira)


[ 
https://issues.apache.org/jira/browse/BEAM-7519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17073706#comment-17073706
 ] 

Alexey Romanenko commented on BEAM-7519:


I just found this issue by chance, so the answer is quite late, but required 
functionality in KafkaIO can implemented by writing custom consumer client. 
More details can be found here BEAM-9627

> Improve work with SSL/TLS certificates in SDK IO, for Google Dataflow
> -
>
> Key: BEAM-7519
> URL: https://issues.apache.org/jira/browse/BEAM-7519
> Project: Beam
>  Issue Type: Wish
>  Components: io-ideas, io-java-elasticsearch, io-java-gcp, 
> io-java-kafka
>Reporter: Oleksandr Priadko
>Priority: Major
>
> Sinks like Kafka or Elasticsearch have worked with SSL/TLS. But all of these 
> can read only certificates from a local machine (use 
> java.io.File#File(java.lang.String). 
> In the case of Google Dataflow, we can not easily copy files to a worker, I 
> see two possible variants to read certificates from the worker:
> 1) somehow copy certificates to a worker from GC Storage 
> 2) in Kafka IO or Elasticsearch IO read certificates directly from GC Storage 
> Any thoughts about how I can solve this issue easily or mb other variants 
> will be helpful.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)