Hi Michael,
You are using the right one, it just lacks the support for Opensearch REST
client customization at the moment. It would make sense to provide this
functionality.
Thank you.
Best Regards,
Andriy Redko
MHJ> Hi Andriy,
MHJ> we are currently use the OpensearchSink[1] connector, as we assumed that
that was the one to use going forwards. We are not quite sure what the
difference is and if any will become deprecated soon?
MHJ> Regards,
MHJ> Michael
MHJ> [1]
https://github.com/apache/flink-connector-opensearch/blob/main/flink-connector-opensearch/src/main/java/org/apache/flink/connector/opensearch/sink/OpensearchSink.java
MHJ> ________________________________
MHJ> From: Andriy Redko <[email protected]>
MHJ> Sent: Tuesday, 18 April 2023 4:16:45 PM
MHJ> To: Martijn Visser; Michael Hempel Jørgensen
MHJ> Cc: [email protected]
MHJ> Subject: Re: Opensearch connector and oauth
MHJ> Hi Michael,
MHJ> Indeed the Opensearch REST client supports only basic auth at the moment.
I don't know which sink are you using, but the
MHJ> streaming/connectors/opensearch/OpensearchSink provides the capability to
configure the underlying client using
MHJ> RestClientFactory [1] so you could add the request / response interceptors
to propagate OAuth2 tokens etc. This
MHJ> is not possible with other sinks yet but I think it should not be
difficult to implement. If this is the case
MHJ> for you, could you please open the JIRA ticket?
MHJ> Thank you.
MHJ> [1]
https://github.com/apache/flink-connector-opensearch/blob/main/flink-connector-opensearch/src/main/java/org/apache/flink/streaming/connectors/opensearch/RestClientFactory.java
MHJ> Best Regards,
MHJ> Andriy Redko
MHJ> Tuesday, April 18, 2023, 3:47:22 AM, you wrote:
>> Hi Michael,
>> I'm looping in Andrey since he has worked a lot on the Opensearch connector.
>> A contribution is very welcome in case this can be improved.
>> Best regards,
>> Martijn
>> On Tue, Apr 18, 2023 at 8:45 AM Michael Hempel Jørgensen <[email protected]>
>> wrote:
>>> Hi,
>>> we need to use OAuth2 (Client Credentials Flow) in Flink to authenticate
>>> and authorise against different services, initially Kafka and Opensearch.
>>> We have it working with Kafka using however it doesn't seem to be possible
>>> with the Opensearch Flink Connector
>>> (https://github.com/apache/flink-connector-opensearch).
>>> It looks like the connector only supports basic auth (user/pass) at the
>>> moment, and doesn't have an option for manually setting custom headers
>>> either. Is this correct?
>>> Is there any work in progress towards making this possible?
>>> We are willing to help contribute to code/discussions if necessary to get
>>> this implemented somehow.
>>> Regards,
>>> Michael
