Thanks, will have a look through! -----Original Message----- From: Yangze Guo <karma...@gmail.com> Sent: Wednesday, September 15, 2021 11:25 AM To: Osada Paranaliyanage <osada.paranaliyan...@dialog.lk> Cc: David Morávek <d...@apache.org>; user@flink.apache.org Subject: Re: Streaming SQL support for redis streaming connector
[EXTERNAL EMAIL] This email has been received from an external source – please review before actioning, clicking on links, or opening attachments. > What about the bahir streaming connectors? Are they considered canonical? > Would they be merged in to the main project at some point? Iiuc we can use > table API etc to write data to redis using that right? I think it still can be used with Flink 1.12 and 1.13. However, it has not been updated for 5 months and is not implemented based on the new interfaces introduced in FLIP-95. AFAIK, the community does not plan to merge it. > What more will be required for us to use SQL? (as in specify the > connector in the WITH clause as redis) To use the bahir with SQL, you need to build the project and move the uber jar to the lib directory of your flink dist and fill in necessary properties[1][2]. [1] https://github.com/apache/bahir-flink/blob/master/flink-connector-redis/src/main/java/org/apache/flink/streaming/connectors/redis/RedisTableSinkFactory.java [2] https://github.com/apache/bahir-flink/blob/master/flink-connector-redis/src/main/java/org/apache/flink/streaming/connectors/redis/descriptor/RedisValidator.java Best, Yangze Guo On Wed, Sep 15, 2021 at 11:34 AM Osada Paranaliyanage <osada.paranaliyan...@dialog.lk> wrote: > > Hi David, > > > > What about the bahir streaming connectors? Are they considered > canonical? Would they be merged in to the main project at some point? > Iiuc we can use table API etc to write data to redis using that right? > What more will be required for us to use SQL? (as in specify the > connector in the WITH clause as redis) > > > > Thanks, > > Osada. > > > > From: David Morávek <d...@apache.org> > Sent: Tuesday, September 14, 2021 7:53 PM > To: Osada Paranaliyanage <osada.paranaliyan...@dialog.lk> > Cc: user@flink.apache.org > Subject: Re: Streaming SQL support for redis streaming connector > > > > [EXTERNAL EMAIL] This email has been received from an external source – > please review before actioning, clicking on links, or opening attachments. > > Hi Osada, > > > > in theory building a Redis table from "CDC stream" should definitely be > doable. Unfortunately Flink currently doesn't have any official Redis Sink > for the Table API and there is currently no on-going effort for adding it, so > it would need to be implemented first. The resulting syntax would be pretty > much the same to what's outlined in the mentioned example. > > > > Best, > > D. > > > > On Tue, Sep 14, 2021 at 2:57 PM Osada Paranaliyanage > <osada.paranaliyan...@dialog.lk> wrote: > > Hi All, We are looking to use flink to build a materialized view of a > relation db and a document db using cdc streams. For this purpose we would > like to use redis for hosting the materialized view. Can we do this in > streaming SQL? We have worked through > https://github.com/ververica/flink-sql-CDC and can see how this will work > with ES as a sink. But can we use redis as the sink? Where do we find the > syntax for that? > > > > Thanks, > > Osada. > > > > > > ________________________________ > > > > This e-mail is confidential. It may also be legally privileged. If you are > not the intended recipient or have received it in error, please delete it and > all copies from your system and notify the sender immediately by return > e-mail. Any unauthorized reading, reproducing, printing or further > dissemination of this e-mail or its contents is strictly prohibited and may > be unlawful. Internet communications cannot be guaranteed to be timely, > secure, error or virus-free. The sender does not accept liability for any > errors or omissions. > > > ________________________________ > > > This e-mail is confidential. It may also be legally privileged. If you are > not the intended recipient or have received it in error, please delete it and > all copies from your system and notify the sender immediately by return > e-mail. Any unauthorized reading, reproducing, printing or further > dissemination of this e-mail or its contents is strictly prohibited and may > be unlawful. Internet communications cannot be guaranteed to be timely, > secure, error or virus-free. The sender does not accept liability for any > errors or omissions. > ________________________________ This e-mail is confidential. It may also be legally privileged. If you are not the intended recipient or have received it in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Any unauthorized reading, reproducing, printing or further dissemination of this e-mail or its contents is strictly prohibited and may be unlawful. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions.
