Hi, Samrat Thanks for driving this FLIP. It looks like supporting flink-connector-redshift is very useful to Flink. I have two question: 1. Regarding the `read.mode` and `write.mode`, you say here provides two modes, respectively, jdbc and `unload or copy`, What is the default value for `read.mode` and `write.mode? 2. For Source, does it both support batch read and streaming read?
Best, Ron Samrat Deb <decordea...@gmail.com> 于2023年5月30日周二 17:15写道： > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift > > [note] Missed the trailing link for previous mail > > > > On Tue, May 30, 2023 at 2:43 PM Samrat Deb <decordea...@gmail.com> wrote: > > > Hi Leonard, > > > > > and I’m glad to help review the design as well as the code review. > > Thank you so much. It would be really great and helpful to bring > > flink-connector-redshift for flink users :) . > > > > I have divided the implementation in 3 phases in the `Scope` Section[1]. > > 1st phase is to > > > > - Integrate with Flink Sink API (*FLIP-171* > > < > https://cwiki.apache.org/confluence/display/FLINK/FLIP-171%3A+Async+Sink> > > ) > > > > > > > About the implementation phases, How about prioritizing support for the > > Datastream Sink API and TableSink API in the first phase? > > I can completely resonate with you to prioritize support for Datastream > > Sink API and TableSink API in the first phase. > > I will update the FLIP[1] as you have suggested. > > > > > It seems that the primary use cases for the Redshift connector are > > acting as a sink for processed data by Flink. > > Yes, majority ask and requirement for Redshift connector is sink for > > processed data by Flink. > > > > Bests, > > Samrat > > > > On Tue, May 30, 2023 at 12:35 PM Leonard Xu <xbjt...@gmail.com> wrote: > > > >> Thanks @Samrat for bringing this discussion. > >> > >> It makes sense to me to introduce AWS Redshift connector for Apache > >> Flink, and I’m glad to help review the design as well as the code > review. > >> > >> About the implementation phases, How about prioritizing support for the > >> Datastream Sink API and TableSink API in the first phase? It seems that > the > >> primary use cases for the Redshift connector are acting as a sink for > >> processed data by Flink. > >> > >> Best, > >> Leonard > >> > >> > >> > On May 29, 2023, at 12:51 PM, Samrat Deb <decordea...@gmail.com> > wrote: > >> > > >> > Hello all , > >> > > >> > Context: > >> > Amazon Redshift [1] is a fully managed, petabyte-scale data warehouse > >> > service in the cloud. It allows analyzing data without all of the > >> > configurations of a provisioned data warehouse. Resources are > >> automatically > >> > provisioned and data warehouse capacity is intelligently scaled to > >> deliver > >> > fast performance for even the most demanding and unpredictable > >> workloads. > >> > Redshift is one of the widely used warehouse solutions in the current > >> > market. > >> > > >> > Building flink connector redshift will allow flink users to have > source > >> and > >> > sink directly to redshift. It will help flink to expand the scope to > >> > redshift as a new connector in the ecosystem. > >> > > >> > I would like to start a discussion on the FLIP-307: Flink connector > >> > redshift [2]. > >> > Looking forward to comments, feedbacks and suggestions from the > >> community > >> > on the proposal. > >> > > >> > [1] https://docs.aws.amazon.com/redshift/latest/mgmt/welcome.html > >> > [2] > >> > > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-307%3A++Flink+Connector+Redshift > >> > > >> > > >> > > >> > Bests, > >> > Samrat > >> > >> >
