Hi Vinoth, I am aligned with the first reason that you mentioned. Better to have a separate tool to take care of this.
On Mon, Apr 3, 2023 at 9:01 PM Vinoth Chandar <mail.vinoth.chan...@gmail.com> wrote: > +1 > > I was thinking that we add a new utility and NOT extend DeltaStreamer by > adding a Sink interface, for the following reasons > > - It will make it look like a generic Source => Sink ETL tool, which is > actually not our intention to support on Hudi. There are plenty of good > tools for that out there. > - the config management can get bit hard to understand, since we overload > ingest and reverse ETL into a single tool. So break it off at use-case > level? > > Thoughts? > > David: PMC does not have control over that. Please see unsubscribe > instructions here. https://hudi.apache.org/community/get-involved > Love to keep this thread about reverse streamer discussion. So kindly fork > another thread if you want to discuss unsubscribing. > > On Fri, Mar 31, 2023 at 1:47 AM Davidiam <david.rosa...@gmail.com> wrote: > > > Hello Vinoth, > > > > Can you please unsubscribe me? I have been trying to unsubscribe for > > months without success. > > > > Kind Regards, > > David > > > > Sent from Outlook for Android<https://aka.ms/AAb9ysg> > > ________________________________ > > From: Vinoth Chandar <vin...@apache.org> > > Sent: Friday, March 31, 2023 5:09:52 AM > > To: dev <dev@hudi.apache.org> > > Subject: [DISCUSS] Hudi Reverse Streamer > > > > Hi all, > > > > Any interest in building a reverse streaming tool, that does the reverse > of > > what the DeltaStreamer tool does? It will read Hudi table incrementally > > (only source) and write out the data to a variety of sinks - Kafka, JDBC > > Databases, DFS. > > > > This has come up many times with data warehouse users. Often times, they > > want to use Hudi to speed up or reduce costs on their data ingestion and > > ETL (using Spark/Flink), but want to move the derived data back into a > data > > warehouse or an operational database for serving. > > > > What do you all think? > > > > Thanks > > Vinoth > > >
