some other transformation conditionally
> based
> > > on
> > > > > whether upsert/insert happened, right?
> > > > > and I could do that without losing any of the existing
> transformation
> > > > > methods I know in Spark. I am not quite clear on
s desirable. Typical scenarios I
> > > have
> > > > seen, job A ingests data into table A, job B
> > > > incrementally queries table A and kicks another ETL to build table B.
> > > Job A
> > > > and B are typically different and written
m tables. What I am trying to say is
> -
> > > there is definitely gaps we would like to improve upon to make
> > incremental
> > > processing mainstream, not sure if the proposed
> > > APIs are the highest on that list.
> > >
> > > Apologies if I
g. Please help me understand if so.
> >
> > Thanks
> > Vinoth
> >
> >
> >
> >
> > On Tue, Sep 1, 2020 at 4:26 AM vino yang wrote:
> >
> > > Hi,
> > >
> > > Does anyone have ideas or disagreements?
> > >
> >
> > > +1
> > > This will give hudi more capabilities besides data ingestion and
> writing,
> > > and make hudi-based data processing more timely!
> > > Best,
> > > wangxianghu
> > >
> > > 发件人: Abhishek Modi
> > > 发送时间: 2020年
> +1
> > This will give hudi more capabilities besides data ingestion and writing,
> > and make hudi-based data processing more timely!
> > Best,
> > wangxianghu
> >
> > 发件人: Abhishek Modi
> > 发送时间: 2020年8月31日 15:01
> > 收件人: dev@hudi.apac
; This will give hudi more capabilities besides data ingestion and writing,
> and make hudi-based data processing more timely!
> Best,
> wangxianghu
>
> 发件人: Abhishek Modi
> 发送时间: 2020年8月31日 15:01
> 收件人: dev@hudi.apache.org
> 主题: Re: [DISCUSS] Introduce incremental processing
+1
This will give hudi more capabilities besides data ingestion and writing, and
make hudi-based data processing more timely!
Best,
wangxianghu
发件人: Abhishek Modi
发送时间: 2020年8月31日 15:01
收件人: dev@hudi.apache.org
主题: Re: [DISCUSS] Introduce incremental processing API in Hudi
+1
This sounds
+1
This sounds really interesting! I like that this implicitly gives Hudi the
ability to do transformations on ingested data :)
On Sun, Aug 30, 2020 at 10:59 PM vino yang wrote:
> Hi everyone,
>
>
> For a long time, in the field of big data, people hope that the tools they
> use can give greate
Hi everyone,
For a long time, in the field of big data, people hope that the tools they
use can give greater play to the processing and analysis capabilities of
big data. At present, from the perspective of API, Hudi mostly provides
APIs related to data ingestion, and relies on various big data q
10 matches
Mail list logo
