d throws exception when received the
> > filtering data source event.
> > >
> > > Best,
> > > Yun Gao
> > >
> > >
> > >
> > > [1]
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-168%3A+Speculative+Execution+for+Batch+Job
> > >
> > >
> > >
> > > --
t; > Best,
> > Yun Gao
> >
> >
> >
> > [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-168%3A+Speculative+Execution+for+Batch+Job
> >
> >
> >
> > ------
> &g
> > Yun Gao
> > >
> > >
> > >
> > > [1]
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-168%3A+Speculative+Execution+for+Batch+Job
> > >
> > >
> > >
> > >
st,
> > Yun Gao
> >
> >
> >
> > [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-168%3A+Speculative+Execution+for+Batch+Job
> >
> >
> >
> > ------
> > From:Jing Ge
>
---
> From:Jing Ge
> Send Time:2022 Jul. 21 (Thu.) 18:56
> To:dev
> Subject:Re: [DISCUSS] FLIP-248: Introduce dynamic partition pruning
>
> Hi,
>
> Thanks for the informative discussion! Looking forward to using dynamic
> filtering provided by Flink.
>
> Best regar
--
From:Jing Ge
Send Time:2022 Jul. 21 (Thu.) 18:56
To:dev
Subject:Re: [DISCUSS] FLIP-248: Introduce dynamic partition pruning
Hi,
Thanks for the informative discussion! Looking forward to using dynamic
filtering provided by Flink.
Best regards,
Jing
Hi,
Thanks for the informative discussion! Looking forward to using dynamic
filtering provided by Flink.
Best regards,
Jing
On Tue, Jul 19, 2022 at 3:22 AM godfrey he wrote:
> Hi, Jingong, Jark, Jing,
>
> Thanks for for the important inputs.
> Lake storage is a very important scenario, and con
Hi, Jingong, Jark, Jing,
Thanks for for the important inputs.
Lake storage is a very important scenario, and consider more generic
and extended case,
I also would like to use "dynamic filtering" concept instead of
"dynamic partition".
>maybe the FLIP should also demonstrate the EXPLAIN result, wh
Hi Godfrey,
Thanks for driving this discussion.
This is an important improvement for batch sql jobs.
I agree with Jingsong to expand the capability to more than just partitions.
Besides, I have two points:
1. Based on FLIP-248[1],
> Dynamic partition pruning mechanism can improve performance by av
I agree with Jingsong. DPP is a particular case of Dynamic Filter Pushdown
that the join key contains partition fields. Extending this FLIP to
general filter
pushdown can benefit more optimizations, and they can share the same
interface.
For example, Trino Hive Connector leverages dynamic filteri
Thanks Godfrey for driving.
I like this FLIP.
We can restrict this capability to more than just partitions.
Here are some inputs from Lake Storage.
The format of the splits generated by Lake Storage is roughly as follows:
Split {
Path filePath;
Statistics[] fieldStats;
}
Stats contain the
Hi all,
I would like to open a discussion on FLIP-248: Introduce dynamic
partition pruning.
Currently, Flink supports static partition pruning: the conditions in
the WHERE clause are analyzed
to determine in advance which partitions can be safely skipped in the
optimization phase.
Another common
12 matches
Mail list logo
