Re: [DISCUSS] SPIP: Structured Streaming - Arbitrary State API v2

2023-11-27 Thread Jungtaek Lim
Kindly bump for better reach after the long holiday. Please kindly review the proposal which opens the chance to address complex use cases of streaming. Thanks! On Thu, Nov 23, 2023 at 8:19 AM Jungtaek Lim wrote: > Thanks Anish for proposing SPIP and initiating this thread! I believe this >

Re: [VOTE] SPIP: Testing Framework for Spark UI Javascript files

2023-11-27 Thread Binjie Yang
+ 1 Thanks, Binjie Yang On 2023/11/27 02:27:22 Ruifeng Zheng wrote: > +1 > > On Sun, Nov 26, 2023 at 6:58 AM Gengliang Wang wrote: > > > +1 > > > > On Sat, Nov 25, 2023 at 2:50 AM yangjie01 > > wrote: > > > >> +1 > >> > >> > >> > >> *发件人**: *Reynold Xin > >> *日期**: *2023年11月25日 星期六 14:35 >

Join push down in DSv2

2023-11-27 Thread Stefan Hagedorn
Hi, At the Spark Summit 2017 Ioana Delaney presented an approach for join pushdown in Apache Spark [1]. Is there any intent to actually bring this into Spark, especially in the DSv2 interface? Does anyone know if there's ongoing work or a document about this? [1]