Re: mapPartitions

2021-03-04 Thread Debajyoti Roy
ke the mapPartitions can be implemented as a SELECT + GROUP BY, > where GROUP BY is to partition the data, then per partition computation is > handled by the SELECT. > > -Rui > > On Thu, Mar 4, 2021 at 11:56 AM Debajyoti Roy wrote: > > > Thanks again Julian. > > > >

Re: mapPartitions

2021-03-04 Thread Debajyoti Roy
on sorted or > partitioned data). mapPartitions is in this category. Of course a physical > implementation of one of SQL’s logical operators might use mapPartitions. > > Julian > > > > > > > On Mar 4, 2021, at 10:44 AM, Debajyoti Roy wrote: > > > > Thanks

Re: mapPartitions

2021-03-04 Thread Debajyoti Roy
) FROM my_twitter_data... So what is a better usage example for mapPartitions expressed as SQL ? I am really struggling with that part and I agree with Julian that is the key. Regards, Debajyoti Roy On Thu, Mar 4, 2021 at 12:01 AM Julian Hyde wrote: > I searched for mapPartitions and fla

mapPartitions

2021-03-03 Thread Debajyoti Roy
, (disclaimer: I am new to Calcite) Debajyoti Roy