@Haisheng, could you please clarify what you mean by these points? > - the poor-design of column pruning, > - lack of group properties etc.
I guess I'm not aware of these problems. -- Kind Regards Roman Kondakov On 08.01.2020 02:21, Haisheng Yuan wrote: >> @Haisheng, are you doing something like that? > Kind of, but not exactly. It is about on-demand trait propagation. > > @Roman seems to be keen on space pruning for Calcite. But IMHO, for now, the > main reason of Calcite's poor performance is not lack of branch & bound space > puning, but > - rule applying on physical nodes, > - randomness of rule matching, > - the poor-design of column pruning, > - lack of on-demand trait propagation, > - lack of group properties etc. > > We tried a similar change with Roman's on our product. We totally removed > rule match importance and its comparison, split it into exploration, > implementation, enforcement 3 phases with specific top-down/bottom-up order, > it achieved almost 100% speedup. > Even @vlsi's RexNode normalization can improve it to some degree. > > Calcite currently generates only 1 join-order alternative for 6-way joins in > testJoinManyWay, not even top 10, 100 or N! ordering alternatives, but it > still can't finish within reasonable amount of time when abstract converter > is allowed. If there is only 1 join order alternative, the query optimizer > should finish the optimization quickly even for clique or chain queries with > 20 way joins, without space pruning. But this is not the case for Calcite. > > Simply put it, space pruning is important for optimization, especially for > join-reordering, but not an urgent issue for Calcite. > > - Haisheng > > ------------------------------------------------------------------ > 发件人：Roman Kondakov<kondako...@mail.ru.INVALID> > 日 期：2020年01月08日 02:39:19 > 收件人：<dev@calcite.apache.org> > 主 题：Re: [DISCUSS] Proposal to add API to force rules matching specific rels > > I forgot to mention that this approach was inspired by Stamatis's idea [1] > > [1] > https://ponymail-vm.apache.org/_GUI_/thread.html/d8f8bc0efd091c0750534ca5cd224f4dfe8940c9d0a99ce486516fd5@%3Cdev.calcite.apache.org%3E > >
