+1 On Mon, Nov 14, 2016, 3:54 PM sujith chacko <sujithchacko.2...@gmail.com> wrote:
> Hi liang, > Yes, its for high cardinality columns. > Thanks, > Sujith > > On Nov 14, 2016 2:01 PM, "Liang Chen" <chenliang6...@gmail.com> wrote: > > > Hi > > > > I have one query : for no dictionary columns which are high cardinality > > like phone number, Whether the pruning cost is hight,or not ? > > > > Regards > > Liang > > > > 2016-11-14 15:18 GMT+08:00 sujith chacko <sujithchacko.2...@gmail.com>: > > > > > Hi All, > > > > > > I am going to optimize the LIKE Filter query flow for no-dictionary > > > columns, please find the details mentioned below. > > > > > > *Current design:* > > > For Like filter queries no push down is happening to carbon layer, > > because > > > of this there will be no block/blocklet level pruning which can happen > > > before applying the LIKE filters, this can add overhead while scanning > > > since the system has to scan all the blocks and blocklets in order to > > apply > > > filters. > > > > > > *Proposed design/solution:* > > > Like filters(startsWith,endsWith,contains) can be pushed to carbon > > engine > > > layer so that carbon can perform block and blocklet level pruning > inorder > > > before applying filters. > > > Block level pruning will be happening in driver side and blocklet level > > > pruning will be done in executer as per existing design. > > > > > > Requesting all to please provide valuable feedback and vote for > > > implementing the above solution inorder to improve Like Filter > Queries. > > > > > > Thanks, > > > Sujith > > > > > > > > > > > -- > > Regards > > Liang > > >