Is it the traditional bitmap indexing? I would not recommend it for big data. You could use bloom filters and min/max indexes in-memory which look to be more appropriate. However, if you want to use bitmap indexes then you would have to do it as you say. However, bitmap indexes may consume a lot of memory, so I am not sure that simply caching them in-memory is desired.
> On 29 Jun 2016, at 19:49, Nishadi Kirielle <ndime...@gmail.com> wrote: > > Hi All, > > I am a CSE undergraduate and as for our final year project, we are expecting > to construct a cluster based, bit-oriented analytic platform (storage engine) > to provide fast query performance when used for OLAP with the use of novel > bitmap indexing techniques when and where appropriate. > > For that we are expecting to use Spark SQL. We will need to implement a way > to cache the bit map indexes and in-cooperate the use of bitmap indexing at > the catalyst optimizer level when it is possible. > > I would highly appreciate your feedback regarding the proposed approach. > > Thank you & Regards > > Nishadi Kirielle > Department of Computer Science and Engineering > University of Moratuwa > Sri Lanka --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
