@Raghunandan subramanya <raghunandan.subrama...@gmail.com> We have tested with *80 string columns with 10 high cardinality columns(fallback happened for these columns)*, please find the stats:
*Test result is with 1 billion records 385 Gb size* *1. Load time without local dictionary:* 66 minutes *2. Load time without fallback local dictionary:* 72 minutes *3. Load time with fallback local dictionary:* 74 minutes *Without fallback local dictionary:* 9.09% degradation *With fallback local dictionary:* 13.63% -Regards Kumar Vishal On Fri, Dec 7, 2018 at 12:59 PM Raghunandan S < carbondatacontributi...@gmail.com> wrote: > @kumar vishal what is the fallback performance if more number of columns > need to fallback. Would it not increase the overhead of generating > temporary dictionary and discarding it? > > On Fri, 7 Dec 2018, 12:56 pm ravipesala, <ravi.pes...@gmail.com> wrote: > > > > > I agree with @kumarvishal , better not add more options as it confuses > the > > user. We better fallback automatically depends on the size of the > > dictionary. > > > > > > > > -- > > Sent from: > > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/ > > >
