Re: [DISCUSS]Query performance Multiple and possible memory leaks

2019-01-16 Thread Mohammad Shahid Khan
@xuchuanyin Count star on carbon rely over the carbon index and carbonindexmerge so it will load the index files into the driver LRU cache. @litao If there is no other query and executing the same sql again then increasing memory footprint from 14g to 20g really looks like a memory leak. -Regards

Re: [DISCUSS]Query performance Multiple and possible memory leaks

2019-01-16 Thread litao
Do you mean for the same query, even if the memory configured for LRU is big enough, some of the cached entries for index still get purged? --yes i think it is. However, since your query is purely count-star without any conditions, I suppose the query may not use the index are all, which means th

Re: [DISCUSS]Query performance Multiple and possible memory leaks

2019-01-16 Thread ChuanYin Xu
Do you mean for the same query, even if the memory configured for LRU is big enough, some of the cached entries for index still get purged? If it is so, then there maybe some problems with the LRU cache I think. However, since your query is purely count-star without any conditions, I suppose th

Re: [DISCUSS]Query performance Multiple and possible memory leaks

2019-01-16 Thread ChuanYin Xu
Do you mean for the same query, even if the memory configured for LRU is big enough, some of the cached entries for index still get purged? If it is so, then there maybe some problems with the LRU cache I think. However, since your query is purely count-star without any conditions, I suppose th

[DISCUSS]Query performance Multiple and possible memory leaks

2019-01-16 Thread litao
I did an experiment these two days.Insert thousands of batches of data into a table use global_sort.I found that the first query use 4800s JDBC memery is 14BG;when first finish i do the second query immediately use the same sql "select count(*) from sameTable". The result is second query take 3700