Abhilash L L created KYLIN-1844: ----------------------------------- Summary: High cardinality dimensions in memory Key: KYLIN-1844 URL: https://issues.apache.org/jira/browse/KYLIN-1844 Project: Kylin Issue Type: Improvement Components: Query Engine Affects Versions: v1.5.2, v1.2 Reporter: Abhilash L L Assignee: liyang
A whole dimension is kept in memory. We should have a way to keep only certain number / size of total rows to be kept in memory. A LRU cache for rows in the dimension will help keep memory in check. Why not store all the dimensions data in hbase in a different table with a prefix of dimensionid, and all calls to the dimensions (get based on dim key), is mapped to hbase. This does mean it will cost more time on a miss. -- This message was sent by Atlassian JIRA (v6.3.4#6332)