[
https://issues.apache.org/jira/browse/LENS-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajat Khandelwal updated LENS-281:
----------------------------------
Description:
For each fact-storage table, for each of fact-storage table's update periods,
for each of fact-storage table's time partitioning column, maintain a Timeline
data structure which records all the time partitions existing for the
combination in memory. This info will be cached in memory for server lifecycle.
For the first time, it will be evaluated from all existing partitions and
stored in table properties. For next server restarts, this will be read from
the properties. all partition operations(add/delete/latest/...) will update the
timeline and use it as best as they can.
For the Timeline Data structure, the following implementations come to mind:
* Store list of all partitions.
* Store first, last and in-between holes.
* Store as an interval tree/segment tree.
As part of this jira, I intend to provide an interface, atleast one
implementation and (maybe) configurability of the implementation.
was:To reduce number of look ups, we need to keep track of what the first
partition is for each storage table of fact table.
> Create Partition Timeline and keep it cached and stored as storage table
> properties
> -----------------------------------------------------------------------------------
>
> Key: LENS-281
> URL: https://issues.apache.org/jira/browse/LENS-281
> Project: Apache Lens
> Issue Type: Sub-task
> Components: cube
> Reporter: Amareshwari Sriramadasu
> Fix For: 2.1
>
>
> For each fact-storage table, for each of fact-storage table's update periods,
> for each of fact-storage table's time partitioning column, maintain a
> Timeline data structure which records all the time partitions existing for
> the combination in memory. This info will be cached in memory for server
> lifecycle. For the first time, it will be evaluated from all existing
> partitions and stored in table properties. For next server restarts, this
> will be read from the properties. all partition
> operations(add/delete/latest/...) will update the timeline and use it as best
> as they can.
> For the Timeline Data structure, the following implementations come to mind:
> * Store list of all partitions.
> * Store first, last and in-between holes.
> * Store as an interval tree/segment tree.
> As part of this jira, I intend to provide an interface, atleast one
> implementation and (maybe) configurability of the implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
