[ https://issues.apache.org/jira/browse/PHOENIX-5943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chinmay Kulkarni updated PHOENIX-5943: -------------------------------------- Fix Version/s: (was: 4.16.0) 4.16.1 > Consider splitting up SYSTEM.TASK into 2 tables - one for queued jobs and one > for task history > ---------------------------------------------------------------------------------------------- > > Key: PHOENIX-5943 > URL: https://issues.apache.org/jira/browse/PHOENIX-5943 > Project: Phoenix > Issue Type: Improvement > Affects Versions: 4.15.0 > Reporter: Chinmay Kulkarni > Priority: Major > Fix For: 5.1.0, 4.16.1 > > > We can split SYSTEM.TASK into 2 tables: > # SYSTEM.TASK_QUEUE: used as an ephemeral task queue with potentially a > smaller TTL which contains tasks that are new and yet to be picked up. > # SYSTEM.TASK_HISTORY: used to store tasks that have already been picked up > and may or may not have succeeded, along with more data about the task (stack > trace, job IDs, etc.). This table can have a longer TTL and can be used for > auditing/debugging purposes. > The advantages of doing this are: > * Makes SYSTEM.TASK_QUEUE more lightweight. > * Less data scanned when picking up new tasks. > * Combined with a PK change, we can introduce a natural priority queue using > the TASK_PRIORITY field. > * Different TTL value on the TASK_HISTORY table so we can maintain past > tasks for a longer period of time. > * Potentially makes this more in-line with other use-cases that want to > maintain a similar kind of task and task history table (for ex: View TTL's > async delete functionality). -- This message was sent by Atlassian Jira (v8.3.4#803005)