[ https://issues.apache.org/jira/browse/UNOMI-739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Griffon updated UNOMI-739: -------------------------------- Description: Currently the configuration is set by: {code} # Purge all monthly indexes (sessions/events) that have been created for a specific number of months org.apache.unomi.monthly.index.purge.existTime=${env:UNOMI_MONTHLY_INDEX_PURGE_EXISTTIME:-12} {code} it has to be changed to allow a purge by type: session and event {code} org.apache.unomi.sessions.purge.existTime=${env:UNOMI_SESSIONS_PURGE_EXISTTIME:-} org.apache.unomi.events.purge.existTime=${env:UNOMI_EVENTS_PURGE_EXISTTIME:-} {code} *Old value should be deprecated and use as default if the new one are not set.* h2. Implementation 2 mechanisms need to be implemented in the purge of sessions / events: h3. Deleting old documents: There is a job that currently purge monthly indices and profiles, the goal is to replace the [monthly indices purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L402] by a dedicated method to remove events and sessions. This method will work as the [profile purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L333] calling {{persistenceService.removeByQuery(..)}}. Like in here: https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L365 *A log need to be added to mention the number of document removed.* h3. Removing empty indices. When documents are deleted, it will happen that some indices contain no document, we need to have a dedicated execution to remove the empty indices. *A log need to be added to mention the deleted indices and the remaining indices with the number of documents in each.* h2. Handle TODOs Look for UNOMI-726 todos within the code: - Remove index deletion from the rollup configuration. - Enable back and adapt integration test related to the purge. was: Currently the configuration is set by: {code} # Purge all monthly indexes (sessions/events) that have been created for a specific number of months org.apache.unomi.monthly.index.purge.existTime=${env:UNOMI_MONTHLY_INDEX_PURGE_EXISTTIME:-12} {code} it has to be changed to allow a purge by type: session and event {code} org.apache.unomi.sessions.purge.existTime=${env:UNOMI_SESSIONS_PURGE_EXISTTIME:-} org.apache.unomi.events.purge.existTime=${env:UNOMI_EVENTS_PURGE_EXISTTIME:-} {code} *Old value should be deprecated and use as default if the new one are not set.* h2. Implementation 2 mechanisms need to be implemented in the purge of sessions / events: h3. Deleting old documents: There is a job that currently purge monthly indices and profiles, the goal is to replace the [monthly indices purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L402] by a dedicated method to remove events and sessions. This method will work as the [profile purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L333] calling {{persistenceService.removeByQuery(..)}}. Like in here: https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L365 *A log need to be added to mention the number of document removed.* h3. Removing empty indices. When documents are deleted, it will happen that some indices contain no document, we need to have a dedicated execution to remove the empty indices. *A log need to be added to mention the deleted indices and the remaining indices with the number of documents in each.* h2. Handle TODOs - Remove index deletion from the rollup configuration. - Enable back and adapt integration test related to the purge. > Implemenation of purge of sessions / events following rollover usage. > --------------------------------------------------------------------- > > Key: UNOMI-739 > URL: https://issues.apache.org/jira/browse/UNOMI-739 > Project: Apache Unomi > Issue Type: New Feature > Reporter: David Griffon > Priority: Major > > Currently the configuration is set by: > {code} > # Purge all monthly indexes (sessions/events) that have been created for a > specific number of months > org.apache.unomi.monthly.index.purge.existTime=${env:UNOMI_MONTHLY_INDEX_PURGE_EXISTTIME:-12} > {code} > it has to be changed to allow a purge by type: session and event > {code} > org.apache.unomi.sessions.purge.existTime=${env:UNOMI_SESSIONS_PURGE_EXISTTIME:-} > org.apache.unomi.events.purge.existTime=${env:UNOMI_EVENTS_PURGE_EXISTTIME:-} > {code} > *Old value should be deprecated and use as default if the new one are not > set.* > h2. Implementation > 2 mechanisms need to be implemented in the purge of sessions / events: > h3. Deleting old documents: > There is a job that currently purge monthly indices and profiles, the goal is > to replace the [monthly indices > purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L402] > by a dedicated method to remove events and sessions. > This method will work as the [profile > purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L333] > calling {{persistenceService.removeByQuery(..)}}. > Like in here: > https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L365 > *A log need to be added to mention the number of document removed.* > h3. Removing empty indices. > When documents are deleted, it will happen that some indices contain no > document, we need to have a dedicated execution to remove the empty indices. > *A log need to be added to mention the deleted indices and the remaining > indices with the number of documents in each.* > h2. Handle TODOs > Look for UNOMI-726 todos within the code: > - Remove index deletion from the rollup configuration. > - Enable back and adapt integration test related to the purge. -- This message was sent by Atlassian Jira (v8.20.10#820010)