[ https://issues.apache.org/jira/browse/ATLAS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Mestry reassigned ATLAS-2774: -------------------------------------- Assignee: Ashutosh Mestry > Options for hard and soft delete of instances > --------------------------------------------- > > Key: ATLAS-2774 > URL: https://issues.apache.org/jira/browse/ATLAS-2774 > Project: Atlas > Issue Type: New Feature > Reporter: Graham Wallis > Assignee: Ashutosh Mestry > Priority: Major > Attachments: ATLAS-2774-Soft-and-hard-delete-impl.patch > > > *Overview* > For Entities and Relationships, the following delete behaviors are desired. > Atlas can be configured to offer hard delete (only). Alternatively it can be > configured to offer soft-delete. Configuration is achieved by setting the > delete handler configuration property (see atlas-application.properties) to > either the SoftDeleteHandlerV1 or the HardDeleteHandlerV1. The default (for > when the property is not set) is to use the SoftDeleteHandlerV1. > The AtlasEntityStoreV2 deleteById() and deleteByIds() and methods and the > AtlasRelationshipStoreV2 deleteById() method are sensitive to the above > configuration. When the configuration is soft these methods will perform a > soft delete; when it is hard they will perform a hard delete. > In addition to the above methods, the AtlasEntityStoreV2 and > AtlasRelationshipStoreV2 offer a (new) purgeById() method that ALWAYS > performs a hard delete. This is true regardless of which delete handler has > been configured. When the configuration is hard, the purgeById() methods and > deleteById() methods are essentially equivalent. When the configuration is > soft, the purgeById() methods continue to provide a hard delete capability. > The AtlasEntityStoreV2 deleteById(), deleteByIds() and purgeById() methods > will delete the specified entities and any relationships to which they are > connected. The AtlasRelationshipStoreV2 deleteById() and purgeById() methods > will delete the specified relationship. Deletion of a relationship may cause > an upate to an entity to which the relationhsip is connected, if it changes > the propagation of classifications, for example. > In a hard delete or purge operation, an affected entity and relationship will > be removed from the graph and will not be returned in response to any future > queries. This is true regardless of whether the affected entity or > relationship has already been soft deleted or not. i.e. it's status could > initially be ACTIVE or DELETED. Following the operation it will not exist. > In a soft delete, the affected entities and relationships (that initially > have status ACTIVE) are updated to set the status to DELETED. These instances > can still be returned from queries and searches if the appropriate control is > selected (to include deleted instances). Without that control selected they > will not be returned. > *Use Case for soft delete* > Soft delete provides a 'safe' means of deleting instances from the repository > whilst preserving an audit trail and (if supported by the repository) > enabling a deleted instance to be restored (to ACTIVE status). > *Use Cases for hard delete* > One use case for hard delete is to permanently remove soft-deleted items some > period of time after they were soft deleted. > Another use case for hard delete is to is to correct (clean up after) a > mistake in which a potentially large batch of incomplete/corrupted/wrong > metadata is loaded and needs to be fully removed. > *Use Scenario for an OMAS user* > An OMAS provides a delete function that does NOT offer the user a choice of > hard/soft. The OMAS's delete function will invoke the relevant soft delete > method provided by the OMRS - deleteEntity or deleteRelationship. These are > both soft delete methods. > The Atlas OMRS Connector's deleteEntity/deleteRelationship methods will check > whether Atlas is configured for hard or soft delete. If Atlas is configured > for hard delete then soft-delete is not possible, and the method will throw a > FunctionNotSupported exception which is caught by the OMAS. > On catching this exception the OMAS author should automatically escalate the > soft delete to a hard delete by calling either of the mandatory OMRS > purgeEntity() or purgeRelationship() methods. In the Atlas OMRS Connector > these methods will invoke AtlasEntityStoreV2 purgeById() or > AtlasRelationshipStoreV2 purgeById(). The effect of this is that if soft > delete is not possible, a hard delete is used instead. > An OMAS does NOT provide a purge capability to a non-privileged user, but a > suitably expert OMRS or Atlas repository administrator can explicitly issue a > purge call at the OMRS or Atlas interface, by calling purgeEntity() or > purgeRelationship() (or the relevant Atlas store's purgeById() method). > *Use Scenario for an Administrator* > As highlighted above, a suitably expert repository Administrator can invoke > the purgeEntity(), purgeRelationship() methods directly. They may need to do > this to permanently remove instances following an earlier soft-delete and the > timeout of a period of grace, or following a failed import or batch update. > In addition to the above direct use by a repository administrator of the > 'purge' methods, an Administrator could alternatively use the deleteEntity(), > deleteRelationship() methods to perform soft deletes. The invoked method will > behave as described above for an OMAS - performing a soft-delete if possible > and throwing a FunctionNotSupported exception otherwise. In the latter case > (no support for soft-delete), the Administrator COULD then choose to issue a > purgeEntity(), purgeRelationship() call. > *Restore of an entity or relationship* > If an instance has been deleted using a soft-delete, the object still exists > but has a status of 'DELETED'. Whilst in the DELETED state, the object can be: > * optionally included/excluded from search results > * restored using the OMRS method for restoreEntity or restoreRelationship. > * purged from the repository using the purgeEntity() or purgeRelationship() > method (or the underlying Atlas store purgeById() method). > The use case for restoring a soft-deleted entity or relationship is to > correct an accidental deletion, or possibly a deliberately deletion that was > performed for impact analysis. > To restore an entity or relationship that has been soft-deleted, an OMAS user > or repository administrator can use the restoreEntity() or > restoreRealtionship() method of the OMRS connector. > In the case of the Atlas OMRS Connector, the restoreEntity() and > restoreRealtionship() methods will invoke (new) Atlas store methods to update > the affected entities and relationships, leaving them in ACTIVE state and > with classifications that reflect the propagation options between all the > resulting active entities and relationships. > *Within the Atlas Repository* > The AtlasEntityStoreV2 and AtlasRelationshipStoreV2 provide additional > methods for purgeById() that perform a hard delete, regardless of the choice > of configured delete handler. > The stores also provide additional methods to enable restore of an entity or > relationship that has been soft-deleted. > *Within the Atlas OMRS Connector* > The Atlas OMRS Connector will interrogate the Atlas repository configuration > to determine whether it is configuraed for soft or hard deletes. The Atlas > OMRS Connector implementations of deleteEntity() and deleteRelationship() > methods will therefore have prior knowledge as to whether it is feasible to > attempt to perform a soft-delete. If feasible, the soft-delete will be issued > to the relevant Atlas store. If not feasible, the methods will throw a > FunctionNotSupported exception. > Also within the Atlas OMRS Connector, the mandatory OMRS purgeEntity(), > purgeRelationship() methods will invoke the Atlas stores' purgeById() methods > to perform permanent (hard) removal of an entity or relationship from the > repository. As outline above, these methods may be called either by an OMAS > that has attempted a soft-delete and been 'bounced' or called directly by an > Adminstrator. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)