[
https://issues.apache.org/jira/browse/ATLAS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16532886#comment-16532886
]
Graham Wallis commented on ATLAS-2774:
--------------------------------------
Thanks David - useful comments
One slight change to the original description above, that became apparent when
I started to write some initial implementation and asked for clarification from
Mandy -
If Atlas is configured for soft delete, then an OMRS purgeEntity() or
purgeRelationship() will insist that the entity or relationship has been
soft-deleted before doing the purge. In this configuration an entity or
relationship must be in DELETED state otherwise purge will refuse to hard
delete. If Atlas is configured for hard delete, then purge will remove the
entity or relationship regardless of its state - so it will remove an entity or
relationship that is in ACTIVE (or DELETED state).
> Options for hard and soft delete of instances
> ---------------------------------------------
>
> Key: ATLAS-2774
> URL: https://issues.apache.org/jira/browse/ATLAS-2774
> Project: Atlas
> Issue Type: New Feature
> Reporter: Graham Wallis
> Priority: Major
>
> *Overview*
> For Entities and Relationships, the following delete behaviors are desired.
> Atlas can be configured to offer hard delete (only). Alternatively it can be
> configured to offer soft-delete. Configuration is achieved by setting the
> delete handler configuration property (see atlas-application.properties) to
> either the SoftDeleteHandlerV1 or the HardDeleteHandlerV1. The default (for
> when the property is not set) is to use the SoftDeleteHandlerV1.
> The AtlasEntityStoreV2 deleteById() and deleteByIds() and methods and the
> AtlasRelationshipStoreV2 deleteById() method are sensitive to the above
> configuration. When the configuration is soft these methods will perform a
> soft delete; when it is hard they will perform a hard delete.
> In addition to the above methods, the AtlasEntityStoreV2 and
> AtlasRelationshipStoreV2 offer a (new) purgeById() method that ALWAYS
> performs a hard delete. This is true regardless of which delete handler has
> been configured. When the configuration is hard, the purgeById() methods and
> deleteById() methods are essentially equivalent. When the configuration is
> soft, the purgeById() methods continue to provide a hard delete capability.
> The AtlasEntityStoreV2 deleteById(), deleteByIds() and purgeById() methods
> will delete the specified entities and any relationships to which they are
> connected. The AtlasRelationshipStoreV2 deleteById() and purgeById() methods
> will delete the specified relationship. Deletion of a relationship may cause
> an upate to an entity to which the relationhsip is connected, if it changes
> the propagation of classifications, for example.
> In a hard delete or purge operation, an affected entity and relationship will
> be removed from the graph and will not be returned in response to any future
> queries. This is true regardless of whether the affected entity or
> relationship has already been soft deleted or not. i.e. it's status could
> initially be ACTIVE or DELETED. Following the operation it will not exist.
> In a soft delete, the affected entities and relationships (that initially
> have status ACTIVE) are updated to set the status to DELETED. These instances
> can still be returned from queries and searches if the appropriate control is
> selected (to include deleted instances). Without that control selected they
> will not be returned.
> *Use Case for soft delete*
> Soft delete provides a 'safe' means of deleting instances from the repository
> whilst preserving an audit trail and (if supported by the repository)
> enabling a deleted instance to be restored (to ACTIVE status).
> *Use Cases for hard delete*
> One use case for hard delete is to permanently remove soft-deleted items some
> period of time after they were soft deleted.
> Another use case for hard delete is to is to correct (clean up after) a
> mistake in which a potentially large batch of incomplete/corrupted/wrong
> metadata is loaded and needs to be fully removed.
> *Use Scenario for an OMAS user*
> An OMAS provides a delete function that does NOT offer the user a choice of
> hard/soft. The OMAS's delete function will invoke the relevant soft delete
> method provided by the OMRS - deleteEntity or deleteRelationship. These are
> both soft delete methods.
> The Atlas OMRS Connector's deleteEntity/deleteRelationship methods will check
> whether Atlas is configured for hard or soft delete. If Atlas is configured
> for hard delete then soft-delete is not possible, and the method will throw a
> FunctionNotSupported exception which is caught by the OMAS.
> On catching this exception the OMAS author should automatically escalate the
> soft delete to a hard delete by calling either of the mandatory OMRS
> purgeEntity() or purgeRelationship() methods. In the Atlas OMRS Connector
> these methods will invoke AtlasEntityStoreV2 purgeById() or
> AtlasRelationshipStoreV2 purgeById(). The effect of this is that if soft
> delete is not possible, a hard delete is used instead.
> An OMAS does NOT provide a purge capability to a non-privileged user, but a
> suitably expert OMRS or Atlas repository administrator can explicitly issue a
> purge call at the OMRS or Atlas interface, by calling purgeEntity() or
> purgeRelationship() (or the relevant Atlas store's purgeById() method).
> *Use Scenario for an Administrator*
> As highlighted above, a suitably expert repository Administrator can invoke
> the purgeEntity(), purgeRelationship() methods directly. They may need to do
> this to permanently remove instances following an earlier soft-delete and the
> timeout of a period of grace, or following a failed import or batch update.
> In addition to the above direct use by a repository administrator of the
> 'purge' methods, an Administrator could alternatively use the deleteEntity(),
> deleteRelationship() methods to perform soft deletes. The invoked method will
> behave as described above for an OMAS - performing a soft-delete if possible
> and throwing a FunctionNotSupported exception otherwise. In the latter case
> (no support for soft-delete), the Administrator COULD then choose to issue a
> purgeEntity(), purgeRelationship() call.
> *Restore of an entity or relationship*
> If an instance has been deleted using a soft-delete, the object still exists
> but has a status of 'DELETED'. Whilst in the DELETED state, the object can be:
> * optionally included/excluded from search results
> * restored using the OMRS method for restoreEntity or restoreRelationship.
> * purged from the repository using the purgeEntity() or purgeRelationship()
> method (or the underlying Atlas store purgeById() method).
> The use case for restoring a soft-deleted entity or relationship is to
> correct an accidental deletion, or possibly a deliberately deletion that was
> performed for impact analysis.
> To restore an entity or relationship that has been soft-deleted, an OMAS user
> or repository administrator can use the restoreEntity() or
> restoreRealtionship() method of the OMRS connector.
> In the case of the Atlas OMRS Connector, the restoreEntity() and
> restoreRealtionship() methods will invoke (new) Atlas store methods to update
> the affected entities and relationships, leaving them in ACTIVE state and
> with classifications that reflect the propagation options between all the
> resulting active entities and relationships.
> *Within the Atlas Repository*
> The AtlasEntityStoreV2 and AtlasRelationshipStoreV2 provide additional
> methods for purgeById() that perform a hard delete, regardless of the choice
> of configured delete handler.
> The stores also provide additional methods to enable restore of an entity or
> relationship that has been soft-deleted.
> *Within the Atlas OMRS Connector*
> The Atlas OMRS Connector will interrogate the Atlas repository configuration
> to determine whether it is configuraed for soft or hard deletes. The Atlas
> OMRS Connector implementations of deleteEntity() and deleteRelationship()
> methods will therefore have prior knowledge as to whether it is feasible to
> attempt to perform a soft-delete. If feasible, the soft-delete will be issued
> to the relevant Atlas store. If not feasible, the methods will throw a
> FunctionNotSupported exception.
> Also within the Atlas OMRS Connector, the mandatory OMRS purgeEntity(),
> purgeRelationship() methods will invoke the Atlas stores' purgeById() methods
> to perform permanent (hard) removal of an entity or relationship from the
> repository. As outline above, these methods may be called either by an OMAS
> that has attempted a soft-delete and been 'bounced' or called directly by an
> Adminstrator.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
