----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73076/ -----------------------------------------------------------
(Updated Dec. 16, 2020, 5:18 a.m.) Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian. Changes ------- Updates include: Updated design approach. Bugs: ATLAS-3919 https://issues.apache.org/jira/browse/ATLAS-3919 Repository: atlas Description (updated) ------- **Approach** Model: _TaskDef_ - Definition of task. - Contains status of the task for querying. _AbstractTask_ - Takes TaskDef as input and calls run. - The concrete tasks has logic for doing real work. _TaskFactory_ - Creates tasks given task type. - Enumerates all the tasks that are supported. _TaskManagement_ - Component/Service. Constructed after all existing components are constructed. - Provides a simple interface for adding tasks. - Processes tasks that are in pending state. - Supports HA mode, where the tasks are not executed on passive instance. _TaskManagement.Registry_ - CRUD operations with TaskDef vertex. _TaskManagement.Processor_ - Provides mechanism for provides single queue for processing tasks in a way that is non-blocking. - Uses task type and the associated TaskFactory to construct concrete task. **Concurrent Changes to Classifications** Via REST APIs and operations from web UI, it is possible to make changes to the same entity. This situation will be more pronounced in the new setup. The existing framework has _GraphTransactionInterceptor.lockObjectAndReleasePostCommit_. This provides ReentrantLock on an entity that is identified by the supplied GUID. This will cause a lock on the entity and all the impacted vertices. This has potential for adversely impacting scalability. Diffs ----- intg/src/main/java/org/apache/atlas/AtlasConfiguration.java ea9f26d47 intg/src/main/java/org/apache/atlas/model/tasks/TaskDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasType.java b30b483e0 intg/src/main/java/org/apache/atlas/utils/AtlasJson.java abeddf640 repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java 86b369fc9 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java b80e42eed repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 2cfcc0bc1 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapperWithTasks.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationPropagateTaskFactory.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationPropagationTasks.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java PRE-CREATION repository/src/main/java/org/apache/atlas/tasks/AbstractTask.java PRE-CREATION repository/src/main/java/org/apache/atlas/tasks/TaskExecutor.java PRE-CREATION repository/src/main/java/org/apache/atlas/tasks/TaskFactory.java PRE-CREATION repository/src/main/java/org/apache/atlas/tasks/TaskManagement.java PRE-CREATION repository/src/main/java/org/apache/atlas/util/BeanUtilRepo.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java 84e9bfa04 repository/src/test/java/org/apache/atlas/tasks/TaskExecutorTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/tasks/TaskManagementTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/tasks/TaskRegistryTest.java PRE-CREATION server-api/src/main/java/org/apache/atlas/RequestContext.java 32ffddde1 server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java bb7f8fcc4 webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java b20b40474 Diff: https://reviews.apache.org/r/73076/diff/6/ Testing (updated) ------- **Unit tests** New tests. **Manual tests** In-progress. **Volume test** Pending. **Pre-commit build** https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/279/ Thanks, Ashutosh Mestry