Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2024-04-24 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/#review226400
---



Consider adding few samples and test cases, to make it easier to understand 
audit filter functioanlity.


intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java
Lines 221 (patched)


Consider annotating isDiscarded() and setDiscarded() with @JsonIgnore.



repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
Lines 462 (patched)


prepareDataObject() seems expensive - especially given it will be executed 
for each entity operation in Atlas. Are all fields of AtlasEntity used in rules?



repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java
Lines 74 (patched)


this.rules could be updated between #74 and #81, with the call to 
setRules() (#41). Consider adding the following line before #74:

final List rules = this.rules;



repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java
Lines 83 (patched)


This conversion seems expensive. Is this conversion necessary for every 
evalation of a rule?



repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java
Lines 97 (patched)


Rule evaluation continues even after ACCEPT result from a rule; is this 
necessary?



webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
Lines 1327 (patched)


Doesn't "/audits/rules/all" conflict with "/audits/rules/{ruleGuid}" (line 
#1269 above)?


- Madhan Neethiraj


On April 4, 2024, 5:14 a.m., Sheetal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74608/
> ---
> 
> (Updated April 4, 2024, 5:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
> Pawar, Radhika Kundam, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-4797
> https://issues.apache.org/jira/browse/ATLAS-4797
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently, Atlas has to persist all audit events without any constraints 
> which can make audit data grow exponentially after some time.
>  
> This feature is specifically to reduce Atlas audit storage where generation 
> of audit events can be controlled based on the user's use-case/criteria
> 
> Note: Audit data mentioned here is exclusive of Admin Audit Data which 
> persists in atlas_janus table. Filtering will be applicable to the entity 
> audit data which persists in ATLAS_ENTITY_AUDIT_EVENTS table.
> 
> Precommit details :
> Latest PC build (PASSED) : 
> build:https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1592/
> 
> 
> Diffs
> -
> 
>   addons/hbase-bridge/pom.xml 278b6c6d3 
>   addons/hbase-testing-util/pom.xml c4f3a99ee 
>   addons/hive-bridge-shim/pom.xml 39f16ceee 
>   addons/hive-bridge/pom.xml 356ac9542 
>   addons/kafka-bridge/pom.xml 093db7229 
>   addons/models/-Area0/0010-base_model.json a4a9248ec 
>   addons/sqoop-bridge/pom.xml 4b6eac98f 
>   authorization/pom.xml 7a1108487 
>   distro/src/conf/atlas-application.properties b5734d7a8 
>   graphdb/api/pom.xml 4ba89b20f 
>   graphdb/janus-hbase2/pom.xml 9ec087dbb 
>   intg/pom.xml 43a172c1b 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 0868f8b6e 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> 3afd27e7d 
>   repository/pom.xml 8fd744b44 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  9b011ab06 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  b7f0dd5e2 
>   repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
> e63152475 
>   
> repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
>  PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
> PRE-CREATION 
>   server-api/pom.xml de311c329 
>   test-tools/src/main/resources/solr/cor

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2024-04-24 Thread Jayendra Parab 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/#review226399
---


Ship it!




Ship It!

- Jayendra Parab


On April 4, 2024, 5:14 a.m., Sheetal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74608/
> ---
> 
> (Updated April 4, 2024, 5:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
> Pawar, Radhika Kundam, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-4797
> https://issues.apache.org/jira/browse/ATLAS-4797
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently, Atlas has to persist all audit events without any constraints 
> which can make audit data grow exponentially after some time.
>  
> This feature is specifically to reduce Atlas audit storage where generation 
> of audit events can be controlled based on the user's use-case/criteria
> 
> Note: Audit data mentioned here is exclusive of Admin Audit Data which 
> persists in atlas_janus table. Filtering will be applicable to the entity 
> audit data which persists in ATLAS_ENTITY_AUDIT_EVENTS table.
> 
> Precommit details :
> Latest PC build (PASSED) : 
> build:https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1592/
> 
> 
> Diffs
> -
> 
>   addons/hbase-bridge/pom.xml 278b6c6d3 
>   addons/hbase-testing-util/pom.xml c4f3a99ee 
>   addons/hive-bridge-shim/pom.xml 39f16ceee 
>   addons/hive-bridge/pom.xml 356ac9542 
>   addons/kafka-bridge/pom.xml 093db7229 
>   addons/models/-Area0/0010-base_model.json a4a9248ec 
>   addons/sqoop-bridge/pom.xml 4b6eac98f 
>   authorization/pom.xml 7a1108487 
>   distro/src/conf/atlas-application.properties b5734d7a8 
>   graphdb/api/pom.xml 4ba89b20f 
>   graphdb/janus-hbase2/pom.xml 9ec087dbb 
>   intg/pom.xml 43a172c1b 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 0868f8b6e 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> 3afd27e7d 
>   repository/pom.xml 8fd744b44 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  9b011ab06 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  b7f0dd5e2 
>   repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
> e63152475 
>   
> repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
>  PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
> PRE-CREATION 
>   server-api/pom.xml de311c329 
>   test-tools/src/main/resources/solr/core-template/solrconfig.xml 7a0e8dd16 
>   webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
> e19beccb1 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 0025276bb 
>   webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
> 268fca717 
> 
> 
> Diff: https://reviews.apache.org/r/74608/diff/13/
> 
> 
> Testing
> ---
> 
> Manual testing done
> 
> 
> Thanks,
> 
> Sheetal Shah
> 
>

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2024-04-24 Thread Radhika Kundam via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/#review226398
---


Ship it!




Ship It!

- Radhika Kundam


On April 3, 2024, 10:14 p.m., Sheetal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74608/
> ---
> 
> (Updated April 3, 2024, 10:14 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
> Pawar, Radhika Kundam, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-4797
> https://issues.apache.org/jira/browse/ATLAS-4797
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently, Atlas has to persist all audit events without any constraints 
> which can make audit data grow exponentially after some time.
>  
> This feature is specifically to reduce Atlas audit storage where generation 
> of audit events can be controlled based on the user's use-case/criteria
> 
> Note: Audit data mentioned here is exclusive of Admin Audit Data which 
> persists in atlas_janus table. Filtering will be applicable to the entity 
> audit data which persists in ATLAS_ENTITY_AUDIT_EVENTS table.
> 
> Precommit details :
> Latest PC build (PASSED) : 
> build:https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1592/
> 
> 
> Diffs
> -
> 
>   addons/hbase-bridge/pom.xml 278b6c6d3 
>   addons/hbase-testing-util/pom.xml c4f3a99ee 
>   addons/hive-bridge-shim/pom.xml 39f16ceee 
>   addons/hive-bridge/pom.xml 356ac9542 
>   addons/kafka-bridge/pom.xml 093db7229 
>   addons/models/-Area0/0010-base_model.json a4a9248ec 
>   addons/sqoop-bridge/pom.xml 4b6eac98f 
>   authorization/pom.xml 7a1108487 
>   distro/src/conf/atlas-application.properties b5734d7a8 
>   graphdb/api/pom.xml 4ba89b20f 
>   graphdb/janus-hbase2/pom.xml 9ec087dbb 
>   intg/pom.xml 43a172c1b 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 0868f8b6e 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> 3afd27e7d 
>   repository/pom.xml 8fd744b44 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  9b011ab06 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  b7f0dd5e2 
>   repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
> e63152475 
>   
> repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
>  PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
> PRE-CREATION 
>   server-api/pom.xml de311c329 
>   test-tools/src/main/resources/solr/core-template/solrconfig.xml 7a0e8dd16 
>   webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
> e19beccb1 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 0025276bb 
>   webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
> 268fca717 
> 
> 
> Diff: https://reviews.apache.org/r/74608/diff/13/
> 
> 
> Testing
> ---
> 
> Manual testing done
> 
> 
> Thanks,
> 
> Sheetal Shah
> 
>

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2024-04-03 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated April 4, 2024, 10:44 a.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, Radhika Kundam, and Sidharth Mishra.


Changes
---

corrected operator case sensitivity logic


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description (updated)
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.

Precommit details :
Latest PC build (PASSED) : 
build:https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1592/


Diffs (updated)
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml 9ec087dbb 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 0868f8b6e 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 b7f0dd5e2 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 7a0e8dd16 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
e19beccb1 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
0025276bb 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
268fca717 


Diff: https://reviews.apache.org/r/74608/diff/13/

Changes: https://reviews.apache.org/r/74608/diff/12-13/


Testing
---

Manual testing done


Thanks,

Sheetal Shah

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2024-03-28 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated March 29, 2024, 8:01 a.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, Radhika Kundam, and Sidharth Mishra.


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.

Precommit details :
Latest PC build: 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1473
Failed tests: EntityV2JerseyResourceIT.testSetLabelsByTypeName:986 expected [2] 
but found [1]


Diffs (updated)
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml 9ec087dbb 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 0868f8b6e 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 b7f0dd5e2 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 7a0e8dd16 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
e19beccb1 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
0025276bb 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
268fca717 


Diff: https://reviews.apache.org/r/74608/diff/11/

Changes: https://reviews.apache.org/r/74608/diff/10-11/


Testing
---

Manual testing done


Thanks,

Sheetal Shah

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2023-10-25 Thread Sidharth Mishra 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/#review225896
---




intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java
Lines 122 (patched)


Kindly format with the correct indentation



repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java
Lines 39 (patched)


Kindly format with the correct indentation



repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java
Lines 117 (patched)


Kindly format with the correct indentation.



repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
Lines 209 (patched)


isValid can be avoided here like below for better readability.

private void validateRuleAction(String action) throws AtlasBaseException {
if (StringUtils.isNotEmpty(action)) {
for (RuleAction.Result res : RuleAction.Result.values()){
if (res.name().equals(action)) {
return;
}
}
throw new AtlasBaseException(AtlasErrorCode.INVALID_RULE_ACTION);
}
}



repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
Lines 224 (patched)


Its very big function and should be broken down like - 

private void validateRuleExprFormat(AtlasRule.RuleExpr ruleExpr) throws 
AtlasBaseException {
if (ruleExpr == null) {
return;
}

List allExpressions = 
ruleExpr.getRuleExprObjList();
List> recordedTypeNamesList = new ArrayList<>();

for (AtlasRule.RuleExprObject ruleExprObj : allExpressions) {
validateTypeName(ruleExprObj.getTypeName(), recordedTypeNamesList);

AtlasRule.Condition condition = ruleExprObj.getCondition();
List criterion = 
ruleExprObj.getCriterion();
validateConditionAndCriteria(condition, criterion);

validateAttributes(ruleExprObj.getTypeName(), condition, criterion);
}
}

private void validateTypeName(String typeName, List> 
recordedTypeNamesList) throws AtlasBaseException {
if (Strings.isNullOrEmpty(typeName) || "null".equals(typeName)) {
throw new 
AtlasBaseException(AtlasErrorCode.MISSING_MANDATORY_TYPENAME_IN_RULE_EXPR);
}

if (isDuplicateTypeNameValue(recordedTypeNamesList, typeName)) {
throw new 
AtlasBaseException(AtlasErrorCode.DUPLICATE_TYPENAME_IN_RULE_EXPR, typeName);
}

recordedTypeNamesList.add(Arrays.asList(typeName.split(","));

getEntityTypes(typeName);
}

private void validateConditionAndCriteria(AtlasRule.Condition condition, 
List criterion) throws AtlasBaseException {
if (condition == null && CollectionUtils.isEmpty(criterion)) {
throw new 
AtlasBaseException(AtlasErrorCode.MISSING_CRITERIA_CONDITION, "condition and 
criteria");
}
}

private void validateAttributes(String typeName, AtlasRule.Condition 
condition, List criterion) throws 
AtlasBaseException {
Set entityTypes = getEntityTypes(typeName);

for (AtlasEntityType entityType : entityTypes) {
if (condition != null && CollectionUtils.isNotEmpty(criterion)) {
validateCriteriaList(entityType, criterion);
} else {
validateExpression(entityType, ruleExprObj.getOperator(), 
ruleExprObj.getAttributeName(),  ruleExprObj.getAttributeValue() );
}
}
}



repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
Lines 292 (patched)


Kinldy consider to refactor this like-

private void validateExternalAttribute(String attrName, String attrValue, 
AtlasRule.RuleExprObject.Operator operator) throws AtlasBaseException {
if (ATTR_OPERATION_TYPE.equals(attrName)) {
EntityAuditEventV2.EntityAuditActionV2[] enumConstants = 
EntityAuditEventV2.EntityAuditActionV2.class.getEnumConstants();

if (isValidOperator(enumConstants, operator, attrValue)) {
return;
}
}

throw new 
AtlasBaseException(AtlasErrorCode.INVALID_OPERATOR_ON_ATTRIBUTE, 
operator.getSymbol(), ATTR_OPERATION_TYPE);
}

private boolean isValidOperator(EntityAuditEventV2.EntityAuditActionV2[] 
enumConstants, AtlasRule.RuleExprObject.Operator operator, String attrValue) {
switch (operator) {
case EQ:
return

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2023-10-09 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated Oct. 10, 2023, 9:16 a.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, and Radhika Kundam.


Changes
---

Code improvements done


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.

Precommit details :
Latest PC build: 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1473
Failed tests: EntityV2JerseyResourceIT.testSetLabelsByTypeName:986 expected [2] 
but found [1]


Diffs (updated)
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml c2a2e74f0 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java df886753f 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 a99f9b383 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 1550052b4 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
a25a51b5a 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
b19095b48 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
a4d794615 


Diff: https://reviews.apache.org/r/74608/diff/10/

Changes: https://reviews.apache.org/r/74608/diff/9-10/


Testing
---

Manual testing done


Thanks,

Sheetal Shah

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2023-10-04 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated Oct. 4, 2023, 1:21 p.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, and Radhika Kundam.


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description (updated)
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.

Precommit details :
Latest PC build: 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1473
Failed tests: EntityV2JerseyResourceIT.testSetLabelsByTypeName:986 expected [2] 
but found [1]


Diffs
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml c2a2e74f0 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java df886753f 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 a99f9b383 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 1550052b4 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
a25a51b5a 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
b19095b48 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
a4d794615 


Diff: https://reviews.apache.org/r/74608/diff/9/


Testing
---

Manual testing done


Thanks,

Sheetal Shah

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2023-10-04 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated Oct. 4, 2023, 1:19 p.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, and Radhika Kundam.


Changes
---

Latest PC build: 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1473
Failed tests: EntityV2JerseyResourceIT.testSetLabelsByTypeName:986 expected [2] 
but found [1]


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.


Diffs (updated)
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml c2a2e74f0 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java df886753f 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 a99f9b383 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 1550052b4 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
a25a51b5a 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
b19095b48 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
a4d794615 


Diff: https://reviews.apache.org/r/74608/diff/9/

Changes: https://reviews.apache.org/r/74608/diff/8-9/


Testing
---

Manual testing done


Thanks,

Sheetal Shah

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2023-09-27 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated Sept. 27, 2023, 6:06 p.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, and Radhika Kundam.


Changes
---

changed catch exception mesage


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.


Diffs (updated)
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml c2a2e74f0 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java df886753f 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 a99f9b383 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 1550052b4 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
a25a51b5a 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
b19095b48 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
a4d794615 


Diff: https://reviews.apache.org/r/74608/diff/8/

Changes: https://reviews.apache.org/r/74608/diff/7-8/


Testing
---

Manual testing done


Thanks,

Sheetal Shah

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2023-09-27 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated Sept. 27, 2023, 2:26 p.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, and Radhika Kundam.


Changes
---

Continue evaluating remaining rules if error occurs in any rule execution


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.


Diffs (updated)
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml c2a2e74f0 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java df886753f 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 a99f9b383 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 1550052b4 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
a25a51b5a 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
b19095b48 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
a4d794615 


Diff: https://reviews.apache.org/r/74608/diff/7/

Changes: https://reviews.apache.org/r/74608/diff/6-7/


Testing
---

Manual testing done


Thanks,

Sheetal Shah

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2023-09-26 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated Sept. 26, 2023, 3:36 p.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, and Radhika Kundam.


Changes
---

isNull and notNull operations corrected


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.


Diffs (updated)
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml c2a2e74f0 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java df886753f 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 a99f9b383 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 1550052b4 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
a25a51b5a 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
b19095b48 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
a4d794615 


Diff: https://reviews.apache.org/r/74608/diff/6/

Changes: https://reviews.apache.org/r/74608/diff/5-6/


Testing
---

Manual testing done


Thanks,

Sheetal Shah

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2023-09-25 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated Sept. 26, 2023, 9:04 a.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, and Radhika Kundam.


Changes
---

solrconfig.xml corrected


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.


Diffs (updated)
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml c2a2e74f0 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java df886753f 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 a99f9b383 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 1550052b4 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
a25a51b5a 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
b19095b48 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
a4d794615 


Diff: https://reviews.apache.org/r/74608/diff/5/

Changes: https://reviews.apache.org/r/74608/diff/4-5/


Testing
---

Manual testing done


Thanks,

Sheetal Shah

Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2023-09-25 Thread Sheetal Shah 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/
---

(Updated Sept. 25, 2023, 6:07 p.m.)


Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
Pawar, and Radhika Kundam.


Changes
---

solrconfig.xml changed for FreeTextSearchProcessor; isIndexable set to false 
for ruleExpr


Bugs: ATLAS-4797
https://issues.apache.org/jira/browse/ATLAS-4797


Repository: atlas


Description
---

Currently, Atlas has to persist all audit events without any constraints which 
can make audit data grow exponentially after some time.
 
This feature is specifically to reduce Atlas audit storage where generation of 
audit events can be controlled based on the user's use-case/criteria

Note: Audit data mentioned here is exclusive of Admin Audit Data which persists 
in atlas_janus table. Filtering will be applicable to the entity audit data 
which persists in ATLAS_ENTITY_AUDIT_EVENTS table.


Diffs (updated)
-

  addons/hbase-bridge/pom.xml 278b6c6d3 
  addons/hbase-testing-util/pom.xml c4f3a99ee 
  addons/hive-bridge-shim/pom.xml 39f16ceee 
  addons/hive-bridge/pom.xml 356ac9542 
  addons/kafka-bridge/pom.xml 093db7229 
  addons/models/-Area0/0010-base_model.json a4a9248ec 
  addons/sqoop-bridge/pom.xml 4b6eac98f 
  authorization/pom.xml 7a1108487 
  distro/src/conf/atlas-application.properties b5734d7a8 
  graphdb/api/pom.xml 4ba89b20f 
  graphdb/janus-hbase2/pom.xml c2a2e74f0 
  intg/pom.xml 43a172c1b 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java df886753f 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
3afd27e7d 
  repository/pom.xml 8fd744b44 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 9b011ab06 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 a99f9b383 
  repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
e63152475 
  
repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
 PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java 
PRE-CREATION 
  repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java 
PRE-CREATION 
  server-api/pom.xml de311c329 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml 1550052b4 
  webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
a25a51b5a 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
b19095b48 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
a4d794615 


Diff: https://reviews.apache.org/r/74608/diff/4/

Changes: https://reviews.apache.org/r/74608/diff/3-4/


Testing
---

Manual testing done


Thanks,

Sheetal Shah