[jira] [Updated] (RANGER-2170) Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-12-12 Thread Qiang Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/RANGER-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Zhang updated RANGER-2170:

Attachment: 0001-RANGER-2170-Ranger-supports-plugin-to-enable-monitor.patch

> Ranger supports plugin to enable, monitor and manage Elasticsearch
> --
>
> Key: RANGER-2170
> URL: https://issues.apache.org/jira/browse/RANGER-2170
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>Priority: Major
>  Labels: new-feature, patch
> Attachments: 
> 0001-RANGER-2170-Ranger-supports-plugin-to-enable-monitor.patch, 
> 1_ElasticSearchServiceEntry.jpg, 2_EditElasticSearchService.jpg, 
> 3_ElasticSearchPolicies.jpg, 4_EditElasticSearchPolicy.jpg, 
> 5_ElasticSearchAuditLog.jpg, 6_ElasticSearchPlugins.jpg, 
> 7_ElasticSearchPluginStatus.jpg
>
>
> Elasticsearch is a distributed, RESTful search and analytics engine capable 
> of solving a growing number of use cases. 
> Like Apache Solr, it is also an index server based on Lucence.
> Ranger supports plugin to enable, monitor and manage Elasticsearch,
> to control index security of Elasticsearch.
> As there is X-Pack plugin for the Elasticsearch, but it is not free.
> X-Pack is an Elastic Stack extension that bundles security, alerting, 
> monitoring, reporting, 
> and graph capabilities into one easy-to-install package.
> We refer to the Indices Privileges design of X-Pack,
> by keeping the permissions consistent,
> to make user use ranger Elasticsearch plugin easily.
> Reference X-Pack Indices Privileges:
> https://www.elastic.co/guide/en/x-pack/current/security-privileges.html
> Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
> 6.2.2.
> Elasticsearch 6.2.2 was released in February 20, 2018, reference 
> release-notes:
> https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
> Not like other system, Elasticsearch has no basic authentication, 
> it uses X-pack plugin to support basic authentication, 
> role-based access control, SSL/TLS encryption, LDAP and so on.
> Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
> authorization,
> it is to control index of Elasticsearch without authentication,
> this plugin should work with other Elasticsearch plugin to authenticate users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2291) Make optimized db schema script idempotent for all DB Flavors

2018-12-12 Thread Pradeep Agrawal (JIRA) 


[ 
https://issues.apache.org/jira/browse/RANGER-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16719835#comment-16719835
 ] 

Pradeep Agrawal commented on RANGER-2291:
-

Committed to master branch: 
[https://github.com/apache/ranger/commit/5b07a8dfd7a3ab0d0f10690657d74345e964f163]

Committed to ranger-1.2 branch: 58a777212703c219b544ac3b304f089ce2522be2

Committed to ranger-1.1 branch: a605fd1c8e152fc715f45d4dfd5ef3e8da47d6a8

Committed to ranger-1 branch: d0bf545bd1c4f05e4ef5e8f05f7373127df70300

> Make optimized db schema script idempotent for all DB Flavors
> -
>
> Key: RANGER-2291
> URL: https://issues.apache.org/jira/browse/RANGER-2291
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: 1.0.0, 1.1.0, 1.2.0
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 1.0.1, 2.0.0, 1.1.1, 1.2.1
>
> Attachments: 
> 0001-RANGER-2291-Make-optimized-db-schema-script-idempote.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2308) User role user should not able to access usersync audit report if it does not have permissions on the audit module.

2018-12-12 Thread Pradeep Agrawal (JIRA) 


[ 
https://issues.apache.org/jira/browse/RANGER-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16719834#comment-16719834
 ] 

Pradeep Agrawal commented on RANGER-2308:
-

Committed to master branch: 
[https://github.com/apache/ranger/commit/51461487f72e0959aa01d657ef77a1a4176b4738]

Committed to ranger-1.2 branch: 638a3bc323d6b66a453d5c0438315d2351a8f1a5

Committed to ranger-1.1 branch: f96b8a7587341281d6f0e2145de8feb60602d685

Committed to ranger-1 branch: 7b851f74fac23bda4bdbbfd82aae56df40ea9e32

> User role user should not able to access usersync audit report if it does not 
> have permissions on the audit module.
> ---
>
> Key: RANGER-2308
> URL: https://issues.apache.org/jira/browse/RANGER-2308
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: 1.0.0, 1.1.0, 1.2.0
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Minor
> Fix For: 1.0.1, 1.1.1, 1.2.1
>
> Attachments: 
> 0001-RANGER-2308-User-role-user-should-not-able-to-access.patch
>
>
> User role user should not able to access usersync audit report if it does not 
> have permissions on the audit module.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (RANGER-2170) Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-12-12 Thread Qiang Zhang (JIRA) 


[ 
https://issues.apache.org/jira/browse/RANGER-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16563379#comment-16563379
 ] 

Qiang Zhang edited comment on RANGER-2170 at 12/13/18 6:54 AM:
---

Review Requet:
https://reviews.apache.org/r/68128/

Solution patch, please see attachment:
[patch|https://issues.apache.org/jira/secure/attachment/12951614/0001-RANGER-2170-Ranger-supports-plugin-to-enable-monitor.patch]

Implementation details:
[1_ElasticSearchServiceEntry|https://issues.apache.org/jira/secure/attachment/12933730/1_ElasticSearchServiceEntry.jpg]
[2_EditElasticSearchService|https://issues.apache.org/jira/secure/attachment/12933731/2_EditElasticSearchService.jpg]
[3_ElasticSearchPolicies|https://issues.apache.org/jira/secure/attachment/12933732/3_ElasticSearchPolicies.jpg]
[4_EditElasticSearchPolicy|https://issues.apache.org/jira/secure/attachment/12933733/4_EditElasticSearchPolicy.jpg]
[5_ElasticSearchAuditLog|https://issues.apache.org/jira/secure/attachment/12933734/5_ElasticSearchAuditLog.jpg]
[6_ElasticSearchPlugins|https://issues.apache.org/jira/secure/attachment/12933735/6_ElasticSearchPlugins.jpg]
[7_ElasticSearchPluginStatus.jpg|https://issues.apache.org/jira/secure/attachment/12933736/7_ElasticSearchPluginStatus.jpg]


was (Author: zhangqiang2):
Review Requet:
https://reviews.apache.org/r/68128/

Solution patch, please see attachment:
[patch|https://issues.apache.org/jira/secure/attachment/12935505/0001-RANGER-2170-Ranger-supports-plugin-to-enable-monitor.patch]

Implementation details:
[1_ElasticSearchServiceEntry|https://issues.apache.org/jira/secure/attachment/12933730/1_ElasticSearchServiceEntry.jpg]
[2_EditElasticSearchService|https://issues.apache.org/jira/secure/attachment/12933731/2_EditElasticSearchService.jpg]
[3_ElasticSearchPolicies|https://issues.apache.org/jira/secure/attachment/12933732/3_ElasticSearchPolicies.jpg]
[4_EditElasticSearchPolicy|https://issues.apache.org/jira/secure/attachment/12933733/4_EditElasticSearchPolicy.jpg]
[5_ElasticSearchAuditLog|https://issues.apache.org/jira/secure/attachment/12933734/5_ElasticSearchAuditLog.jpg]
[6_ElasticSearchPlugins|https://issues.apache.org/jira/secure/attachment/12933735/6_ElasticSearchPlugins.jpg]
[7_ElasticSearchPluginStatus.jpg|https://issues.apache.org/jira/secure/attachment/12933736/7_ElasticSearchPluginStatus.jpg]

> Ranger supports plugin to enable, monitor and manage Elasticsearch
> --
>
> Key: RANGER-2170
> URL: https://issues.apache.org/jira/browse/RANGER-2170
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>Priority: Major
>  Labels: new-feature, patch
> Attachments: 
> 0001-RANGER-2170-Ranger-supports-plugin-to-enable-monitor.patch, 
> 1_ElasticSearchServiceEntry.jpg, 2_EditElasticSearchService.jpg, 
> 3_ElasticSearchPolicies.jpg, 4_EditElasticSearchPolicy.jpg, 
> 5_ElasticSearchAuditLog.jpg, 6_ElasticSearchPlugins.jpg, 
> 7_ElasticSearchPluginStatus.jpg
>
>
> Elasticsearch is a distributed, RESTful search and analytics engine capable 
> of solving a growing number of use cases. 
> Like Apache Solr, it is also an index server based on Lucence.
> Ranger supports plugin to enable, monitor and manage Elasticsearch,
> to control index security of Elasticsearch.
> As there is X-Pack plugin for the Elasticsearch, but it is not free.
> X-Pack is an Elastic Stack extension that bundles security, alerting, 
> monitoring, reporting, 
> and graph capabilities into one easy-to-install package.
> We refer to the Indices Privileges design of X-Pack,
> by keeping the permissions consistent,
> to make user use ranger Elasticsearch plugin easily.
> Reference X-Pack Indices Privileges:
> https://www.elastic.co/guide/en/x-pack/current/security-privileges.html
> Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
> 6.2.2.
> Elasticsearch 6.2.2 was released in February 20, 2018, reference 
> release-notes:
> https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
> Not like other system, Elasticsearch has no basic authentication, 
> it uses X-pack plugin to support basic authentication, 
> role-based access control, SSL/TLS encryption, LDAP and so on.
> Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
> authorization,
> it is to control index of Elasticsearch without authentication,
> this plugin should work with other Elasticsearch plugin to authenticate users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-2170) Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-12-12 Thread Qiang Zhang (JIRA) 


 [ 
https://issues.apache.org/jira/browse/RANGER-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Zhang updated RANGER-2170:

Attachment: (was: 
0001-RANGER-2170-Ranger-supports-plugin-to-enable-monitor.patch)

> Ranger supports plugin to enable, monitor and manage Elasticsearch
> --
>
> Key: RANGER-2170
> URL: https://issues.apache.org/jira/browse/RANGER-2170
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>Priority: Major
>  Labels: new-feature, patch
> Attachments: 1_ElasticSearchServiceEntry.jpg, 
> 2_EditElasticSearchService.jpg, 3_ElasticSearchPolicies.jpg, 
> 4_EditElasticSearchPolicy.jpg, 5_ElasticSearchAuditLog.jpg, 
> 6_ElasticSearchPlugins.jpg, 7_ElasticSearchPluginStatus.jpg
>
>
> Elasticsearch is a distributed, RESTful search and analytics engine capable 
> of solving a growing number of use cases. 
> Like Apache Solr, it is also an index server based on Lucence.
> Ranger supports plugin to enable, monitor and manage Elasticsearch,
> to control index security of Elasticsearch.
> As there is X-Pack plugin for the Elasticsearch, but it is not free.
> X-Pack is an Elastic Stack extension that bundles security, alerting, 
> monitoring, reporting, 
> and graph capabilities into one easy-to-install package.
> We refer to the Indices Privileges design of X-Pack,
> by keeping the permissions consistent,
> to make user use ranger Elasticsearch plugin easily.
> Reference X-Pack Indices Privileges:
> https://www.elastic.co/guide/en/x-pack/current/security-privileges.html
> Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
> 6.2.2.
> Elasticsearch 6.2.2 was released in February 20, 2018, reference 
> release-notes:
> https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
> Not like other system, Elasticsearch has no basic authentication, 
> it uses X-pack plugin to support basic authentication, 
> role-based access control, SSL/TLS encryption, LDAP and so on.
> Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
> authorization,
> it is to control index of Elasticsearch without authentication,
> this plugin should work with other Elasticsearch plugin to authenticate users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-12-12 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/
---

(Updated 十二月 13, 2018, 6:52 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.


Changes
---

Update to resolve file confilict~


Bugs: RANGER-2170
https://issues.apache.org/jira/browse/RANGER-2170


Repository: ranger


Description
---

Elasticsearch is a distributed, RESTful search and analytics engine capable of 
solving a growing number of use cases. 
Like Apache Solr, it is also an index server based on Lucence.
Ranger supports plugin to enable, monitor and manage Elasticsearch,
to control index security of Elasticsearch.

As there is X-Pack plugin for the Elasticsearch, but it is not free.
X-Pack is an Elastic Stack extension that bundles security, alerting, 
monitoring, reporting, 
and graph capabilities into one easy-to-install package.
We refer to the Indices Privileges design of X-Pack,
by keeping the permissions consistent,
to make user use ranger Elasticsearch plugin easily.
Reference X-Pack Indices Privileges:
https://www.elastic.co/guide/en/x-pack/current/security-privileges.html

Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
6.2.2.
Elasticsearch 6.2.2 was released in February 20, 2018, reference release-notes:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
Not like other system, Elasticsearch has no basic authentication, 
it uses X-pack plugin to support basic authentication, 
role-based access control, SSL/TLS encryption, LDAP and so on.
Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
authorization,
it is to control index of Elasticsearch without authentication,
this plugin should work with other Elasticsearch plugin to authenticate users.


Diffs (updated)
-

  agents-common/scripts/enable-agent.sh ce0dc8c 
  agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java 
e654f2b 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 118af1f 
  
agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json
 PRE-CREATION 
  plugin-elasticsearch/.gitignore PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg 
PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-elasticsearch/pom.xml PRE-CREATION 
  plugin-elasticsearch/scripts/install.properties PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java
 PRE-CREATION 
  pom.xml a11cf51 
  ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties 
PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION 
  ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticsearchPlugin.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/authc/user/UsernamePasswordToken.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization

Re: Review Request 69526: RANGER-2308: User role user should not able to access usersync audit report if it does not have permissions on the audit module.

2018-12-12 Thread Mehul Parikh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69526/#review211273
---


Ship it!




Ship It!

- Mehul Parikh


On Dec. 10, 2018, 6 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69526/
> ---
> 
> (Updated Dec. 10, 2018, 6 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2308
> https://issues.apache.org/jira/browse/RANGER-2308
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently user is having default access to usersync audit report but it be 
> should able to access the report only when he is having access in audit 
> module. without that user can't see details in the UI which is not as per the 
> default behaviour of dashboard for user role users.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java 941691aaa 
>   security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java e1a6b5859 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> 865e115d3 
>   security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 
> 471052f62 
>   security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java 
> a8e6e61a0 
> 
> 
> Diff: https://reviews.apache.org/r/69526/diff/2/
> 
> 
> Testing
> ---
> 
> Tested at local with patch.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 69453: RANGER-2291: Make optimized db schema script idempotent for all DB Flavors

2018-12-12 Thread Mehul Parikh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69453/#review211272
---


Ship it!




Ship It!

- Mehul Parikh


On Nov. 28, 2018, 9:43 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69453/
> ---
> 
> (Updated Nov. 28, 2018, 9:43 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2291
> https://issues.apache.org/jira/browse/RANGER-2291
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** Currently Ranger core db schema is not idempotent for 
> all db flavors. Ranger core DB schema for Oracle and SQL anywhere flavor may 
> fail to execute if we execute them again for the same DB flavor.
> 
> **Proposed Solution:** I have added drop statements before the create 
> statements for the various objects(table/constraints etc)
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> a4fa1305e 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> 0949cbd1d 
>   security-admin/db/oracle/patches/009-updated_schema.sql 7e21f69e1 
>   security-admin/db/oracle/patches/013-permissionmodel.sql 4ac7901ba 
>   
> security-admin/db/oracle/patches/016-updated-schema-for-tag-based-policy.sql 
> 12627f589 
>   security-admin/db/oracle/patches/020-datamask-policy.sql 8448a8568 
>   security-admin/db/oracle/patches/022-split-service-table.sql 9b4f69c4c 
>   security-admin/db/oracle/patches/025-create-schema-for-plugin-info.sql 
> bedd0a2ef 
>   security-admin/db/oracle/patches/030-policy-labels-schema.sql 894b9346f 
>   
> security-admin/db/oracle/patches/031-create-schema-for-usersync-audit-info.sql
>  cb52065c6 
>   security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql 
> c75e62089 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> a0e02e0e0 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  db8ebc343 
>   
> security-admin/db/sqlanywhere/patches/016-updated-schema-for-tag-based-policy.sql
>  f3b64d003 
>   security-admin/db/sqlanywhere/patches/020-datamask-policy.sql fe6fa9f61 
>   security-admin/db/sqlanywhere/patches/022-split-service-table.sql d32966d8c 
>   security-admin/db/sqlanywhere/patches/025-create-schema-for-plugin-info.sql 
> 6e9477984 
>   security-admin/db/sqlanywhere/patches/030-policy-labels-schema.sql 
> b2ed2386d 
>   
> security-admin/db/sqlanywhere/patches/031-create-schema-for-usersync-audit-info.sql
>  8ed84e302 
>   security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql 
> c079014df 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> 522b57b03 
> 
> 
> Diff: https://reviews.apache.org/r/69453/diff/1/
> 
> 
> Testing
> ---
> 
> **Steps Performed (with patch) :**
> 1. After Build untar the Ranger module and updated install.properties for 
> Oracle DB flavor.
> 2. Called setup.sh to install Ranger.
> 3. Started Ranger admin and logged in to check the installation is working or 
> not.
> 4. create a user 'testuser1'.
> 5. Stopped Ranger admin.
> 6. Executed given JISQL command again to import core db schema with the same 
> config (for the same ranger db and user):
> 
> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.security.egd=file:///dev/urandom  -cp 
> /usr/share/java/ojdbc6.jar:/tmp/ranger-2.0.0-SNAPSHOT-admin/jisql/lib/* 
> org.apache.util.sql.Jisql -driver oraclethin -cstring 
> jdbc:oracle:thin:@localhost -u 'ranger112701' -p '' -noheader -trim 
> -input 
> /tmp/ranger-2.0.0-SNAPSHOT-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql
>  -c \;
> 
> **Expected behavior:** Command should able to execute core db schema file 
> again and should not fail. user testuser1 should not appear in user/groups 
> page of ranger admin
> 
> **Actual behavior:** Command executed successfully and recreated all the 
> tables again, was able to see new entries and able to login to ranger admin. 
> 'testuser1' was not seen in the ranger admin.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 69468: RANGER-2295: Set specific Ranger version in patches status entry table

2018-12-12 Thread Pradeep Agrawal 


> On Dec. 12, 2018, 5:36 p.m., Velmurugan Periasamy wrote:
> > security-admin/scripts/db_setup.py
> > Lines 1024 (patched)
> > 
> >
> > Is it enough to consider only localhost?

Yes, because the code suppose to update only those entries which are made by 
ranger core db schema. Ranger core db schema has updated_by='localhost' as hard 
coded value in the script. 
Ref: 
https://github.com/apache/ranger/blob/master/security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql#L1359


- Pradeep


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69468/#review211246
---


On Nov. 28, 2018, 10:19 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69468/
> ---
> 
> (Updated Nov. 28, 2018, 10:19 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2295
> https://issues.apache.org/jira/browse/RANGER-2295
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** DB setup script(db_setup.py) looks for a specific 
> version (For example: "Ranger 2.0.0-SNAPSHOT") to decide if patches need to 
> be applied or not. 
> 
> For example:
> select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 
> 'Ranger 2.0.0-SNAPSHOT' and active = 'Y';
> select version from x_db_version_h where version = 'JAVA_PATCHES' and inst_by 
> = 'Ranger 2.0.0-SNAPSHOT' and active = 'Y';
> 
> 
> However, the optimized schema creation script comes with a generic version 
> (For example: "Ranger 1.0.0"):
> 
> 
> INSERT INTO x_db_version_h 
> (version,inst_at,inst_by,updated_at,updated_by,active) VALUES 
> ('DB_PATCHES',CURRENT_TIMESTAMP,'Ranger 
> 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
> INSERT INTO x_db_version_h 
> (version,inst_at,inst_by,updated_at,updated_by,active) VALUES 
> ('JAVA_PATCHES',CURRENT_TIMESTAMP,'Ranger 
> 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
> 
> The result is that a separate check is executed for each patch, which takes 
> time. It will be good if the status entries have the exact ranger version 
> rather a base version.
> 
> **Proposed Solution:** The propsed solution includes following changes:
> After core db schema file(ranger_core_db_*.sql) is imported patch shall 
> update the sql/java patches entry version with the exact version+build being 
> used. Once the exact version is updated then when DB setup script will look 
> for a specific version then it will find a matching entry and skip the all 
> patches check.
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/db_setup.py 2bda1a8e7 
> 
> 
> Diff: https://reviews.apache.org/r/69468/diff/1/
> 
> 
> Testing
> ---
> 
> **Steps performed for Ranger-admin(with patch):**
> 
> 1. Created Build with patch and untar the build.
> 2. Opened install.properties and provided db configuration in 
> install.properties
> 3. Called setup.sh
> 
> **Expected Behavior:**
> 1. There should be a single call of db schema setup and it should not try to 
> apply/check all the db patches entries.
> 
> **Actual Behavior:**
> 2. After importing the db schema file, ranger checked for entries of 
> 'DB_PATCHES' and 'JAVA_PATCHES' for the current ranger version and skipped 
> checking entries of every db and java patches.
> 
> 
> **Note:**
> Patch has been tested for all the db flavor.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Resolved] (RANGER-2163) Spelling error for "Persmission" in the PatchPersmissionModel_J10003.java.

2018-12-12 Thread xiehaihong (JIRA) 


 [ 
https://issues.apache.org/jira/browse/RANGER-2163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xiehaihong resolved RANGER-2163.

   Resolution: Fixed
Fix Version/s: 1.2.0
   master

> Spelling error for "Persmission" in the PatchPersmissionModel_J10003.java.
> --
>
> Key: RANGER-2163
> URL: https://issues.apache.org/jira/browse/RANGER-2163
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: master
>Reporter: xiehaihong
>Assignee: Qiang Zhang
>Priority: Minor
>  Labels: patch
> Fix For: master, 1.2.0
>
> Attachments: 
> 0001-RANGER-2163-Spelling-error-in-the-PatchPersmissionMo.patch
>
>
> Spelling error for "Persmission" in the 
> ranger/security-admin/src/main/java/org/apache/ranger/patch/PatchPersmissionModel_J10003.java.
>  "PatchPermissionModel" instead of "PatchPersmissionModel".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69509: RANGER-2304 : Need to add property dfs.permissions.ContentSummary.subAccess when enabling Ranger HDFS plugin manually.

2018-12-12 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69509/#review211268
---


Ship it!




Ship It!

- Qiang Zhang


On 十二月 5, 2018, 11:29 a.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69509/
> ---
> 
> (Updated 十二月 5, 2018, 11:29 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, bhavik patel, Colm O hEigeartaigh, 
> Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, pengjianhua, 
> Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, Velmurugan Periasamy, and 
> Qiang Zhang.
> 
> 
> Bugs: RANGER-2304
> https://issues.apache.org/jira/browse/RANGER-2304
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> As part of fixes in HDFS-14112 and RANGER-2297, need to update the script 
> that handles setting up HDFS authorizer when Ranger HDFS plugin is 
> enabled/disabled, as below:
>* Set the property dfs.permissions.ContentSummary.subAccess in 
> hdfs-site.xml to ‘true’ when Ranger plugin is   
>  enabled.
>* Remove the property dfs.permissions.ContentSummary.subAccess in 
> hdfs-site.xml or set to ‘false’ when Ranger plugin
>  is disabled.
> 
> 
> Diffs
> -
> 
>   hdfs-agent/conf/hdfs-site-changes.cfg 8088b43f8 
>   hdfs-agent/disable-conf/hdfs-site-changes.cfg 652bf2ee8 
> 
> 
> Diff: https://reviews.apache.org/r/69509/diff/1/
> 
> 
> Testing
> ---
> 
> Tested with a fresh install on Cent-OS, the property 
> dfs.permissions.ContentSummary.subAccess is set to true when Ranger HDFS 
> plugin is enabled manually.
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>

[jira] [Commented] (RANGER-2306) Knox Plugin doesn't pass X-Forwarded-for remote address to Ranger

2018-12-12 Thread Vipin Rathor (JIRA) 


[ 
https://issues.apache.org/jira/browse/RANGER-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16719581#comment-16719581
 ] 

Vipin Rathor commented on RANGER-2306:
--

Thanks [~rmani] for review and commit. Appreciate it!

> Knox Plugin doesn't pass X-Forwarded-for remote address to Ranger
> -
>
> Key: RANGER-2306
> URL: https://issues.apache.org/jira/browse/RANGER-2306
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins
>Affects Versions: 1.2.0
>Reporter: Vipin Rathor
>Priority: Major
> Fix For: 2.0.0, 1.2.1
>
> Attachments: 
> 0001-RANGER-2306-Add-support-for-X-Forwarded-for-header-i.patch
>
>
> *Problem Description:*
>  IP-based Knox policies doesn't work when Knox is behind a Load Balancer. 
> Because currently Ranger Knox plugin doesn't accept & pass on the 
> "X-Forwarded-for" header to Ranger policy engine.
> *Impact:*
> In an environment where Knox is running behind a Load Balancer and Knox has a 
> Ranger policy to allow/deny access to Hadoop services based on client IP 
> addresses, this won't work as expected due to this bug.
> *Expected Behavior:*
>  1. Knox plugin should process "X-Forwarded-for" header received from Load 
> Balancer and pass it on to policy engine in the form of 
> 'RangerAccessRequestImpl.forwardedAdresses'.
> *Steps to reproduce:*
>  1. Install & configure Knox behind a Load Balancer
> 2. Enable Ranger Knox plugin
> 3. Also Set "ranger.plugin.knox.use.x-forwarded-for.ipaddress=true" and 
> "ranger.plugin.knox.trusted.proxy.ipaddresses="
> 4. Define a Knox policy to allow access to user from designated client IP(s)
> 5. Try to access any WebHDFS (for example) resource via Knox via Load 
> Balancer for designated client host.
> *Workaround:*
> None



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69524: RANGER-2307 - native authentication improvements

2018-12-12 Thread Sailaja Polavarapu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69524/#review211261
---


Ship it!




Ship It!

- Sailaja Polavarapu


On Dec. 7, 2018, 10 a.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69524/
> ---
> 
> (Updated Dec. 7, 2018, 10 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-2307
> https://issues.apache.org/jira/browse/RANGER-2307
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently credValidator and pamCredValidator don't handle well configuration 
> problems - when the user doesn't have permission to read /etc/shadow or 
> access pam, or when the shadow file is not filled properly. This could cause 
> 'core dumps', and hard to fix deployment issues.
> 
> With this change, at least, it doesn't segfaults, when the crypt function 
> returns null, and it shows the underlying error message in its response.
> 
> 
> Diffs
> -
> 
>   unixauthnative/src/main/c/credValidator.c e426bdd2f 
>   unixauthpam/src/main/c/pamCredValidator.c 60d38aebd 
> 
> 
> Diff: https://reviews.apache.org/r/69524/diff/1/
> 
> 
> Testing
> ---
> 
> Tryied to call both method on live cluster - as root / non-root, with wrong 
> username, missing password from shadow file, etc.
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

[jira] [Commented] (RANGER-2305) When Audit spooling to local filesystem is enabled, log files of the component have show a wrong error message

2018-12-12 Thread Ramesh Mani (JIRA) 


[ 
https://issues.apache.org/jira/browse/RANGER-2305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16719395#comment-16719395
 ] 

Ramesh Mani commented on RANGER-2305:
-

committed to branches 
[1.0.1|https://issues.apache.org/jira/issues/?jql=project+%3D+RANGER+AND+fixVersion+%3D+1.0.1],
 
[2.0.0|https://issues.apache.org/jira/issues/?jql=project+%3D+RANGER+AND+fixVersion+%3D+2.0.0],
 
[1.1.1|https://issues.apache.org/jira/issues/?jql=project+%3D+RANGER+AND+fixVersion+%3D+1.1.1],
 
[1.2.1|https://issues.apache.org/jira/issues/?jql=project+%3D+RANGER+AND+fixVersion+%3D+1.2.1]

> When Audit spooling to local filesystem is enabled, log files of the 
> component have show a wrong error message
> --
>
> Key: RANGER-2305
> URL: https://issues.apache.org/jira/browse/RANGER-2305
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 1.0.0, 1.1.0, 2.0.0, 1.2.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 1.0.1, 2.0.0, 1.1.1, 1.2.1
>
> Attachments: 
> 0001-RANGER-2305-When-Audit-spooling-to-local-filesystem-.patch
>
>
> When Audit spooling to the local filesystem is enabled, log files of the 
> component shows a wrong error message.
> Audit spooling to local filesystem before it gets into destination is done by 
>  xasecure.audit.provider.filecache.is.enabled=true parameter. When this is 
> done the component logs show an error message "fail to log audit event 
> AuthzAuditEvent" even though the audits are successful. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2306) Knox Plugin doesn't pass X-Forwarded-for remote address to Ranger

2018-12-12 Thread Ramesh Mani (JIRA) 


[ 
https://issues.apache.org/jira/browse/RANGER-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16719371#comment-16719371
 ] 

Ramesh Mani commented on RANGER-2306:
-

commit id - master:

[http://git-wip-us.apache.org/repos/asf/ranger/commit/3d282ccb]

commit id - ranger-1.2

[http://git-wip-us.apache.org/repos/asf/ranger/commit/9916e1ba]

> Knox Plugin doesn't pass X-Forwarded-for remote address to Ranger
> -
>
> Key: RANGER-2306
> URL: https://issues.apache.org/jira/browse/RANGER-2306
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins
>Affects Versions: 1.2.0
>Reporter: Vipin Rathor
>Priority: Major
> Fix For: 2.0.0, 1.2.1
>
> Attachments: 
> 0001-RANGER-2306-Add-support-for-X-Forwarded-for-header-i.patch
>
>
> *Problem Description:*
>  IP-based Knox policies doesn't work when Knox is behind a Load Balancer. 
> Because currently Ranger Knox plugin doesn't accept & pass on the 
> "X-Forwarded-for" header to Ranger policy engine.
> *Impact:*
> In an environment where Knox is running behind a Load Balancer and Knox has a 
> Ranger policy to allow/deny access to Hadoop services based on client IP 
> addresses, this won't work as expected due to this bug.
> *Expected Behavior:*
>  1. Knox plugin should process "X-Forwarded-for" header received from Load 
> Balancer and pass it on to policy engine in the form of 
> 'RangerAccessRequestImpl.forwardedAdresses'.
> *Steps to reproduce:*
>  1. Install & configure Knox behind a Load Balancer
> 2. Enable Ranger Knox plugin
> 3. Also Set "ranger.plugin.knox.use.x-forwarded-for.ipaddress=true" and 
> "ranger.plugin.knox.trusted.proxy.ipaddresses="
> 4. Define a Knox policy to allow access to user from designated client IP(s)
> 5. Try to access any WebHDFS (for example) resource via Knox via Load 
> Balancer for designated client host.
> *Workaround:*
> None



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69453: RANGER-2291: Make optimized db schema script idempotent for all DB Flavors

2018-12-12 Thread Velmurugan Periasamy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69453/#review211247
---


Ship it!




Ship It!

- Velmurugan Periasamy


On Nov. 28, 2018, 9:43 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69453/
> ---
> 
> (Updated Nov. 28, 2018, 9:43 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2291
> https://issues.apache.org/jira/browse/RANGER-2291
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** Currently Ranger core db schema is not idempotent for 
> all db flavors. Ranger core DB schema for Oracle and SQL anywhere flavor may 
> fail to execute if we execute them again for the same DB flavor.
> 
> **Proposed Solution:** I have added drop statements before the create 
> statements for the various objects(table/constraints etc)
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> a4fa1305e 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> 0949cbd1d 
>   security-admin/db/oracle/patches/009-updated_schema.sql 7e21f69e1 
>   security-admin/db/oracle/patches/013-permissionmodel.sql 4ac7901ba 
>   
> security-admin/db/oracle/patches/016-updated-schema-for-tag-based-policy.sql 
> 12627f589 
>   security-admin/db/oracle/patches/020-datamask-policy.sql 8448a8568 
>   security-admin/db/oracle/patches/022-split-service-table.sql 9b4f69c4c 
>   security-admin/db/oracle/patches/025-create-schema-for-plugin-info.sql 
> bedd0a2ef 
>   security-admin/db/oracle/patches/030-policy-labels-schema.sql 894b9346f 
>   
> security-admin/db/oracle/patches/031-create-schema-for-usersync-audit-info.sql
>  cb52065c6 
>   security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql 
> c75e62089 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> a0e02e0e0 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  db8ebc343 
>   
> security-admin/db/sqlanywhere/patches/016-updated-schema-for-tag-based-policy.sql
>  f3b64d003 
>   security-admin/db/sqlanywhere/patches/020-datamask-policy.sql fe6fa9f61 
>   security-admin/db/sqlanywhere/patches/022-split-service-table.sql d32966d8c 
>   security-admin/db/sqlanywhere/patches/025-create-schema-for-plugin-info.sql 
> 6e9477984 
>   security-admin/db/sqlanywhere/patches/030-policy-labels-schema.sql 
> b2ed2386d 
>   
> security-admin/db/sqlanywhere/patches/031-create-schema-for-usersync-audit-info.sql
>  8ed84e302 
>   security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql 
> c079014df 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> 522b57b03 
> 
> 
> Diff: https://reviews.apache.org/r/69453/diff/1/
> 
> 
> Testing
> ---
> 
> **Steps Performed (with patch) :**
> 1. After Build untar the Ranger module and updated install.properties for 
> Oracle DB flavor.
> 2. Called setup.sh to install Ranger.
> 3. Started Ranger admin and logged in to check the installation is working or 
> not.
> 4. create a user 'testuser1'.
> 5. Stopped Ranger admin.
> 6. Executed given JISQL command again to import core db schema with the same 
> config (for the same ranger db and user):
> 
> /usr/jdk64/jdk1.8.0_112/bin/java -Djava.security.egd=file:///dev/urandom  -cp 
> /usr/share/java/ojdbc6.jar:/tmp/ranger-2.0.0-SNAPSHOT-admin/jisql/lib/* 
> org.apache.util.sql.Jisql -driver oraclethin -cstring 
> jdbc:oracle:thin:@localhost -u 'ranger112701' -p '' -noheader -trim 
> -input 
> /tmp/ranger-2.0.0-SNAPSHOT-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql
>  -c \;
> 
> **Expected behavior:** Command should able to execute core db schema file 
> again and should not fail. user testuser1 should not appear in user/groups 
> page of ranger admin
> 
> **Actual behavior:** Command executed successfully and recreated all the 
> tables again, was able to see new entries and able to login to ranger admin. 
> 'testuser1' was not seen in the ranger admin.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 69468: RANGER-2295: Set specific Ranger version in patches status entry table

2018-12-12 Thread Velmurugan Periasamy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69468/#review211246
---




security-admin/scripts/db_setup.py
Lines 1024 (patched)


Is it enough to consider only localhost?


- Velmurugan Periasamy


On Nov. 28, 2018, 10:19 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69468/
> ---
> 
> (Updated Nov. 28, 2018, 10:19 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2295
> https://issues.apache.org/jira/browse/RANGER-2295
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** DB setup script(db_setup.py) looks for a specific 
> version (For example: "Ranger 2.0.0-SNAPSHOT") to decide if patches need to 
> be applied or not. 
> 
> For example:
> select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 
> 'Ranger 2.0.0-SNAPSHOT' and active = 'Y';
> select version from x_db_version_h where version = 'JAVA_PATCHES' and inst_by 
> = 'Ranger 2.0.0-SNAPSHOT' and active = 'Y';
> 
> 
> However, the optimized schema creation script comes with a generic version 
> (For example: "Ranger 1.0.0"):
> 
> 
> INSERT INTO x_db_version_h 
> (version,inst_at,inst_by,updated_at,updated_by,active) VALUES 
> ('DB_PATCHES',CURRENT_TIMESTAMP,'Ranger 
> 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
> INSERT INTO x_db_version_h 
> (version,inst_at,inst_by,updated_at,updated_by,active) VALUES 
> ('JAVA_PATCHES',CURRENT_TIMESTAMP,'Ranger 
> 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
> 
> The result is that a separate check is executed for each patch, which takes 
> time. It will be good if the status entries have the exact ranger version 
> rather a base version.
> 
> **Proposed Solution:** The propsed solution includes following changes:
> After core db schema file(ranger_core_db_*.sql) is imported patch shall 
> update the sql/java patches entry version with the exact version+build being 
> used. Once the exact version is updated then when DB setup script will look 
> for a specific version then it will find a matching entry and skip the all 
> patches check.
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/db_setup.py 2bda1a8e7 
> 
> 
> Diff: https://reviews.apache.org/r/69468/diff/1/
> 
> 
> Testing
> ---
> 
> **Steps performed for Ranger-admin(with patch):**
> 
> 1. Created Build with patch and untar the build.
> 2. Opened install.properties and provided db configuration in 
> install.properties
> 3. Called setup.sh
> 
> **Expected Behavior:**
> 1. There should be a single call of db schema setup and it should not try to 
> apply/check all the db patches entries.
> 
> **Actual Behavior:**
> 2. After importing the db schema file, ranger checked for entries of 
> 'DB_PATCHES' and 'JAVA_PATCHES' for the current ranger version and skipped 
> checking entries of every db and java patches.
> 
> 
> **Note:**
> Patch has been tested for all the db flavor.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 69526: RANGER-2308: User role user should not able to access usersync audit report if it does not have permissions on the audit module.

2018-12-12 Thread Velmurugan Periasamy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69526/#review211245
---


Ship it!




Ship It!

- Velmurugan Periasamy


On Dec. 10, 2018, 6 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69526/
> ---
> 
> (Updated Dec. 10, 2018, 6 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2308
> https://issues.apache.org/jira/browse/RANGER-2308
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Currently user is having default access to usersync audit report but it be 
> should able to access the report only when he is having access in audit 
> module. without that user can't see details in the UI which is not as per the 
> default behaviour of dashboard for user role users.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java 941691aaa 
>   security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java e1a6b5859 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> 865e115d3 
>   security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 
> 471052f62 
>   security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java 
> a8e6e61a0 
> 
> 
> Diff: https://reviews.apache.org/r/69526/diff/2/
> 
> 
> Testing
> ---
> 
> Tested at local with patch.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Updated] (RANGER-2309) Improve group search on policy edit page.

2018-12-12 Thread Nikhil Purbhe (JIRA) 


 [ 
https://issues.apache.org/jira/browse/RANGER-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Purbhe updated RANGER-2309:
--
Attachment: RANGER-2309-Improve-group-search-on-policy-edit-page.patch

> Improve group search on policy edit page.
> -
>
> Key: RANGER-2309
> URL: https://issues.apache.org/jira/browse/RANGER-2309
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Nikhil Purbhe
>Assignee: Nikhil Purbhe
>Priority: Major
> Fix For: master, 2.0.0, 1.1.1, 1.2.1
>
> Attachments: 
> RANGER-2309-Improve-group-search-on-policy-edit-page.patch
>
>
> Improve group search on policy edit page.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69509: RANGER-2304 : Need to add property dfs.permissions.ContentSummary.subAccess when enabling Ranger HDFS plugin manually.

2018-12-12 Thread Velmurugan Periasamy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69509/#review211229
---


Ship it!




Ship It!

- Velmurugan Periasamy


On Dec. 5, 2018, 11:29 a.m., Vishal Suvagia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69509/
> ---
> 
> (Updated Dec. 5, 2018, 11:29 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, bhavik patel, Colm O hEigeartaigh, 
> Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, pengjianhua, 
> Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, Velmurugan Periasamy, and 
> Qiang Zhang.
> 
> 
> Bugs: RANGER-2304
> https://issues.apache.org/jira/browse/RANGER-2304
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> As part of fixes in HDFS-14112 and RANGER-2297, need to update the script 
> that handles setting up HDFS authorizer when Ranger HDFS plugin is 
> enabled/disabled, as below:
>* Set the property dfs.permissions.ContentSummary.subAccess in 
> hdfs-site.xml to ‘true’ when Ranger plugin is   
>  enabled.
>* Remove the property dfs.permissions.ContentSummary.subAccess in 
> hdfs-site.xml or set to ‘false’ when Ranger plugin
>  is disabled.
> 
> 
> Diffs
> -
> 
>   hdfs-agent/conf/hdfs-site-changes.cfg 8088b43f8 
>   hdfs-agent/disable-conf/hdfs-site-changes.cfg 652bf2ee8 
> 
> 
> Diff: https://reviews.apache.org/r/69509/diff/1/
> 
> 
> Testing
> ---
> 
> Tested with a fresh install on Cent-OS, the property 
> dfs.permissions.ContentSummary.subAccess is set to true when Ranger HDFS 
> plugin is enabled manually.
> 
> 
> Thanks,
> 
> Vishal Suvagia
> 
>

[jira] [Created] (RANGER-2309) Improve group search on policy edit page.

2018-12-12 Thread Nikhil Purbhe (JIRA) 
Nikhil Purbhe created RANGER-2309:
-

 Summary: Improve group search on policy edit page.
 Key: RANGER-2309
 URL: https://issues.apache.org/jira/browse/RANGER-2309
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Nikhil Purbhe
Assignee: Nikhil Purbhe
 Fix For: master, 2.0.0, 1.1.1, 1.2.1


Improve group search on policy edit page.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69549: RANGER-2234 : Cannot add or update a child row, a foreign key constraint fails when installing ranger-admin

2018-12-12 Thread Akash Pawale 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69549/
---

(Updated Dec. 12, 2018, 9:59 a.m.)


Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Haihui Xu, 
Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nitin Galave, Pradeep Agrawal, 
Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy.


Bugs: RANGER-2234
https://issues.apache.org/jira/browse/RANGER-2234


Repository: ranger


Description
---

Added functions getXportalUIdByLoginId[ to get x_portal_user.id by 
x_portal_user.login_id] and getModulesIdByName
and calling this functions while updating child rows to avoid foreign key 
constraint fails errors


Diffs (updated)
-

  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql a4fa130 
  security-admin/db/mysql/patches/013-permissionmodel.sql 381bb6f 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 0949cbd 
  security-admin/db/oracle/patches/013-permissionmodel.sql 4ac7901 
  security-admin/db/oracle/patches/016-updated-schema-for-tag-based-policy.sql 
12627f5 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
a0e02e0 
  
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 
db8ebc3 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
522b57b 
  
security-admin/db/sqlserver/patches/016-updated-schema-for-tag-based-policy.sql 
4b856d7 


Diff: https://reviews.apache.org/r/69549/diff/2/

Changes: https://reviews.apache.org/r/69549/diff/1-2/


Testing
---

Testing done with all supported db flavours


Thanks,

Akash Pawale

[jira] [Updated] (RANGER-2234) Cannot add or update a child row,a foreign key constraint fails when installing ranger-admin

2018-12-12 Thread Akash Pawale (JIRA) 


 [ 
https://issues.apache.org/jira/browse/RANGER-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akash Pawale updated RANGER-2234:
-
Attachment: RANGER-2234-03.patch

> Cannot add or update a child row,a foreign key constraint fails when 
> installing ranger-admin
> 
>
> Key: RANGER-2234
> URL: https://issues.apache.org/jira/browse/RANGER-2234
> Project: Ranger
>  Issue Type: Bug
>  Components: admin, Ranger
>Affects Versions: 1.1.0
>Reporter: Haihui Xu
>Assignee: Akash Pawale
>Priority: Major
>  Labels: patch
> Fix For: 2.0.0
>
> Attachments: RANGER-2234-02.patch, RANGER-2234-03.patch, 
> RANGER-2234_Cannot add or update a child row,a foreign key constraint fails 
> when installing ranger-admin.patch
>
>
> Installing ranger-admin use mysql as the database,execute setup.sh, in 
> progress something happend, the error logs are as flowing:
> 2018-09-26 17:17:08,539 [I] Table xa_access_audit does not exist in database 
> ranger
> 2018-09-26 17:17:08,539 [I] Importing db schema to database ranger from file: 
> ranger_core_db_mysql.sql
> 2018-09-26 17:17:08,540 [JISQL] /home/ranger/jdk1.8.0_121/bin/java -cp 
> /usr/share/java/mysql-connector-java.jar:/home/ranger/ranger-1.1.0-admin/jisql/lib/*
>  org.apache.util.sql.Jisql -driver mysqlconj -cstring 
> jdbc:mysql://10.139.16.75/ranger -u 'root' -p '' -noheader -trim -c 
> \; -input 
> /home/ranger/ranger-1.1.0-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> Error executing: INSERT INTO 
> x_portal_user_role(create_time,update_time,added_by_id,upd_by_id,user_id,user_role,status)
>  VALUES (UTC_TIMESTAMP(),UTC_TIMESTAMP(),NULL,NULL,2,'ROLE_SYS_ADMIN',1); 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`))
> SQLException : SQL state: 23000 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`)) ErrorCode: 1452
> 2018-09-26 17:22:20,882 [E] ranger_core_db_mysql.sql file import failed!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Subject: [NOTICE] Mandatory relocation of Apache git repositories on git-wip-us.apache.org

2018-12-12 Thread Gautam Borad 
+1

On Wed, Dec 12, 2018 at 2:45 AM Zs.  wrote:

> +1
>
> Regards,
>  Zsombor
>
> On Tue, Dec 11, 2018 at 7:28 PM Sailaja Polavarapu <
> spolavar...@hortonworks.com> wrote:
>
> > +1
> >
> > On 12/10/18, 1:22 PM, "Abhay Kulkarni" 
> wrote:
> >
> > +1
> >
> > On 12/10/18, 12:06 PM, "Velmurugan Periasamy" 
> wrote:
> >
> > >Rangers:
> > >
> > >Please see below. I propose to move ranger repos over to gitbox
> (from
> > >git-wip-us.apache.org ) during the
> > first
> > >phase (voluntary coordination). Please share your thoughts.
> > >
> > >Once there is agreement in dev list, I can open INFRA ticket to
> > complete
> > >the move. Tentative target next week (week of Dec 17).
> > >
> > >Here¹s my +1.
> > >
> > >Thank you,
> > >Vel
> > >
> > >
> > >From: Daniel Gruno 
> > >Sent: Friday, December 7, 2018 11:52 AM
> > >To: us...@infra.apache.org
> > >Subject: [NOTICE] Mandatory relocation of Apache git repositories on
> > >git-wip-us.apache.org
> > >
> > >[IF YOUR PROJECT DOES NOT HAVE GIT REPOSITORIES ON GIT-WIP-US PLEASE
> > >  DISREGARD THIS EMAIL; IT WAS MASS-MAILED TO ALL APACHE PROJECTS]
> > >
> > >Hello Apache projects,
> > >
> > >I am writing to you because you may have git repositories on the
> > >git-wip-us server, which is slated to be decommissioned in the
> coming
> > >months. All repositories will be moved to the new gitbox service
> which
> > >includes direct write access on github as well as the standard ASF
> > >commit access via gitbox.apache.org.
> > >
> > >## Why this move? ##
> > >The move comes as a result of retiring the git-wip service, as the
> > >hardware it runs on is longing for retirement. In lieu of this, we
> > >have decided to consolidate the two services (git-wip and gitbox),
> to
> > >ease the management of our repository systems and future-proof the
> > >underlying hardware. The move is fully automated, and ideally,
> nothing
> > >will change in your workflow other than added features and access to
> > >GitHub.
> > >
> > >## Timeframe for relocation ##
> > >Initially, we are asking that projects voluntarily request to move
> > >their repositories to gitbox, hence this email. The voluntary
> > >timeframe is between now and January 9th 2019, during which projects
> > >are free to either move over to gitbox or stay put on git-wip. After
> > >this phase, we will be requiring the remaining projects to move
> within
> > >one month, after which we will move the remaining projects over.
> > >
> > >To have your project moved in this initial phase, you will need:
> > >
> > >- Consensus in the project (documented via the mailing list)
> > >- File a JIRA ticket with INFRA to voluntarily move your project
> repos
> > >   over to gitbox (as stated, this is highly automated and will take
> > >   between a minute and an hour, depending on the size and number of
> > >   your repositories)
> > >
> > >To sum up the preliminary timeline;
> > >
> > >- December 9th 2018 -> January 9th 2019: Voluntary (coordinated)
> > >   relocation
> > >- January 9th -> February 6th: Mandated (coordinated) relocation
> > >- February 7th: All remaining repositories are mass migrated.
> > >
> > >This timeline may change to accommodate various scenarios.
> > >
> > >## Using GitHub with ASF repositories ##
> > >When your project has moved, you are free to use either the ASF
> > >repository system (gitbox.apache.org) OR GitHub for your
> development
> > >and code pushes. To be able to use GitHub, please follow the primer
> > >at: https://reference.apache.org/committer/github
> > >
> > >
> > >We appreciate your understanding of this issue, and hope that your
> > >project can coordinate voluntarily moving your repositories in a
> > >timely manner.
> > >
> > >All settings, such as commit mail targets, issue linking, PR
> > >notification schemes etc will automatically be migrated to gitbox as
> > >well.
> > >
> > >With regards, Daniel on behalf of ASF Infra.
> > >
> > >PS:For inquiries, please reply to us...@infra.apache.org, not your
> > >project's dev list :-).
> > >
> > >
> >
> >
> >
> >
>