Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-12-12 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/
---

(Updated 十二月 13, 2018, 6:52 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.


Changes
---

Update to resolve file confilict~


Bugs: RANGER-2170
https://issues.apache.org/jira/browse/RANGER-2170


Repository: ranger


Description
---

Elasticsearch is a distributed, RESTful search and analytics engine capable of 
solving a growing number of use cases. 
Like Apache Solr, it is also an index server based on Lucence.
Ranger supports plugin to enable, monitor and manage Elasticsearch,
to control index security of Elasticsearch.

As there is X-Pack plugin for the Elasticsearch, but it is not free.
X-Pack is an Elastic Stack extension that bundles security, alerting, 
monitoring, reporting, 
and graph capabilities into one easy-to-install package.
We refer to the Indices Privileges design of X-Pack,
by keeping the permissions consistent,
to make user use ranger Elasticsearch plugin easily.
Reference X-Pack Indices Privileges:
https://www.elastic.co/guide/en/x-pack/current/security-privileges.html

Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
6.2.2.
Elasticsearch 6.2.2 was released in February 20, 2018, reference release-notes:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
Not like other system, Elasticsearch has no basic authentication, 
it uses X-pack plugin to support basic authentication, 
role-based access control, SSL/TLS encryption, LDAP and so on.
Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
authorization,
it is to control index of Elasticsearch without authentication,
this plugin should work with other Elasticsearch plugin to authenticate users.


Diffs (updated)
-

  agents-common/scripts/enable-agent.sh ce0dc8c 
  agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java 
e654f2b 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 118af1f 
  
agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json
 PRE-CREATION 
  plugin-elasticsearch/.gitignore PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg 
PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-elasticsearch/pom.xml PRE-CREATION 
  plugin-elasticsearch/scripts/install.properties PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java
 PRE-CREATION 
  pom.xml a11cf51 
  ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties 
PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION 
  ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticsearchPlugin.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/authc/user/UsernamePasswordToken.java
 PRE-CREATION

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-09-12 Thread Qiang Zhang 


> On 九月 6, 2018, 6:30 p.m., Ramesh Mani wrote:
> > ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties
> > Lines 46 (patched)
> > 
> >
> > Why is -shim- is having the conf and classes which are core to the 
> > plugin it should part of /plugin-elasticseach/conf folder.
> > -shim is to hold the ranger-classloader.
> > Please refer the existing plugin and see if that can be done.
> 
> Qiang Zhang wrote:
> Because unlike other Hadoop components, Elasticseach is designed to be 
> pluggable.
> To implement a new extension function, 
> we need to organize code and configuration files according to the 
> requirements of Elasticseach.
> Some classes in ranger-elasticsearch-plugin-shim is necessary to mount on 
> Elasticseach.
> Other classes is due to the fact that Elasticseach itself does not 
> support user authentication,
> so this plugin should work with other Elasticsearch plugin to 
> authenticate users,
> such as Basic Authentication, Kerberos, LDAP, etc.
> Or, in the future, we can realize user authentication in 
> ranger-elasticsearch-plugin-shim.
> So these classes can't sink into plugin-elasticseach.
> And plugin-descriptor.properties is for Elasticseach to start the Ranger 
> Elasticseach plugin.
> But the files in the plugin-elasticseach/conf directory are for Ranger 
> Elasticseach plugin.
> 
> The related directory structure after plugin installed is as follows:
> 1.ranger-elasticsearch-plugin-shim/conf/?
> ```
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
> 
> /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
> -rwxrwxrwx. 1 elasticsearch hadoop 588337 4?  25 2017 
> commons-collections-3.2.2.jar
> -rwxrwxrwx. 1 elasticsearch hadoop 284220 4?  25 2017 commons-lang-2.6.jar
> -rwxrwxrwx. 1 elasticsearch hadoop   2547 6?  26 09:41 
> plugin-descriptor.properties
> -rwxrwxrwx. 1 elasticsearch hadoop   1754 6?  26 09:27 
> plugin-security.policy
> drwxrwxrwx. 2 elasticsearch hadoop   4096 7?  13 09:40 
> ranger-elasticsearch-plugin-impl
> -rwxrwxrwx. 1 elasticsearch hadoop  20627 6?  26 09:36 
> ranger-elasticsearch-plugin-shim-1.1.0-SNAPSHOT.jar
> -rwxrwxrwx. 1 elasticsearch hadoop  16799 6?  26 09:35 
> ranger-plugin-classloader-1.1.0-SNAPSHOT.jar
> -rwxrwxrwx. 1 elasticsearch hadoop  26084 4?  25 2017 slf4j-api-1.7.5.jar
> -rwxrwxrwx. 1 elasticsearch hadoop   8866 6?  26 15:30 
> slf4j-log4j12-1.7.10.jar
> ```
> 
> 2.plugin-elasticseach/conf?
> ```
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
> /home/elasticsearch/elasticsearch-6.2.2/config/ranger-elasticsearch-plugin
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
> -rwxrwxrwx. 1 elasticsearch hadoop 9548 6?  26 14:15 
> ranger-elasticsearch-audit.xml
> -rwxrwxrwx. 1 elasticsearch hadoop 2773 6?  26 14:15 
> ranger-elasticsearch-security.xml
> -rwxrwxrwx. 1 elasticsearch hadoop 1917 6?  26 14:15 
> ranger-policymgr-ssl.xml
> -rwxrwxrwx. 1 elasticsearch hadoop   83 6?  26 14:15 ranger-security.xml
> ```
> 
> In addition, I have developed 2 plugins: Ranger Kylin Plugin and Ranger 
> Sqoop2 Plugin.
> In order to meet the requirements of Elasticsearch plugins 
> and take into account the general design principles of Ranger plugins,
> I think the implementation of Ranger Elasticsearch plugin is reasonable 
> at present.
> Try install this plugin , and you can get a better understanding of the 
> implementation of it.
> 
> Ramesh Mani wrote:
> If you are moving the configs from ranger-elasticsearch-plugin-shim/conf 
> to 
> /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin, 
> you can have all the configs in plugin-elasticsearch/conf and during 
> installtion you can move it 
> /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin. 
> Now you have config folder in both shim and plugin folder and this is 
> confusing.
> 
> I know that there are elastic-search specific classes which needs to be 
> there in shim folder which is fine.

The configs of ranger-elasticsearch-plugin-shim/conf can not be moved to 
/home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin,
because they are the necessary components of Elasticsearch plugin,
If they are removed, it will result in Ranger Elasticsearch plugin not loaded 
or startup failure.
Especially this plugin-descriptor.properties,
please refer the following official description:
```
# Elasticsearch plugin descriptor file
# This file must exist as 'plugin-descriptor.properties' in a folder named 
`elasticsearch`
# inside all plugins.
#
### example plugin for "foo"
#
# foo.zip <-- zip file for the plugin, with this structure:
#|elasticsearch/

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-09-10 Thread Ramesh Mani 


> On Sept. 6, 2018, 6:30 p.m., Ramesh Mani wrote:
> > ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties
> > Lines 46 (patched)
> > 
> >
> > Why is -shim- is having the conf and classes which are core to the 
> > plugin it should part of /plugin-elasticseach/conf folder.
> > -shim is to hold the ranger-classloader.
> > Please refer the existing plugin and see if that can be done.
> 
> Qiang Zhang wrote:
> Because unlike other Hadoop components, Elasticseach is designed to be 
> pluggable.
> To implement a new extension function, 
> we need to organize code and configuration files according to the 
> requirements of Elasticseach.
> Some classes in ranger-elasticsearch-plugin-shim is necessary to mount on 
> Elasticseach.
> Other classes is due to the fact that Elasticseach itself does not 
> support user authentication,
> so this plugin should work with other Elasticsearch plugin to 
> authenticate users,
> such as Basic Authentication, Kerberos, LDAP, etc.
> Or, in the future, we can realize user authentication in 
> ranger-elasticsearch-plugin-shim.
> So these classes can't sink into plugin-elasticseach.
> And plugin-descriptor.properties is for Elasticseach to start the Ranger 
> Elasticseach plugin.
> But the files in the plugin-elasticseach/conf directory are for Ranger 
> Elasticseach plugin.
> 
> The related directory structure after plugin installed is as follows:
> 1.ranger-elasticsearch-plugin-shim/conf/?
> ```
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
> 
> /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
> -rwxrwxrwx. 1 elasticsearch hadoop 588337 4?  25 2017 
> commons-collections-3.2.2.jar
> -rwxrwxrwx. 1 elasticsearch hadoop 284220 4?  25 2017 commons-lang-2.6.jar
> -rwxrwxrwx. 1 elasticsearch hadoop   2547 6?  26 09:41 
> plugin-descriptor.properties
> -rwxrwxrwx. 1 elasticsearch hadoop   1754 6?  26 09:27 
> plugin-security.policy
> drwxrwxrwx. 2 elasticsearch hadoop   4096 7?  13 09:40 
> ranger-elasticsearch-plugin-impl
> -rwxrwxrwx. 1 elasticsearch hadoop  20627 6?  26 09:36 
> ranger-elasticsearch-plugin-shim-1.1.0-SNAPSHOT.jar
> -rwxrwxrwx. 1 elasticsearch hadoop  16799 6?  26 09:35 
> ranger-plugin-classloader-1.1.0-SNAPSHOT.jar
> -rwxrwxrwx. 1 elasticsearch hadoop  26084 4?  25 2017 slf4j-api-1.7.5.jar
> -rwxrwxrwx. 1 elasticsearch hadoop   8866 6?  26 15:30 
> slf4j-log4j12-1.7.10.jar
> ```
> 
> 2.plugin-elasticseach/conf?
> ```
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
> /home/elasticsearch/elasticsearch-6.2.2/config/ranger-elasticsearch-plugin
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
> -rwxrwxrwx. 1 elasticsearch hadoop 9548 6?  26 14:15 
> ranger-elasticsearch-audit.xml
> -rwxrwxrwx. 1 elasticsearch hadoop 2773 6?  26 14:15 
> ranger-elasticsearch-security.xml
> -rwxrwxrwx. 1 elasticsearch hadoop 1917 6?  26 14:15 
> ranger-policymgr-ssl.xml
> -rwxrwxrwx. 1 elasticsearch hadoop   83 6?  26 14:15 ranger-security.xml
> ```
> 
> In addition, I have developed 2 plugins: Ranger Kylin Plugin and Ranger 
> Sqoop2 Plugin.
> In order to meet the requirements of Elasticsearch plugins 
> and take into account the general design principles of Ranger plugins,
> I think the implementation of Ranger Elasticsearch plugin is reasonable 
> at present.
> Try install this plugin , and you can get a better understanding of the 
> implementation of it.

If you are moving the configs from ranger-elasticsearch-plugin-shim/conf to 
/home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin, 
you can have all the configs in plugin-elasticsearch/conf and during 
installtion you can move it 
/home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin. 
Now you have config folder in both shim and plugin folder and this is confusing.

I know that there are elastic-search specific classes which needs to be there 
in shim folder which is fine.


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/#review208418
---


On Aug. 14, 2018, 8:46 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68128/
> ---
> 
> (Updated Aug. 14, 2018, 8:46 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-09-06 Thread Qiang Zhang 


> On 九月 6, 2018, 6:30 p.m., Ramesh Mani wrote:
> > ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties
> > Lines 46 (patched)
> > 
> >
> > Why is -shim- is having the conf and classes which are core to the 
> > plugin it should part of /plugin-elasticseach/conf folder.
> > -shim is to hold the ranger-classloader.
> > Please refer the existing plugin and see if that can be done.

Because unlike other Hadoop components, Elasticseach is designed to be 
pluggable.
To implement a new extension function, 
we need to organize code and configuration files according to the requirements 
of Elasticseach.
Some classes in ranger-elasticsearch-plugin-shim is necessary to mount on 
Elasticseach.
Other classes is due to the fact that Elasticseach itself does not support user 
authentication,
so this plugin should work with other Elasticsearch plugin to authenticate 
users,
such as Basic Authentication, Kerberos, LDAP, etc.
Or, in the future, we can realize user authentication in 
ranger-elasticsearch-plugin-shim.
So these classes can't sink into plugin-elasticseach.
And plugin-descriptor.properties is for Elasticseach to start the Ranger 
Elasticseach plugin.
But the files in the plugin-elasticseach/conf directory are for Ranger 
Elasticseach plugin.

The related directory structure after plugin installed is as follows:
1.ranger-elasticsearch-plugin-shim/conf/?
```
[elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
/home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin
[elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
-rwxrwxrwx. 1 elasticsearch hadoop 588337 4?  25 2017 
commons-collections-3.2.2.jar
-rwxrwxrwx. 1 elasticsearch hadoop 284220 4?  25 2017 commons-lang-2.6.jar
-rwxrwxrwx. 1 elasticsearch hadoop   2547 6?  26 09:41 
plugin-descriptor.properties
-rwxrwxrwx. 1 elasticsearch hadoop   1754 6?  26 09:27 plugin-security.policy
drwxrwxrwx. 2 elasticsearch hadoop   4096 7?  13 09:40 
ranger-elasticsearch-plugin-impl
-rwxrwxrwx. 1 elasticsearch hadoop  20627 6?  26 09:36 
ranger-elasticsearch-plugin-shim-1.1.0-SNAPSHOT.jar
-rwxrwxrwx. 1 elasticsearch hadoop  16799 6?  26 09:35 
ranger-plugin-classloader-1.1.0-SNAPSHOT.jar
-rwxrwxrwx. 1 elasticsearch hadoop  26084 4?  25 2017 slf4j-api-1.7.5.jar
-rwxrwxrwx. 1 elasticsearch hadoop   8866 6?  26 15:30 slf4j-log4j12-1.7.10.jar
```

2.plugin-elasticseach/conf?
```
[elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
/home/elasticsearch/elasticsearch-6.2.2/config/ranger-elasticsearch-plugin
[elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
-rwxrwxrwx. 1 elasticsearch hadoop 9548 6?  26 14:15 
ranger-elasticsearch-audit.xml
-rwxrwxrwx. 1 elasticsearch hadoop 2773 6?  26 14:15 
ranger-elasticsearch-security.xml
-rwxrwxrwx. 1 elasticsearch hadoop 1917 6?  26 14:15 ranger-policymgr-ssl.xml
-rwxrwxrwx. 1 elasticsearch hadoop   83 6?  26 14:15 ranger-security.xml
```

In addition, I have developed 2 plugins: Ranger Kylin Plugin and Ranger Sqoop2 
Plugin.
In order to meet the requirements of Elasticsearch plugins 
and take into account the general design principles of Ranger plugins,
I think the implementation of Ranger Elasticsearch plugin is reasonable at 
present.
Try install this plugin , and you can get a better understanding of the 
implementation of it.


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/#review208418
---


On 八月 14, 2018, 8:46 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68128/
> ---
> 
> (Updated 八月 14, 2018, 8:46 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2170
> https://issues.apache.org/jira/browse/RANGER-2170
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Elasticsearch is a distributed, RESTful search and analytics engine capable 
> of solving a growing number of use cases. 
> Like Apache Solr, it is also an index server based on Lucence.
> Ranger supports plugin to enable, monitor and manage Elasticsearch,
> to control index security of Elasticsearch.
> 
> As there is X-Pack plugin for the Elasticsearch, but it is not free.
> X-Pack is an Elastic Stack extension that bundles security, alerting, 
> monitoring, reporting, 
> and graph capabilities into one easy-to-install package.
> We refer to the Indices Privileges design of X-Pack,
> by keeping the permissions consistent,
> to make user use ranger Elasticsearch plugin

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-09-06 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/#review208418
---




ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties
Lines 46 (patched)


Why is -shim- is having the conf and classes which are core to the plugin 
it should part of /plugin-elasticseach/conf folder.
-shim is to hold the ranger-classloader.
Please refer the existing plugin and see if that can be done.


- Ramesh Mani


On Aug. 14, 2018, 8:46 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68128/
> ---
> 
> (Updated Aug. 14, 2018, 8:46 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2170
> https://issues.apache.org/jira/browse/RANGER-2170
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Elasticsearch is a distributed, RESTful search and analytics engine capable 
> of solving a growing number of use cases. 
> Like Apache Solr, it is also an index server based on Lucence.
> Ranger supports plugin to enable, monitor and manage Elasticsearch,
> to control index security of Elasticsearch.
> 
> As there is X-Pack plugin for the Elasticsearch, but it is not free.
> X-Pack is an Elastic Stack extension that bundles security, alerting, 
> monitoring, reporting, 
> and graph capabilities into one easy-to-install package.
> We refer to the Indices Privileges design of X-Pack,
> by keeping the permissions consistent,
> to make user use ranger Elasticsearch plugin easily.
> Reference X-Pack Indices Privileges:
> https://www.elastic.co/guide/en/x-pack/current/security-privileges.html
> 
> Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
> 6.2.2.
> Elasticsearch 6.2.2 was released in February 20, 2018, reference 
> release-notes:
> https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
> Not like other system, Elasticsearch has no basic authentication, 
> it uses X-pack plugin to support basic authentication, 
> role-based access control, SSL/TLS encryption, LDAP and so on.
> Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
> authorization,
> it is to control index of Elasticsearch without authentication,
> this plugin should work with other Elasticsearch plugin to authenticate users.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh ce0dc8c 
>   agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java 
> e654f2b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  5e74da8 
>   
> agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json
>  PRE-CREATION 
>   plugin-elasticsearch/.gitignore PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg 
> PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg 
> PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-elasticsearch/pom.xml PRE-CREATION 
>   plugin-elasticsearch/scripts/install.properties PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java
>  PRE-CREATION 
>   pom.xml 8d7ea13 
>   ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION 
>   ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties 
> PRE-CREATION 
>   ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION 
>   ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION 
>   
>

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-08-14 Thread Qiang Zhang 


> On 八月 10, 2018, 12:50 p.m., Zsombor Gegesy wrote:
> > plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
> > Lines 99 (patched)
> > 
> >
> > I don't get it, why this clientIPAddress is used. This is not a client 
> > IP address, but the current ElasticServer's address. This could confuse the 
> > users. Why not leaving just empty, if ES won't provide it automatically?

OK, I will get client IP address from rest request.


> On 八月 10, 2018, 12:50 p.m., Zsombor Gegesy wrote:
> > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java
> > Lines 139 (patched)
> > 
> >
> > I guess, joptsimple.internal.Strings.EMPTY is just "". I don't think, 
> > it is good idea to depend on a library just for this - and this library is 
> > not directly included in the pom.

OK, thanks.


> On 八月 10, 2018, 12:50 p.m., Zsombor Gegesy wrote:
> > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticserachPlugin.java
> > Lines 50 (patched)
> > 
> >
> > RangerElasticserachPlugin -> RangerElasticsearchPlugin

OK, thanks.


> On 八月 10, 2018, 12:50 p.m., Zsombor Gegesy wrote:
> > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
> > Lines 76 (patched)
> > 
> >
> > Isn't it possible to get the user's IP address from 
> > action.remoteAddress()? I'm not too familiar with ES inner workings.

Thanks, I find the user's IP address can be getted from RestRequest in the 
following method:
org.apache.ranger.authorization.elasticsearch.plugin.rest.filter.RangerSecurityRestFilter.handleRequest(RestRequest,
 RestChannel, NodeClient)


> On 八月 10, 2018, 12:50 p.m., Zsombor Gegesy wrote:
> > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
> > Lines 88 (patched)
> > 
> >
> > Isn't it a security problem, if no user name is there, the plugin 
> > allows everything?

1.It is designed for elasticsearch internal call, such as elasticsearch start 
up with no user name, 
otherwise elasticsearch cluster could not work.

2.Restful API call with no user name would throw exception.
Please refer the following test:
curl -X GET "localhost:9200/twitter/_stats?pretty"
{
  "error" : {
"root_cause" : [
  {
"type" : "status_exception",
"reason" : "Error: User is null, the request requires user 
authentication."
  }
],
"type" : "status_exception",
"reason" : "Error: User is null, the request requires user authentication."
  },
  "status" : 401
}

3.May be there is better design in the future,
to do more rigorous verification for elasticsearch cluster.


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/#review207073
---


On 八月 14, 2018, 8:46 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68128/
> ---
> 
> (Updated 八月 14, 2018, 8:46 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2170
> https://issues.apache.org/jira/browse/RANGER-2170
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Elasticsearch is a distributed, RESTful search and analytics engine capable 
> of solving a growing number of use cases. 
> Like Apache Solr, it is also an index server based on Lucence.
> Ranger supports plugin to enable, monitor and manage Elasticsearch,
> to control index security of Elasticsearch.
> 
> As there is X-Pack plugin for the Elasticsearch, but it is not free.
> X-Pack is an Elastic Stack extension that bundles security, alerting, 
> monitoring, reporting, 
> and graph capabilities into one easy-to-install package.
> We refer to the Indices Privileges design of X-Pack,
> by keeping the permissions consistent,
> to make user use ranger Elasticsearch plugin easily.
> Reference X-Pack Indices Privileges:
> https://www.elastic.co/guide/en/x-pack/current/security-privileges.html
> 
> Here we

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-08-14 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/
---

(Updated 八月 14, 2018, 8:46 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.


Changes
---

Changes:
1.Get client IP address from rest request for audit log.
2.Fix spelling error for "RangerElasticsearchPlugin".
3.Remove joptsimple.internal.Strings.EMPTY, and use 
org.apache.commons.lang.StringUtils.EMPTY instead.


Bugs: RANGER-2170
https://issues.apache.org/jira/browse/RANGER-2170


Repository: ranger


Description
---

Elasticsearch is a distributed, RESTful search and analytics engine capable of 
solving a growing number of use cases. 
Like Apache Solr, it is also an index server based on Lucence.
Ranger supports plugin to enable, monitor and manage Elasticsearch,
to control index security of Elasticsearch.

As there is X-Pack plugin for the Elasticsearch, but it is not free.
X-Pack is an Elastic Stack extension that bundles security, alerting, 
monitoring, reporting, 
and graph capabilities into one easy-to-install package.
We refer to the Indices Privileges design of X-Pack,
by keeping the permissions consistent,
to make user use ranger Elasticsearch plugin easily.
Reference X-Pack Indices Privileges:
https://www.elastic.co/guide/en/x-pack/current/security-privileges.html

Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
6.2.2.
Elasticsearch 6.2.2 was released in February 20, 2018, reference release-notes:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
Not like other system, Elasticsearch has no basic authentication, 
it uses X-pack plugin to support basic authentication, 
role-based access control, SSL/TLS encryption, LDAP and so on.
Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
authorization,
it is to control index of Elasticsearch without authentication,
this plugin should work with other Elasticsearch plugin to authenticate users.


Diffs (updated)
-

  agents-common/scripts/enable-agent.sh ce0dc8c 
  agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java 
e654f2b 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 5e74da8 
  
agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json
 PRE-CREATION 
  plugin-elasticsearch/.gitignore PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg 
PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-elasticsearch/pom.xml PRE-CREATION 
  plugin-elasticsearch/scripts/install.properties PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java
 PRE-CREATION 
  pom.xml 8d7ea13 
  ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties 
PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION 
  ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticsearchPlugin.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
 PRE-CREATION

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-08-10 Thread Zsombor Gegesy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/#review207073
---




plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
Lines 99 (patched)


I don't get it, why this clientIPAddress is used. This is not a client IP 
address, but the current ElasticServer's address. This could confuse the users. 
Why not leaving just empty, if ES won't provide it automatically?



plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java
Lines 139 (patched)


I guess, joptsimple.internal.Strings.EMPTY is just "". I don't think, it is 
good idea to depend on a library just for this - and this library is not 
directly included in the pom.



ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticserachPlugin.java
Lines 50 (patched)


RangerElasticserachPlugin -> RangerElasticsearchPlugin



ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
Lines 76 (patched)


Isn't it possible to get the user's IP address from action.remoteAddress()? 
I'm not too familiar with ES inner workings.



ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
Lines 88 (patched)


Isn't it a security problem, if no user name is there, the plugin allows 
everything?


- Zsombor Gegesy


On Aug. 1, 2018, 9:27 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68128/
> ---
> 
> (Updated Aug. 1, 2018, 9:27 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2170
> https://issues.apache.org/jira/browse/RANGER-2170
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Elasticsearch is a distributed, RESTful search and analytics engine capable 
> of solving a growing number of use cases. 
> Like Apache Solr, it is also an index server based on Lucence.
> Ranger supports plugin to enable, monitor and manage Elasticsearch,
> to control index security of Elasticsearch.
> 
> As there is X-Pack plugin for the Elasticsearch, but it is not free.
> X-Pack is an Elastic Stack extension that bundles security, alerting, 
> monitoring, reporting, 
> and graph capabilities into one easy-to-install package.
> We refer to the Indices Privileges design of X-Pack,
> by keeping the permissions consistent,
> to make user use ranger Elasticsearch plugin easily.
> Reference X-Pack Indices Privileges:
> https://www.elastic.co/guide/en/x-pack/current/security-privileges.html
> 
> Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
> 6.2.2.
> Elasticsearch 6.2.2 was released in February 20, 2018, reference 
> release-notes:
> https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
> Not like other system, Elasticsearch has no basic authentication, 
> it uses X-pack plugin to support basic authentication, 
> role-based access control, SSL/TLS encryption, LDAP and so on.
> Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
> authorization,
> it is to control index of Elasticsearch without authentication,
> this plugin should work with other Elasticsearch plugin to authenticate users.
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh ce0dc8c 
>   agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java 
> e654f2b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  5e74da8 
>   
> agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json
>  PRE-CREATION 
>   plugin-elasticsearch/.gitignore PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg 
> PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg 
> PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-08-01 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/
---

(Updated 八月 1, 2018, 9:27 a.m.)


Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.


Bugs: RANGER-2170
https://issues.apache.org/jira/browse/RANGER-2170


Repository: ranger


Description
---

Elasticsearch is a distributed, RESTful search and analytics engine capable of 
solving a growing number of use cases. 
Like Apache Solr, it is also an index server based on Lucence.
Ranger supports plugin to enable, monitor and manage Elasticsearch,
to control index security of Elasticsearch.

As there is X-Pack plugin for the Elasticsearch, but it is not free.
X-Pack is an Elastic Stack extension that bundles security, alerting, 
monitoring, reporting, 
and graph capabilities into one easy-to-install package.
We refer to the Indices Privileges design of X-Pack,
by keeping the permissions consistent,
to make user use ranger Elasticsearch plugin easily.
Reference X-Pack Indices Privileges:
https://www.elastic.co/guide/en/x-pack/current/security-privileges.html

Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
6.2.2.
Elasticsearch 6.2.2 was released in February 20, 2018, reference release-notes:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
Not like other system, Elasticsearch has no basic authentication, 
it uses X-pack plugin to support basic authentication, 
role-based access control, SSL/TLS encryption, LDAP and so on.
Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
authorization,
it is to control index of Elasticsearch without authentication,
this plugin should work with other Elasticsearch plugin to authenticate users.


Diffs
-

  agents-common/scripts/enable-agent.sh ce0dc8c 
  agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java 
e654f2b 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 5e74da8 
  
agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json
 PRE-CREATION 
  plugin-elasticsearch/.gitignore PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg 
PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-elasticsearch/pom.xml PRE-CREATION 
  plugin-elasticsearch/scripts/install.properties PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java
 PRE-CREATION 
  pom.xml 8d7ea13 
  ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties 
PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION 
  ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticserachPlugin.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/authc/user/UsernamePasswordToken.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/rest/filter/RangerSecurityRestFilter.java

Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-07-31 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/
---

Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, 
Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, 
sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.


Bugs: RANGER-2170
https://issues.apache.org/jira/browse/RANGER-2170


Repository: ranger


Description
---

Elasticsearch is a distributed, RESTful search and analytics engine capable of 
solving a growing number of use cases. 
Like Apache Solr, it is also an index server based on Lucence.
Ranger supports plugin to enable, monitor and manage Elasticsearch,
to control index security of Elasticsearch.

As there is X-Pack plugin for the Elasticsearch, but it is not free.
X-Pack is an Elastic Stack extension that bundles security, alerting, 
monitoring, reporting, 
and graph capabilities into one easy-to-install package.
We refer to the Indices Privileges design of X-Pack,
by keeping the permissions consistent,
to make user use ranger Elasticsearch plugin easily.
Reference X-Pack Indices Privileges:
https://www.elastic.co/guide/en/x-pack/current/security-privileges.html

Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
6.2.2.
Elasticsearch 6.2.2 was released in February 20, 2018, reference release-notes:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
Not like other system, Elasticsearch has no basic authentication, 
it uses X-pack plugin to support basic authentication, 
role-based access control, SSL/TLS encryption, LDAP and so on.
Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
authorization,
it is to control index of Elasticsearch without authentication,
this plugin should work with other Elasticsearch plugin to authenticate users.


Diffs
-

  agents-common/scripts/enable-agent.sh ce0dc8c 
  agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java 
e654f2b 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
 5e74da8 
  
agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json
 PRE-CREATION 
  plugin-elasticsearch/.gitignore PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg 
PRE-CREATION 
  plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
  plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION 
  plugin-elasticsearch/pom.xml PRE-CREATION 
  plugin-elasticsearch/scripts/install.properties PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java
 PRE-CREATION 
  
plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java
 PRE-CREATION 
  pom.xml 8d7ea13 
  ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties 
PRE-CREATION 
  ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION 
  ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticserachPlugin.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/authc/user/UsernamePasswordToken.java
 PRE-CREATION 
  
ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/rest/filter/RangerSecurityRestFilter.java
 PRE-CREATION