> On 九月 6, 2018, 6:30 p.m., Ramesh Mani wrote:
> > ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties
> > Lines 46 (patched)
> > <https://reviews.apache.org/r/68128/diff/2/?file=2072368#file2072368line46>
> >
> >     Why is -shim- is having the conf and classes which are core to the 
> > plugin it should part of /plugin-elasticseach/conf folder.
> >     -shim is to hold the ranger-classloader.
> >     Please refer the existing plugin and see if that can be done.

Because unlike other Hadoop components, Elasticseach is designed to be 
pluggable.
To implement a new extension function, 
we need to organize code and configuration files according to the requirements 
of Elasticseach.
Some classes in ranger-elasticsearch-plugin-shim is necessary to mount on 
Elasticseach.
Other classes is due to the fact that Elasticseach itself does not support user 
authentication,
so this plugin should work with other Elasticsearch plugin to authenticate 
users,
such as Basic Authentication, Kerberos, LDAP, etc.
Or, in the future, we can realize user authentication in 
ranger-elasticsearch-plugin-shim.
So these classes can't sink into plugin-elasticseach.
And plugin-descriptor.properties is for Elasticseach to start the Ranger 
Elasticseach plugin.
But the files in the plugin-elasticseach/conf directory are for Ranger 
Elasticseach plugin.

The related directory structure after plugin installed is as follows:
1.ranger-elasticsearch-plugin-shim/conf/?
```
[elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
/home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin
[elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
-rwxrwxrwx. 1 elasticsearch hadoop 588337 4?  25 2017 
commons-collections-3.2.2.jar
-rwxrwxrwx. 1 elasticsearch hadoop 284220 4?  25 2017 commons-lang-2.6.jar
-rwxrwxrwx. 1 elasticsearch hadoop   2547 6?  26 09:41 
plugin-descriptor.properties
-rwxrwxrwx. 1 elasticsearch hadoop   1754 6?  26 09:27 plugin-security.policy
drwxrwxrwx. 2 elasticsearch hadoop   4096 7?  13 09:40 
ranger-elasticsearch-plugin-impl
-rwxrwxrwx. 1 elasticsearch hadoop  20627 6?  26 09:36 
ranger-elasticsearch-plugin-shim-1.1.0-SNAPSHOT.jar
-rwxrwxrwx. 1 elasticsearch hadoop  16799 6?  26 09:35 
ranger-plugin-classloader-1.1.0-SNAPSHOT.jar
-rwxrwxrwx. 1 elasticsearch hadoop  26084 4?  25 2017 slf4j-api-1.7.5.jar
-rwxrwxrwx. 1 elasticsearch hadoop   8866 6?  26 15:30 slf4j-log4j12-1.7.10.jar
```

2.plugin-elasticseach/conf?
```
[elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
/home/elasticsearch/elasticsearch-6.2.2/config/ranger-elasticsearch-plugin
[elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
-rwxrwxrwx. 1 elasticsearch hadoop 9548 6?  26 14:15 
ranger-elasticsearch-audit.xml
-rwxrwxrwx. 1 elasticsearch hadoop 2773 6?  26 14:15 
ranger-elasticsearch-security.xml
-rwxrwxrwx. 1 elasticsearch hadoop 1917 6?  26 14:15 ranger-policymgr-ssl.xml
-rwxrwxrwx. 1 elasticsearch hadoop   83 6?  26 14:15 ranger-security.xml
```

In addition, I have developed 2 plugins: Ranger Kylin Plugin and Ranger Sqoop2 
Plugin.
In order to meet the requirements of Elasticsearch plugins 
and take into account the general design principles of Ranger plugins,
I think the implementation of Ranger Elasticsearch plugin is reasonable at 
present.
Try install this plugin , and you can get a better understanding of the 
implementation of it.


- Qiang


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68128/#review208418
-----------------------------------------------------------


On 八月 14, 2018, 8:46 a.m., Qiang Zhang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68128/
> -----------------------------------------------------------
> 
> (Updated 八月 14, 2018, 8:46 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2170
>     https://issues.apache.org/jira/browse/RANGER-2170
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> Elasticsearch is a distributed, RESTful search and analytics engine capable 
> of solving a growing number of use cases. 
> Like Apache Solr, it is also an index server based on Lucence.
> Ranger supports plugin to enable, monitor and manage Elasticsearch,
> to control index security of Elasticsearch.
> 
> As there is X-Pack plugin for the Elasticsearch, but it is not free.
> X-Pack is an Elastic Stack extension that bundles security, alerting, 
> monitoring, reporting, 
> and graph capabilities into one easy-to-install package.
> We refer to the Indices Privileges design of X-Pack,
> by keeping the permissions consistent,
> to make user use ranger Elasticsearch plugin easily.
> Reference X-Pack Indices Privileges:
> https://www.elastic.co/guide/en/x-pack/current/security-privileges.html
> 
> Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version 
> 6.2.2.
> Elasticsearch 6.2.2 was released in February 20, 2018, reference 
> release-notes:
> https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html
> Not like other system, Elasticsearch has no basic authentication, 
> it uses X-pack plugin to support basic authentication, 
> role-based access control, SSL/TLS encryption, LDAP and so on.
> Not like X-pack, our Ranger Elasticsearch plugin is designed to do 
> authorization,
> it is to control index of Elasticsearch without authentication,
> this plugin should work with other Elasticsearch plugin to authenticate users.
> 
> 
> Diffs
> -----
> 
>   agents-common/scripts/enable-agent.sh ce0dc8c 
>   agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java 
> e654f2b 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  5e74da8 
>   
> agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json
>  PRE-CREATION 
>   plugin-elasticsearch/.gitignore PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg 
> PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg 
> PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-elasticsearch/pom.xml PRE-CREATION 
>   plugin-elasticsearch/scripts/install.properties PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java
>  PRE-CREATION 
>   
> plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java
>  PRE-CREATION 
>   pom.xml 8d7ea13 
>   ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION 
>   ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties 
> PRE-CREATION 
>   ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION 
>   ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java
>  PRE-CREATION 
>   
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java
>  PRE-CREATION 
>   
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticsearchPlugin.java
>  PRE-CREATION 
>   
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java
>  PRE-CREATION 
>   
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/authc/user/UsernamePasswordToken.java
>  PRE-CREATION 
>   
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/rest/filter/RangerSecurityRestFilter.java
>  PRE-CREATION 
>   
> ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/utils/RequestUtils.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml d0f3545 
>   src/main/assembly/plugin-elasticsearch.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68128/diff/2/
> 
> 
> Testing
> -------
> 
> #Test Steps:
> 
> 1.Intall
> Ranger Elasticsearch Plugin Installation Guide        
> https://cwiki.apache.org/confluence/display/RANGER/Elasticsearch+Plugin
> Include install Elasticsearch and Ranger Elasticsearch Plugin,
> and verify install result.
> 
> 2.Create policy in Ranger Admin
> User "elasticsearch" has all permissions on all indices.
> User "yuwen" has permission "read" on index "twitter".
> 
> 3.Test permission
> 
> 3.1 successful:
> curl -u elasticsearch:xxx -X GET "localhost:9200/twitter/_stats?pretty"
> curl -u elasticsearch:xxx -X GET "localhost:9200/twitter2/_stats?pretty"
> curl -u yuwen:xxx -X GET "localhost:9200/twitter/_stats?pretty"
> 
> 3.2 failed:
> curl -X GET "localhost:9200/twitter/_stats?pretty"
> {
>   "error" : {
>     "root_cause" : [
>       {
>         "type" : "status_exception",
>         "reason" : "Error: User is null, the request requires user 
> authentication."
>       }
>     ],
>     "type" : "status_exception",
>     "reason" : "Error: User is null, the request requires user 
> authentication."
>   },
>   "status" : 401
> }
> 
> curl -u yuwen:xxx -X GET "localhost:9200/twitter2/_stats?pretty"
> {
>   "error" : {
>     "root_cause" : [
>       {
>         "type" : "status_exception",
>         "reason" : "Error: User[yuwen] could not do 
> action[indices:monitor/stats] on index[twitter2]"
>       }
>     ],
>     "type" : "status_exception",
>     "reason" : "Error: User[yuwen] could not do action[indices:monitor/stats] 
> on index[twitter2]"
>   },
>   "status" : 403
> }
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

Reply via email to