> On 九月 6, 2018, 6:30 p.m., Ramesh Mani wrote: > > ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties > > Lines 46 (patched) > > <https://reviews.apache.org/r/68128/diff/2/?file=2072368#file2072368line46> > > > > Why is -shim- is having the conf and classes which are core to the > > plugin it should part of /plugin-elasticseach/conf folder. > > -shim is to hold the ranger-classloader. > > Please refer the existing plugin and see if that can be done.
Because unlike other Hadoop components, Elasticseach is designed to be pluggable. To implement a new extension function, we need to organize code and configuration files according to the requirements of Elasticseach. Some classes in ranger-elasticsearch-plugin-shim is necessary to mount on Elasticseach. Other classes is due to the fact that Elasticseach itself does not support user authentication, so this plugin should work with other Elasticsearch plugin to authenticate users, such as Basic Authentication, Kerberos, LDAP, etc. Or, in the future, we can realize user authentication in ranger-elasticsearch-plugin-shim. So these classes can't sink into plugin-elasticseach. And plugin-descriptor.properties is for Elasticseach to start the Ranger Elasticseach plugin. But the files in the plugin-elasticseach/conf directory are for Ranger Elasticseach plugin. The related directory structure after plugin installed is as follows: 1.ranger-elasticsearch-plugin-shim/conf/? ``` [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll -rwxrwxrwx. 1 elasticsearch hadoop 588337 4? 25 2017 commons-collections-3.2.2.jar -rwxrwxrwx. 1 elasticsearch hadoop 284220 4? 25 2017 commons-lang-2.6.jar -rwxrwxrwx. 1 elasticsearch hadoop 2547 6? 26 09:41 plugin-descriptor.properties -rwxrwxrwx. 1 elasticsearch hadoop 1754 6? 26 09:27 plugin-security.policy drwxrwxrwx. 2 elasticsearch hadoop 4096 7? 13 09:40 ranger-elasticsearch-plugin-impl -rwxrwxrwx. 1 elasticsearch hadoop 20627 6? 26 09:36 ranger-elasticsearch-plugin-shim-1.1.0-SNAPSHOT.jar -rwxrwxrwx. 1 elasticsearch hadoop 16799 6? 26 09:35 ranger-plugin-classloader-1.1.0-SNAPSHOT.jar -rwxrwxrwx. 1 elasticsearch hadoop 26084 4? 25 2017 slf4j-api-1.7.5.jar -rwxrwxrwx. 1 elasticsearch hadoop 8866 6? 26 15:30 slf4j-log4j12-1.7.10.jar ``` 2.plugin-elasticseach/conf? ``` [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd /home/elasticsearch/elasticsearch-6.2.2/config/ranger-elasticsearch-plugin [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll -rwxrwxrwx. 1 elasticsearch hadoop 9548 6? 26 14:15 ranger-elasticsearch-audit.xml -rwxrwxrwx. 1 elasticsearch hadoop 2773 6? 26 14:15 ranger-elasticsearch-security.xml -rwxrwxrwx. 1 elasticsearch hadoop 1917 6? 26 14:15 ranger-policymgr-ssl.xml -rwxrwxrwx. 1 elasticsearch hadoop 83 6? 26 14:15 ranger-security.xml ``` In addition, I have developed 2 plugins: Ranger Kylin Plugin and Ranger Sqoop2 Plugin. In order to meet the requirements of Elasticsearch plugins and take into account the general design principles of Ranger plugins, I think the implementation of Ranger Elasticsearch plugin is reasonable at present. Try install this plugin , and you can get a better understanding of the implementation of it. - Qiang ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68128/#review208418 ----------------------------------------------------------- On 八月 14, 2018, 8:46 a.m., Qiang Zhang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68128/ > ----------------------------------------------------------- > > (Updated 八月 14, 2018, 8:46 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy. > > > Bugs: RANGER-2170 > https://issues.apache.org/jira/browse/RANGER-2170 > > > Repository: ranger > > > Description > ------- > > Elasticsearch is a distributed, RESTful search and analytics engine capable > of solving a growing number of use cases. > Like Apache Solr, it is also an index server based on Lucence. > Ranger supports plugin to enable, monitor and manage Elasticsearch, > to control index security of Elasticsearch. > > As there is X-Pack plugin for the Elasticsearch, but it is not free. > X-Pack is an Elastic Stack extension that bundles security, alerting, > monitoring, reporting, > and graph capabilities into one easy-to-install package. > We refer to the Indices Privileges design of X-Pack, > by keeping the permissions consistent, > to make user use ranger Elasticsearch plugin easily. > Reference X-Pack Indices Privileges: > https://www.elastic.co/guide/en/x-pack/current/security-privileges.html > > Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version > 6.2.2. > Elasticsearch 6.2.2 was released in February 20, 2018, reference > release-notes: > https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html > Not like other system, Elasticsearch has no basic authentication, > it uses X-pack plugin to support basic authentication, > role-based access control, SSL/TLS encryption, LDAP and so on. > Not like X-pack, our Ranger Elasticsearch plugin is designed to do > authorization, > it is to control index of Elasticsearch without authentication, > this plugin should work with other Elasticsearch plugin to authenticate users. > > > Diffs > ----- > > agents-common/scripts/enable-agent.sh ce0dc8c > agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java > e654f2b > > agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java > 5e74da8 > > agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json > PRE-CREATION > plugin-elasticsearch/.gitignore PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg > PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg > PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION > plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION > plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION > plugin-elasticsearch/pom.xml PRE-CREATION > plugin-elasticsearch/scripts/install.properties PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java > PRE-CREATION > pom.xml 8d7ea13 > ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION > ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties > PRE-CREATION > ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION > ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticsearchPlugin.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/authc/user/UsernamePasswordToken.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/rest/filter/RangerSecurityRestFilter.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/utils/RequestUtils.java > PRE-CREATION > src/main/assembly/admin-web.xml d0f3545 > src/main/assembly/plugin-elasticsearch.xml PRE-CREATION > > > Diff: https://reviews.apache.org/r/68128/diff/2/ > > > Testing > ------- > > #Test Steps: > > 1.Intall > Ranger Elasticsearch Plugin Installation Guide > https://cwiki.apache.org/confluence/display/RANGER/Elasticsearch+Plugin > Include install Elasticsearch and Ranger Elasticsearch Plugin, > and verify install result. > > 2.Create policy in Ranger Admin > User "elasticsearch" has all permissions on all indices. > User "yuwen" has permission "read" on index "twitter". > > 3.Test permission > > 3.1 successful: > curl -u elasticsearch:xxx -X GET "localhost:9200/twitter/_stats?pretty" > curl -u elasticsearch:xxx -X GET "localhost:9200/twitter2/_stats?pretty" > curl -u yuwen:xxx -X GET "localhost:9200/twitter/_stats?pretty" > > 3.2 failed: > curl -X GET "localhost:9200/twitter/_stats?pretty" > { > "error" : { > "root_cause" : [ > { > "type" : "status_exception", > "reason" : "Error: User is null, the request requires user > authentication." > } > ], > "type" : "status_exception", > "reason" : "Error: User is null, the request requires user > authentication." > }, > "status" : 401 > } > > curl -u yuwen:xxx -X GET "localhost:9200/twitter2/_stats?pretty" > { > "error" : { > "root_cause" : [ > { > "type" : "status_exception", > "reason" : "Error: User[yuwen] could not do > action[indices:monitor/stats] on index[twitter2]" > } > ], > "type" : "status_exception", > "reason" : "Error: User[yuwen] could not do action[indices:monitor/stats] > on index[twitter2]" > }, > "status" : 403 > } > > > Thanks, > > Qiang Zhang > >
