> On 九月 6, 2018, 6:30 p.m., Ramesh Mani wrote: > > ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties > > Lines 46 (patched) > > <https://reviews.apache.org/r/68128/diff/2/?file=2072368#file2072368line46> > > > > Why is -shim- is having the conf and classes which are core to the > > plugin it should part of /plugin-elasticseach/conf folder. > > -shim is to hold the ranger-classloader. > > Please refer the existing plugin and see if that can be done. > > Qiang Zhang wrote: > Because unlike other Hadoop components, Elasticseach is designed to be > pluggable. > To implement a new extension function, > we need to organize code and configuration files according to the > requirements of Elasticseach. > Some classes in ranger-elasticsearch-plugin-shim is necessary to mount on > Elasticseach. > Other classes is due to the fact that Elasticseach itself does not > support user authentication, > so this plugin should work with other Elasticsearch plugin to > authenticate users, > such as Basic Authentication, Kerberos, LDAP, etc. > Or, in the future, we can realize user authentication in > ranger-elasticsearch-plugin-shim. > So these classes can't sink into plugin-elasticseach. > And plugin-descriptor.properties is for Elasticseach to start the Ranger > Elasticseach plugin. > But the files in the plugin-elasticseach/conf directory are for Ranger > Elasticseach plugin. > > The related directory structure after plugin installed is as follows: > 1.ranger-elasticsearch-plugin-shim/conf/? > ``` > [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd > > /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin > [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll > -rwxrwxrwx. 1 elasticsearch hadoop 588337 4? 25 2017 > commons-collections-3.2.2.jar > -rwxrwxrwx. 1 elasticsearch hadoop 284220 4? 25 2017 commons-lang-2.6.jar > -rwxrwxrwx. 1 elasticsearch hadoop 2547 6? 26 09:41 > plugin-descriptor.properties > -rwxrwxrwx. 1 elasticsearch hadoop 1754 6? 26 09:27 > plugin-security.policy > drwxrwxrwx. 2 elasticsearch hadoop 4096 7? 13 09:40 > ranger-elasticsearch-plugin-impl > -rwxrwxrwx. 1 elasticsearch hadoop 20627 6? 26 09:36 > ranger-elasticsearch-plugin-shim-1.1.0-SNAPSHOT.jar > -rwxrwxrwx. 1 elasticsearch hadoop 16799 6? 26 09:35 > ranger-plugin-classloader-1.1.0-SNAPSHOT.jar > -rwxrwxrwx. 1 elasticsearch hadoop 26084 4? 25 2017 slf4j-api-1.7.5.jar > -rwxrwxrwx. 1 elasticsearch hadoop 8866 6? 26 15:30 > slf4j-log4j12-1.7.10.jar > ``` > > 2.plugin-elasticseach/conf? > ``` > [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd > /home/elasticsearch/elasticsearch-6.2.2/config/ranger-elasticsearch-plugin > [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll > -rwxrwxrwx. 1 elasticsearch hadoop 9548 6? 26 14:15 > ranger-elasticsearch-audit.xml > -rwxrwxrwx. 1 elasticsearch hadoop 2773 6? 26 14:15 > ranger-elasticsearch-security.xml > -rwxrwxrwx. 1 elasticsearch hadoop 1917 6? 26 14:15 > ranger-policymgr-ssl.xml > -rwxrwxrwx. 1 elasticsearch hadoop 83 6? 26 14:15 ranger-security.xml > ``` > > In addition, I have developed 2 plugins: Ranger Kylin Plugin and Ranger > Sqoop2 Plugin. > In order to meet the requirements of Elasticsearch plugins > and take into account the general design principles of Ranger plugins, > I think the implementation of Ranger Elasticsearch plugin is reasonable > at present. > Try install this plugin , and you can get a better understanding of the > implementation of it. > > Ramesh Mani wrote: > If you are moving the configs from ranger-elasticsearch-plugin-shim/conf > to > /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin, > you can have all the configs in plugin-elasticsearch/conf and during > installtion you can move it > /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin. > Now you have config folder in both shim and plugin folder and this is > confusing. > > I know that there are elastic-search specific classes which needs to be > there in shim folder which is fine.
The configs of ranger-elasticsearch-plugin-shim/conf can not be moved to /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin, because they are the necessary components of Elasticsearch plugin, If they are removed, it will result in Ranger Elasticsearch plugin not loaded or startup failure. Especially this plugin-descriptor.properties, please refer the following official description: ``` # Elasticsearch plugin descriptor file # This file must exist as 'plugin-descriptor.properties' in a folder named `elasticsearch` # inside all plugins. # ### example plugin for "foo" # # foo.zip <-- zip file for the plugin, with this structure: #|____elasticsearch/ #| |____ <arbitrary name1>.jar <-- classes, resources, dependencies #| |____ <arbitrary nameN>.jar <-- any number of jars #| |____ plugin-descriptor.properties <-- example contents below: # # classname=foo.bar.BazPlugin # description=My cool plugin # version=2.0 # elasticsearch.version=2.0 # java.version=1.7 # ``` These configuration files are strongly related to Elasticsearch, which ard loaded by Elasticsearch. and usually they will not be changed. But the configs of plugin-elasticsearch/conf are related to Ranger, which ard loaded by Ranger plugin. So they should be separated, but putting them together will be confusing. - Qiang ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68128/#review208418 ----------------------------------------------------------- On 八月 14, 2018, 8:46 a.m., Qiang Zhang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68128/ > ----------------------------------------------------------- > > (Updated 八月 14, 2018, 8:46 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy. > > > Bugs: RANGER-2170 > https://issues.apache.org/jira/browse/RANGER-2170 > > > Repository: ranger > > > Description > ------- > > Elasticsearch is a distributed, RESTful search and analytics engine capable > of solving a growing number of use cases. > Like Apache Solr, it is also an index server based on Lucence. > Ranger supports plugin to enable, monitor and manage Elasticsearch, > to control index security of Elasticsearch. > > As there is X-Pack plugin for the Elasticsearch, but it is not free. > X-Pack is an Elastic Stack extension that bundles security, alerting, > monitoring, reporting, > and graph capabilities into one easy-to-install package. > We refer to the Indices Privileges design of X-Pack, > by keeping the permissions consistent, > to make user use ranger Elasticsearch plugin easily. > Reference X-Pack Indices Privileges: > https://www.elastic.co/guide/en/x-pack/current/security-privileges.html > > Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version > 6.2.2. > Elasticsearch 6.2.2 was released in February 20, 2018, reference > release-notes: > https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html > Not like other system, Elasticsearch has no basic authentication, > it uses X-pack plugin to support basic authentication, > role-based access control, SSL/TLS encryption, LDAP and so on. > Not like X-pack, our Ranger Elasticsearch plugin is designed to do > authorization, > it is to control index of Elasticsearch without authentication, > this plugin should work with other Elasticsearch plugin to authenticate users. > > > Diffs > ----- > > agents-common/scripts/enable-agent.sh ce0dc8c > agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java > e654f2b > > agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java > 5e74da8 > > agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json > PRE-CREATION > plugin-elasticsearch/.gitignore PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg > PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-security-changes.cfg > PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-security.xml PRE-CREATION > plugin-elasticsearch/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION > plugin-elasticsearch/conf/ranger-policymgr-ssl.xml PRE-CREATION > plugin-elasticsearch/pom.xml PRE-CREATION > plugin-elasticsearch/scripts/install.properties PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/RangerServiceElasticsearch.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchResourceMgr.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilege.java > PRE-CREATION > > plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/privilege/IndexPrivilegeUtils.java > PRE-CREATION > pom.xml 8d7ea13 > ranger-elasticsearch-plugin-shim/.gitignore PRE-CREATION > ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties > PRE-CREATION > ranger-elasticsearch-plugin-shim/conf/plugin-security.policy PRE-CREATION > ranger-elasticsearch-plugin-shim/pom.xml PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAccessControl.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticsearchPlugin.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/authc/user/UsernamePasswordToken.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/rest/filter/RangerSecurityRestFilter.java > PRE-CREATION > > ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/utils/RequestUtils.java > PRE-CREATION > src/main/assembly/admin-web.xml d0f3545 > src/main/assembly/plugin-elasticsearch.xml PRE-CREATION > > > Diff: https://reviews.apache.org/r/68128/diff/2/ > > > Testing > ------- > > #Test Steps: > > 1.Intall > Ranger Elasticsearch Plugin Installation Guide > https://cwiki.apache.org/confluence/display/RANGER/Elasticsearch+Plugin > Include install Elasticsearch and Ranger Elasticsearch Plugin, > and verify install result. > > 2.Create policy in Ranger Admin > User "elasticsearch" has all permissions on all indices. > User "yuwen" has permission "read" on index "twitter". > > 3.Test permission > > 3.1 successful: > curl -u elasticsearch:xxx -X GET "localhost:9200/twitter/_stats?pretty" > curl -u elasticsearch:xxx -X GET "localhost:9200/twitter2/_stats?pretty" > curl -u yuwen:xxx -X GET "localhost:9200/twitter/_stats?pretty" > > 3.2 failed: > curl -X GET "localhost:9200/twitter/_stats?pretty" > { > "error" : { > "root_cause" : [ > { > "type" : "status_exception", > "reason" : "Error: User is null, the request requires user > authentication." > } > ], > "type" : "status_exception", > "reason" : "Error: User is null, the request requires user > authentication." > }, > "status" : 401 > } > > curl -u yuwen:xxx -X GET "localhost:9200/twitter2/_stats?pretty" > { > "error" : { > "root_cause" : [ > { > "type" : "status_exception", > "reason" : "Error: User[yuwen] could not do > action[indices:monitor/stats] on index[twitter2]" > } > ], > "type" : "status_exception", > "reason" : "Error: User[yuwen] could not do action[indices:monitor/stats] > on index[twitter2]" > }, > "status" : 403 > } > > > Thanks, > > Qiang Zhang > >
