[jira] [Updated] (RANGER-2210) Ranger support for Apache Kafka 2.0.0

2018-09-12 Thread Velmurugan Periasamy (JIRA) 


 [ 
https://issues.apache.org/jira/browse/RANGER-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Velmurugan Periasamy updated RANGER-2210:
-
Fix Version/s: (was: master)

> Ranger support for Apache Kafka 2.0.0
> -
>
> Key: RANGER-2210
> URL: https://issues.apache.org/jira/browse/RANGER-2210
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: master, 2.0.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 
> 0001-RANGER-2210-Ranger-support-for-Apache-Kafka-2.0.0.patch
>
>
> Ranger support for Apache Kafka version 2.0.0.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch

2018-09-12 Thread Qiang Zhang 


> On 九月 6, 2018, 6:30 p.m., Ramesh Mani wrote:
> > ranger-elasticsearch-plugin-shim/conf/plugin-descriptor.properties
> > Lines 46 (patched)
> > 
> >
> > Why is -shim- is having the conf and classes which are core to the 
> > plugin it should part of /plugin-elasticseach/conf folder.
> > -shim is to hold the ranger-classloader.
> > Please refer the existing plugin and see if that can be done.
> 
> Qiang Zhang wrote:
> Because unlike other Hadoop components, Elasticseach is designed to be 
> pluggable.
> To implement a new extension function, 
> we need to organize code and configuration files according to the 
> requirements of Elasticseach.
> Some classes in ranger-elasticsearch-plugin-shim is necessary to mount on 
> Elasticseach.
> Other classes is due to the fact that Elasticseach itself does not 
> support user authentication,
> so this plugin should work with other Elasticsearch plugin to 
> authenticate users,
> such as Basic Authentication, Kerberos, LDAP, etc.
> Or, in the future, we can realize user authentication in 
> ranger-elasticsearch-plugin-shim.
> So these classes can't sink into plugin-elasticseach.
> And plugin-descriptor.properties is for Elasticseach to start the Ranger 
> Elasticseach plugin.
> But the files in the plugin-elasticseach/conf directory are for Ranger 
> Elasticseach plugin.
> 
> The related directory structure after plugin installed is as follows:
> 1.ranger-elasticsearch-plugin-shim/conf/?
> ```
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
> 
> /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
> -rwxrwxrwx. 1 elasticsearch hadoop 588337 4?  25 2017 
> commons-collections-3.2.2.jar
> -rwxrwxrwx. 1 elasticsearch hadoop 284220 4?  25 2017 commons-lang-2.6.jar
> -rwxrwxrwx. 1 elasticsearch hadoop   2547 6?  26 09:41 
> plugin-descriptor.properties
> -rwxrwxrwx. 1 elasticsearch hadoop   1754 6?  26 09:27 
> plugin-security.policy
> drwxrwxrwx. 2 elasticsearch hadoop   4096 7?  13 09:40 
> ranger-elasticsearch-plugin-impl
> -rwxrwxrwx. 1 elasticsearch hadoop  20627 6?  26 09:36 
> ranger-elasticsearch-plugin-shim-1.1.0-SNAPSHOT.jar
> -rwxrwxrwx. 1 elasticsearch hadoop  16799 6?  26 09:35 
> ranger-plugin-classloader-1.1.0-SNAPSHOT.jar
> -rwxrwxrwx. 1 elasticsearch hadoop  26084 4?  25 2017 slf4j-api-1.7.5.jar
> -rwxrwxrwx. 1 elasticsearch hadoop   8866 6?  26 15:30 
> slf4j-log4j12-1.7.10.jar
> ```
> 
> 2.plugin-elasticseach/conf?
> ```
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ pwd
> /home/elasticsearch/elasticsearch-6.2.2/config/ranger-elasticsearch-plugin
> [elasticsearch@zdh-11 ranger-elasticsearch-plugin]$ ll
> -rwxrwxrwx. 1 elasticsearch hadoop 9548 6?  26 14:15 
> ranger-elasticsearch-audit.xml
> -rwxrwxrwx. 1 elasticsearch hadoop 2773 6?  26 14:15 
> ranger-elasticsearch-security.xml
> -rwxrwxrwx. 1 elasticsearch hadoop 1917 6?  26 14:15 
> ranger-policymgr-ssl.xml
> -rwxrwxrwx. 1 elasticsearch hadoop   83 6?  26 14:15 ranger-security.xml
> ```
> 
> In addition, I have developed 2 plugins: Ranger Kylin Plugin and Ranger 
> Sqoop2 Plugin.
> In order to meet the requirements of Elasticsearch plugins 
> and take into account the general design principles of Ranger plugins,
> I think the implementation of Ranger Elasticsearch plugin is reasonable 
> at present.
> Try install this plugin , and you can get a better understanding of the 
> implementation of it.
> 
> Ramesh Mani wrote:
> If you are moving the configs from ranger-elasticsearch-plugin-shim/conf 
> to 
> /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin, 
> you can have all the configs in plugin-elasticsearch/conf and during 
> installtion you can move it 
> /home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin. 
> Now you have config folder in both shim and plugin folder and this is 
> confusing.
> 
> I know that there are elastic-search specific classes which needs to be 
> there in shim folder which is fine.

The configs of ranger-elasticsearch-plugin-shim/conf can not be moved to 
/home/elasticsearch/elasticsearch-6.2.2/plugins/ranger-elasticsearch-plugin,
because they are the necessary components of Elasticsearch plugin,
If they are removed, it will result in Ranger Elasticsearch plugin not loaded 
or startup failure.
Especially this plugin-descriptor.properties,
please refer the following official description:
```
# Elasticsearch plugin descriptor file
# This file must exist as 'plugin-descriptor.properties' in a folder named 
`elasticsearch`
# inside all plugins.
#
### example plugin for "foo"
#
# foo.zip <-- zip file for the plugin, with this structure:
#|elasticsearch/
#

Review Request 68707: RANGER-2210:Ranger support for Apache Kafka 2.0.0

2018-09-12 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68707/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, and Velmurugan 
Periasamy.


Bugs: RANGER-2210
https://issues.apache.org/jira/browse/RANGER-2210


Repository: ranger


Description
---

RANGER-2210:Ranger support for Apache Kafka 2.0.0


Diffs
-

  
plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
 b5d151e 
  
plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerTest.java
 bccdb80 
  pom.xml ae3f4be 


Diff: https://reviews.apache.org/r/68707/diff/1/


Testing
---

Verified in local VM
 - Policy download is successful and authorization is fine.


Thanks,

Ramesh Mani

[jira] [Updated] (RANGER-2210) Ranger support for Apache Kafka 2.0.0

2018-09-12 Thread Ramesh Mani (JIRA) 


 [ 
https://issues.apache.org/jira/browse/RANGER-2210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramesh Mani updated RANGER-2210:

Attachment: 0001-RANGER-2210-Ranger-support-for-Apache-Kafka-2.0.0.patch

> Ranger support for Apache Kafka 2.0.0
> -
>
> Key: RANGER-2210
> URL: https://issues.apache.org/jira/browse/RANGER-2210
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: master, 2.0.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Critical
> Fix For: master, 2.0.0
>
> Attachments: 
> 0001-RANGER-2210-Ranger-support-for-Apache-Kafka-2.0.0.patch
>
>
> Ranger support for Apache Kafka version 2.0.0.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (RANGER-2219) De-normalize schema for storing tags and related objects

2018-09-12 Thread Abhay Kulkarni (JIRA) 
Abhay Kulkarni created RANGER-2219:
--

 Summary: De-normalize schema for storing tags and related objects
 Key: RANGER-2219
 URL: https://issues.apache.org/jira/browse/RANGER-2219
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Affects Versions: master
Reporter: Abhay Kulkarni
Assignee: Abhay Kulkarni
 Fix For: master


Currently, tag-definitions, tags and service-resources are stored in database 
using a normalized form. When constructing resource->tag mappings, this schema 
design may lead to a large number of database accesses, thereby causing a major 
performance bottleneck when the number of resource->tag associations is large.

Denormalized schema will reduce the number of database accesses, and improve 
overall performance significantly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (RANGER-2218) Service-Definition update should not allow updates to names of resources, access-types, conditions or data-masks

2018-09-12 Thread Abhay Kulkarni (JIRA) 
Abhay Kulkarni created RANGER-2218:
--

 Summary: Service-Definition update should not allow updates to 
names of resources, access-types,  conditions or data-masks
 Key: RANGER-2218
 URL: https://issues.apache.org/jira/browse/RANGER-2218
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Affects Versions: master
Reporter: Abhay Kulkarni
 Fix For: master


Updates to service definitions should not allow updating names of the following 
components: 
 * Resources
 * Access types
 * Policy conditions
 * Data Masks

In general, these updates are seldom needed and can be avoided by careful 
design of service definition. Also, with a de-normalized database schema for 
storing policies, it is expensive and inefficient to maintain and lookup 
mapping from internal IDs to names for each of these components. By not 
allowing updates to these names, there is no need to maintain ( or reference) 
such mappings after updating (or reading) policy when using de-normalized 
database schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2217) Ranger upgrade from 0.7.1 to 1.1.0

2018-09-12 Thread Sushruth Bharath HG (JIRA) 


[ 
https://issues.apache.org/jira/browse/RANGER-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612083#comment-16612083
 ] 

Sushruth Bharath HG commented on RANGER-2217:
-

Hi Nitin,

Yes it worked

Thank you

Sushruth

> Ranger upgrade from 0.7.1 to 1.1.0
> --
>
> Key: RANGER-2217
> URL: https://issues.apache.org/jira/browse/RANGER-2217
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: 1.1.0
>Reporter: Sushruth Bharath HG
>Priority: Blocker
> Fix For: 1.1.0
>
>
> I upgraded Ranger from 0.7.1 to 1.1.0, after that only ranger UI is showing 
> up, it's not logging in.  
> What are the pre-requisites to upgrade ranger from 0.7.1?
> Will existing policies will carry forward to upgraded version?
> Does ranger 1.1 support hadoop 2.7.1 version?
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68676: RANGER-2212 Add multiple urls tips for the ‘Kylin URL’ configuration item when creating the kylin-plugin service

2018-09-12 Thread Nitin Galave 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68676/#review208556
---


Ship it!




Ship It!

- Nitin Galave


On Sept. 10, 2018, 7:43 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68676/
> ---
> 
> (Updated Sept. 10, 2018, 7:43 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2212
> https://issues.apache.org/jira/browse/RANGER-2212
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Add multiple urls tips for the ‘Kylin URL’ configuration item:
> 1.For one url, eg.
> 'http://:7070'
> 2.For multiple urls (use , or ; delimiter), eg.
> 'http://:7070,http://:7070'
> 
> 
> Diffs
> -
> 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json 
> cda352665 
> 
> 
> Diff: https://reviews.apache.org/r/68676/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

[jira] [Commented] (RANGER-2217) Ranger upgrade from 0.7.1 to 1.1.0

2018-09-12 Thread Nitin Galave (JIRA) 


[ 
https://issues.apache.org/jira/browse/RANGER-2217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16612042#comment-16612042
 ] 

Nitin Galave commented on RANGER-2217:
--

Can you please clear browser cache and try logging again?

Will existing policies will carry forward to upgraded version?
< Ranger upgrade from 0.7.1 to 1.1.0
> --
>
> Key: RANGER-2217
> URL: https://issues.apache.org/jira/browse/RANGER-2217
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: 1.1.0
>Reporter: Sushruth Bharath HG
>Priority: Blocker
> Fix For: 1.1.0
>
>
> I upgraded Ranger from 0.7.1 to 1.1.0, after that only ranger UI is showing 
> up, it's not logging in.  
> What are the pre-requisites to upgrade ranger from 0.7.1?
> Will existing policies will carry forward to upgraded version?
> Does ranger 1.1 support hadoop 2.7.1 version?
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 68681: RANGER-2213 Tomcat Security Vulnerability Alert. The version of the tomcat for ranger should upgrade to 7.0.90.

2018-09-12 Thread Vishal Suvagia via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68681/#review208553
---



@Qiang Zhang, Kindly add the testing done with this patch ?

- Vishal Suvagia


On Sept. 11, 2018, 3:07 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68681/
> ---
> 
> (Updated Sept. 11, 2018, 3:07 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Nitin Galave, pengjianhua, 
> Ramesh Mani, Selvamohan Neethiraj, sam  rome, Venkat Ranganathan, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2213
> https://issues.apache.org/jira/browse/RANGER-2213
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> [SECURITY] CVE-2018-1336
> Severity: High 
> Versions Affected: Apache Tomcat 9.0.0.M9 to 9.0.7, 8.5.0 to 8.5.30, 
> 8.0.0.RC1 to 8.0.51, and 7.0.28 to 7.0.86.
> Description: An improper handing of overflow in the UTF-8 decoder with 
> supplementary characters can lead to an infinite loop in the decoder causing 
> a Denial of Service.
> 
> CVE-2018-8014
> Description: The defaults settings for the CORS filter provided in Apache 
> Tomcat 9.0.0.M1 to 9.0.8, 8.5.0 to 8.5.31, 8.0.0.RC1 to 8.0.52, 7.0.41 to 
> 7.0.88 are insecure and enable 'supportsCredentials' for all origins. It is 
> expected that users of the CORS filter will have configured it appropriately 
> for their environment rather than using it in the default configuration. 
> Therefore, it is expected that most users will not be impacted by this issue.
> 
> CVE-2018-8034
> Description: The host name verification when using TLS with the WebSocket 
> client was missing. It is now enabled by default. 
> Versions Affected: Apache Tomcat 9.0.0.M1 to 9.0.9, 8.5.0 to 8.5.31, 
> 8.0.0.RC1 to 8.0.52, and 7.0.35 to 7.0.88.
> 
> 
> Diffs
> -
> 
>   pom.xml ae3f4be4c 
> 
> 
> Diff: https://reviews.apache.org/r/68681/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>