FW: New Defects reported by Coverity Scan for Apache Ranger

2017-10-19 Thread Don Bosco Durai 
Please review and fix if needed.

Thanks

Bosco


On 10/19/17, 12:41 AM, "scan-ad...@coverity.com"  
wrote:


Hi,

Please find the latest report on new defect(s) introduced to Apache Ranger 
found with Coverity Scan.

6 new defect(s) introduced to Apache Ranger found with Coverity Scan.
5 defect(s), reported by Coverity Scan earlier, were marked fixed in the 
recent build analyzed by Coverity Scan.

New defect(s) Reported-by: Coverity Scan
Showing 6 of 6 defect(s)


** CID 168486:  Code maintainability issues  (UNUSED_VALUE)
/security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java: 
1606 in 
org.apache.ranger.biz.ServiceDBStore.updateService(org.apache.ranger.plugin.model.RangerService,
 java.util.Map)()




*** CID 168486:  Code maintainability issues  (UNUSED_VALUE)
/security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java: 
1606 in 
org.apache.ranger.biz.ServiceDBStore.updateService(org.apache.ranger.plugin.model.RangerService,
 java.util.Map)()
1600}
1601 
1602if (StringUtils.equalsIgnoreCase(configKey, 
CONFIG_KEY_PASSWORD)) {
1603if 
(StringUtils.equalsIgnoreCase(configValue, HIDDEN_PASSWORD_STR)) {
1604 String[] crypt_algo_array 
= null;
1605 if 
(configValue.contains(",")) {
>>> CID 168486:  Code maintainability issues  (UNUSED_VALUE)
>>> Assigning value from "configValue.split(",")" to "crypt_algo_array" 
here, but that stored value is overwritten before it can be used.
1606 crypt_algo_array = 
configValue.split(",");
1607 }
1608 if (oldPassword != null && 
oldPassword.contains(",")) {
1609String encryptKey = 
null;
1610String salt = null;
1611int iterationCount = 0;

** CID 168485:  High impact security  (CSRF)
/security-admin/src/main/java/org/apache/ranger/rest/PublicAPIsv2.java: 212 
in 
org.apache.ranger.rest.PublicAPIsv2.updateService(org.apache.ranger.plugin.model.RangerService,
 java.lang.Long, javax.servlet.http.HttpServletRequest)()




*** CID 168485:  High impact security  (CSRF)
/security-admin/src/main/java/org/apache/ranger/rest/PublicAPIsv2.java: 212 
in 
org.apache.ranger.rest.PublicAPIsv2.updateService(org.apache.ranger.plugin.model.RangerService,
 java.lang.Long, javax.servlet.http.HttpServletRequest)()
206 }
207 
208 @PUT
209 @Path("/api/service/{id}")
210 
@PreAuthorize("@rangerPreAuthSecurityHandler.isAPISpnegoAccessible()")
211 @Produces({ "application/json", "application/xml" })
>>> CID 168485:  High impact security  (CSRF)
>>> No CSRF protection was detected anywhere in this application. If 
this is not correct, please refer to the CSRF checker reference on how to 
specify it via checker option.
212 public RangerService updateService(RangerService service, 
@PathParam("id") Long id,
213@Context HttpServletRequest 
request) {
214 // if service.id is specified, it should be same as the 
param 'id'
215 if(service.getId() == null) {
216 service.setId(id);
217 } else if(!service.getId().equals(id)) {

** CID 168484:  Incorrect expression  (COPY_PASTE_ERROR)
/security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java: 
1625 in 
org.apache.ranger.biz.ServiceDBStore.updateService(org.apache.ranger.plugin.model.RangerService,
 java.util.Map)()




*** CID 168484:  Incorrect expression  (COPY_PASTE_ERROR)
/security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java: 
1625 in 
org.apache.ranger.biz.ServiceDBStore.updateService(org.apache.ranger.plugin.model.RangerService,
 java.util.Map)()
1619 if 
(!OLD_CRYPT_ALGO.equalsIgnoreCase(CRYPT_ALGO)) {
1620 String 
decryptedPwd = PasswordUtils.decryptPassword(oldPassword);
1621 String 
paddingString = CRYPT_ALGO + "," +  encryp

Re: Review Request 61021: RANGER-1672:Ranger supports plugin to enable, monitor and manage apache kylin

2017-10-19 Thread Qiang Zhang 


> On Sept. 27, 2017, 9:33 a.m., pengjianhua wrote:
> > The https://issues.apache.org/jira/browse/KYLIN-2703 I submitted had been 
> > resolved. Our test experts have rigorously tested this issue. At the same 
> > time the Kyligence company which is Apache kylin's main contributor will 
> > soon use this feature.
> 
> Colm O hEigeartaigh wrote:
> When will Kylin 2.2.0 be released? If we merge a dependency on a SNAPSHOT 
> version in Ranger, we can't release Ranger until Kylin is released. Do you 
> have any documentation on how to set up a simple test-scenario?
> 
> pengjianhua wrote:
> Ok. The Kylin 2.2.0 will be released  in late October.
> 
> Qiang Zhang wrote:
> Ok. I am writing install guide for the feature.
> 
> Qiang Zhang wrote:
> Hi Colm, Can you tell me how to build a document patch? Thanks!
> 
> Colm O hEigeartaigh wrote:
> I'm not sure what you mean by "document patch". What do you want to do 
> exactly?
> 
> Qiang Zhang wrote:
> I build patch for RANGER-1809 after writed installing guide for 
> RANGER-1672. I want to know which path the document is placed and how to 
> build the document patch. Thanks!
> 
> Colm O hEigeartaigh wrote:
> Well you could add it to the website, but it's probably easier just to 
> create a new wiki page, which is where all the rest of the Ranger 
> documentation lives:
> 
> https://cwiki.apache.org/confluence/display/RANGER/Index

I do not have permission to add or modify wiki page. How I submit the 
installation guide documentation and operation manual for Kylin plugin and 
Sqoop2 plugin?


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61021/#review186391
---


On Sept. 27, 2017, 9:19 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61021/
> ---
> 
> (Updated Sept. 27, 2017, 9:19 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1672
> https://issues.apache.org/jira/browse/RANGER-1672
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger supports plugin to enable, monitor and manage apache kylin
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
> 58cdd35 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json 
> PRE-CREATION 
>   plugin-kylin/.gitignore PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-kylin/pom.xml PRE-CREATION 
>   plugin-kylin/scripts/install.properties PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/RangerServiceKylin.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinClient.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinResourceMgr.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinCubeResponse.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinProjectResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-kylin-plugin-shim/.gitignore PRE-CREATION 
>   ranger-kylin-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-kylin-plugin-shim/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-kylin.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61021/diff/3/
> 
> 
> Testing
> ---
> 
> Tested
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

Re: Review Request 61021: RANGER-1672:Ranger supports plugin to enable, monitor and manage apache kylin

2017-10-19 Thread Colm O hEigeartaigh 


> On Sept. 27, 2017, 9:33 a.m., pengjianhua wrote:
> > The https://issues.apache.org/jira/browse/KYLIN-2703 I submitted had been 
> > resolved. Our test experts have rigorously tested this issue. At the same 
> > time the Kyligence company which is Apache kylin's main contributor will 
> > soon use this feature.
> 
> Colm O hEigeartaigh wrote:
> When will Kylin 2.2.0 be released? If we merge a dependency on a SNAPSHOT 
> version in Ranger, we can't release Ranger until Kylin is released. Do you 
> have any documentation on how to set up a simple test-scenario?
> 
> pengjianhua wrote:
> Ok. The Kylin 2.2.0 will be released  in late October.
> 
> Qiang Zhang wrote:
> Ok. I am writing install guide for the feature.
> 
> Qiang Zhang wrote:
> Hi Colm, Can you tell me how to build a document patch? Thanks!
> 
> Colm O hEigeartaigh wrote:
> I'm not sure what you mean by "document patch". What do you want to do 
> exactly?
> 
> Qiang Zhang wrote:
> I build patch for RANGER-1809 after writed installing guide for 
> RANGER-1672. I want to know which path the document is placed and how to 
> build the document patch. Thanks!
> 
> Colm O hEigeartaigh wrote:
> Well you could add it to the website, but it's probably easier just to 
> create a new wiki page, which is where all the rest of the Ranger 
> documentation lives:
> 
> https://cwiki.apache.org/confluence/display/RANGER/Index
> 
> Qiang Zhang wrote:
> I do not have permission to add or modify wiki page. How I submit the 
> installation guide documentation and operation manual for Kylin plugin and 
> Sqoop2 plugin?

I can add/modify pages, but I can't grant permission. Can you ask Madhan to 
grant you access?


- Colm


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61021/#review186391
---


On Sept. 27, 2017, 9:19 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61021/
> ---
> 
> (Updated Sept. 27, 2017, 9:19 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1672
> https://issues.apache.org/jira/browse/RANGER-1672
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger supports plugin to enable, monitor and manage apache kylin
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
> 58cdd35 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json 
> PRE-CREATION 
>   plugin-kylin/.gitignore PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-kylin/pom.xml PRE-CREATION 
>   plugin-kylin/scripts/install.properties PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/RangerServiceKylin.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinClient.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinResourceMgr.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinCubeResponse.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinProjectResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-kylin-plugin-shim/.gitignore PRE-CREATION 
>   ranger-kylin-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-kylin-plugin-shim/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-kylin.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61021/diff/3/
> 
> 
> Testing
> ---
> 
> Tested
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

Re: Review Request 61021: RANGER-1672:Ranger supports plugin to enable, monitor and manage apache kylin

2017-10-19 Thread Qiang Zhang 


> On Sept. 27, 2017, 9:33 a.m., pengjianhua wrote:
> > The https://issues.apache.org/jira/browse/KYLIN-2703 I submitted had been 
> > resolved. Our test experts have rigorously tested this issue. At the same 
> > time the Kyligence company which is Apache kylin's main contributor will 
> > soon use this feature.
> 
> Colm O hEigeartaigh wrote:
> When will Kylin 2.2.0 be released? If we merge a dependency on a SNAPSHOT 
> version in Ranger, we can't release Ranger until Kylin is released. Do you 
> have any documentation on how to set up a simple test-scenario?
> 
> pengjianhua wrote:
> Ok. The Kylin 2.2.0 will be released  in late October.
> 
> Qiang Zhang wrote:
> Ok. I am writing install guide for the feature.
> 
> Qiang Zhang wrote:
> Hi Colm, Can you tell me how to build a document patch? Thanks!
> 
> Colm O hEigeartaigh wrote:
> I'm not sure what you mean by "document patch". What do you want to do 
> exactly?
> 
> Qiang Zhang wrote:
> I build patch for RANGER-1809 after writed installing guide for 
> RANGER-1672. I want to know which path the document is placed and how to 
> build the document patch. Thanks!
> 
> Colm O hEigeartaigh wrote:
> Well you could add it to the website, but it's probably easier just to 
> create a new wiki page, which is where all the rest of the Ranger 
> documentation lives:
> 
> https://cwiki.apache.org/confluence/display/RANGER/Index
> 
> Qiang Zhang wrote:
> I do not have permission to add or modify wiki page. How I submit the 
> installation guide documentation and operation manual for Kylin plugin and 
> Sqoop2 plugin?
> 
> Colm O hEigeartaigh wrote:
> I can add/modify pages, but I can't grant permission. Can you ask Madhan 
> to grant you access?

Ok. Thanks a lot!


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61021/#review186391
---


On Sept. 27, 2017, 9:19 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61021/
> ---
> 
> (Updated Sept. 27, 2017, 9:19 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, sam  rome, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-1672
> https://issues.apache.org/jira/browse/RANGER-1672
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger supports plugin to enable, monitor and manage apache kylin
> 
> 
> Diffs
> -
> 
>   agents-common/scripts/enable-agent.sh d31a264 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
>  9463ab8 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 
> 58cdd35 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-kylin.json 
> PRE-CREATION 
>   plugin-kylin/.gitignore PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-audit.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-kylin-security.xml PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION 
>   plugin-kylin/conf/ranger-policymgr-ssl.xml PRE-CREATION 
>   plugin-kylin/pom.xml PRE-CREATION 
>   plugin-kylin/scripts/install.properties PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/RangerServiceKylin.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinClient.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/KylinResourceMgr.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinCubeResponse.java
>  PRE-CREATION 
>   
> plugin-kylin/src/main/java/org/apache/ranger/services/kylin/client/json/model/KylinProjectResponse.java
>  PRE-CREATION 
>   pom.xml 3958014 
>   ranger-kylin-plugin-shim/.gitignore PRE-CREATION 
>   ranger-kylin-plugin-shim/pom.xml PRE-CREATION 
>   
> ranger-kylin-plugin-shim/src/main/java/org/apache/ranger/authorization/kylin/authorizer/RangerKylinAuthorizer.java
>  PRE-CREATION 
>   src/main/assembly/admin-web.xml 0e97818 
>   src/main/assembly/plugin-kylin.xml PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61021/diff/3/
> 
> 
> Testing
> ---
> 
> Tested
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

[jira] [Commented] (RANGER-1672) Ranger supports plugin to enable, monitor and manage apache kylin

2017-10-19 Thread Qiang Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16210808#comment-16210808
 ] 

Qiang Zhang commented on RANGER-1672:
-

Hi [~madhan.neethiraj], Could you grant me to add and modify the wiki page? I 
will submit the installation guide documentation and operation manual for Kylin 
plugin and Sqoop2 plugin. Thanks a lot!

> Ranger supports plugin to enable, monitor and manage apache kylin
> -
>
> Key: RANGER-1672
> URL: https://issues.apache.org/jira/browse/RANGER-1672
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>  Labels: newbie, patch
> Attachments: 
> 0001-RANGER-1672-Ranger-supports-plugin-to-enable-monitor.patch, 
> KylinAuditLog.jpg, KylinPlugins.jpg, KylinPolicies.jpg, 
> KylinServiceEntry.jpg, NewKylinPolicy.jpg, NewKylinService.jpg
>
>
> Apache Kylin is an open source Distributed Analytics Engine designed to 
> provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop 
> supporting extremely large datasets, original contributed from eBay Inc. 
> Apache Kylin lets user query massive data set at sub-second latency in 3 
> steps.
> 1. Identify a Star Schema on Hadoop.
> 2. Build Cube from the identified tables.
> 3. Query with ANSI-SQL and get results in sub-second, via ODBC, JDBC or 
> RESTful API.
> We should support that using Ranger to control kylin's access rights for 
> project and cube.
> Specific implementation plan is as following:
> On the ranger website, administrators can configure policies to control user 
> access to projects and cube permissions.
> Kylin provides an abstract class and authorization interfaces for use by the 
> ranger plugin. kylin instantiates ranger plugin’s implementation class when 
> starting(this class extends the abstract class provided by kylin).
> Ranger plugin periodically polls ranger admin, updates the policy to the 
> local, and updates project and cube access rights based on policy information.
> In the Kylin side：
> 1. Kylin provides an abstract class that enables the ranger plugin's 
> implementation class to extend.
> 2. Add configuration item. 1) ranger authorization switch, 2) ranger plugin 
> implementation class's name.
> 3. Instantiate the ranger plugin implementation class when starting kylin.
> 4. kylin provides authorization interfaces for ranger plugin calls.
> 5. According to the ranger authorization configuration item, hide kylin's 
> authorization management page.
> 6. Using ranger manager access rights of the kylin does not affect kylin's 
> existing permissions functions and logic.
> In the Ranger side：
> 1. Ranger plugin will periodically polls ranger admin, updates the policy to 
> the local.
> 2. The ranger plugin invoking the authorization interfaces provided by kylin 
> to updates the project and cube access rights based on the policy information.
> reference link:https://issues.apache.org/jira/browse/KYLIN-2703



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (RANGER-1844) Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Qiang Zhang (JIRA) 
Qiang Zhang created RANGER-1844:
---

 Summary: Ranger admin support hdfs HA configuration when creating 
hdfs service.
 Key: RANGER-1844
 URL: https://issues.apache.org/jira/browse/RANGER-1844
 Project: Ranger
  Issue Type: Bug
  Components: plugins, Ranger
Affects Versions: master
Reporter: Qiang Zhang
Assignee: Qiang Zhang


In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
mode,you have to config a lot of configs in hdfs_dev,such as:
Namenode URL *=hdfs://hdfscluster
===Add New Configurations===
dfs.nameservices=hdfscluster
dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.ha.namenodes.hdfscluster=nn1,nn2
dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
===End of add New Configurations===

And other big data components such as hbase,hive and so on can support HA 
without config
lots of "Add New Configurations",it is easy to config a url. like zk queue 
configuration in hbase,like jdbc url in hive.  In hdfs service, only need 
to config "fs.default.name" :

Namenode URL *=hdfs://hdfscluster   
   old 
Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000new

 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (RANGER-1844) Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Qiang Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Zhang updated RANGER-1844:

Description: 
In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
mode,you have to config a lot of configs in hdfs_dev,such as:
Namenode URL *=hdfs://hdfscluster
===Add New Configurations===
dfs.nameservices=hdfscluster
dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.ha.namenodes.hdfscluster=nn1,nn2
dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
===End of add New Configurations===

And other big data components such as hbase,hive and so on can support HA 
without config
lots of "Add New Configurations",it is easy to config a url. like zk queue 
configuration in hbase,like jdbc url in hive.  In hdfs service, only need 
to config "fs.default.name" :

Namenode URL *=hdfs://hdfscluster   
   【old】 
Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000【new】

 

  was:
In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
mode,you have to config a lot of configs in hdfs_dev,such as:
Namenode URL *=hdfs://hdfscluster
===Add New Configurations===
dfs.nameservices=hdfscluster
dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.ha.namenodes.hdfscluster=nn1,nn2
dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
===End of add New Configurations===

And other big data components such as hbase,hive and so on can support HA 
without config
lots of "Add New Configurations",it is easy to config a url. like zk queue 
configuration in hbase,like jdbc url in hive.  In hdfs service, only need 
to config "fs.default.name" :

Namenode URL *=hdfs://hdfscluster   
   old 
Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000new

 


> Ranger admin support hdfs HA configuration when creating hdfs service.
> --
>
> Key: RANGER-1844
> URL: https://issues.apache.org/jira/browse/RANGER-1844
> Project: Ranger
>  Issue Type: Bug
>  Components: plugins, Ranger
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>
> In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
> mode,you have to config a lot of configs in hdfs_dev,such as:
> Namenode URL *=hdfs://hdfscluster
> ===Add New Configurations===
> dfs.nameservices=hdfscluster
> dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> dfs.ha.namenodes.hdfscluster=nn1,nn2
> dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
> dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
> ===End of add New Configurations===
> And other big data components such as hbase,hive and so on can support HA 
> without config
> lots of "Add New Configurations",it is easy to config a url. like zk queue 
> configuration in hbase,like jdbc url in hive.  In hdfs service, only need 
> to config "fs.default.name" :
> Namenode URL *=hdfs://hdfscluster 
>  【old】 
> Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000【new】
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (RANGER-1844) Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Qiang Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Zhang updated RANGER-1844:

Issue Type: New Feature  (was: Bug)

> Ranger admin support hdfs HA configuration when creating hdfs service.
> --
>
> Key: RANGER-1844
> URL: https://issues.apache.org/jira/browse/RANGER-1844
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins, Ranger
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>
> In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
> mode,you have to config a lot of configs in hdfs_dev,such as:
> Namenode URL *=hdfs://hdfscluster
> ===Add New Configurations===
> dfs.nameservices=hdfscluster
> dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> dfs.ha.namenodes.hdfscluster=nn1,nn2
> dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
> dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
> ===End of add New Configurations===
> And other big data components such as hbase,hive and so on can support HA 
> without config
> lots of "Add New Configurations",it is easy to config a url. like zk queue 
> configuration in hbase,like jdbc url in hive.  In hdfs service, only need 
> to config "fs.default.name" :
> Namenode URL *=hdfs://hdfscluster 
>  【old】 
> Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000【new】
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 62971: LOG class is imported error for RangerServiceService class

2017-10-19 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62971/#review188666
---


Ship it!




Ship It!

- Qiang Zhang


On Oct. 13, 2017, 10 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62971/
> ---
> 
> (Updated Oct. 13, 2017, 10 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1836
> https://issues.apache.org/jira/browse/RANGER-1836
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> LOG class is imported error for RangerServiceService class
> RangerServiceService.java import:
> import java.util.logging.Logger;
> I think java.util.logging.Logger class should be repalced with:
> import org.apache.commons.logging.Log;
> import org.apache.commons.logging.LogFactory;
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerServiceService.java
>  3dd761a2 
> 
> 
> Diff: https://reviews.apache.org/r/62971/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

Review Request 63142: Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63142/
---

Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
Selvamohan Neethiraj, Velmurugan Periasamy, and Qiang Zhang.


Bugs: RANGER-1844
https://issues.apache.org/jira/browse/RANGER-1844


Repository: ranger


Description
---

In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
mode,you have to config a lot of configs in hdfs_dev,such as:
Namenode URL *=hdfs://hdfscluster
===Add New Configurations===
dfs.nameservices=hdfscluster
dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.ha.namenodes.hdfscluster=nn1,nn2
dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
===End of add New Configurations===
And other big data components such as hbase,hive and so on can support HA 
without config
lots of "Add New Configurations",it is easy to config a url. like zk queue 
configuration in hbase ,like jdbc url in hive. In hdfs service, only need to 
config "fs.default.name" :
Namenode URL *=hdfs://hdfscluster ?old? 
Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000 ?new?


Diffs
-

  
hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/client/HdfsClient.java 
c252213f 


Diff: https://reviews.apache.org/r/63142/diff/1/


Testing
---


Thanks,

Qiang Zhang

[jira] [Updated] (RANGER-1844) Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Qiang Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Zhang updated RANGER-1844:

Attachment: 0001-RANGER-1844-Ranger-admin-support-hdfs-HA-configurati.patch

> Ranger admin support hdfs HA configuration when creating hdfs service.
> --
>
> Key: RANGER-1844
> URL: https://issues.apache.org/jira/browse/RANGER-1844
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins, Ranger
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
> Attachments: 
> 0001-RANGER-1844-Ranger-admin-support-hdfs-HA-configurati.patch
>
>
> In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
> mode,you have to config a lot of configs in hdfs_dev,such as:
> Namenode URL *=hdfs://hdfscluster
> ===Add New Configurations===
> dfs.nameservices=hdfscluster
> dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> dfs.ha.namenodes.hdfscluster=nn1,nn2
> dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
> dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
> ===End of add New Configurations===
> And other big data components such as hbase,hive and so on can support HA 
> without config
> lots of "Add New Configurations",it is easy to config a url. like zk queue 
> configuration in hbase,like jdbc url in hive.  In hdfs service, only need 
> to config "fs.default.name" :
> Namenode URL *=hdfs://hdfscluster 
>  【old】 
> Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000【new】
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (RANGER-1845) Add support to configure JWT signature algorithms

2017-10-19 Thread Colm O hEigeartaigh (JIRA) 
Colm O hEigeartaigh created RANGER-1845:
---

 Summary: Add support to configure JWT signature algorithms
 Key: RANGER-1845
 URL: https://issues.apache.org/jira/browse/RANGER-1845
 Project: Ranger
  Issue Type: Improvement
  Components: admin
Reporter: Colm O hEigeartaigh
Assignee: Colm O hEigeartaigh
Priority: Minor
 Fix For: 1.0.0


The next Knox release will feature the ability to change the signature 
algorithm from the default RS256. This task is to add support to specify a 
signature algorithm in Ranger, which the received token must match. The new 
configuration parameter is "ranger.sso.expected.sigalg" with a default value of 
"RS256".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (RANGER-1846) This JAVA_VERSION_REQUIRED configuration item is invalid in security admin installer, we should enable it to control the necessary java version.

2017-10-19 Thread peng.jianhua (JIRA) 
peng.jianhua created RANGER-1846:


 Summary: This JAVA_VERSION_REQUIRED configuration item is invalid 
in security admin installer, we should enable it to control the necessary java 
version.
 Key: RANGER-1846
 URL: https://issues.apache.org/jira/browse/RANGER-1846
 Project: Ranger
  Issue Type: Bug
  Components: admin
Affects Versions: 1.0.0, master
Reporter: peng.jianhua
Assignee: peng.jianhua


The java version must be equal to or more than 1.8 when we set db_ssl_enabled 
equal to true.
This JAVA_VERSION_REQUIRED configuration item is invalid in security admin 
installer, we should enable it to control the necessary java version.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (RANGER-1846) This JAVA_VERSION_REQUIRED configuration item is invalid in security admin installer, we should enable it to control the necessary java version.

2017-10-19 Thread peng.jianhua (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

peng.jianhua updated RANGER-1846:
-
Attachment: 0001-RANGER-1846-This-JAVA_VERSION_REQUIRED-configuration.patch

> This JAVA_VERSION_REQUIRED configuration item is invalid in security admin 
> installer, we should enable it to control the necessary java version.
> 
>
> Key: RANGER-1846
> URL: https://issues.apache.org/jira/browse/RANGER-1846
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Affects Versions: 1.0.0, master
>Reporter: peng.jianhua
>Assignee: peng.jianhua
>  Labels: patch
> Attachments: 
> 0001-RANGER-1846-This-JAVA_VERSION_REQUIRED-configuration.patch
>
>
> The java version must be equal to or more than 1.8 when we set db_ssl_enabled 
> equal to true.
> This JAVA_VERSION_REQUIRED configuration item is invalid in security admin 
> installer, we should enable it to control the necessary java version.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 63145: This JAVA_VERSION_REQUIRED configuration item is invalid in security admin installer, we should enable it to control the necessary java version.

2017-10-19 Thread pengjianhua 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63145/
---

Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
Neethiraj, Velmurugan Periasamy, and Qiang Zhang.


Bugs: RANGER-1846
https://issues.apache.org/jira/browse/RANGER-1846


Repository: ranger


Description
---

The java version must be equal to or more than 1.8 when we set db_ssl_enabled 
equal to true.
This JAVA_VERSION_REQUIRED configuration item is invalid in security admin 
installer, we should enable it to control the necessary java version.


Diffs
-

  security-admin/scripts/setup.sh e2a69b71 


Diff: https://reviews.apache.org/r/63145/diff/1/


Testing
---


Thanks,

pengjianhua

Re: Review Request 62751: RANGER-1824 - Upgrade Spring Framework to 3.2.18

2017-10-19 Thread pengjianhua 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62751/#review188674
---


Ship it!




Ship It!

- pengjianhua


On 十月 3, 2017, 2:09 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62751/
> ---
> 
> (Updated 十月 3, 2017, 2:09 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1824
> https://issues.apache.org/jira/browse/RANGER-1824
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> When starting the Admin console, the following appears in the logs:
> 
> 2017-10-02 10:00:35,651 [localhost-startStop-1] WARN 
> org.springframework.security.core.SpringSecurityCoreVersion 
> (SpringSecurityCoreVersion.java:60) -  You are advised to use Spring 
> 3.2.18.RELEASE or later with this version. You are running: 3.2.10.RELEASE
> 
> We should update Spring to 3.2.18
> 
> 
> Diffs
> -
> 
>   pom.xml 3958014c 
> 
> 
> Diff: https://reviews.apache.org/r/62751/diff/1/
> 
> 
> Testing
> ---
> 
> Tested the admin console started properly + can load policies etc.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 63068: RANGER-1842 - Don't catch Throwables in the test code

2017-10-19 Thread pengjianhua 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63068/#review188675
---


Ship it!




Ship It!

- pengjianhua


On 十月 17, 2017, 10:04 a.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63068/
> ---
> 
> (Updated 十月 17, 2017, 10:04 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1842
> https://issues.apache.org/jira/browse/RANGER-1842
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> In a couple of the tests, we are catching Throwables. This means that the 
> Assert.fail() conditions inside the try statement are ineffective.
> 
> 
> Diffs
> -
> 
>   
> kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/RangerMasterKeyTest.java
>  cac22502 
>   
> storm-agent/src/test/java/org/apache/ranger/authorization/storm/StormRangerAuthorizerTest.java
>  53173078 
> 
> 
> Diff: https://reviews.apache.org/r/63068/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

[jira] [Updated] (RANGER-1845) Add support to configure JWT signature algorithms

2017-10-19 Thread Colm O hEigeartaigh (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colm O hEigeartaigh updated RANGER-1845:

Attachment: 0001-RANGER-1845-Add-support-to-configure-JWT-signature-a.patch

> Add support to configure JWT signature algorithms
> -
>
> Key: RANGER-1845
> URL: https://issues.apache.org/jira/browse/RANGER-1845
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin
>Reporter: Colm O hEigeartaigh
>Assignee: Colm O hEigeartaigh
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: 
> 0001-RANGER-1845-Add-support-to-configure-JWT-signature-a.patch
>
>
> The next Knox release will feature the ability to change the signature 
> algorithm from the default RS256. This task is to add support to specify a 
> signature algorithm in Ranger, which the received token must match. The new 
> configuration parameter is "ranger.sso.expected.sigalg" with a default value 
> of "RS256".



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 63147: RANGER-1845 - Add support to configure JWT signature algorithms

2017-10-19 Thread Colm O hEigeartaigh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63147/
---

Review request for ranger.


Bugs: RANGER-1845
https://issues.apache.org/jira/browse/RANGER-1845


Repository: ranger


Description
---

The next Knox release will feature the ability to change the signature 
algorithm from the default RS256. This task is to add support to specify a 
signature algorithm in Ranger, which the received token must match. The new 
configuration parameter is "ranger.sso.expected.sigalg" with a default value of 
"RS256".


Diffs
-

  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
 5e4207c9 
  
security-admin/src/main/java/org/apache/ranger/security/web/filter/SSOAuthenticationProperties.java
 b8246a9a 


Diff: https://reviews.apache.org/r/63147/diff/1/


Testing
---

Tested it with Knox.


Thanks,

Colm O hEigeartaigh

Re: Review Request 63142: Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63142/#review188706
---




hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/client/HdfsClient.java
Lines 300 (patched)


Doesn't this also have to set 
dfs.namenode.http-address.$cluster.$nn_id ?

and potentially https instead if SSL is enabled.


- Alejandro Fernandez


On Oct. 19, 2017, 11:41 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63142/
> ---
> 
> (Updated Oct. 19, 2017, 11:41 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1844
> https://issues.apache.org/jira/browse/RANGER-1844
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
> mode,you have to config a lot of configs in hdfs_dev,such as:
> Namenode URL *=hdfs://hdfscluster
> ===Add New Configurations===
> dfs.nameservices=hdfscluster
> dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> dfs.ha.namenodes.hdfscluster=nn1,nn2
> dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
> dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
> ===End of add New Configurations===
> And other big data components such as hbase,hive and so on can support HA 
> without config
> lots of "Add New Configurations",it is easy to config a url. like zk queue 
> configuration in hbase ,like jdbc url in hive. In hdfs service, only need to 
> config "fs.default.name" :
> Namenode URL *=hdfs://hdfscluster ?old? 
> Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000 ?new?
> 
> 
> Diffs
> -
> 
>   
> hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/client/HdfsClient.java
>  c252213f 
> 
> 
> Diff: https://reviews.apache.org/r/63142/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

Re: Review Request 63145: This JAVA_VERSION_REQUIRED configuration item is invalid in security admin installer, we should enable it to control the necessary java version.

2017-10-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63145/#review188713
---




security-admin/scripts/setup.sh
Line 305 (original), 304 (patched)


This will not work if the minor version ever changes to 2 digits, e.g., 
1.10.0_1

Instead, concatenate 
export java_major_minor="$major.$minor"

if [[ "$java_major_minor" != "$JAVA_VERSION_REQUIRED" ]]; then
   ...


- Alejandro Fernandez


On Oct. 19, 2017, 1 p.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63145/
> ---
> 
> (Updated Oct. 19, 2017, 1 p.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1846
> https://issues.apache.org/jira/browse/RANGER-1846
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The java version must be equal to or more than 1.8 when we set db_ssl_enabled 
> equal to true.
> This JAVA_VERSION_REQUIRED configuration item is invalid in security admin 
> installer, we should enable it to control the necessary java version.
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/setup.sh e2a69b71 
> 
> 
> Diff: https://reviews.apache.org/r/63145/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 63147: RANGER-1845 - Add support to configure JWT signature algorithms

2017-10-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63147/#review188714
---


Ship it!




Ship It!

- Alejandro Fernandez


On Oct. 19, 2017, 1:18 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63147/
> ---
> 
> (Updated Oct. 19, 2017, 1:18 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1845
> https://issues.apache.org/jira/browse/RANGER-1845
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The next Knox release will feature the ability to change the signature 
> algorithm from the default RS256. This task is to add support to specify a 
> signature algorithm in Ranger, which the received token must match. The new 
> configuration parameter is "ranger.sso.expected.sigalg" with a default value 
> of "RS256".
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
>  5e4207c9 
>   
> security-admin/src/main/java/org/apache/ranger/security/web/filter/SSOAuthenticationProperties.java
>  b8246a9a 
> 
> 
> Diff: https://reviews.apache.org/r/63147/diff/1/
> 
> 
> Testing
> ---
> 
> Tested it with Knox.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

[jira] [Commented] (RANGER-1672) Ranger supports plugin to enable, monitor and manage apache kylin

2017-10-19 Thread Madhan Neethiraj (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212028#comment-16212028
 ] 

Madhan Neethiraj commented on RANGER-1672:
--

[~vperiasamy]/[~sneethiraj] - can you please update wiki permissions to enable 
[~zhangqiang2] add pages?

> Ranger supports plugin to enable, monitor and manage apache kylin
> -
>
> Key: RANGER-1672
> URL: https://issues.apache.org/jira/browse/RANGER-1672
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>  Labels: newbie, patch
> Attachments: 
> 0001-RANGER-1672-Ranger-supports-plugin-to-enable-monitor.patch, 
> KylinAuditLog.jpg, KylinPlugins.jpg, KylinPolicies.jpg, 
> KylinServiceEntry.jpg, NewKylinPolicy.jpg, NewKylinService.jpg
>
>
> Apache Kylin is an open source Distributed Analytics Engine designed to 
> provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop 
> supporting extremely large datasets, original contributed from eBay Inc. 
> Apache Kylin lets user query massive data set at sub-second latency in 3 
> steps.
> 1. Identify a Star Schema on Hadoop.
> 2. Build Cube from the identified tables.
> 3. Query with ANSI-SQL and get results in sub-second, via ODBC, JDBC or 
> RESTful API.
> We should support that using Ranger to control kylin's access rights for 
> project and cube.
> Specific implementation plan is as following:
> On the ranger website, administrators can configure policies to control user 
> access to projects and cube permissions.
> Kylin provides an abstract class and authorization interfaces for use by the 
> ranger plugin. kylin instantiates ranger plugin’s implementation class when 
> starting(this class extends the abstract class provided by kylin).
> Ranger plugin periodically polls ranger admin, updates the policy to the 
> local, and updates project and cube access rights based on policy information.
> In the Kylin side：
> 1. Kylin provides an abstract class that enables the ranger plugin's 
> implementation class to extend.
> 2. Add configuration item. 1) ranger authorization switch, 2) ranger plugin 
> implementation class's name.
> 3. Instantiate the ranger plugin implementation class when starting kylin.
> 4. kylin provides authorization interfaces for ranger plugin calls.
> 5. According to the ranger authorization configuration item, hide kylin's 
> authorization management page.
> 6. Using ranger manager access rights of the kylin does not affect kylin's 
> existing permissions functions and logic.
> In the Ranger side：
> 1. Ranger plugin will periodically polls ranger admin, updates the policy to 
> the local.
> 2. The ranger plugin invoking the authorization interfaces provided by kylin 
> to updates the project and cube access rights based on the policy information.
> reference link:https://issues.apache.org/jira/browse/KYLIN-2703



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 63145: This JAVA_VERSION_REQUIRED configuration item is invalid in security admin installer, we should enable it to control the necessary java version.

2017-10-19 Thread pengjianhua 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63145/
---

(Updated 十月 20, 2017, 2:02 a.m.)


Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
Neethiraj, Velmurugan Periasamy, and Qiang Zhang.


Bugs: RANGER-1846
https://issues.apache.org/jira/browse/RANGER-1846


Repository: ranger


Description
---

The java version must be equal to or more than 1.8 when we set db_ssl_enabled 
equal to true.
This JAVA_VERSION_REQUIRED configuration item is invalid in security admin 
installer, we should enable it to control the necessary java version.


Diffs (updated)
-

  security-admin/scripts/setup.sh e2a69b71 


Diff: https://reviews.apache.org/r/63145/diff/2/

Changes: https://reviews.apache.org/r/63145/diff/1-2/


Testing
---


Thanks,

pengjianhua

Re: Review Request 63145: This JAVA_VERSION_REQUIRED configuration item is invalid in security admin installer, we should enable it to control the necessary java version.

2017-10-19 Thread pengjianhua 


> On 十月 19, 2017, 5:51 p.m., Alejandro Fernandez wrote:
> > security-admin/scripts/setup.sh
> > Line 305 (original), 304 (patched)
> > 
> >
> > This will not work if the minor version ever changes to 2 digits, e.g., 
> > 1.10.0_1
> > 
> > Instead, concatenate 
> > export java_major_minor="$major.$minor"
> > 
> > if [[ "$java_major_minor" != "$JAVA_VERSION_REQUIRED" ]]; then
> >...

You are right. I fixed it and updated the patch. Thanks!


- pengjianhua


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63145/#review188713
---


On 十月 19, 2017, 1 p.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63145/
> ---
> 
> (Updated 十月 19, 2017, 1 p.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1846
> https://issues.apache.org/jira/browse/RANGER-1846
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The java version must be equal to or more than 1.8 when we set db_ssl_enabled 
> equal to true.
> This JAVA_VERSION_REQUIRED configuration item is invalid in security admin 
> installer, we should enable it to control the necessary java version.
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/setup.sh e2a69b71 
> 
> 
> Diff: https://reviews.apache.org/r/63145/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

Re: Review Request 63145: This JAVA_VERSION_REQUIRED configuration item is invalid in security admin installer, we should enable it to control the necessary java version.

2017-10-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63145/#review188803
---


Ship it!




Ship It!

- Alejandro Fernandez


On Oct. 20, 2017, 2:02 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63145/
> ---
> 
> (Updated Oct. 20, 2017, 2:02 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1846
> https://issues.apache.org/jira/browse/RANGER-1846
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The java version must be equal to or more than 1.8 when we set db_ssl_enabled 
> equal to true.
> This JAVA_VERSION_REQUIRED configuration item is invalid in security admin 
> installer, we should enable it to control the necessary java version.
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/setup.sh e2a69b71 
> 
> 
> Diff: https://reviews.apache.org/r/63145/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

[jira] [Commented] (RANGER-1672) Ranger supports plugin to enable, monitor and manage apache kylin

2017-10-19 Thread Velmurugan Periasamy (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212065#comment-16212065
 ] 

Velmurugan Periasamy commented on RANGER-1672:
--

[~zhangqiang2] - I have given you permissions, check it out and let me know. 

> Ranger supports plugin to enable, monitor and manage apache kylin
> -
>
> Key: RANGER-1672
> URL: https://issues.apache.org/jira/browse/RANGER-1672
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>  Labels: newbie, patch
> Attachments: 
> 0001-RANGER-1672-Ranger-supports-plugin-to-enable-monitor.patch, 
> KylinAuditLog.jpg, KylinPlugins.jpg, KylinPolicies.jpg, 
> KylinServiceEntry.jpg, NewKylinPolicy.jpg, NewKylinService.jpg
>
>
> Apache Kylin is an open source Distributed Analytics Engine designed to 
> provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop 
> supporting extremely large datasets, original contributed from eBay Inc. 
> Apache Kylin lets user query massive data set at sub-second latency in 3 
> steps.
> 1. Identify a Star Schema on Hadoop.
> 2. Build Cube from the identified tables.
> 3. Query with ANSI-SQL and get results in sub-second, via ODBC, JDBC or 
> RESTful API.
> We should support that using Ranger to control kylin's access rights for 
> project and cube.
> Specific implementation plan is as following:
> On the ranger website, administrators can configure policies to control user 
> access to projects and cube permissions.
> Kylin provides an abstract class and authorization interfaces for use by the 
> ranger plugin. kylin instantiates ranger plugin’s implementation class when 
> starting(this class extends the abstract class provided by kylin).
> Ranger plugin periodically polls ranger admin, updates the policy to the 
> local, and updates project and cube access rights based on policy information.
> In the Kylin side：
> 1. Kylin provides an abstract class that enables the ranger plugin's 
> implementation class to extend.
> 2. Add configuration item. 1) ranger authorization switch, 2) ranger plugin 
> implementation class's name.
> 3. Instantiate the ranger plugin implementation class when starting kylin.
> 4. kylin provides authorization interfaces for ranger plugin calls.
> 5. According to the ranger authorization configuration item, hide kylin's 
> authorization management page.
> 6. Using ranger manager access rights of the kylin does not affect kylin's 
> existing permissions functions and logic.
> In the Ranger side：
> 1. Ranger plugin will periodically polls ranger admin, updates the policy to 
> the local.
> 2. The ranger plugin invoking the authorization interfaces provided by kylin 
> to updates the project and cube access rights based on the policy information.
> reference link:https://issues.apache.org/jira/browse/KYLIN-2703



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 63142: Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Qiang Zhang 


> On 十月 19, 2017, 5:36 p.m., Alejandro Fernandez wrote:
> > hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/client/HdfsClient.java
> > Lines 300 (patched)
> > 
> >
> > Doesn't this also have to set 
> > dfs.namenode.http-address.$cluster.$nn_id ?
> > 
> > and potentially https instead if SSL is enabled.

Here only use hdfs ipc, no use https (port: 50070 or 50470).


- Qiang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63142/#review188706
---


On 十月 19, 2017, 11:41 a.m., Qiang Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63142/
> ---
> 
> (Updated 十月 19, 2017, 11:41 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
> Selvamohan Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1844
> https://issues.apache.org/jira/browse/RANGER-1844
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
> mode,you have to config a lot of configs in hdfs_dev,such as:
> Namenode URL *=hdfs://hdfscluster
> ===Add New Configurations===
> dfs.nameservices=hdfscluster
> dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> dfs.ha.namenodes.hdfscluster=nn1,nn2
> dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
> dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
> ===End of add New Configurations===
> And other big data components such as hbase,hive and so on can support HA 
> without config
> lots of "Add New Configurations",it is easy to config a url. like zk queue 
> configuration in hbase ,like jdbc url in hive. In hdfs service, only need to 
> config "fs.default.name" :
> Namenode URL *=hdfs://hdfscluster ?old? 
> Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000 ?new?
> 
> 
> Diffs
> -
> 
>   
> hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/client/HdfsClient.java
>  c252213f 
> 
> 
> Diff: https://reviews.apache.org/r/63142/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Qiang Zhang
> 
>

[jira] [Commented] (RANGER-1672) Ranger supports plugin to enable, monitor and manage apache kylin

2017-10-19 Thread Qiang Zhang (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212077#comment-16212077
 ] 

Qiang Zhang commented on RANGER-1672:
-

[~vperiasamy], I  can not  still add and modify the wiki page. Thanks!

> Ranger supports plugin to enable, monitor and manage apache kylin
> -
>
> Key: RANGER-1672
> URL: https://issues.apache.org/jira/browse/RANGER-1672
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>  Labels: newbie, patch
> Attachments: 
> 0001-RANGER-1672-Ranger-supports-plugin-to-enable-monitor.patch, 
> KylinAuditLog.jpg, KylinPlugins.jpg, KylinPolicies.jpg, 
> KylinServiceEntry.jpg, NewKylinPolicy.jpg, NewKylinService.jpg
>
>
> Apache Kylin is an open source Distributed Analytics Engine designed to 
> provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop 
> supporting extremely large datasets, original contributed from eBay Inc. 
> Apache Kylin lets user query massive data set at sub-second latency in 3 
> steps.
> 1. Identify a Star Schema on Hadoop.
> 2. Build Cube from the identified tables.
> 3. Query with ANSI-SQL and get results in sub-second, via ODBC, JDBC or 
> RESTful API.
> We should support that using Ranger to control kylin's access rights for 
> project and cube.
> Specific implementation plan is as following:
> On the ranger website, administrators can configure policies to control user 
> access to projects and cube permissions.
> Kylin provides an abstract class and authorization interfaces for use by the 
> ranger plugin. kylin instantiates ranger plugin’s implementation class when 
> starting(this class extends the abstract class provided by kylin).
> Ranger plugin periodically polls ranger admin, updates the policy to the 
> local, and updates project and cube access rights based on policy information.
> In the Kylin side：
> 1. Kylin provides an abstract class that enables the ranger plugin's 
> implementation class to extend.
> 2. Add configuration item. 1) ranger authorization switch, 2) ranger plugin 
> implementation class's name.
> 3. Instantiate the ranger plugin implementation class when starting kylin.
> 4. kylin provides authorization interfaces for ranger plugin calls.
> 5. According to the ranger authorization configuration item, hide kylin's 
> authorization management page.
> 6. Using ranger manager access rights of the kylin does not affect kylin's 
> existing permissions functions and logic.
> In the Ranger side：
> 1. Ranger plugin will periodically polls ranger admin, updates the policy to 
> the local.
> 2. The ranger plugin invoking the authorization interfaces provided by kylin 
> to updates the project and cube access rights based on the policy information.
> reference link:https://issues.apache.org/jira/browse/KYLIN-2703



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 63145: This JAVA_VERSION_REQUIRED configuration item is invalid in security admin installer, we should enable it to control the necessary java version.

2017-10-19 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63145/#review188804
---


Ship it!




Ship It!

- Qiang Zhang


On Oct. 20, 2017, 2:02 a.m., pengjianhua wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63145/
> ---
> 
> (Updated Oct. 20, 2017, 2:02 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
> hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan 
> Neethiraj, Velmurugan Periasamy, and Qiang Zhang.
> 
> 
> Bugs: RANGER-1846
> https://issues.apache.org/jira/browse/RANGER-1846
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The java version must be equal to or more than 1.8 when we set db_ssl_enabled 
> equal to true.
> This JAVA_VERSION_REQUIRED configuration item is invalid in security admin 
> installer, we should enable it to control the necessary java version.
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/setup.sh e2a69b71 
> 
> 
> Diff: https://reviews.apache.org/r/63145/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> pengjianhua
> 
>

[jira] [Created] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 
Ronald van de Kuil created RANGER-1847:
--

 Summary: Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
 Key: RANGER-1847
 URL: https://issues.apache.org/jira/browse/RANGER-1847
 Project: Ranger
  Issue Type: Test
  Components: plugins
Affects Versions: 0.7.1, 0.6.3
 Environment: ubuntu stand-alone hobby environment
Reporter: Ronald van de Kuil
Priority: Minor


I am such a NOOB hobby-ing away. And I like it. ;)

I figured I would give it a try to setup Kafka to use the 
sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
the auditing (instead of GSSAPI).

I got it to work pretty far. KafkaServer gets into state SaslAuthenticated with 
Zookeeper. 

Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer complains 
about Kerberos. 

I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212230#comment-16212230
 ] 

Ronald van de Kuil commented on RANGER-1847:


I think that the relevant settings in the kafka server are: 

listeners=SASL_PLAINTEXT://:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN
authorizer.class.name=org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer
super.users=User:kafka
allow.everyone.if.no.acl.found=false

> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212232#comment-16212232
 ] 

Ronald van de Kuil commented on RANGER-1847:


[2017-10-14 13:16:51,905] INFO Default UGI before using new Subject:root 
(auth:SIMPLE) (org.apache.ranger.audit.provider.MiscUtil)
[2017-10-14 13:16:51,905] ERROR Error getting principal. 
(org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
java.io.IOException: Provided Subject must contain a KerberosPrincipal
at 
org.apache.hadoop.security.UserGroupInformation.getUGIFromSubject(UserGroupInformation.java:735)
at 
org.apache.ranger.audit.provider.MiscUtil.createUGIFromSubject(MiscUtil.java:457)
at 
org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:85)
at 
org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:96)
at 
kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:209)
at 
kafka.server.KafkaServer$$anonfun$startup$3.apply(KafkaServer.scala:207)
at scala.Option.map(Option.scala:145)
at kafka.server.KafkaServer.startup(KafkaServer.scala:207)
at 
kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
at kafka.Kafka$.main(Kafka.scala:67)
at kafka.Kafka.main(Kafka.scala)

> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212233#comment-16212233
 ] 

Ronald van de Kuil commented on RANGER-1847:


[2017-10-14 13:16:51,911] INFO Calling plugin.init() 
(org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
[2017-10-14 13:16:51,911] ERROR addResourceIfReadable(ranger-kafka-audit.xml): 
couldn't find resource file location 
(org.apache.ranger.authorization.hadoop.config.RangerConfiguration)
[2017-10-14 13:16:51,914] ERROR 
addResourceIfReadable(ranger-kafka-security.xml): couldn't find resource file 
location (org.apache.ranger.authorization.hadoop.config.RangerConfiguration)

Well, they are present:
ronald@ronald-VirtualBox:/usr/local/kafka/config$ ls -lrt
total 96
-rw-r--r-- 1 root  root   1023 aug  4  2016 zookeeper.properties.old
-rw-r--r-- 1 root  root   1032 aug  4  2016 tools-log4j.properties
-rw-r--r-- 1 root  root   1900 aug  4  2016 producer.properties
-rw-r--r-- 1 root  root   4369 aug  4  2016 log4j.properties
-rw-r--r-- 1 root  root   1199 aug  4  2016 consumer.properties
-rw-r--r-- 1 root  root   2061 aug  4  2016 connect-standalone.properties
-rw-r--r-- 1 root  root   1074 aug  4  2016 connect-log4j.properties
-rw-r--r-- 1 root  root881 aug  4  2016 connect-file-source.properties
-rw-r--r-- 1 root  root883 aug  4  2016 connect-file-sink.properties
-rw-r--r-- 1 root  root   2760 aug  4  2016 connect-distributed.properties
-rw-r--r-- 1 root  root909 aug  4  2016 connect-console-source.properties
-rw-r--r-- 1 root  root906 aug  4  2016 connect-console-sink.properties
-rw-r--r-- 1 root  root   5243 aug  4  2016 server.properties.old
-rw-r--r-- 1 kafka hadoop   69 okt 14 12:19 ranger-security.xml
-rwxr--r-- 1 kafka hadoop 9642 okt 14 12:19 ranger-kafka-audit.xml
-rwxr--r-- 1 kafka hadoop 2928 okt 14 12:19 ranger-kafka-security.xml
-rwxr--r-- 1 kafka hadoop 2272 okt 14 12:19 ranger-policymgr-ssl.xml
-rw-r--r-- 1 root  root   1270 okt 14 12:43 zookeeper.properties
-rw-r--r-- 1 root  root   6020 okt 14 13:03 server.properties


> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212234#comment-16212234
 ] 

Ronald van de Kuil commented on RANGER-1847:


[2017-10-14 13:16:52,388] WARN cache file does not exist or not readable 'null' 
(org.apache.ranger.plugin.util.PolicyRefresher)
[2017-10-14 13:16:52,394] ERROR PolicyRefresher(serviceName=null): failed to 
refresh policies. Will continue to use last known version of policies (-1) 
(org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: 
java.lang.IllegalArgumentException: URI is not absolute
at 
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at 
com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at 
org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:115)
at 
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:232)
at 
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:188)
at 
org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:158)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at 
com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at 
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2017-10-14 13:16:52,395] WARN cache file does not exist or not readable 'null' 
(org.apache.ranger.plugin.util.PolicyRefresher)
[2017-10-14 13:16:52,426] INFO [ThrottledRequestReaper-Produce], Starting  
(kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-10-14 13:16:52,429] INFO [ThrottledRequestReaper-Fetch], Starting  
(kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-10-14 13:16:52,443] INFO Will not load MX4J, mx4j-tools.jar is not in the 
classpath (kafka.utils.Mx4jLoader$)
[2017-10-14 13:16:52,474] INFO Creating /brokers/ids/0 (is it secure? false) 
(kafka.utils.ZKCheckedEphemeral)
[2017-10-14 13:16:52,485] INFO Result of znode creation is: OK 
(kafka.utils.ZKCheckedEphemeral)
[2017-10-14 13:16:52,486] INFO Registered broker 0 at path /brokers/ids/0 with 
addresses: SASL_PLAINTEXT -> EndPoint(ronald-VirtualBox,9092,SASL_PLAINTEXT) 
(kafka.utils.ZkUtils)
[2017-10-14 13:16:52,486] WARN No meta.properties file under dir 
/var/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2017-10-14 13:16:52,580] INFO Kafka version : 0.10.0.1 
(org.apache.kafka.common.utils.AppInfoParser)
[2017-10-14 13:16:52,581] INFO Kafka commitId : a7a17cdec9eaa6c5 
(org.apache.kafka.common.utils.AppInfoParser)
[2017-10-14 13:16:52,581] INFO [Kafka Server 0], started 
(kafka.server.KafkaServer)
[2017-10-14 13:16:52,750] ERROR Ranger Plugin returned null. Returning false 
(org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)

> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212235#comment-16212235
 ] 

Ronald van de Kuil commented on RANGER-1847:


[2017-10-14 13:17:22,396] ERROR PolicyRefresher(serviceName=null): failed to 
refresh policies. Will continue to use last known version of policies (-1) 
(org.apache.ranger.plugin.util.PolicyRefresher)
com.sun.jersey.api.client.ClientHandlerException: 
java.lang.IllegalArgumentException: URI is not absolute
at 
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:151)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at 
com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:507)
at 
org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:115)
at 
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:232)
at 
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:188)
at 
org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:158)
Caused by: java.lang.IllegalArgumentException: URI is not absolute
at java.net.URI.toURL(URI.java:1088)
at 
com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at 
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
... 8 more
[2017-10-14 13:17:22,396] WARN cache file does not exist or not readable 'null' 
(org.apache.ranger.plugin.util.PolicyRefresher)

And then it repeats

> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Review Request 63142: Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Qiang Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63142/
---

(Updated 十月 20, 2017, 6:18 a.m.)


Review request for ranger, Alok Lal, Ankita Sinha, Don Bosco Durai, Colm O 
hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, 
Selvamohan Neethiraj, Velmurugan Periasamy, and Qiang Zhang.


Bugs: RANGER-1844
https://issues.apache.org/jira/browse/RANGER-1844


Repository: ranger


Description
---

In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
mode,you have to config a lot of configs in hdfs_dev,such as:
Namenode URL *=hdfs://hdfscluster
===Add New Configurations===
dfs.nameservices=hdfscluster
dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.ha.namenodes.hdfscluster=nn1,nn2
dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
===End of add New Configurations===
And other big data components such as hbase,hive and so on can support HA 
without config
lots of "Add New Configurations",it is easy to config a url. like zk queue 
configuration in hbase ,like jdbc url in hive. In hdfs service, only need to 
config "fs.default.name" :
Namenode URL *=hdfs://hdfscluster ?old? 
Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000 ?new?


Diffs (updated)
-

  
hdfs-agent/src/main/java/org/apache/ranger/services/hdfs/client/HdfsClient.java 
c252213f 
  security-admin/src/main/webapp/scripts/models/BackboneFormDataType.js 
3f8697e7 
  security-admin/src/main/webapp/scripts/modules/globalize/message/en.js 
811db0c9 


Diff: https://reviews.apache.org/r/63142/diff/2/

Changes: https://reviews.apache.org/r/63142/diff/1-2/


Testing
---


Thanks,

Qiang Zhang

[jira] [Updated] (RANGER-1844) Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Qiang Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Zhang updated RANGER-1844:

Attachment: (was: 
0001-RANGER-1844-Ranger-admin-support-hdfs-HA-configurati.patch)

> Ranger admin support hdfs HA configuration when creating hdfs service.
> --
>
> Key: RANGER-1844
> URL: https://issues.apache.org/jira/browse/RANGER-1844
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins, Ranger
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
>
> In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
> mode,you have to config a lot of configs in hdfs_dev,such as:
> Namenode URL *=hdfs://hdfscluster
> ===Add New Configurations===
> dfs.nameservices=hdfscluster
> dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> dfs.ha.namenodes.hdfscluster=nn1,nn2
> dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
> dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
> ===End of add New Configurations===
> And other big data components such as hbase,hive and so on can support HA 
> without config
> lots of "Add New Configurations",it is easy to config a url. like zk queue 
> configuration in hbase,like jdbc url in hive.  In hdfs service, only need 
> to config "fs.default.name" :
> Namenode URL *=hdfs://hdfscluster 
>  【old】 
> Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000【new】
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (RANGER-1844) Ranger admin support hdfs HA configuration when creating hdfs service.

2017-10-19 Thread Qiang Zhang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Qiang Zhang updated RANGER-1844:

Attachment: 0001-RANGER-1844-Ranger-admin-support-hdfs-HA-configurati.patch

> Ranger admin support hdfs HA configuration when creating hdfs service.
> --
>
> Key: RANGER-1844
> URL: https://issues.apache.org/jira/browse/RANGER-1844
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins, Ranger
>Affects Versions: master
>Reporter: Qiang Zhang
>Assignee: Qiang Zhang
> Attachments: 
> 0001-RANGER-1844-Ranger-admin-support-hdfs-HA-configurati.patch
>
>
> In ranger admin, when creating a hdfs service, if hdfs cluster is in HA 
> mode,you have to config a lot of configs in hdfs_dev,such as:
> Namenode URL *=hdfs://hdfscluster
> ===Add New Configurations===
> dfs.nameservices=hdfscluster
> dfs.client.failover.proxy.provider.hdfscluster=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> dfs.ha.namenodes.hdfscluster=nn1,nn2
> dfs.namenode.rpc-address.hdfscluster.nn1=hdfs://10.43.159.240:9000
> dfs.namenode.rpc-address.hdfscluster.nn2=hdfs://10.43.159.245:9000
> ===End of add New Configurations===
> And other big data components such as hbase,hive and so on can support HA 
> without config
> lots of "Add New Configurations",it is easy to config a url. like zk queue 
> configuration in hbase,like jdbc url in hive.  In hdfs service, only need 
> to config "fs.default.name" :
> Namenode URL *=hdfs://hdfscluster 
>  【old】 
> Namenode URL *=hdfs://dap230-183:9000,hdfs://dap229-183:9000【new】
>  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212246#comment-16212246
 ] 

Ronald van de Kuil commented on RANGER-1847:


I also ran a setup with ranger-0.7.1-kafka-plugin and kafka_2.12-0.11.0.1 
server. 

Then I get the following:
[2017-10-14 15:54:41,167] ERROR Error getting principal. 
(org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer)
java.lang.NoClassDefFoundError: org/apache/kafka/common/network/LoginType
at 
org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:82)
at 
org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:94)
at kafka.server.KafkaServer.$anonfun$startup$4(KafkaServer.scala:247)
at scala.Option.map(Option.scala:146)
at kafka.server.KafkaServer.startup(KafkaServer.scala:245)
at 
kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
at kafka.Kafka$.main(Kafka.scala:65)
at kafka.Kafka.main(Kafka.scala)
Caused by: java.lang.ClassNotFoundException: 
org.apache.kafka.common.network.LoginType
at java.lang.ClassLoader.findClass(ClassLoader.java:530)
at 
org.apache.ranger.plugin.classloader.RangerPluginClassLoader$MyClassLoader.findClass(RangerPluginClassLoader.java:272)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at 
org.apache.ranger.plugin.classloader.RangerPluginClassLoader.loadClass(RangerPluginClassLoader.java:125)

> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212249#comment-16212249
 ] 

Ronald van de Kuil commented on RANGER-1847:


Followed by similar exceptions:
- couldn't find resource file location;
- failed to refresh policies.


> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212258#comment-16212258
 ] 

Ronald van de Kuil commented on RANGER-1847:


Hmm, ... after reviewing the logs of the kafka server I realized that I had to 
add the Kafka Server's config directory to the CLASSPATH. It gives me different 
errors.

[2017-10-14 14:16:50,357] ERROR Error getting policies. secureMode=false, 
user=root (auth:SIMPLE), 
response={"httpStatusCode":400,"statusCode":1,"msgDesc":"Service:kafkadev not 
found","messageList":[{"name":"DATA_NOT_FOUND","rbKey":"xa.error.data_not_found","message":"Data
 not found"}]}, serviceName=kafkadev 
(org.apache.ranger.admin.client.RangerAdminRESTClient)

[2017-10-14 14:16:50,372] WARN cache file does not exist or not readable 
'/etc/ranger/kafkadev/policycache/kafka_kafkadev.json' 
(org.apache.ranger.plugin.util.PolicyRefresher)

> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

[ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16212260#comment-16212260
 ] 

Ronald van de Kuil commented on RANGER-1847:


I feel I am pretty close. 

It would be nice if someone could give me some re-assurance that this setup 
should work.

> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (RANGER-1847) Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN

2017-10-19 Thread Ronald van de Kuil (JIRA) 

 [ 
https://issues.apache.org/jira/browse/RANGER-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ronald van de Kuil updated RANGER-1847:
---
Description: 
I am such a NOOB hobby-ing away. And I like it. ;)

I figured I would give it a try to setup Kafka to use the 
sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
the auditing (instead of GSSAPI).

I got it to work pretty far. KafkaServer gets into state SaslAuthenticated with 
Zookeeper. 

Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer complains 
about Kerberos. 

I then updated the CLASSPATH and it complains about something else.

I am not sure how to classify this issue. 

  was:
I am such a NOOB hobby-ing away. And I like it. ;)

I figured I would give it a try to setup Kafka to use the 
sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
the auditing (instead of GSSAPI).

I got it to work pretty far. KafkaServer gets into state SaslAuthenticated with 
Zookeeper. 

Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer complains 
about Kerberos. 

I am not sure how to classify this issue. 


> Ranger Kafka Plugin sasl.enabled.mechanisms=PLAIN
> -
>
> Key: RANGER-1847
> URL: https://issues.apache.org/jira/browse/RANGER-1847
> Project: Ranger
>  Issue Type: Test
>  Components: plugins
>Affects Versions: 0.6.3, 0.7.1
> Environment: ubuntu stand-alone hobby environment
>Reporter: Ronald van de Kuil
>Priority: Minor
>
> I am such a NOOB hobby-ing away. And I like it. ;)
> I figured I would give it a try to setup Kafka to use the 
> sasl.enabled.mechanisms of type PLAIN with ranger to do the authorisation and 
> the auditing (instead of GSSAPI).
> I got it to work pretty far. KafkaServer gets into state SaslAuthenticated 
> with Zookeeper. 
> Next it loads the ranger kafka plugin. Then the RangerKafkaAuthorizer 
> complains about Kerberos. 
> I then updated the CLASSPATH and it complains about something else.
> I am not sure how to classify this issue. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)