[jira] [Commented] (RANGER-2599) Add more audit data to HBase grant/revoke events
[
https://issues.apache.org/jira/browse/RANGER-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945922#comment-16945922
]
Zsombor Gegesy commented on RANGER-2599:
Thanks for your patch, the commit is merged to
[master|https://github.com/apache/ranger/commit/0011a441d0151c9eeec78ab1e5b9e3e3262129b8]
> Add more audit data to HBase grant/revoke events
>
>
> Key: RANGER-2599
> URL: https://issues.apache.org/jira/browse/RANGER-2599
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Affects Versions: 2.0.0
>Reporter: Andor Molnar
>Priority: Major
> Labels: patch-available
> Fix For: 2.1.0
>
>
> Currently {{RangerAuthorizationCoprocessor}} correctly captures all data from
> HBase grant and revoke events, but {{RangerBasePlugin}} only copies certain
> fields to RangerAccessEvent.
> {{RequestData}} is one the fields which are copied to the final entity and
> currently not being used by the co-processor. I'd like to add some missing
> information to this field and make it available on the UI similarly how Hive
> queries are shown in a small pop-up.
> First, I change the co-processor to populate {{RequestData}} with additional
> grant/revoke information.
> Second, I modify JS to show the RequestData pop-up on HBase audit events too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Review Request 71566: RANGER-2599. Add more audit data to HBase grant/revoke events
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71566/#review218114 --- Ship it! Ship It! - Zsombor Gegesy On Oct. 4, 2019, 12:17 p.m., Andor Molnar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71566/ > --- > > (Updated Oct. 4, 2019, 12:17 p.m.) > > > Review request for ranger. > > > Repository: ranger > > > Description > --- > > Currently RangerAuthorizationCoprocessor correctly captures all data from > HBase grant and revoke events, but RangerBasePlugin only copies certain > fields to RangerAccessEvent. > > RequestData is one the fields which are copied to the final entity and > currently not being used by the co-processor. I'd like to add some missing > information to this field and make it available on the UI similarly how Hive > queries are shown in a small pop-up. > > First, I change the co-processor to populate RequestData with additional > grant/revoke information. > > Second, I modify JS to show the RequestData pop-up on HBase audit events too. > > https://issues.apache.org/jira/browse/RANGER-2599 > > > Diffs > - > > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java > 364a415943bed22890dc5773e162962bb30147f9 > security-admin/src/main/webapp/scripts/utils/XAViewUtils.js > 6ef33435af7e7cbed7fbc5dc4174c8fb46be3aaf > > > Diff: https://reviews.apache.org/r/71566/diff/3/ > > > Testing > --- > > Tested on private test cluster. > > > Thanks, > > Andor Molnar > >
Re: Review Request 71566: RANGER-2599. Add more audit data to HBase grant/revoke events
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71566/#review218018 --- Ship it! Ship It! - Zsombor Gegesy On Oct. 2, 2019, 12:33 p.m., Andor Molnar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71566/ > --- > > (Updated Oct. 2, 2019, 12:33 p.m.) > > > Review request for ranger. > > > Repository: ranger > > > Description > --- > > Currently RangerAuthorizationCoprocessor correctly captures all data from > HBase grant and revoke events, but RangerBasePlugin only copies certain > fields to RangerAccessEvent. > > RequestData is one the fields which are copied to the final entity and > currently not being used by the co-processor. I'd like to add some missing > information to this field and make it available on the UI similarly how Hive > queries are shown in a small pop-up. > > First, I change the co-processor to populate RequestData with additional > grant/revoke information. > > Second, I modify JS to show the RequestData pop-up on HBase audit events too. > > https://issues.apache.org/jira/browse/RANGER-2599 > > > Diffs > - > > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java > 364a415943bed22890dc5773e162962bb30147f9 > security-admin/src/main/webapp/scripts/utils/XAViewUtils.js > 6ef33435af7e7cbed7fbc5dc4174c8fb46be3aaf > > > Diff: https://reviews.apache.org/r/71566/diff/2/ > > > Testing > --- > > Tested on private test cluster. > > > Thanks, > > Andor Molnar > >
Re: Review Request 71566: RANGER-2599. Add more audit data to HBase grant/revoke events
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71566/#review218008
---
security-admin/src/main/webapp/scripts/utils/XAViewUtils.js
Lines 72 (patched)
<https://reviews.apache.org/r/71566/#comment305469>
You could have a js object:
var titleMap = {
XAEnums.ServiceType.Service_HIVE.label: 'Hive Query',
AEnums.ServiceType.Service_HBASE.label: 'HBase Audit Data'
}
and you can use it like:
var title = titleMap[model.get('serviceType')] || 'Request Data';
- Zsombor Gegesy
On Oct. 1, 2019, 1:45 p.m., Andor Molnar wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71566/
> ---
>
> (Updated Oct. 1, 2019, 1:45 p.m.)
>
>
> Review request for ranger.
>
>
> Repository: ranger
>
>
> Description
> ---
>
> Currently RangerAuthorizationCoprocessor correctly captures all data from
> HBase grant and revoke events, but RangerBasePlugin only copies certain
> fields to RangerAccessEvent.
>
> RequestData is one the fields which are copied to the final entity and
> currently not being used by the co-processor. I'd like to add some missing
> information to this field and make it available on the UI similarly how Hive
> queries are shown in a small pop-up.
>
> First, I change the co-processor to populate RequestData with additional
> grant/revoke information.
>
> Second, I modify JS to show the RequestData pop-up on HBase audit events too.
>
> https://issues.apache.org/jira/browse/RANGER-2599
>
>
> Diffs
> -
>
>
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
> 364a415943bed22890dc5773e162962bb30147f9
> security-admin/src/main/webapp/scripts/utils/XAViewUtils.js
> 6ef33435af7e7cbed7fbc5dc4174c8fb46be3aaf
>
>
> Diff: https://reviews.apache.org/r/71566/diff/1/
>
>
> Testing
> ---
>
> Tested on private test cluster.
>
>
> Thanks,
>
> Andor Molnar
>
>
[jira] [Commented] (RANGER-2599) Add more audit data to HBase grant/revoke events
[
https://issues.apache.org/jira/browse/RANGER-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941845#comment-16941845
]
Zsombor Gegesy commented on RANGER-2599:
Sorry, I can't assign this ticket to you, Andor.
> Add more audit data to HBase grant/revoke events
>
>
> Key: RANGER-2599
> URL: https://issues.apache.org/jira/browse/RANGER-2599
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Affects Versions: 2.0.0
>Reporter: Andor Molnar
>Priority: Major
> Fix For: 2.1.0
>
>
> Currently {{RangerAuthorizationCoprocessor}} correctly captures all data from
> HBase grant and revoke events, but {{RangerBasePlugin}} only copies certain
> fields to RangerAccessEvent.
> {{RequestData}} is one the fields which are copied to the final entity and
> currently not being used by the co-processor. I'd like to add some missing
> information to this field and make it available on the UI similarly how Hive
> queries are shown in a small pop-up.
> First, I change the co-processor to populate {{RequestData}} with additional
> grant/revoke information.
> Second, I modify JS to show the RequestData pop-up on HBase audit events too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (RANGER-2599) Add more audit data to HBase grant/revoke events
[
https://issues.apache.org/jira/browse/RANGER-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsombor Gegesy updated RANGER-2599:
---
Fix Version/s: 2.1.0
> Add more audit data to HBase grant/revoke events
>
>
> Key: RANGER-2599
> URL: https://issues.apache.org/jira/browse/RANGER-2599
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Affects Versions: 2.0.0
>Reporter: Andor Molnar
>Priority: Major
> Fix For: 2.1.0
>
>
> Currently {{RangerAuthorizationCoprocessor}} correctly captures all data from
> HBase grant and revoke events, but {{RangerBasePlugin}} only copies certain
> fields to RangerAccessEvent.
> {{RequestData}} is one the fields which are copied to the final entity and
> currently not being used by the co-processor. I'd like to add some missing
> information to this field and make it available on the UI similarly how Hive
> queries are shown in a small pop-up.
> First, I change the co-processor to populate {{RequestData}} with additional
> grant/revoke information.
> Second, I modify JS to show the RequestData pop-up on HBase audit events too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (RANGER-2599) Add more audit data to HBase grant/revoke events
[
https://issues.apache.org/jira/browse/RANGER-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsombor Gegesy updated RANGER-2599:
---
Affects Version/s: 2.0.0
> Add more audit data to HBase grant/revoke events
>
>
> Key: RANGER-2599
> URL: https://issues.apache.org/jira/browse/RANGER-2599
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Affects Versions: 2.0.0
>Reporter: Andor Molnar
>Priority: Major
>
> Currently {{RangerAuthorizationCoprocessor}} correctly captures all data from
> HBase grant and revoke events, but {{RangerBasePlugin}} only copies certain
> fields to RangerAccessEvent.
> {{RequestData}} is one the fields which are copied to the final entity and
> currently not being used by the co-processor. I'd like to add some missing
> information to this field and make it available on the UI similarly how Hive
> queries are shown in a small pop-up.
> First, I change the co-processor to populate {{RequestData}} with additional
> grant/revoke information.
> Second, I modify JS to show the RequestData pop-up on HBase audit events too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Review Request 71527: Upgrade jackson-databind
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71527/#review217892 --- Ship it! Ship It! - Zsombor Gegesy On Sept. 20, 2019, 1:39 p.m., László Terjéki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71527/ > --- > > (Updated Sept. 20, 2019, 1:39 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2583 > https://issues.apache.org/jira/browse/RANGER-2583 > > > Repository: ranger > > > Description > --- > > Upgrade jackson-databind version to 2.9.9.3 > > > Diffs > - > > kms/pom.xml d344d16f0 > pom.xml abc5d59c0 > security-admin/pom.xml fbc24f27a > src/main/assembly/tagsync.xml 80188e82c > tagsync/pom.xml 16f6183d6 > > > Diff: https://reviews.apache.org/r/71527/diff/1/ > > > Testing > --- > > > Thanks, > > László Terjéki > >
[jira] [Resolved] (RANGER-2394) Filter/exclude multiple users in audit search
[ https://issues.apache.org/jira/browse/RANGER-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy resolved RANGER-2394. Resolution: Fixed Merged into [master|https://github.com/apache/ranger/commit/444db6edb092f23de89813503dae04f909e5325e] > Filter/exclude multiple users in audit search > - > > Key: RANGER-2394 > URL: https://issues.apache.org/jira/browse/RANGER-2394 > Project: Ranger > Issue Type: Improvement > Components: admin >Affects Versions: master >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Labels: admin, solr > Fix For: 2.0.0 > > Attachments: RANGER-2394-5.patch, RANGER-2394-6.patch, > RANGER-2394.patch > > > Currently the audit search only allows to: > * filter to one user's activity > * exclude all 'service users' from every user's activity. > If there were way to search for multiple users or exclude multiple users from > the search list, it would make debugging complex interactions simpler, for > example only look for actions for 'alice' and 'hive' and 'yarn' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70389: RANGER-2394 - filter multiple users or exclude multiple users in audit search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70389/ --- (Updated June 5, 2019, 12:03 p.m.) Review request for ranger. Changes --- One more js issue Bugs: RANGER-2394 https://issues.apache.org/jira/browse/RANGER-2394 Repository: ranger Description --- Currently the audit search only allows to: * filter to one user's activity * exclude all 'service users' from every user's activity. If there were way to search for multiple users or exclude multiple users from the search list, it would make debugging complex interactions simpler, for example only look for actions for 'alice' and 'hive' and 'yarn' The frontend tweaked a bit, so if multiple users are passed to the jquery layer, the user names are always converted as requestUser=aaa=bbb=ccc instead of changing to requestUser[]=aaa[]=bbb[]=ccc, which would be an incompatible change between the server and to any potential client code. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java 037888e8cd33f7a9bae9720a3c3180222758bda8 security-admin/src/main/java/org/apache/ranger/rest/XAuditREST.java fdf5ad86bff91e811c7d56ceb87845f158f8c789 security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java a517d7627af47adacb5bd8f90a104499981beeb8 security-admin/src/main/webapp/scripts/utils/XAUtils.js a7c4497feea13f06ddf8b92c05344301fca5a9a0 security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js 9c01eb76e410b45b19056f2638e23b8722cfb0fe security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java ef149d5fa01b72f1ec3631012db4cf3433570547 Diff: https://reviews.apache.org/r/70389/diff/6/ Changes: https://reviews.apache.org/r/70389/diff/5-6/ Testing --- Tested on a live cluster that: * searching for one user * searching for multiple users * excluding one user * excluding multiple users * searching for one user + 'excluding service users' * searching for multiple users + 'excluding service users' * excluding one user + 'excluding service users' * excluding multiple users + 'excluding service users' works as expected. Thanks, Zsombor Gegesy
[jira] [Updated] (RANGER-2394) Filter/exclude multiple users in audit search
[ https://issues.apache.org/jira/browse/RANGER-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2394: --- Attachment: RANGER-2394-6.patch > Filter/exclude multiple users in audit search > - > > Key: RANGER-2394 > URL: https://issues.apache.org/jira/browse/RANGER-2394 > Project: Ranger > Issue Type: Improvement > Components: admin >Affects Versions: master >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Labels: admin, solr > Fix For: 2.0.0 > > Attachments: RANGER-2394-5.patch, RANGER-2394-6.patch, > RANGER-2394.patch > > > Currently the audit search only allows to: > * filter to one user's activity > * exclude all 'service users' from every user's activity. > If there were way to search for multiple users or exclude multiple users from > the search list, it would make debugging complex interactions simpler, for > example only look for actions for 'alice' and 'hive' and 'yarn' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RANGER-2394) Filter/exclude multiple users in audit search
[ https://issues.apache.org/jira/browse/RANGER-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2394: --- Attachment: RANGER-2394-5.patch > Filter/exclude multiple users in audit search > - > > Key: RANGER-2394 > URL: https://issues.apache.org/jira/browse/RANGER-2394 > Project: Ranger > Issue Type: Improvement > Components: admin >Affects Versions: master >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Labels: admin, solr > Fix For: 2.0.0 > > Attachments: RANGER-2394-5.patch, RANGER-2394.patch > > > Currently the audit search only allows to: > * filter to one user's activity > * exclude all 'service users' from every user's activity. > If there were way to search for multiple users or exclude multiple users from > the search list, it would make debugging complex interactions simpler, for > example only look for actions for 'alice' and 'hive' and 'yarn' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70389: RANGER-2394 - filter multiple users or exclude multiple users in audit search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70389/ --- (Updated May 30, 2019, 10:24 a.m.) Review request for ranger. Changes --- Fix js issues Bugs: RANGER-2394 https://issues.apache.org/jira/browse/RANGER-2394 Repository: ranger Description --- Currently the audit search only allows to: * filter to one user's activity * exclude all 'service users' from every user's activity. If there were way to search for multiple users or exclude multiple users from the search list, it would make debugging complex interactions simpler, for example only look for actions for 'alice' and 'hive' and 'yarn' The frontend tweaked a bit, so if multiple users are passed to the jquery layer, the user names are always converted as requestUser=aaa=bbb=ccc instead of changing to requestUser[]=aaa[]=bbb[]=ccc, which would be an incompatible change between the server and to any potential client code. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java 037888e8c security-admin/src/main/java/org/apache/ranger/rest/XAuditREST.java fdf5ad86b security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java a517d7627 security-admin/src/main/webapp/scripts/utils/XAUtils.js 18e86c9cc security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js 18dba7ace security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java ef149d5fa Diff: https://reviews.apache.org/r/70389/diff/5/ Changes: https://reviews.apache.org/r/70389/diff/4-5/ Testing --- Tested on a live cluster that: * searching for one user * searching for multiple users * excluding one user * excluding multiple users * searching for one user + 'excluding service users' * searching for multiple users + 'excluding service users' * excluding one user + 'excluding service users' * excluding multiple users + 'excluding service users' works as expected. Thanks, Zsombor Gegesy
Review Request 70615: RANGER-2421 - add http client to atlas plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70615/ --- Review request for ranger. Bugs: RANGER-2421 https://issues.apache.org/jira/browse/RANGER-2421 Repository: ranger Description --- Due to http client version difference, and the lack of httpclient-4.5.3.jar inside the ranger-atlas-plugin/lib/ranger-atlas-plugin-impl/ folder, the following exception can be seen: ava.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.evictIdleConnections(JLjava/util/concurrent/TimeUnit;)Lorg/apache/http/impl/client/HttpClientBuilder; at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:311) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:255) at org.apache.solr.client.solrj.impl.CloudSolrClient.(CloudSolrClient.java:280) at org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1600) at org.apache.ranger.audit.destination.SolrAuditDestination$1.run(SolrAuditDestination.java:126) at org.apache.ranger.audit.destination.SolrAuditDestination$1.run(SolrAuditDestination.java:123) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) at org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:516) at org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:123) at org.apache.ranger.audit.destination.SolrAuditDestination.init(SolrAuditDestination.java:72) at org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:179) at org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:217) Atlas has a httpclient-4.4.x, which lacks the needed method. Diffs - src/main/assembly/plugin-atlas.xml 4de27b071 Diff: https://reviews.apache.org/r/70615/diff/1/ Testing --- Added the necessary jars to a deployed Atlas server, and noticed that the exception disapears, and audit correctly sent. Thanks, Zsombor Gegesy
[jira] [Updated] (RANGER-2421) Solr audit fails in Atlas plugin
[
https://issues.apache.org/jira/browse/RANGER-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsombor Gegesy updated RANGER-2421:
---
Attachment: RANGER-2421.patch
> Solr audit fails in Atlas plugin
>
>
> Key: RANGER-2421
> URL: https://issues.apache.org/jira/browse/RANGER-2421
> Project: Ranger
> Issue Type: Bug
> Components: audit, plugins
> Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Major
> Labels: atlas, audit, plugin
> Attachments: RANGER-2421.patch
>
>
> Due to http client version difference, and the lack of httpclient-4.5.3.jar
> inside the ranger-atlas-plugin/lib/ranger-atlas-plugin-impl/ folder, the
> following exception can be seen:
> {code}
> ava.lang.NoSuchMethodError:
> org.apache.http.impl.client.HttpClientBuilder.evictIdleConnections(JLjava/util/concurrent/TimeUnit;)Lorg/apache/http/impl/client/HttpClientBuilder;
> at
> org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:311)
> at
> org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
> at
> org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
> at
> org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:255)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.(CloudSolrClient.java:280)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1600)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination$1.run(SolrAuditDestination.java:126)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination$1.run(SolrAuditDestination.java:123)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:516)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:123)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination.init(SolrAuditDestination.java:72)
> at
> org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:179)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:217)
> {code}
> Atlas has a httpclient-4.4.x, which lacks the needed method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (RANGER-2421) Solr audit fails in Atlas plugin
Zsombor Gegesy created RANGER-2421:
--
Summary: Solr audit fails in Atlas plugin
Key: RANGER-2421
URL: https://issues.apache.org/jira/browse/RANGER-2421
Project: Ranger
Issue Type: Bug
Components: audit, plugins
Reporter: Zsombor Gegesy
Assignee: Zsombor Gegesy
Due to http client version difference, and the lack of httpclient-4.5.3.jar
inside the ranger-atlas-plugin/lib/ranger-atlas-plugin-impl/ folder, the
following exception can be seen:
{code}
ava.lang.NoSuchMethodError:
org.apache.http.impl.client.HttpClientBuilder.evictIdleConnections(JLjava/util/concurrent/TimeUnit;)Lorg/apache/http/impl/client/HttpClientBuilder;
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:311)
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at
org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:255)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.(CloudSolrClient.java:280)
at
org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1600)
at
org.apache.ranger.audit.destination.SolrAuditDestination$1.run(SolrAuditDestination.java:126)
at
org.apache.ranger.audit.destination.SolrAuditDestination$1.run(SolrAuditDestination.java:123)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at
org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:516)
at
org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:123)
at
org.apache.ranger.audit.destination.SolrAuditDestination.init(SolrAuditDestination.java:72)
at
org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:179)
at
org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:217)
{code}
Atlas has a httpclient-4.4.x, which lacks the needed method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 70389: RANGER-2394 - filter multiple users or exclude multiple users in audit search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70389/ --- (Updated May 3, 2019, 6:18 p.m.) Review request for ranger. Changes --- Fix rebase problems Bugs: RANGER-2394 https://issues.apache.org/jira/browse/RANGER-2394 Repository: ranger Description --- Currently the audit search only allows to: * filter to one user's activity * exclude all 'service users' from every user's activity. If there were way to search for multiple users or exclude multiple users from the search list, it would make debugging complex interactions simpler, for example only look for actions for 'alice' and 'hive' and 'yarn' The frontend tweaked a bit, so if multiple users are passed to the jquery layer, the user names are always converted as requestUser=aaa=bbb=ccc instead of changing to requestUser[]=aaa[]=bbb[]=ccc, which would be an incompatible change between the server and to any potential client code. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java 703d30beb security-admin/src/main/java/org/apache/ranger/rest/XAuditREST.java fdf5ad86b security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java 9be2ef480 security-admin/src/main/webapp/scripts/utils/XAUtils.js 6da76d5bf security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js 5ae06d5cd security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java 1f7370955 Diff: https://reviews.apache.org/r/70389/diff/4/ Changes: https://reviews.apache.org/r/70389/diff/3-4/ Testing --- Tested on a live cluster that: * searching for one user * searching for multiple users * excluding one user * excluding multiple users * searching for one user + 'excluding service users' * searching for multiple users + 'excluding service users' * excluding one user + 'excluding service users' * excluding multiple users + 'excluding service users' works as expected. Thanks, Zsombor Gegesy
Re: Review Request 70389: RANGER-2394 - filter multiple users or exclude multiple users in audit search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70389/ --- (Updated April 27, 2019, 6:17 p.m.) Review request for ranger. Bugs: RANGER-2394 https://issues.apache.org/jira/browse/RANGER-2394 Repository: ranger Description --- Currently the audit search only allows to: * filter to one user's activity * exclude all 'service users' from every user's activity. If there were way to search for multiple users or exclude multiple users from the search list, it would make debugging complex interactions simpler, for example only look for actions for 'alice' and 'hive' and 'yarn' The frontend tweaked a bit, so if multiple users are passed to the jquery layer, the user names are always converted as requestUser=aaa=bbb=ccc instead of changing to requestUser[]=aaa[]=bbb[]=ccc, which would be an incompatible change between the server and to any potential client code. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java 703d30beb security-admin/src/main/java/org/apache/ranger/rest/XAuditREST.java fdf5ad86b security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java 9be2ef480 security-admin/src/main/webapp/scripts/utils/XAUtils.js 6da76d5bf security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js 5ae06d5cd security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java 1f7370955 Diff: https://reviews.apache.org/r/70389/diff/3/ Changes: https://reviews.apache.org/r/70389/diff/2-3/ Testing --- Tested on a live cluster that: * searching for one user * searching for multiple users * excluding one user * excluding multiple users * searching for one user + 'excluding service users' * searching for multiple users + 'excluding service users' * excluding one user + 'excluding service users' * excluding multiple users + 'excluding service users' works as expected. Thanks, Zsombor Gegesy
Re: Review Request 70389: RANGER-2394 - filter multiple users or exclude multiple users in audit search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70389/ --- (Updated April 11, 2019, 8:24 a.m.) Review request for ranger. Changes --- Unit test fix Bugs: RANGER-2394 https://issues.apache.org/jira/browse/RANGER-2394 Repository: ranger Description --- Currently the audit search only allows to: * filter to one user's activity * exclude all 'service users' from every user's activity. If there were way to search for multiple users or exclude multiple users from the search list, it would make debugging complex interactions simpler, for example only look for actions for 'alice' and 'hive' and 'yarn' The frontend tweaked a bit, so if multiple users are passed to the jquery layer, the user names are always converted as requestUser=aaa=bbb=ccc instead of changing to requestUser[]=aaa[]=bbb[]=ccc, which would be an incompatible change between the server and to any potential client code. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java ce577e0fc73a02962f923b13317021cd1feef38b security-admin/src/main/java/org/apache/ranger/rest/XAuditREST.java fdf5ad86bff91e811c7d56ceb87845f158f8c789 security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java cf2a3b4d8862d738f882b6157cd1e9a317613aa7 security-admin/src/main/webapp/scripts/utils/XAUtils.js b14f4b918a23be4f8ae1ae95308c90ebb3e4bcf6 security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js daf6924ebb8b3f6dc0935736275d3e8d1c1f3f0b security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java ef149d5fa01b72f1ec3631012db4cf3433570547 Diff: https://reviews.apache.org/r/70389/diff/2/ Changes: https://reviews.apache.org/r/70389/diff/1-2/ Testing --- Tested on a live cluster that: * searching for one user * searching for multiple users * excluding one user * excluding multiple users * searching for one user + 'excluding service users' * searching for multiple users + 'excluding service users' * excluding one user + 'excluding service users' * excluding multiple users + 'excluding service users' works as expected. Thanks, Zsombor Gegesy
[jira] [Updated] (RANGER-2394) Filter/exclude multiple users in audit search
[ https://issues.apache.org/jira/browse/RANGER-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2394: --- Attachment: RANGER-2394.patch > Filter/exclude multiple users in audit search > - > > Key: RANGER-2394 > URL: https://issues.apache.org/jira/browse/RANGER-2394 > Project: Ranger > Issue Type: Improvement > Components: admin >Affects Versions: master >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Labels: admin, solr > Fix For: 2.0.0 > > Attachments: RANGER-2394.patch > > > Currently the audit search only allows to: > * filter to one user's activity > * exclude all 'service users' from every user's activity. > If there were way to search for multiple users or exclude multiple users from > the search list, it would make debugging complex interactions simpler, for > example only look for actions for 'alice' and 'hive' and 'yarn' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Review Request 70389: RANGER-2394 - filter multiple users or exclude multiple users in audit search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70389/ --- Review request for ranger. Bugs: RANGER-2394 https://issues.apache.org/jira/browse/RANGER-2394 Repository: ranger Description --- Currently the audit search only allows to: * filter to one user's activity * exclude all 'service users' from every user's activity. If there were way to search for multiple users or exclude multiple users from the search list, it would make debugging complex interactions simpler, for example only look for actions for 'alice' and 'hive' and 'yarn' The frontend tweaked a bit, so if multiple users are passed to the jquery layer, the user names are always converted as requestUser=aaa=bbb=ccc instead of changing to requestUser[]=aaa[]=bbb[]=ccc, which would be an incompatible change between the server and to any potential client code. Diffs - security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java ce577e0fc security-admin/src/main/java/org/apache/ranger/rest/XAuditREST.java fdf5ad86b security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java cf2a3b4d8 security-admin/src/main/webapp/scripts/utils/XAUtils.js b14f4b918 security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js e7b45d992 Diff: https://reviews.apache.org/r/70389/diff/1/ Testing --- Tested on a live cluster that: * searching for one user * searching for multiple users * excluding one user * excluding multiple users * searching for one user + 'excluding service users' * searching for multiple users + 'excluding service users' * excluding one user + 'excluding service users' * excluding multiple users + 'excluding service users' works as expected. Thanks, Zsombor Gegesy
[jira] [Created] (RANGER-2394) Filter/exclude multiple users in audit search
Zsombor Gegesy created RANGER-2394: -- Summary: Filter/exclude multiple users in audit search Key: RANGER-2394 URL: https://issues.apache.org/jira/browse/RANGER-2394 Project: Ranger Issue Type: Improvement Components: admin Affects Versions: master Reporter: Zsombor Gegesy Assignee: Zsombor Gegesy Fix For: 2.0.0 Currently the audit search only allows to: * filter to one user's activity * exclude all 'service users' from every user's activity. If there were way to search for multiple users or exclude multiple users from the search list, it would make debugging complex interactions simpler, for example only look for actions for 'alice' and 'hive' and 'yarn' -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RANGER-2386) Code duplication due to RangerCredentialProvider.getCredentialString returns char[]
[ https://issues.apache.org/jira/browse/RANGER-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2386: --- Attachment: RANGER-2386.patch > Code duplication due to RangerCredentialProvider.getCredentialString returns > char[] > --- > > Key: RANGER-2386 > URL: https://issues.apache.org/jira/browse/RANGER-2386 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: master >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 2.0.0 > > Attachments: RANGER-2386.patch > > > The same code appears in lot's of places, because > RangerCredentialProvider.getCredentialString returns a char array, which > needs to be converted to String. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Review Request 70339: RANGER-2386: Code duplication due to RangerCredentialProvider.getCredentialString returns char[]
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70339/ --- Review request for ranger. Bugs: RANGER-2386 https://issues.apache.org/jira/browse/RANGER-2386 Repository: ranger Description --- The same code appears in lot's of places, because RangerCredentialProvider.getCredentialString returns a char array, which needs to be converted to String - remove this duplication. Diffs - agents-audit/src/main/java/org/apache/ranger/audit/provider/DbAuditProvider.java 1490c2782a9d603b8ad1392a1aafef7655e56562 agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java eff38249e044a910b747aa7e81003028d2b3 agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java 2a4b9c90186effdcc2d9bc5b8726a5351ea668bd agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSslHelper.java 2036661e81201b15fc296d53f2a831a8a1770b36 agents-cred/src/main/java/org/apache/ranger/authorization/hadoop/utils/RangerCredentialProvider.java 44cadfba2dc2f209a94b11f6d3ad7e7582f3d8cb agents-cred/src/test/java/org/apache/ranger/authorization/hadoop/utils/RangerCredentialProviderTest.java 971ee681a33dfcf5993e4d79e25c648c90646df2 Diff: https://reviews.apache.org/r/70339/diff/1/ Testing --- Tested with plugins on a cluster Thanks, Zsombor Gegesy
[jira] [Created] (RANGER-2386) Code duplication due to RangerCredentialProvider.getCredentialString returns char[]
Zsombor Gegesy created RANGER-2386: -- Summary: Code duplication due to RangerCredentialProvider.getCredentialString returns char[] Key: RANGER-2386 URL: https://issues.apache.org/jira/browse/RANGER-2386 Project: Ranger Issue Type: Improvement Components: plugins Affects Versions: master Reporter: Zsombor Gegesy Assignee: Zsombor Gegesy Fix For: 2.0.0 The same code appears in lot's of places, because RangerCredentialProvider.getCredentialString returns a char array, which needs to be converted to String. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70057: RANGER-2324 Bootstrapping Solr in Ranger service start-up
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70057/#review214200 --- Ship it! Ship It! - Zsombor Gegesy On March 28, 2019, 4:10 p.m., bhavik patel wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70057/ > --- > > (Updated March 28, 2019, 4:10 p.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Oliver Szabo, Pradeep Agrawal, Ramesh Mani, > Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2324 > https://issues.apache.org/jira/browse/RANGER-2324 > > > Repository: ranger > > > Description > --- > > We are handling the solr bootstrapping in below mentioned manner for Ranger > 1.) Connection to solr > 2.) Upload Configuration > 3.) Create Collection > 4.) Setting ACL > > > Diffs > - > > > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java > 8d32352 > > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java > PRE-CREATION > security-admin/scripts/install.properties fdcee1b > security-admin/scripts/setup.sh bd4bd4c > security-admin/src/main/resources/conf.dist/ranger-admin-default-site.xml > 686f720 > security-admin/src/main/resources/conf.dist/ranger-admin-site.xml 4d4a1de > src/main/assembly/admin-web.xml 0296652 > > > Diff: https://reviews.apache.org/r/70057/diff/11/ > > > Testing > --- > > Tested Below Scenario on ranger manual start / restart > 1.) Solr configuration were uploaded successfully > 2.) Solr collections were created successfully > 3.) ACL were setup as required. > > > Thanks, > > bhavik patel > >
Re: Review Request 70057: RANGER-2324 Bootstrapping Solr in Ranger service start-up
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70057/#review214169
---
embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 176 (patched)
<https://reviews.apache.org/r/70057/#comment300361>
You can move this repeating pattern to a separate function:
void logAndWait(String msg, Exception e) {
logger.severe(msg + "[retrying after " + time_interval + " ms]", e);
try {
Thread.sleep(time_interval);
} catch (InterruptedException e) {
logger.info("sleep interrupted: " + e.getMessage());
}
}
embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
Lines 237 (patched)
<https://reviews.apache.org/r/70057/#comment300360>
In this case, when config is not specified correctly, I don't think
re-trying the same would make sense
- Zsombor Gegesy
On March 27, 2019, 1:04 p.m., bhavik patel wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70057/
> ---
>
> (Updated March 27, 2019, 1:04 p.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay
> Kulkarni, Madhan Neethiraj, Oliver Szabo, Pradeep Agrawal, Ramesh Mani,
> Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2324
> https://issues.apache.org/jira/browse/RANGER-2324
>
>
> Repository: ranger
>
>
> Description
> ---
>
> We are handling the solr bootstrapping in below mentioned manner for Ranger
> 1.) Connection to solr
> 2.) Upload Configuration
> 3.) Create Collection
> 4.) Setting ACL
>
>
> Diffs
> -
>
>
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java
> 8d32352
>
> embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBoostrapper.java
> PRE-CREATION
> security-admin/scripts/install.properties fdcee1b
> security-admin/scripts/setup.sh bd4bd4c
> security-admin/src/main/resources/conf.dist/ranger-admin-default-site.xml
> 686f720
> security-admin/src/main/resources/conf.dist/ranger-admin-site.xml 4d4a1de
> src/main/assembly/admin-web.xml 0296652
>
>
> Diff: https://reviews.apache.org/r/70057/diff/10/
>
>
> Testing
> ---
>
> Tested Below Scenario on ranger manual start / restart
> 1.) Solr configuration were uploaded successfully
> 2.) Solr collections were created successfully
> 3.) ACL were setup as required.
>
>
> Thanks,
>
> bhavik patel
>
>
[jira] [Created] (RANGER-2378) KeySecure HSM Integration is not compatible with Java9
Zsombor Gegesy created RANGER-2378:
--
Summary: KeySecure HSM Integration is not compatible with Java9
Key: RANGER-2378
URL: https://issues.apache.org/jira/browse/RANGER-2378
Project: Ranger
Issue Type: Bug
Components: kms
Reporter: Zsombor Gegesy
The patch introduced in RANGER-2331 relies on internal
sun.security.pkcs11.SunPKCS11 class, unfortunately this class changed between
Java 8 and 9, so the code no longer compiles on Java9+.
The Java8 way of doing (documented
[here|https://docs.oracle.com/javase/8/docs/technotes/guides/security/p11guide.html]
) is:
{code}
Provider p = new sun.security.pkcs11.SunPKCS11(configName);
Security.addProvider(p);
{code}
However, in Java 9, sun.security.pkcs11.SunPKCS11 doesn't have a constructor
with a String parameter, and the documentation
[suggests|https://docs.oracle.com/javase/9/security/pkcs11-reference-guide1.htm]
suggest to use:
{code}
Provider p = Security.getProvider("SunPKCS11");
p = p.configure(configName);
Security.addProvider(p);
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 70254: RANGER-2373 : User creation POST and PUT response not showing groupIdList and groupNameList with expected data
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70254/#review213910 --- Ship it! Ship It! - Zsombor Gegesy On March 22, 2019, 7:17 a.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70254/ > --- > > (Updated March 22, 2019, 7:17 a.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, > Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-2373 > https://issues.apache.org/jira/browse/RANGER-2373 > > > Repository: ranger > > > Description > --- > > User creation POST and PUT response should show groupIdList and groupNameList > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java fc40917 > security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java > 49c57a6 > > > Diff: https://reviews.apache.org/r/70254/diff/2/ > > > Testing > --- > > validated If user creation and updation response is showing expected list of > group Id / group Name. > > 1.Tested CRUD for users. > 2.Tested CRUD for groups. > 3.Checked adding users to group. > 4.Checked users are getting deleted when groups are assigned to it. > > > File Attachments > > > RANGER-2373-2.patch > > https://reviews.apache.org/media/uploaded/files/2019/03/22/f1f0b2d5-68d1-4413-b5f7-f8836b2dc2a8__RANGER-2373-2.patch > > > Thanks, > > Nikhil P > >
Re: Review Request 70257: RANGER-2375: RangerAuthContext is not correctly initialized
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70257/#review213850 --- Could you add a simple unit test for RangerAuthContext which checks that addOrReplaceRequestContextEnricher and preProcess(RangerAccessRequest) works as expected? - Zsombor Gegesy On March 20, 2019, 7:52 p.m., Abhay Kulkarni wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70257/ > --- > > (Updated March 20, 2019, 7:52 p.m.) > > > Review request for ranger, Madhan Neethiraj and Ramesh Mani. > > > Bugs: RANGER-2375 > https://issues.apache.org/jira/browse/RANGER-2375 > > > Repository: ranger > > > Description > --- > > RangerAuthContext object may be used to provide consistent view of > authorization policies database across multiple authorization API calls. It > maintains a list of context enrichers, if defined, to provide context > enrichment for access requests. When this list is not maintained correctly, > context for authorization will not be initialized correctly, and > authorization calls made with RangerAuthContext will fail. > > This seems to a regression introduced by RANGER-2341. > > The fix consists of creating a RangerAuthContext object before a > RangerPolicyEngine is created from policies. > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java > b2cccef5c > > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java > e52d4de28 > > > Diff: https://reviews.apache.org/r/70257/diff/1/ > > > Testing > --- > > Tested with policies containing policy conditions that depended on context of > access-request to be populated correctly. Verified that context is populated > correctly and policies with conditions work as expected. > > > Thanks, > > Abhay Kulkarni > >
Re: Review Request 70187: RANGER-2353:Upgrade Apache Thrift Java client library to 0.12.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70187/#review213849 --- Ship it! Ship It! - Zsombor Gegesy On March 11, 2019, 7:49 p.m., Ramesh Mani wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70187/ > --- > > (Updated March 11, 2019, 7:49 p.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2353 > https://issues.apache.org/jira/browse/RANGER-2353 > > > Repository: ranger > > > Description > --- > > RANGER-2353:Upgrade Apache Thrift Java client library to 0.12.0 > > > Diffs > - > > pom.xml d5c4e92 > security-admin/pom.xml 2c58760 > src/main/assembly/admin-web.xml 0296652 > > > Diff: https://reviews.apache.org/r/70187/diff/1/ > > > Testing > --- > > Verified in Local VM with test. > > > Thanks, > > Ramesh Mani > >
Re: Review Request 70226: RANGER-2354:Add custom condition at policy level
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70226/#review213848 --- agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyEvaluator.java Lines 1224 (patched) <https://reviews.apache.org/r/70226/#comment299948> perf is always null, it never logs anything. - Zsombor Gegesy On March 20, 2019, 5:03 a.m., Ramesh Mani wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70226/ > --- > > (Updated March 20, 2019, 5:03 a.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Repository: ranger > > > Description > --- > > RANGER-2354:Add custom condition at policy level > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerAbstractConditionEvaluator.java > ddd1a54 > > agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerConditionEvaluator.java > 16f9a3c > > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java > 327d37b > > agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerAbstractPolicyEvaluator.java > bc459e3 > > agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyEvaluator.java > 3e7c34c > > agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerPolicyEvaluator.java > 5400f71 > > agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyEngine.java > 9bd5e24 > > agents-common/src/test/resources/policyengine/test_policyengine_policylevel_conditions.json > PRE-CREATION > > ranger-examples/conditions-enrichers/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerPolicyConditionSampleSimpleMatcher.java > PRE-CREATION > security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java > 25b48bb > > security-admin/src/main/java/org/apache/ranger/service/RangerPolicyService.java > d0f1d93 > > > Diff: https://reviews.apache.org/r/70226/diff/2/ > > > Testing > --- > > - Tested in local VM with existing RangerSampleProjectProvider condition > enricher and RangerSimpleMatch for policyLevel Custom condition evaluation. > > > Thanks, > > Ramesh Mani > >
Re: Review Request 70254: User creation POST and PUT response not showing groupIdList and groupNameList with expected data
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70254/#review213847
---
security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java
Lines 485 (patched)
<https://reviews.apache.org/r/70254/#comment299946>
Modifying a function argument is not the cleanest solution - it's a bit
suprising for the casual reader, that the function is called for side effects.
trxLogList.addAll(createOrDelGrpUserWithUpdatedGrpId(vXUser,
groupIdList, userId, groupUsersToRemove));
private List createOrDelGrpUserWithUpdatedGrpId(VXUser
vXUser, Collection groupIdList,
List trxLogList, Long userId, List
groupUsersToRemove) {
List trxLogList = new ArrayList<>();
...
}
security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java
Line 610 (original), 616 (patched)
<https://reviews.apache.org/r/70254/#comment299947>
Could you add a check that getGroupIdList() is not empty, but contains a
group id?
- Zsombor Gegesy
On March 20, 2019, 1:26 p.m., Nikhil P wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70254/
> ---
>
> (Updated March 20, 2019, 1:26 p.m.)
>
>
> Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj,
> Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and
> Velmurugan Periasamy.
>
>
> Bugs: RANGER-2373
> https://issues.apache.org/jira/browse/RANGER-2373
>
>
> Repository: ranger
>
>
> Description
> ---
>
> User creation POST and PUT response should show groupIdList and groupNameList
>
>
> Diffs
> -
>
> security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java fc40917
> security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java
> 49c57a6
>
>
> Diff: https://reviews.apache.org/r/70254/diff/1/
>
>
> Testing
> ---
>
> validated If user creation and updation response is showing expected list of
> group Id / group Name.
>
> 1.Tested CRUD for users.
> 2.Tested CRUD for groups.
> 3.Checked adding users to group.
> 4.Checked users are getting deleted when groups are assigned to it.
>
>
> Thanks,
>
> Nikhil P
>
>
Re: Review Request 70204: RANGER-2358: Upgrade Jackson Databind to 2.9.8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70204/#review213667 --- Ship it! Ship It! - Zsombor Gegesy On March 13, 2019, 12:41 p.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70204/ > --- > > (Updated March 13, 2019, 12:41 p.m.) > > > Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Gautam > Borad, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, > Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2358 > https://issues.apache.org/jira/browse/RANGER-2358 > > > Repository: ranger > > > Description > --- > > It seems there are different versions of same jar is being pulled by > different parent libraries during the ranger build. This issue shall address > following libraries versions. > 1) jackson-core > 2) jackson-annotations > 3) jackson-databind > 4) commons.codec > 5) commons.io.version > 6) commons.net.version > 7) netty-all.version > 8) zookeeper.version > > > Diffs > - > > kms/pom.xml 59dd2761f > pom.xml d5c4e924a > ranger-examples/src/main/assembly/plugin-sampleapp.xml 42d2e2d62 > security-admin/pom.xml 2c587605b > src/main/assembly/kms.xml 8a7c6a7c4 > src/main/assembly/tagsync.xml dd7580b01 > > > Diff: https://reviews.apache.org/r/70204/diff/1/ > > > Testing > --- > > Tested Ranger installation and user/policy CRUD operations. > > > Thanks, > > Pradeep Agrawal > >
[jira] [Resolved] (RANGER-2317) Enable compilation on JDK11
[ https://issues.apache.org/jira/browse/RANGER-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy resolved RANGER-2317. Resolution: Fixed Merged to [master|https://github.com/apache/ranger/commit/08f32cd35824399eaac573f47338fbe8433ed97e] > Enable compilation on JDK11 > --- > > Key: RANGER-2317 > URL: https://issues.apache.org/jira/browse/RANGER-2317 > Project: Ranger > Issue Type: Improvement > Components: admin, plugins > Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: master > > Attachments: RANGER-2317-2.patch, RANGER-2317-3.patch > > > Currently, Ranger can be compiled only with JDK 8, however JDK 11 is the > current LTS release for Java, it is essential to support it. As a first step, > we need to ensure that Ranger can be compiled on JDK 11. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-2356) External user's email address can be edited
[ https://issues.apache.org/jira/browse/RANGER-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy resolved RANGER-2356. Resolution: Fixed Merged to [master|https://github.com/apache/ranger/commit/3e04f089c9ad5b8e749c3faa08447cbe04be6dba] - Thanks for the fix ! > External user's email address can be edited > --- > > Key: RANGER-2356 > URL: https://issues.apache.org/jira/browse/RANGER-2356 > Project: Ranger > Issue Type: Bug > Components: admin >Affects Versions: master >Reporter: László Dénes Terjéki >Priority: Major > Labels: email > Attachments: > 0001-RANGER-2356-Ranger-UI-disable-email-editing-for-Exte.patch, Screenshot > 2019-03-12 at 13.30.46.png > > > In Settings -> Users/Groups clicking on an external user the email field is > editable while the "User Name", "First Name" and "Last Name" fields are > disabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70194: RANGER-2356: External user's email address can be edited
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70194/#review213624 --- Ship it! Ship It! - Zsombor Gegesy On March 12, 2019, 1:35 p.m., László Terjéki wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70194/ > --- > > (Updated March 12, 2019, 1:35 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2356 > https://issues.apache.org/jira/browse/RANGER-2356 > > > Repository: ranger > > > Description > --- > > The disabled attribute was added to an invalid element not to the input > element. > > > Diffs > - > > security-admin/src/main/webapp/scripts/views/users/UserForm.js b11264f94 > > > Diff: https://reviews.apache.org/r/70194/diff/1/ > > > Testing > --- > > > Thanks, > > László Terjéki > >
Re: Review Request 69655: RANGER-2317 : make Ranger buildable on newer JDKs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69655/ --- (Updated March 11, 2019, 7:10 p.m.) Review request for ranger. Bugs: RANGER-2317 https://issues.apache.org/jira/browse/RANGER-2317 Repository: ranger Description --- Upgrade librarias to become compatible with JDK11, and JAXB and other librarias to the modules, and exclude jdk.tools. (These modules was previously part of the JDK, now they are independent). Add JDK10/11 to the travis build - with excluding the Hive tests, as we can't even start the Hive tests on JDK 9 - because internal Hive code cast the system classloader to URLClassLoader. Removed one empty test class file, which triggered test failures with newer mockito version. Diffs (updated) - .travis.yml b38b892bd embeddedwebserver/scripts/ranger-admin-services.sh c04e1fc72 enunciate.xml f64af1a56 hbase-agent/pom.xml 6b3763f22 kms/pom.xml 59dd2761f knox-agent/pom.xml bd68221cc plugin-solr/pom.xml 619cc9ebd pom.xml d5c4e924a ranger-hbase-plugin-shim/pom.xml 9ed0aebe8 ranger-solr-plugin-shim/pom.xml 46b78218a security-admin/pom.xml 2c587605b security-admin/src/test/java/org/apache/ranger/plugin/store/RangerDBStore.java cd5bb384e ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestUnixUserGroupBuilder.java 2118c8430 Diff: https://reviews.apache.org/r/69655/diff/4/ Changes: https://reviews.apache.org/r/69655/diff/3-4/ Testing --- Tested resulting binaries locally on JDK 8, to stay the same as previously. All the dependency change was either in test code, or in Maven plugin config, or has 'provided' scope. Tested on Travis that build is succesfull in 4 JVMs - oraclejdk8/oraclejdk11/openjdk10/openjdk11: https://travis-ci.org/gzsombor/ranger/builds/474559352 Thanks, Zsombor Gegesy
[jira] [Commented] (RANGER-2336) Ranger HBase plugin should pack guava lib as a dependency.
[
https://issues.apache.org/jira/browse/RANGER-2336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778605#comment-16778605
]
Zsombor Gegesy commented on RANGER-2336:
In [this
commit|https://github.com/apache/ranger/commit/092320830f89bc426e68e4fe8a3e5e3255693571]
(for RANGER-2232), all the MoreObjects usage was removed - so I think, the
original
[fix|https://github.com/apache/ranger/commit/a5d212871588211870b711cd6ec5f3650c14079d]
can be reverted.
> Ranger HBase plugin should pack guava lib as a dependency.
> --
>
> Key: RANGER-2336
> URL: https://issues.apache.org/jira/browse/RANGER-2336
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: master, 2.0.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments:
> 0001-RANGER-2336-Ranger-HBase-plugin-should-pack-guava-li.patch
>
>
> Ranger HBase plugin should pack guava lib as a dependency. This is avoid run
> time exception in debug mode, which crashes HBase
>
> {code:java}
> ABORTING region server
> ctr-e139-1542663976389-66118-01-09.hwx.site,16020,1550008218797: The
> coprocessor
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor threw
> java.lang.NoClassDefFoundError: com/google/common/base/MoreObjects *
> Cause:
> java.lang.NoClassDefFoundError: com/google/common/base/MoreObjects
> at
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor$ColumnFamilyAccessResult.toString(RangerAuthorizationCoprocessor.java:288)
> at
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.evaluateAccess(RangerAuthorizationCoprocessor.java:330)
> at
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.authorizeAccess(RangerAuthorizationCoprocessor.java:531)
> at
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.preGetOp(RangerAuthorizationCoprocessor.java:1130)
> at
> org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor.preGetOp(RangerAuthorizationCoprocessor.java:927)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:831)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$19.call(RegionCoprocessorHost.java:828)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:540)
> at
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:614)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preGet(RegionCoprocessorHost.java:828)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2530)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2470)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:41998)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:413)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:131)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)
> Caused by: java.lang.ClassNotFoundException:
> com.google.common.base.MoreObjects
> at java.lang.ClassLoader.findClass(ClassLoader.java:530)
> at
> org.apache.ranger.plugin.classloader.RangerPluginClassLoader$MyClassLoader.findClass(RangerPluginClassLoader.java:272)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.ranger.plugin.classloader.RangerPluginClassLoader.loadClass(RangerPluginClassLoader.java:125)
> .{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 70057: RANGER-2324 Bootstrapping Solr in Ranger service start-up
057/#comment299114> You can use Path/FileSystem directly as in previous locations embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrSetupUtil.java Lines 374 (patched) <https://reviews.apache.org/r/70057/#comment299115> You can use Path/FileSystem directly as in previous locations embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrSetupUtil.java Lines 415 (patched) <https://reviews.apache.org/r/70057/#comment299119> Catching Throwable is not too good solution - if you really want to log problems here, please pick a more specific Exception, and rethrow it after the log. Or just remove this catch, and throw the exception to the caller as is - I mean, the original, specific exception embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrSetupUtil.java Lines 510 (patched) <https://reviews.apache.org/r/70057/#comment299116> Catching Exception is not too good solution - if you really want to log problems here, please pick a more specific exception, and rethrow it after the log. Or just remove this catch, and throw the exception to the caller as is - I mean, the original, specific exception embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrSetupUtil.java Lines 531 (patched) <https://reviews.apache.org/r/70057/#comment299120> I don't think, that you can do anything, if you can't talk to Solr, and you don't get the collections, I would rather have the exception propagated to the top. - Zsombor Gegesy On Feb. 26, 2019, 10:17 a.m., bhavik patel wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70057/ > --- > > (Updated Feb. 26, 2019, 10:17 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Oliver Szabo, Pradeep Agrawal, Ramesh Mani, > Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2324 > https://issues.apache.org/jira/browse/RANGER-2324 > > > Repository: ranger > > > Description > --- > > We are handling the solr bootstrapping in below mentioned manner for Ranger > 1.) Connection to solr > 2.) Upload Configuration > 3.) Create Collection > 4.) Setting ACL > > > Diffs > - > > > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java > 8d32352 > > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrSetupUtil.java > PRE-CREATION > security-admin/scripts/install.properties fdcee1b > security-admin/scripts/setup.sh bd4bd4c > security-admin/src/main/resources/conf.dist/ranger-admin-site.xml 4d4a1de > src/main/assembly/admin-web.xml 8ea728b > > > Diff: https://reviews.apache.org/r/70057/diff/2/ > > > Testing > --- > > Tested Below Scenario on ranger manual start / restart > 1.) Solr configuration were uploaded successfully > 2.) Solr collections were created successfully > 3.) ACL were setup as required. > > > Thanks, > > bhavik patel > >
Re: Review Request 69984: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69984/#review213161 --- Ship it! Ship It! - Zsombor Gegesy On Feb. 25, 2019, 12:59 p.m., Gautam Borad wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69984/ > --- > > (Updated Feb. 25, 2019, 12:59 p.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2331 > https://issues.apache.org/jira/browse/RANGER-2331 > > > Repository: ranger > > > Description > --- > > User story: As a security admin, I want to manage encryption keys for > securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure > crypto platform. > > For Safenet KeySecure overview refer to: > https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/ > > Acceptance Criteria: > > 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used > for key offload > > 2) Ranger KMS provides ability to provide key management functions (create > keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform > > 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure > platform > > > Diffs > - > > kms/config/kms-webapp/dbks-site.xml ec649a7 > kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION > kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION > kms/scripts/install.properties aea0bb8 > kms/scripts/setup.sh c695e74 > kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java dd4408f > > kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 4f337bb > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > 88a545e > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java eb4f75a > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > PRE-CREATION > src/main/assembly/kms.xml 3adc55c > > > Diff: https://reviews.apache.org/r/69984/diff/2/ > > > Testing > --- > > Verified below scenario: > > 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol) > 2) DB to Key Secure (NAE-XML) master key Migration utility > 3) Key Secure (NAE-XML) to DB master key Migration utility > > > Thanks, > > Gautam Borad > >
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
> On Feb. 20, 2019, 12:08 p.m., Zsombor Gegesy wrote: > > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > > Lines 72 (patched) > > <https://reviews.apache.org/r/69985/diff/2/?file=2125810#file2125810line72> > > > > myStore is never null here - even if KeyStore.getInstance would return > > a null (but it wont do), the myStore.load would trigger an NPE before. > > Gautam Borad wrote: > Added null before myStore.load > > Zsombor Gegesy wrote: > This check is needs to happen earlier, otherwise it's dead code, as if > 'myStore' is null, then a NPE is raised earlier. Sorry, my bad, I haven't noticed the new changeset. - Zsombor --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69985/#review212953 --- On Feb. 21, 2019, 6:30 a.m., Gautam Borad wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69985/ > --- > > (Updated Feb. 21, 2019, 6:30 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2331 > https://issues.apache.org/jira/browse/RANGER-2331 > > > Repository: ranger > > > Description > --- > > User story: As a security admin, I want to manage encryption keys for > securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure > crypto platform. > > > For Safenet KeySecure overview refer to: > https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/ > > > Acceptance Criteria: > > > 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used > for key offload > > > 2) Ranger KMS provides ability to provide key management functions (create > keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform > > > 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure > platform > > > Diffs > - > > kms/config/kms-webapp/dbks-site.xml 0e0f2ec > kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION > kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION > kms/scripts/install.properties ddc779d > kms/scripts/setup.sh 2db05b8 > kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f > > kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > 267fcf0 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > PRE-CREATION > src/main/assembly/kms.xml fca6a32 > > > Diff: https://reviews.apache.org/r/69985/diff/3/ > > > Testing > --- > > Verified below scenario: > > > 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol) > 2) DB to Key Secure (NAE-XML) master key Migration utility > 3) Key Secure (NAE-XML) to DB master key Migration utility > > > Thanks, > > Gautam Borad > >
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69985/#review213155 --- Ship it! Ship It! - Zsombor Gegesy On Feb. 21, 2019, 6:30 a.m., Gautam Borad wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69985/ > --- > > (Updated Feb. 21, 2019, 6:30 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2331 > https://issues.apache.org/jira/browse/RANGER-2331 > > > Repository: ranger > > > Description > --- > > User story: As a security admin, I want to manage encryption keys for > securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure > crypto platform. > > > For Safenet KeySecure overview refer to: > https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/ > > > Acceptance Criteria: > > > 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used > for key offload > > > 2) Ranger KMS provides ability to provide key management functions (create > keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform > > > 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure > platform > > > Diffs > - > > kms/config/kms-webapp/dbks-site.xml 0e0f2ec > kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION > kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION > kms/scripts/install.properties ddc779d > kms/scripts/setup.sh 2db05b8 > kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f > > kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > 267fcf0 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > PRE-CREATION > src/main/assembly/kms.xml fca6a32 > > > Diff: https://reviews.apache.org/r/69985/diff/3/ > > > Testing > --- > > Verified below scenario: > > > 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol) > 2) DB to Key Secure (NAE-XML) master key Migration utility > 3) Key Secure (NAE-XML) to DB master key Migration utility > > > Thanks, > > Gautam Borad > >
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
> On Feb. 20, 2019, 12:08 p.m., Zsombor Gegesy wrote: > > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > > Lines 130 (patched) > > <https://reviews.apache.org/r/69985/diff/2/?file=2125808#file2125808line130> > > > > masterKey is never null, because if rangerMasterKey.getMasterKey(...) > > would return null, the toCharArray call would cause an NPE > > Gautam Borad wrote: > Surrounded it with Try ... Catch This check is needs to happen earlier, otherwise it's dead code, as if 'masterKey' is null, then a NPE is raised earlier. > On Feb. 20, 2019, 12:08 p.m., Zsombor Gegesy wrote: > > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > > Lines 140 (patched) > > <https://reviews.apache.org/r/69985/diff/2/?file=2125808#file2125808line140> > > > > masterKey is never null, because if rangerMasterKey.getMasterKey(...) > > would return null, the toCharArray call would cause an NPE > > Gautam Borad wrote: > Surrounded it with Try ... Catch It's dead code, no need for this check - or this check needs to happen earlier, so the NPE is not raised. > On Feb. 20, 2019, 12:08 p.m., Zsombor Gegesy wrote: > > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > > Lines 72 (patched) > > <https://reviews.apache.org/r/69985/diff/2/?file=2125810#file2125810line72> > > > > myStore is never null here - even if KeyStore.getInstance would return > > a null (but it wont do), the myStore.load would trigger an NPE before. > > Gautam Borad wrote: > Added null before myStore.load This check is needs to happen earlier, otherwise it's dead code, as if 'myStore' is null, then a NPE is raised earlier. - Zsombor --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69985/#review212953 --- On Feb. 21, 2019, 6:30 a.m., Gautam Borad wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69985/ > --- > > (Updated Feb. 21, 2019, 6:30 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2331 > https://issues.apache.org/jira/browse/RANGER-2331 > > > Repository: ranger > > > Description > --- > > User story: As a security admin, I want to manage encryption keys for > securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure > crypto platform. > > > For Safenet KeySecure overview refer to: > https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/ > > > Acceptance Criteria: > > > 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used > for key offload > > > 2) Ranger KMS provides ability to provide key management functions (create > keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform > > > 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure > platform > > > Diffs > - > > kms/config/kms-webapp/dbks-site.xml 0e0f2ec > kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION > kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION > kms/scripts/install.properties ddc779d > kms/scripts/setup.sh 2db05b8 > kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f > > kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > 267fcf0 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > PRE-CREATION > src/main/assembly/kms.xml fca6a32 > > > Diff: https://reviews.apache.org/r/69985/diff/3/ > > > Testing > --- > > Verified below scenario: > > > 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol) > 2) DB to Key Secure (NAE-XML) master key Migration utility > 3) Key Secure (NAE-XML) to DB master key Migration utility > > > Thanks, > > Gautam Borad > >
[jira] [Updated] (RANGER-2317) Enable compilation on JDK11
[ https://issues.apache.org/jira/browse/RANGER-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2317: --- Attachment: RANGER-2317-3.patch > Enable compilation on JDK11 > --- > > Key: RANGER-2317 > URL: https://issues.apache.org/jira/browse/RANGER-2317 > Project: Ranger > Issue Type: Improvement > Components: admin, plugins > Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: master > > Attachments: RANGER-2317-2.patch, RANGER-2317-3.patch > > > Currently, Ranger can be compiled only with JDK 8, however JDK 11 is the > current LTS release for Java, it is essential to support it. As a first step, > we need to ensure that Ranger can be compiled on JDK 11. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 69655: RANGER-2317 : make Ranger buildable on newer JDKs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69655/ --- (Updated Feb. 22, 2019, 8:51 a.m.) Review request for ranger. Changes --- Incorporate review feedback Bugs: RANGER-2317 https://issues.apache.org/jira/browse/RANGER-2317 Repository: ranger Description --- Upgrade librarias to become compatible with JDK11, and JAXB and other librarias to the modules, and exclude jdk.tools. (These modules was previously part of the JDK, now they are independent). Add JDK10/11 to the travis build - with excluding the Hive tests, as we can't even start the Hive tests on JDK 9 - because internal Hive code cast the system classloader to URLClassLoader. Removed one empty test class file, which triggered test failures with newer mockito version. Diffs (updated) - .travis.yml b38b892bd embeddedwebserver/scripts/ranger-admin-services.sh c04e1fc72 enunciate.xml f64af1a56 hbase-agent/pom.xml c044ef0c3 kms/pom.xml 043c67fde knox-agent/pom.xml 0c45c84ba plugin-solr/pom.xml 619cc9ebd pom.xml 2a2540a0d ranger-hbase-plugin-shim/pom.xml 9ed0aebe8 ranger-solr-plugin-shim/pom.xml 46b78218a security-admin/pom.xml 479f46d05 security-admin/src/test/java/org/apache/ranger/plugin/store/RangerDBStore.java cd5bb384e ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestUnixUserGroupBuilder.java 2118c8430 Diff: https://reviews.apache.org/r/69655/diff/3/ Changes: https://reviews.apache.org/r/69655/diff/2-3/ Testing --- Tested resulting binaries locally on JDK 8, to stay the same as previously. All the dependency change was either in test code, or in Maven plugin config, or has 'provided' scope. Tested on Travis that build is succesfull in 4 JVMs - oraclejdk8/oraclejdk11/openjdk10/openjdk11: https://travis-ci.org/gzsombor/ranger/builds/474559352 Thanks, Zsombor Gegesy
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
> On Feb. 14, 2019, 12:02 p.m., Zsombor Gegesy wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
> > Lines 50 (patched)
> > <https://reviews.apache.org/r/69985/diff/1/?file=2125358#file2125358line50>
> >
> > You can mark all variable as final
>
> Gautam Borad wrote:
> I am initializing non final variables in constructor.
>
> Zsombor Gegesy wrote:
> Yes, that's the way to use final variables: you need to initialize them
> in the constructor.
>
> Pradeep Agrawal wrote:
> @Zsombor Gegesy : To me its seems okay as he is reinitializing the mkSize
> variable at line 60. I don't think it need to be final, however we can make
> it static.
>
> Zsombor Gegesy wrote:
> Initializing these variables with their default values just adds noise to
> the code, the code behaves the same:
>
> class X {
> int x;
>
> void checkX() {
> if (x==0) {
> System.out.println("x is 0!");
> }
> }
> }
>
> new X().checkX() // this will print 'x is 0!'
>
> I dont think making an instance variable 'static' would be a good idea.
> In my opinion, you can safely remove the ' = null' and '= 0'
> initializations, and mark everything final, to make it clear, that these are
> constant values through the lifetime of this class.
>
> Pradeep Agrawal wrote:
> I agree that it don't need to be static and it don't need to be
> initialized with 0 at line 50 but it can't be final as per the code of line
> 60.
mkSize is not written anywhere else, only be set on line 60, inside the
constructor, so it can be set to final.
- Zsombor
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69985/#review212827
---
On Feb. 19, 2019, 1:58 p.m., Gautam Borad wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69985/
> ---
>
> (Updated Feb. 19, 2019, 1:58 p.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni,
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja
> Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2331
> https://issues.apache.org/jira/browse/RANGER-2331
>
>
> Repository: ranger
>
>
> Description
> ---
>
> User story: As a security admin, I want to manage encryption keys for
> securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure
> crypto platform.
>
>
> For Safenet KeySecure overview refer to:
> https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/
>
>
> Acceptance Criteria:
>
>
> 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used
> for key offload
>
>
> 2) Ranger KMS provides ability to provide key management functions (create
> keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform
>
>
> 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure
> platform
>
>
> Diffs
> -
>
> kms/config/kms-webapp/dbks-site.xml 0e0f2ec
> kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION
> kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION
> kms/scripts/install.properties ddc779d
> kms/scripts/setup.sh 2db05b8
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f
>
> kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java
> 267fcf0
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
> PRE-CREATION
> src/main/assembly/kms.xml fca6a32
>
>
> Diff: https://reviews.apache.org/r/69985/diff/2/
>
>
> Testing
> ---
>
> Verified below scenario:
>
>
> 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol)
> 2) DB to Key Secure (NAE-XML) master key Migration utility
> 3) Key Secure (NAE-XML) to DB master key Migration utility
>
>
> Thanks,
>
> Gautam Borad
>
>
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
> On Feb. 14, 2019, 12:02 p.m., Zsombor Gegesy wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
> > Lines 50 (patched)
> > <https://reviews.apache.org/r/69985/diff/1/?file=2125358#file2125358line50>
> >
> > You can mark all variable as final
>
> Gautam Borad wrote:
> I am initializing non final variables in constructor.
>
> Zsombor Gegesy wrote:
> Yes, that's the way to use final variables: you need to initialize them
> in the constructor.
>
> Pradeep Agrawal wrote:
> @Zsombor Gegesy : To me its seems okay as he is reinitializing the mkSize
> variable at line 60. I don't think it need to be final, however we can make
> it static.
Initializing these variables with their default values just adds noise to the
code, the code behaves the same:
class X {
int x;
void checkX() {
if (x==0) {
System.out.println("x is 0!");
}
}
}
new X().checkX() // this will print 'x is 0!'
I dont think making an instance variable 'static' would be a good idea.
In my opinion, you can safely remove the ' = null' and '= 0' initializations,
and mark everything final, to make it clear, that these are constant values
through the lifetime of this class.
- Zsombor
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69985/#review212827
---
On Feb. 19, 2019, 1:58 p.m., Gautam Borad wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69985/
> ---
>
> (Updated Feb. 19, 2019, 1:58 p.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni,
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja
> Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2331
> https://issues.apache.org/jira/browse/RANGER-2331
>
>
> Repository: ranger
>
>
> Description
> ---
>
> User story: As a security admin, I want to manage encryption keys for
> securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure
> crypto platform.
>
>
> For Safenet KeySecure overview refer to:
> https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/
>
>
> Acceptance Criteria:
>
>
> 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used
> for key offload
>
>
> 2) Ranger KMS provides ability to provide key management functions (create
> keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform
>
>
> 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure
> platform
>
>
> Diffs
> -
>
> kms/config/kms-webapp/dbks-site.xml 0e0f2ec
> kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION
> kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION
> kms/scripts/install.properties ddc779d
> kms/scripts/setup.sh 2db05b8
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f
>
> kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java
> 267fcf0
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
> PRE-CREATION
> src/main/assembly/kms.xml fca6a32
>
>
> Diff: https://reviews.apache.org/r/69985/diff/2/
>
>
> Testing
> ---
>
> Verified below scenario:
>
>
> 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol)
> 2) DB to Key Secure (NAE-XML) master key Migration utility
> 3) Key Secure (NAE-XML) to DB master key Migration utility
>
>
> Thanks,
>
> Gautam Borad
>
>
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69985/#review212953 --- kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java Lines 130 (patched) <https://reviews.apache.org/r/69985/#comment298827> masterKey is never null, because if rangerMasterKey.getMasterKey(...) would return null, the toCharArray call would cause an NPE kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java Lines 140 (patched) <https://reviews.apache.org/r/69985/#comment298826> masterKey is never null, because if rangerMasterKey.getMasterKey(...) would return null, the toCharArray call would cause an NPE kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java Lines 72 (patched) <https://reviews.apache.org/r/69985/#comment298828> myStore is never null here - even if KeyStore.getInstance would return a null (but it wont do), the myStore.load would trigger an NPE before. - Zsombor Gegesy On Feb. 19, 2019, 1:58 p.m., Gautam Borad wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69985/ > --- > > (Updated Feb. 19, 2019, 1:58 p.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2331 > https://issues.apache.org/jira/browse/RANGER-2331 > > > Repository: ranger > > > Description > --- > > User story: As a security admin, I want to manage encryption keys for > securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure > crypto platform. > > > For Safenet KeySecure overview refer to: > https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/ > > > Acceptance Criteria: > > > 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used > for key offload > > > 2) Ranger KMS provides ability to provide key management functions (create > keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform > > > 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure > platform > > > Diffs > - > > kms/config/kms-webapp/dbks-site.xml 0e0f2ec > kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION > kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION > kms/scripts/install.properties ddc779d > kms/scripts/setup.sh 2db05b8 > kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f > > kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > 267fcf0 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > PRE-CREATION > src/main/assembly/kms.xml fca6a32 > > > Diff: https://reviews.apache.org/r/69985/diff/2/ > > > Testing > --- > > Verified below scenario: > > > 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol) > 2) DB to Key Secure (NAE-XML) master key Migration utility > 3) Key Secure (NAE-XML) to DB master key Migration utility > > > Thanks, > > Gautam Borad > >
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
> On Feb. 14, 2019, 12:02 p.m., Zsombor Gegesy wrote: > > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > > Lines 50 (patched) > > <https://reviews.apache.org/r/69985/diff/1/?file=2125358#file2125358line50> > > > > You can mark all variable as final > > Gautam Borad wrote: > I am initializing non final variables in constructor. Yes, that's the way to use final variables: you need to initialize them in the constructor. - Zsombor --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69985/#review212827 --- On Feb. 19, 2019, 1:58 p.m., Gautam Borad wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69985/ > --- > > (Updated Feb. 19, 2019, 1:58 p.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2331 > https://issues.apache.org/jira/browse/RANGER-2331 > > > Repository: ranger > > > Description > --- > > User story: As a security admin, I want to manage encryption keys for > securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure > crypto platform. > > > For Safenet KeySecure overview refer to: > https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/ > > > Acceptance Criteria: > > > 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used > for key offload > > > 2) Ranger KMS provides ability to provide key management functions (create > keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform > > > 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure > platform > > > Diffs > - > > kms/config/kms-webapp/dbks-site.xml 0e0f2ec > kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION > kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION > kms/scripts/install.properties ddc779d > kms/scripts/setup.sh 2db05b8 > kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f > > kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > 267fcf0 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > PRE-CREATION > src/main/assembly/kms.xml fca6a32 > > > Diff: https://reviews.apache.org/r/69985/diff/2/ > > > Testing > --- > > Verified below scenario: > > > 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol) > 2) DB to Key Secure (NAE-XML) master key Migration utility > 3) Key Secure (NAE-XML) to DB master key Migration utility > > > Thanks, > > Gautam Borad > >
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69985/#review212827
---
kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java
Lines 82 (patched)
<https://reviews.apache.org/r/69985/#comment298708>
I would expect that if the import was failed, the process ends with a
non-zero exit code.
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 50 (patched)
<https://reviews.apache.org/r/69985/#comment298711>
You can mark all variable as final
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 52 (patched)
<https://reviews.apache.org/r/69985/#comment298712>
This variable shouldn't be static.
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 59 (patched)
<https://reviews.apache.org/r/69985/#comment298710>
Unnecessary constructor
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 87 (patched)
<https://reviews.apache.org/r/69985/#comment298709>
Why don't you simply re-throw the exception(s)?
Having a non-usable RangerSafenetKeySecure object for the caller doesn't
make too much sense.
So later, you don't need to check that myStore is not null
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 115 (patched)
<https://reviews.apache.org/r/69985/#comment298713>
Why the e.printStackTrace(), could you just add that 'e' to the
logger.error call?
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 117 (patched)
<https://reviews.apache.org/r/69985/#comment298714>
It's not an issue with your code, but I think RangerKMSKI is a bit
confusing, what's the reason for having a 'Throwable' in the method
declaration, and returning a boolean=false. One of them is unnecessary.
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 135 (patched)
<https://reviews.apache.org/r/69985/#comment298715>
If 'key' is null, then it will throw an NPE from here, get catched in the
'catch (Exception e)' and returned null later. Maybe it's simpler to return
null in the if:
if (key == null) {
logger.warn('getMasterKey(pw) returned null!');
return null;
}
kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 154 (patched)
<https://reviews.apache.org/r/69985/#comment298716>
I don't get, why it throws NoSuchAlgorithmException, CertificateException,
and IOException, but catch KeyStoreException ?
- Zsombor Gegesy
On Feb. 14, 2019, 9:59 a.m., Gautam Borad wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69985/
> ---
>
> (Updated Feb. 14, 2019, 9:59 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni,
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja
> Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2331
> https://issues.apache.org/jira/browse/RANGER-2331
>
>
> Repository: ranger
>
>
> Description
> ---
>
> User story: As a security admin, I want to manage encryption keys for
> securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure
> crypto platform.
>
>
> For Safenet KeySecure overview refer to:
> https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/
>
>
> Acceptance Criteria:
>
>
> 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used
> for key offload
>
>
> 2) Ranger KMS provides ability to provide key management functions (create
> keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform
>
>
> 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure
> platform
>
>
> Diffs
> -
>
> kms/config/kms-webapp/dbks-site.xml 0e0f2ec
> kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION
> kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION
> kms/scripts/install.properties ddc779d
> kms/scripts/setup.sh 2db05b8
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f
>
> kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e
> kms/src/main/java/org/apache/hadoop/crypto/key/
Re: Review Request 69984: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69984/#review212825 --- I see two review request, I guess, one of it is duplicate, could you please close one of it? - Zsombor Gegesy On Feb. 14, 2019, 9:55 a.m., Gautam Borad wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69984/ > --- > > (Updated Feb. 14, 2019, 9:55 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2331 > https://issues.apache.org/jira/browse/RANGER-2331 > > > Repository: ranger > > > Description > --- > > User story: As a security admin, I want to manage encryption keys for > securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure > crypto platform. > > For Safenet KeySecure overview refer to: > https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/ > > Acceptance Criteria: > > 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used > for key offload > > 2) Ranger KMS provides ability to provide key management functions (create > keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform > > 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure > platform > > > Diffs > - > > kms/config/kms-webapp/dbks-site.xml ec649a7 > kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION > kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION > kms/scripts/install.properties aea0bb8 > kms/scripts/setup.sh c695e74 > kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java dd4408f > > kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 4f337bb > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > 88a545e > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java eb4f75a > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > PRE-CREATION > src/main/assembly/kms.xml 3adc55c > > > Diff: https://reviews.apache.org/r/69984/diff/1/ > > > Testing > --- > > Verified below scenario: > > 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol) > 2) DB to Key Secure (NAE-XML) master key Migration utility > 3) Key Secure (NAE-XML) to DB master key Migration utility > > > Thanks, > > Gautam Borad > >
Re: Review Request 69740: RANGER-2322 - Use "TLS" in SSLContext.getInstance
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69740/#review212294
---
Ship it!
Ship It!
- Zsombor Gegesy
On Jan. 14, 2019, 11:20 a.m., Colm O hEigeartaigh wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69740/
> ---
>
> (Updated Jan. 14, 2019, 11:20 a.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2322
> https://issues.apache.org/jira/browse/RANGER-2322
>
>
> Repository: ranger
>
>
> Description
> ---
>
> There are a few instances in the code of SSLContext.getInstance("SSL").
> Instead we should use "TLS".
>
>
> Diffs
> -
>
>
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
> f592ed4e7
>
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSslHelper.java
> 2036661e8
>
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapPolicyMgrUserGroupBuilder.java
> f653b81f3
>
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
> cb9b51c95
>
>
> Diff: https://reviews.apache.org/r/69740/diff/1/
>
>
> Testing
> ---
>
>
> Thanks,
>
> Colm O hEigeartaigh
>
>
[jira] [Commented] (RANGER-2287) Improve and optimize db_setup.py file code
[ https://issues.apache.org/jira/browse/RANGER-2287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749768#comment-16749768 ] Zsombor Gegesy commented on RANGER-2287: Awesome, you removed nearly 4000 lines of code, I'm impressed ! > Improve and optimize db_setup.py file code > -- > > Key: RANGER-2287 > URL: https://issues.apache.org/jira/browse/RANGER-2287 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 2.0.0 > > Attachments: > 0001-RANGER-2287-Improve-and-optimize-db_setup.py-file-co.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69677/#review212231 --- Ship it! Ship It! - Zsombor Gegesy On Jan. 22, 2019, 1:47 p.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69677/ > --- > > (Updated Jan. 22, 2019, 1:47 p.m.) > > > Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, > Nikhil P, Ramesh Mani, and Velmurugan Periasamy. > > > Bugs: RANGER-2287 > https://issues.apache.org/jira/browse/RANGER-2287 > > > Repository: ranger > > > Description > --- > > **Problem Statement:** There are lot of repeated code in db_setup.py which > can be removed which shall help developers to make any changes in db_setup.py > in future. > > **Proposed Solution:** Proposed patch shall remove the db setup methods of > each db flavor and shall use a single method for a specific work for each db > flavor. Based on the db flavor, config values shall be populated and handled > in the code after this patch. > > > Diffs > - > > security-admin/scripts/db_setup.py f1223b38c > > > Diff: https://reviews.apache.org/r/69677/diff/3/ > > > Testing > --- > > **Use Cases covered for all the db flavors:** > *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of > ranger admin. > *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same > db config on Ranger 2.0 installation config and run the setup.sh. Ranger was > upgraded successfully. > > > Thanks, > > Pradeep Agrawal > >
Re: Review Request 69740: RANGER-2322 - Use "TLS" in SSLContext.getInstance
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69740/#review212075
---
What about changing to "TLSv1.2"?
- Zsombor Gegesy
On Jan. 14, 2019, 11:20 a.m., Colm O hEigeartaigh wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69740/
> ---
>
> (Updated Jan. 14, 2019, 11:20 a.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2322
> https://issues.apache.org/jira/browse/RANGER-2322
>
>
> Repository: ranger
>
>
> Description
> ---
>
> There are a few instances in the code of SSLContext.getInstance("SSL").
> Instead we should use "TLS".
>
>
> Diffs
> -
>
>
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
> f592ed4e7
>
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSslHelper.java
> 2036661e8
>
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapPolicyMgrUserGroupBuilder.java
> f653b81f3
>
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
> cb9b51c95
>
>
> Diff: https://reviews.apache.org/r/69740/diff/1/
>
>
> Testing
> ---
>
>
> Thanks,
>
> Colm O hEigeartaigh
>
>
[jira] [Updated] (RANGER-2321) Docker build fails due to PhantomJS dependency
[
https://issues.apache.org/jira/browse/RANGER-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsombor Gegesy updated RANGER-2321:
---
Fix Version/s: 2.0.0
> Docker build fails due to PhantomJS dependency
> --
>
> Key: RANGER-2321
> URL: https://issues.apache.org/jira/browse/RANGER-2321
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 2.0.0
>Reporter: Csaba Koncz
>Assignee: Csaba Koncz
>Priority: Major
> Fix For: 2.0.0
>
> Attachments:
> 0001-RANGER-2321-Docker-build-fails-due-to-PhantomJS-depe.patch
>
>
> Docker build fails in an early phase do to the PhantomJS dependency
> introduced in RANGER-2255. E.g. running
> {code:java}
> ./build_ranger_using_docker.sh mvn clean verify -am -pl security-admin{code}
> results in
> {code:java}
> [INFO] < org.apache.ranger:security-admin-web
> >
> [INFO] Building Security Admin Web Application 2.0.0-SNAPSHOT [7/7]
> [INFO] [ war
> ]-
> [INFO]
> [INFO] --- maven-clean-plugin:2.6.1:clean (default-clean) @
> security-admin-web ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-maven-version) @
> security-admin-web ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @
> security-admin-web ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @
> security-admin-web ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 2 resources
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.5:process
> (process-resource-bundles) @ security-admin-web ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.7:resources (default-resources) @
> security-admin-web ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 22 resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-antrun-plugin:1.7:run (default) @ security-admin-web ---
> [INFO] Executing tasks
> main:
> [INFO] Executed tasks
> [INFO]
> [INFO] --- frontend-maven-plugin:1.6:install-node-and-npm (install node and
> npm) @ security-admin-web ---
> [INFO] Installing node version v8.12.0
> [INFO] Unpacking
> /home/builder/.m2/repository/com/github/eirslett/node/8.12.0/node-8.12.0-linux-x64.tar.gz
> into /ranger/security-admin/target/node/tmp
> [INFO] Copying node binary from
> /ranger/security-admin/target/node/tmp/node-v8.12.0-linux-x64/bin/node to
> /ranger/security-admin/target/node/node
> [INFO] Installed node locally.
> [INFO] Installing npm version 6.4.1
> [INFO] Unpacking
> /home/builder/.m2/repository/com/github/eirslett/npm/6.4.1/npm-6.4.1.tar.gz
> into /ranger/security-admin/target/node/node_modules
> [INFO] Installed npm locally.
> [INFO]
> [INFO] --- frontend-maven-plugin:1.6:npm (npm install) @ security-admin-web
> ---
> [INFO] Running 'npm install' in /ranger/security-admin/target
> [INFO]
> [INFO] > phantomjs-prebuilt@2.1.16 install
> /ranger/security-admin/target/node_modules/karma-phantomjs-launcher/node_modules/phantomjs-prebuilt
> [INFO] > node install.js
> [INFO]
> [INFO] PhantomJS not found on PATH
> [INFO] Downloading
> https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
> [INFO] Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
> [INFO] Receiving...
> [INFO]
> [INFO] Received 22866K total.
> [INFO] Extracting tar contents (via spawned process)
> [ERROR] Error extracting archive
> [ERROR] Phantom installation failed { Error: Command failed: tar jxf
> /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
> [ERROR] tar (child): bzip2: Cannot exec: No such file or directory
> [ERROR] tar (child): Error is not recoverable: exiting now
> [ERROR] tar: Child returned status 2
> [ERROR] tar: Error is not recoverable: exiting now
> [ERROR]
> [ERROR] at ChildProcess.exithandler (child_process.js:276:12)
> [ERROR] at emitTwo (events.js:126:13)
> [ERROR] at ChildProcess.emit (events.js:214:7)
> [ERROR] at maybeClose (internal/child_process.js:915:16)
> [ERROR] at Socket.stream.socket.on (internal/child_process.js:336:11)
> [ERROR] at emitOne (events.js:116:13)
> [ERROR] at Socket.emit (events.js:211:7)
> [ERROR] at Pipe._handle.close [as _onclose] (net.js:561:12)
> [ERROR] killed: false,
> [ERROR] code: 2,
> [ERROR] signal: null,
> [ERROR] cmd: 'tar jxf /tmp/phantomjs/phantomjs-2.1.1-linux-x86_6
[jira] [Resolved] (RANGER-2321) Docker build fails due to PhantomJS dependency
[
https://issues.apache.org/jira/browse/RANGER-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsombor Gegesy resolved RANGER-2321.
Resolution: Fixed
Thanks for the patch, it has been merged to
[master|https://github.com/apache/ranger/commit/2a7782db30f983ddacf9ea3fa53302a37a8b965c]
> Docker build fails due to PhantomJS dependency
> --
>
> Key: RANGER-2321
> URL: https://issues.apache.org/jira/browse/RANGER-2321
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 2.0.0
>Reporter: Csaba Koncz
>Assignee: Csaba Koncz
>Priority: Major
> Attachments:
> 0001-RANGER-2321-Docker-build-fails-due-to-PhantomJS-depe.patch
>
>
> Docker build fails in an early phase do to the PhantomJS dependency
> introduced in RANGER-2255. E.g. running
> {code:java}
> ./build_ranger_using_docker.sh mvn clean verify -am -pl security-admin{code}
> results in
> {code:java}
> [INFO] < org.apache.ranger:security-admin-web
> >
> [INFO] Building Security Admin Web Application 2.0.0-SNAPSHOT [7/7]
> [INFO] [ war
> ]-
> [INFO]
> [INFO] --- maven-clean-plugin:2.6.1:clean (default-clean) @
> security-admin-web ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-maven-version) @
> security-admin-web ---
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-versions) @
> security-admin-web ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @
> security-admin-web ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 2 resources
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.5:process
> (process-resource-bundles) @ security-admin-web ---
> [INFO]
> [INFO] --- maven-resources-plugin:2.7:resources (default-resources) @
> security-admin-web ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 22 resources
> [INFO] Copying 3 resources
> [INFO]
> [INFO] --- maven-antrun-plugin:1.7:run (default) @ security-admin-web ---
> [INFO] Executing tasks
> main:
> [INFO] Executed tasks
> [INFO]
> [INFO] --- frontend-maven-plugin:1.6:install-node-and-npm (install node and
> npm) @ security-admin-web ---
> [INFO] Installing node version v8.12.0
> [INFO] Unpacking
> /home/builder/.m2/repository/com/github/eirslett/node/8.12.0/node-8.12.0-linux-x64.tar.gz
> into /ranger/security-admin/target/node/tmp
> [INFO] Copying node binary from
> /ranger/security-admin/target/node/tmp/node-v8.12.0-linux-x64/bin/node to
> /ranger/security-admin/target/node/node
> [INFO] Installed node locally.
> [INFO] Installing npm version 6.4.1
> [INFO] Unpacking
> /home/builder/.m2/repository/com/github/eirslett/npm/6.4.1/npm-6.4.1.tar.gz
> into /ranger/security-admin/target/node/node_modules
> [INFO] Installed npm locally.
> [INFO]
> [INFO] --- frontend-maven-plugin:1.6:npm (npm install) @ security-admin-web
> ---
> [INFO] Running 'npm install' in /ranger/security-admin/target
> [INFO]
> [INFO] > phantomjs-prebuilt@2.1.16 install
> /ranger/security-admin/target/node_modules/karma-phantomjs-launcher/node_modules/phantomjs-prebuilt
> [INFO] > node install.js
> [INFO]
> [INFO] PhantomJS not found on PATH
> [INFO] Downloading
> https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
> [INFO] Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
> [INFO] Receiving...
> [INFO]
> [INFO] Received 22866K total.
> [INFO] Extracting tar contents (via spawned process)
> [ERROR] Error extracting archive
> [ERROR] Phantom installation failed { Error: Command failed: tar jxf
> /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
> [ERROR] tar (child): bzip2: Cannot exec: No such file or directory
> [ERROR] tar (child): Error is not recoverable: exiting now
> [ERROR] tar: Child returned status 2
> [ERROR] tar: Error is not recoverable: exiting now
> [ERROR]
> [ERROR] at ChildProcess.exithandler (child_process.js:276:12)
> [ERROR] at emitTwo (events.js:126:13)
> [ERROR] at ChildProcess.emit (events.js:214:7)
> [ERROR] at maybeClose (internal/child_process.js:915:16)
> [ERROR] at Socket.stream.socket.on (internal/child_process.js:336:11)
> [ERROR] at emitOne (events.js:116:13)
> [ERROR] at Socket.emit (events.js:211:7)
> [ERROR] at Pipe._handle.close [as _onclose] (net.js:561:12)
> [ERROR] killed: false,
> [ERROR] code: 2,
> [ERROR] signal:
Re: Review Request 69731: RANGER-2321 Docker build fails due to PhantomJS dependency
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69731/#review211940 --- Ship it! Ship It! - Zsombor Gegesy On Jan. 13, 2019, 7:18 p.m., Csaba Koncz wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69731/ > --- > > (Updated Jan. 13, 2019, 7:18 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2321 > https://issues.apache.org/jira/browse/RANGER-2321 > > > Repository: ranger > > > Description > --- > > - Docker image building installs the packages needed by installing and > running PhantomJS > - modified Maven build so that PhantomJS downloading and execution can be > skipped by adding -DskipJSTests to the command line > > > Diffs > - > > README.txt 8aa177927 > build_ranger_using_docker.sh fa77d910f > security-admin/pom.xml 14cafb4b2 > security-admin/src/main/webapp/package-lock.json 9deaf0071 > security-admin/src/main/webapp/package.json 880f6b34f > security-admin/src/test/javascript/karma-common.conf.js 4dc9bb60f > security-admin/src/test/javascript/package-lock.json PRE-CREATION > security-admin/src/test/javascript/package.json PRE-CREATION > > > Diff: https://reviews.apache.org/r/69731/diff/2/ > > > Testing > --- > > The following succeeds and runs JavaScript tests: > > ``` > ./build_ranger_using_docker.sh -build_image mvn clean verify -am -pl > security-admin > ``` > > The following succeeds and does not run JavaScript tests and does not > download PhantomJS: > ``` > mvn clean verify -am -pl security-admin -DskipJSTests > ``` > > > Thanks, > > Csaba Koncz > >
Re: Review Request 69731: RANGER-2321 Docker build fails due to PhantomJS dependency
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69731/#review211937
---
security-admin/pom.xml
Line 608 (original), 611 (patched)
<https://reviews.apache.org/r/69731/#comment297501>
${basedir}/target is the same as ${project.build.directory}, isn't it? Can
you use it everywhere?
- Zsombor Gegesy
On Jan. 12, 2019, 2:41 p.m., Csaba Koncz wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69731/
> ---
>
> (Updated Jan. 12, 2019, 2:41 p.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2321
> https://issues.apache.org/jira/browse/RANGER-2321
>
>
> Repository: ranger
>
>
> Description
> ---
>
> - Docker image building installs the packages needed by installing and
> running PhantomJS
> - modified Maven build so that PhantomJS downloading and execution can be
> skipped by adding -DskipJSTests to the command line
>
>
> Diffs
> -
>
> README.txt 8aa177927
> build_ranger_using_docker.sh fa77d910f
> security-admin/pom.xml 14cafb4b2
> security-admin/src/main/webapp/package-lock.json 9deaf0071
> security-admin/src/main/webapp/package.json 880f6b34f
> security-admin/src/test/javascript/karma-common.conf.js 4dc9bb60f
> security-admin/src/test/javascript/package-lock.json PRE-CREATION
> security-admin/src/test/javascript/package.json PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/69731/diff/1/
>
>
> Testing
> ---
>
> The following succeeds and runs JavaScript tests:
>
> ```
> ./build_ranger_using_docker.sh -build_image mvn clean verify -am -pl
> security-admin
> ```
>
> The following succeeds and does not run JavaScript tests and does not
> download PhantomJS:
> ```
> mvn clean verify -am -pl security-admin -DskipJSTests
> ```
>
>
> Thanks,
>
> Csaba Koncz
>
>
Re: Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code
> On Jan. 8, 2019, 11:48 a.m., Zsombor Gegesy wrote:
> > It's great news, that you could delete thousands of lines of repetitive
> > code, however you could achieve more, if instead of putting everything into
> > one class, and put
> > '''
> > if self.XA_DB_FLAVOR == DB_MYSQL:
> >...
> > elif self.XA_DB_FLAVOR == DB_POSTGRES:
> >...
> > '''
> >
> > You can write
> >self.do_something(...)
> >
> > and implement do_something differently in the MySQL/PostgreSQL/Oracle
> > specific adapter class
>
> Pradeep Agrawal wrote:
> There shall be too many self.do_something(...) function I have to write
> which shall look like the previous code. Can you review it once again and let
> me know with few examples.
Maybe you can add:
'''
def execute_query(self, query):
''' Execute query and return the output as a string '''
get_cmd = self.get_jisql_cmd(self.db_user, self.db_password, self.db_name)
if is_unix:
full_command = get_cmd + " -query \"" + query + "\""
elif os_name == "WINDOWS":
full_command = get_cmd + " -query \"" + query + "\" -c ;"
else:
raise Exception("This OS is not supported!")
jisql_log(full_command, self.db_password)
output = check_output(query)
return output
def execute_update(self, update):
''' Execute the update query and return the error code'''
get_cmd = self.get_jisql_cmd(self.db_user, self.db_password, self.db_name)
if is_unix:
full_command = get_cmd + " -query \"" + update + "\""
jisql_log(full_command, self.db_password)
return subprocess.call(shlex.split(query))
elif os_name == "WINDOWS":
full_command = get_cmd + " -query \"" + update + "\" -c ;"
jisql_log(full_command, self.db_password)
ret = subprocess.call(query)
raise Exception("This OS is not supported!")
'''
So you can get rid of lot's of repeating code around to support Windows.
And for the db changes, I would imagine something like this:
'''
class BaseDB(object):
@abstractmethod
def get_stale_patch_query(self, version, client_host,
stalePatchEntryHoldTimeInMinutes):
pass
class MysqlConf(BaseDB):
def get_stale_patch_query(self, version, client_host,
stalePatchEntryHoldTimeInMinutes):
return "select version from x_db_version_h where version = '%s' and
active = 'N' and updated_by='%s' and
TIMESTAMPDIFF(MINUTE,inst_at,CURRENT_TIMESTAMP)>=%s;" % (version, client_host,
stalePatchEntryHoldTimeInMinutes)
'''
So you can write:
'''
output =
self.execute_query(self.get_stale_patch_query(version,client_host,stalePatchEntryHoldTimeInMinutes))
...
'''
What do you think, does it makes sense?
- Zsombor
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69677/#review211760
---
On Jan. 7, 2019, 6:37 a.m., Pradeep Agrawal wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69677/
> ---
>
> (Updated Jan. 7, 2019, 6:37 a.m.)
>
>
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh,
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2287
> https://issues.apache.org/jira/browse/RANGER-2287
>
>
> Repository: ranger
>
>
> Description
> ---
>
> **Problem Statement:** There are lot of repeated code in db_setup.py which
> can be removed which shall help developers to make any changes in db_setup.py
> in future.
>
> **Proposed Solution:** Proposed patch shall remove the db setup methods of
> each db flavor and shall use a single method for a specific work for each db
> flavor. Based on the db flavor, config values shall be populated and handled
> in the code after this patch.
>
>
> Diffs
> -
>
> security-admin/scripts/db_setup.py f1223b38c
>
>
> Diff: https://reviews.apache.org/r/69677/diff/1/
>
>
> Testing
> ---
>
> **Use Cases covered for all the db flavors:**
> *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of
> ranger admin.
> *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same
> db config on Ranger 2.0 installation config and run the setup.sh. Ranger was
> upgraded successfully.
>
>
> Thanks,
>
> Pradeep Agrawal
>
>
[jira] [Resolved] (RANGER-2319) Remove deprecated phantomjs NPM package
[
https://issues.apache.org/jira/browse/RANGER-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsombor Gegesy resolved RANGER-2319.
Resolution: Fixed
Fix Version/s: 2.0.0
Thanks for the patch, applied to
[master|https://github.com/apache/ranger/commit/a3394a19f98cea04dc2824fdaa4e3a3fdbd6beb3]
> Remove deprecated phantomjs NPM package
> ---
>
> Key: RANGER-2319
> URL: https://issues.apache.org/jira/browse/RANGER-2319
> Project: Ranger
> Issue Type: Wish
> Components: admin
>Affects Versions: 2.0.0
>Reporter: Csaba Koncz
>Assignee: Csaba Koncz
>Priority: Major
> Fix For: 2.0.0
>
>
> [Phantomjs|https://github.com/apache/ranger/blob/e1b0105eee67bb73c56b66b2dda1c3424555ab3e/security-admin/src/main/webapp/package.json#L15]
> NPM package is deprecated.
> {code:java}
> $ npm show phantomjs
> phantomjs@2.1.7 | Apache-2.0 | deps: 8 | versions: 81
> Headless WebKit with JS API
> https://github.com/Medium/phantomjs
> DEPRECATED ⚠️ - Package renamed to phantomjs-prebuilt. Please update
> 'phantomjs' package references to 'phantomjs-prebuilt'{code}
> Worse, karma-phantomjs-launcher brings in its version as phantomjs-prebuilt
> which results in Phantomjs binaries being downloaded twice.
> Build would speed up if only one Phantomjs version is downloaded.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 69693: RANGER-2319 remove deprecated phantomjs package
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69693/#review211824 --- Ship it! Ship It! - Zsombor Gegesy On Jan. 10, 2019, 8:46 a.m., Csaba Koncz wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69693/ > --- > > (Updated Jan. 10, 2019, 8:46 a.m.) > > > Review request for ranger. > > > Repository: ranger > > > Description > --- > > Phantomjs package was renamed to phantomjs-prebuilt. It is automatically > downloaded as a transitive dependency of Karma-phantomjs-launcher > > > Diffs > - > > security-admin/src/main/webapp/package-lock.json fbb3e6bf9 > security-admin/src/main/webapp/package.json 060b765ff > > > Diff: https://reviews.apache.org/r/69693/diff/3/ > > > Testing > --- > > `mvn clean verify -am -pl security-admin` succeeds: > ``` > ... > [INFO] > [INFO] --- frontend-maven-plugin:1.6:karma (karma prd) @ security-admin-web > --- > [INFO] Running 'karma start ../src/test/javascript/karma-prd.conf.js' in > /Users/ckoncz/work/git/hwx/ranger/security-admin/target > [INFO] 08 01 2019 12:21:35.542:INFO [karma-server]: Karma v3.1.4 server > started at http://0.0.0.0:9876/ > [INFO] 08 01 2019 12:21:35.545:INFO [launcher]: Launching browsers PhantomJS > with concurrency unlimited > [INFO] 08 01 2019 12:21:35.560:INFO [launcher]: Starting browser PhantomJS > [INFO] 08 01 2019 12:21:36.592:INFO [PhantomJS 2.1.1 (Mac OS X 0.0.0)]: > Connected on socket IKx-RvUUMj7kLzZl with id 31183876 >LOG: 'just before loading Main' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) > [INFO] 08 01 2019 12:21:36.825:WARN [web-server]: 404: > /service/plugins/csrfconf > [INFO] 08 01 2019 12:21:36.835:WARN [web-server]: 404: > /service/users/profile?_=1546946496706 > [INFO] 08 01 2019 12:21:36.866:WARN [web-server]: 404: /images/avatar.png >LOG: 'initialize a Communicator' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) >LOG: 'Initialize a Region Manager' >LOG: 'initialize a Controller Controller' >LOG: 'initialized a TopNav ItemView' >LOG: 'initialized a ProfileBar ItemView' >LOG: 'initialized a Footer ItemView' >LOG: 'Error in service/plugins/checksso REST call404' >LOG: 'initialized a ErrorView ItemView' >LOG: 'mainModule.success=false' >PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 1 of 1 SUCCESS (0.161 secs > / 0.153 secs) > [INFO] TOTAL: 1 SUCCESS > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] ranger 2.0.0-SNAPSHOT .. SUCCESS [ 4.431 > s] > [INFO] Credential Support . SUCCESS [ 5.325 > s] > [INFO] Audit Component SUCCESS [ 3.359 > s] > [INFO] Common library for Plugins . SUCCESS [ 10.998 > s] > [INFO] Ranger Util SUCCESS [ 2.430 > s] > [INFO] Unix Authentication Client . SUCCESS [ 1.532 > s] > [INFO] Security Admin Web Application 2.0.0-SNAPSHOT .. SUCCESS [01:53 > min] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 02:21 min > [INFO] Finished at: 2019-01-08T12:21:36+01:00 > [INFO] > > ``` > > Then: > ``` > $ cd security-admin/target/ > $ rm -rf node_modules/ > $ ./node/npm install > > > fsevents@1.2.4 install > > /Users/ckoncz/work/git/hwx/ranger/security-admin/target/node_modules/fsevents > > node install > > [fsevents] Success: > "/Users/ckoncz/work/git/hwx/ranger/security-admin/target/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64/fse.node" > already installed > Pass --update-binary to reinstall or --build-from-source to recompile > > > phantomjs-prebuilt@2.1.16 install > > /Users/ckoncz/work/git/hwx/ranger/security-admin/target/node_modules/phantomjs-prebuilt > > node install.js > > PhantomJS not found on PATH > Download already available at > /v
Re: Review Request 69693: RANGER-2319 remove deprecated phantomjs package
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69693/#review211763 --- Ship it! Ship It! - Zsombor Gegesy On Jan. 8, 2019, 11:28 a.m., Csaba Koncz wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69693/ > --- > > (Updated Jan. 8, 2019, 11:28 a.m.) > > > Review request for ranger. > > > Repository: ranger > > > Description > --- > > Phantomjs package was renamed to phantomjs-prebuilt. It is automatically > downloaded as a transitive dependency of Karma-phantomjs-launcher > > > Diffs > - > > security-admin/src/main/webapp/package-lock.json fbb3e6bf9 > security-admin/src/main/webapp/package.json 060b765ff > > > Diff: https://reviews.apache.org/r/69693/diff/1/ > > > Testing > --- > > `mvn clean verify -am -pl security-admin` succeeds: > ``` > ... > [INFO] > [INFO] --- frontend-maven-plugin:1.6:karma (karma prd) @ security-admin-web > --- > [INFO] Running 'karma start ../src/test/javascript/karma-prd.conf.js' in > /Users/ckoncz/work/git/hwx/ranger/security-admin/target > [INFO] 08 01 2019 12:21:35.542:INFO [karma-server]: Karma v3.1.4 server > started at http://0.0.0.0:9876/ > [INFO] 08 01 2019 12:21:35.545:INFO [launcher]: Launching browsers PhantomJS > with concurrency unlimited > [INFO] 08 01 2019 12:21:35.560:INFO [launcher]: Starting browser PhantomJS > [INFO] 08 01 2019 12:21:36.592:INFO [PhantomJS 2.1.1 (Mac OS X 0.0.0)]: > Connected on socket IKx-RvUUMj7kLzZl with id 31183876 >LOG: 'just before loading Main' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) > [INFO] 08 01 2019 12:21:36.825:WARN [web-server]: 404: > /service/plugins/csrfconf > [INFO] 08 01 2019 12:21:36.835:WARN [web-server]: 404: > /service/users/profile?_=1546946496706 > [INFO] 08 01 2019 12:21:36.866:WARN [web-server]: 404: /images/avatar.png >LOG: 'initialize a Communicator' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) >LOG: 'Initialize a Region Manager' >LOG: 'initialize a Controller Controller' >LOG: 'initialized a TopNav ItemView' >LOG: 'initialized a ProfileBar ItemView' >LOG: 'initialized a Footer ItemView' >LOG: 'Error in service/plugins/checksso REST call404' >LOG: 'initialized a ErrorView ItemView' >LOG: 'mainModule.success=false' >PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 1 of 1 SUCCESS (0.161 secs > / 0.153 secs) > [INFO] TOTAL: 1 SUCCESS > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] ranger 2.0.0-SNAPSHOT .. SUCCESS [ 4.431 > s] > [INFO] Credential Support . SUCCESS [ 5.325 > s] > [INFO] Audit Component SUCCESS [ 3.359 > s] > [INFO] Common library for Plugins . SUCCESS [ 10.998 > s] > [INFO] Ranger Util SUCCESS [ 2.430 > s] > [INFO] Unix Authentication Client . SUCCESS [ 1.532 > s] > [INFO] Security Admin Web Application 2.0.0-SNAPSHOT .. SUCCESS [01:53 > min] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 02:21 min > [INFO] Finished at: 2019-01-08T12:21:36+01:00 > [INFO] > > ``` > > Then: > ``` > $ cd security-admin/target/ > $ rm -rf node_modules/ > $ ./node/npm install > > > fsevents@1.2.4 install > > /Users/ckoncz/work/git/hwx/ranger/security-admin/target/node_modules/fsevents > > node install > > [fsevents] Success: > "/Users/ckoncz/work/git/hwx/ranger/security-admin/target/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64/fse.node" > already installed > Pass --update-binary to reinstall or --build-from-source to recompile > > > phantomjs-prebuilt@2.1.16 install > > /Users/ckoncz/work/git/hwx/ranger/security-admin/target/node_modules/phantomjs-prebuilt > > node install.js > > PhantomJS not found on PATH > Download already available at > /v
Re: Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69677/#review211760 --- It's great news, that you could delete thousands of lines of repetitive code, however you could achieve more, if instead of putting everything into one class, and put ''' if self.XA_DB_FLAVOR == DB_MYSQL: ... elif self.XA_DB_FLAVOR == DB_POSTGRES: ... ''' You can write self.do_something(...) and implement do_something differently in the MySQL/PostgreSQL/Oracle specific adapter class - Zsombor Gegesy On Jan. 7, 2019, 6:37 a.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69677/ > --- > > (Updated Jan. 7, 2019, 6:37 a.m.) > > > Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, > Nikhil P, Ramesh Mani, and Velmurugan Periasamy. > > > Bugs: RANGER-2287 > https://issues.apache.org/jira/browse/RANGER-2287 > > > Repository: ranger > > > Description > --- > > **Problem Statement:** There are lot of repeated code in db_setup.py which > can be removed which shall help developers to make any changes in db_setup.py > in future. > > **Proposed Solution:** Proposed patch shall remove the db setup methods of > each db flavor and shall use a single method for a specific work for each db > flavor. Based on the db flavor, config values shall be populated and handled > in the code after this patch. > > > Diffs > - > > security-admin/scripts/db_setup.py f1223b38c > > > Diff: https://reviews.apache.org/r/69677/diff/1/ > > > Testing > --- > > **Use Cases covered for all the db flavors:** > *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of > ranger admin. > *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same > db config on Ranger 2.0 installation config and run the setup.sh. Ranger was > upgraded successfully. > > > Thanks, > > Pradeep Agrawal > >
[jira] [Updated] (RANGER-2317) Enable compilation on JDK11
[ https://issues.apache.org/jira/browse/RANGER-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2317: --- Attachment: (was: RANGER-2317.patch) > Enable compilation on JDK11 > --- > > Key: RANGER-2317 > URL: https://issues.apache.org/jira/browse/RANGER-2317 > Project: Ranger > Issue Type: Improvement > Components: admin, plugins > Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: master > > Attachments: RANGER-2317-2.patch > > > Currently, Ranger can be compiled only with JDK 8, however JDK 11 is the > current LTS release for Java, it is essential to support it. As a first step, > we need to ensure that Ranger can be compiled on JDK 11. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RANGER-2317) Enable compilation on JDK11
[ https://issues.apache.org/jira/browse/RANGER-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2317: --- Attachment: RANGER-2317-2.patch > Enable compilation on JDK11 > --- > > Key: RANGER-2317 > URL: https://issues.apache.org/jira/browse/RANGER-2317 > Project: Ranger > Issue Type: Improvement > Components: admin, plugins > Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: master > > Attachments: RANGER-2317-2.patch > > > Currently, Ranger can be compiled only with JDK 8, however JDK 11 is the > current LTS release for Java, it is essential to support it. As a first step, > we need to ensure that Ranger can be compiled on JDK 11. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 69655: RANGER-2317 : make Ranger buildable on newer JDKs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69655/ --- (Updated Jan. 7, 2019, 8:30 p.m.) Review request for ranger. Changes --- Fix enunciate documentation generation Bugs: RANGER-2317 https://issues.apache.org/jira/browse/RANGER-2317 Repository: ranger Description --- Upgrade librarias to become compatible with JDK11, and JAXB and other librarias to the modules, and exclude jdk.tools. (These modules was previously part of the JDK, now they are independent). Add JDK10/11 to the travis build - with excluding the Hive tests, as we can't even start the Hive tests on JDK 9 - because internal Hive code cast the system classloader to URLClassLoader. Removed one empty test class file, which triggered test failures with newer mockito version. Diffs (updated) - .travis.yml b38b892bd enunciate.xml f64af1a56 hbase-agent/pom.xml c044ef0c3 kms/pom.xml 043c67fde knox-agent/pom.xml 0c45c84ba plugin-solr/pom.xml 619cc9ebd pom.xml 2a2540a0d ranger-hbase-plugin-shim/pom.xml 9ed0aebe8 ranger-solr-plugin-shim/pom.xml 46b78218a security-admin/pom.xml 14cafb4b2 security-admin/src/test/java/org/apache/ranger/plugin/store/RangerDBStore.java cd5bb384e ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestUnixUserGroupBuilder.java 2118c8430 Diff: https://reviews.apache.org/r/69655/diff/2/ Changes: https://reviews.apache.org/r/69655/diff/1-2/ Testing --- Tested resulting binaries locally on JDK 8, to stay the same as previously. All the dependency change was either in test code, or in Maven plugin config, or has 'provided' scope. Tested on Travis that build is succesfull in 4 JVMs - oraclejdk8/oraclejdk11/openjdk10/openjdk11: https://travis-ci.org/gzsombor/ranger/builds/474559352 Thanks, Zsombor Gegesy
[jira] [Resolved] (RANGER-2255) Add JavaScript unit tests
[ https://issues.apache.org/jira/browse/RANGER-2255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy resolved RANGER-2255. Resolution: Fixed Commited to [master|https://github.com/apache/ranger/commit/e1b0105eee67bb73c56b66b2dda1c3424555ab3e], thanks for your contribution [~Koncz] ! > Add JavaScript unit tests > - > > Key: RANGER-2255 > URL: https://issues.apache.org/jira/browse/RANGER-2255 > Project: Ranger > Issue Type: Wish > Components: admin >Affects Versions: 0.7.0, 2.0.0, 1.2.1 >Reporter: Csaba Koncz >Assignee: Csaba Koncz >Priority: Minor > Fix For: 2.0.0 > > > It would be nice if the admin-ui project would have JavaScript unit tests. > As with RANGER-2220 JavaScript minification was introduced, that can lead to > new type of loading errors that were not seen before. > It would be nice if there was an automatic check that validates the minified > output. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (RANGER-2318) Incorrect git url on the homepage
[ https://issues.apache.org/jira/browse/RANGER-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16734101#comment-16734101 ] Zsombor Gegesy commented on RANGER-2318: Commited to [master|https://github.com/apache/ranger/commit/ead451a3383f33630f59c1996c478ddb6f6fb39f] > Incorrect git url on the homepage > - > > Key: RANGER-2318 > URL: https://issues.apache.org/jira/browse/RANGER-2318 > Project: Ranger > Issue Type: Bug > Components: documentation > Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Minor > Labels: homepage > Fix For: master > > Attachments: RANGER-2318.patch > > > On http://ranger.apache.org/ the link for the git repository is pointing to > https://git.apache.org/ranger.git/ instead of gitbox and github. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 69067: RANGER-2255 JavaScript tests for UI loading
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69067/#review211633 --- Ship it! Ship It! - Zsombor Gegesy On Jan. 3, 2019, 1:43 p.m., Csaba Koncz wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69067/ > --- > > (Updated Jan. 3, 2019, 1:43 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2255 > https://issues.apache.org/jira/browse/RANGER-2255 > > > Repository: ranger > > > Description > --- > > Added JavaScript testing infrastructure (Karma, Mocha, PhantomJS) and a basic > test to verify that both the initial uncompressed JavaScript module tree and > the minified version can be loaded by a browser. > > There are two test (Karma) configurations that correspond to the two module > loading methods. > > Later on more sophisticated tests can be added that mock backend services and > do "real" business logic testing. > > > Diffs > - > > security-admin/pom.xml c67860115 > security-admin/src/main/webapp/index.html 4eb97353b > security-admin/src/main/webapp/minify.build.js 4b94bb159 > security-admin/src/main/webapp/package-lock.json f7a72833f > security-admin/src/main/webapp/package.json 9f2274445 > security-admin/src/main/webapp/scripts/Init.js 52b49e5fb > security-admin/src/main/webapp/scripts/Main.js 941414618 > security-admin/src/test/javascript/karma-common.conf.js PRE-CREATION > security-admin/src/test/javascript/karma-dev.conf.js PRE-CREATION > security-admin/src/test/javascript/karma-prd.conf.js PRE-CREATION > security-admin/src/test/javascript/test-main.js PRE-CREATION > security-admin/src/test/javascript/tests/main-test.js PRE-CREATION > > > Diff: https://reviews.apache.org/r/69067/diff/4/ > > > Testing > --- > > Executed mvn clean verify. The output contained the following: > > [INFO] --- frontend-maven-plugin:1.6:npm (npm run karma dev) @ > security-admin-web --- > [INFO] Running 'npm run karma -- start > security-admin-web-2.0.0-SNAPSHOT/tests/karma-dev.conf.js' in > /Users/ckoncz/work/git/hwx/ranger/security-admin/target > [INFO] > [INFO] > security-admin@2.0.0 karma > /Users/ckoncz/work/git/hwx/ranger/security-admin/target > [INFO] > karma "start" > "security-admin-web-2.0.0-SNAPSHOT/tests/karma-dev.conf.js" > [INFO] > [INFO] 17 10 2018 21:32:26.799:WARN [watcher]: Pattern > "/Users/ckoncz/work/git/hwx/ranger/security-admin/target/security-admin-web-2.0.0-SNAPSHOT/node_modules/chai/chai.js" > does not match any file. > [INFO] 17 10 2018 21:32:26.802:WARN [watcher]: Pattern > "/Users/ckoncz/work/git/hwx/ranger/security-admin/target/security-admin-web-2.0.0-SNAPSHOT/node_modules/karma-requirejs/lib/adapter.js" > does not match any file. > [INFO] 17 10 2018 21:32:26.944:INFO [karma]: Karma v3.0.0 server started at > http://0.0.0.0:9876/ > [INFO] 17 10 2018 21:32:26.944:INFO [launcher]: Launching browser PhantomJS > with unlimited concurrency > [INFO] 17 10 2018 21:32:26.949:INFO [launcher]: Starting browser PhantomJS > [INFO] 17 10 2018 21:32:27.792:INFO [PhantomJS 2.1.1 (Mac OS X 0.0.0)]: > Connected on socket PKDEs28ja09HLFyL with id 27321101 >LOG: 'just before loading Main' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) >LOG: 'initialize a Communicator' >LOG: 'Initialize a Region Manager' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) > [INFO] 17 10 2018 21:32:28.788:WARN [web-server]: 404: > /service/users/profile?_=1539804747867 > [INFO] 17 10 2018 21:32:28.827:WARN [web-server]: 404: /images/avatar.png >LOG: 'initialize a Controller Controller' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) >LOG: 'initialized a TopNav ItemView' >LOG: 'initialized a ProfileBar ItemView' >LOG: 'initialized a Footer ItemView' >LOG: 'Error in service/plugins/checksso REST call404' >LOG: 'initialized a ErrorView ItemView' >LOG: 'mainModule.success=false' >PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 1 of 1 SUCCESS (0.966 secs > / 0.958 secs) > [INFO] TOTAL: 1 SUCCESS > [INFO] > [INFO] --- frontend-maven-plugin:1.6:npm (npm run karma prd) @ > security-admin-web --- > [INFO] Running 'npm run karma --
[jira] [Updated] (RANGER-2318) Incorrect git url on the homepage
[ https://issues.apache.org/jira/browse/RANGER-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2318: --- Attachment: RANGER-2318.patch > Incorrect git url on the homepage > - > > Key: RANGER-2318 > URL: https://issues.apache.org/jira/browse/RANGER-2318 > Project: Ranger > Issue Type: Bug > Components: documentation > Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Minor > Labels: homepage > Attachments: RANGER-2318.patch > > > On http://ranger.apache.org/ the link for the git repository is pointing to > https://git.apache.org/ranger.git/ instead of gitbox and github. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (RANGER-2318) Incorrect git url on the homepage
Zsombor Gegesy created RANGER-2318: -- Summary: Incorrect git url on the homepage Key: RANGER-2318 URL: https://issues.apache.org/jira/browse/RANGER-2318 Project: Ranger Issue Type: Bug Components: documentation Reporter: Zsombor Gegesy Assignee: Zsombor Gegesy On http://ranger.apache.org/ the link for the git repository is pointing to https://git.apache.org/ranger.git/ instead of gitbox and github. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Review Request 69659: RANGER-2318 - homepage has a wrong git URL
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69659/ --- Review request for ranger. Bugs: RANGER-2318 https://issues.apache.org/jira/browse/RANGER-2318 Repository: ranger Description --- The homepage contains a wrong link to an old git repository, instead of gitbox, or github. Diffs - docs/src/site/apt/index.apt.vm 054fd3bdf Diff: https://reviews.apache.org/r/69659/diff/1/ Testing --- Built the documentation Thanks, Zsombor Gegesy
Review Request 69655: RANGER-2317 : make Ranger buildable on newer JDKs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69655/ --- Review request for ranger. Bugs: RANGER-2317 https://issues.apache.org/jira/browse/RANGER-2317 Repository: ranger Description --- Upgrade librarias to become compatible with JDK11, and JAXB and other librarias to the modules, and exclude jdk.tools. (These modules was previously part of the JDK, now they are independent). Add JDK10/11 to the travis build - with excluding the Hive tests, as we can't even start the Hive tests on JDK 9 - because internal Hive code cast the system classloader to URLClassLoader. Removed one empty test class file, which triggered test failures with newer mockito version. Diffs - .travis.yml b38b892bd21287b21d575aba862d93fd46952ac6 hbase-agent/pom.xml c044ef0c30f348cd82d3ef37089677c8d83b5db8 kms/pom.xml 043c67fde4eb6617c7e3b8d852f0aa471e032161 knox-agent/pom.xml 0c45c84baae8afb50f7c130f611ccaa00dda4406 plugin-solr/pom.xml 619cc9ebdbac883fb3e981843b82bd0c35f8508c pom.xml 2a2540a0d578911a9f84cffcd861d37767cc156f ranger-hbase-plugin-shim/pom.xml 9ed0aebe8e80a63817f0ba0d18039f5e15a6bab2 ranger-solr-plugin-shim/pom.xml 46b78218a8c2c7ec142da159e867a2d38390e630 security-admin/pom.xml c67860115f55719c244cd82ed823a471a6f25114 security-admin/src/test/java/org/apache/ranger/plugin/store/RangerDBStore.java cd5bb384e4e8f2e51ec59e0dff6134990fa192ff ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestUnixUserGroupBuilder.java 2118c8430d885252590d7bcde4216ce9505ae7bc Diff: https://reviews.apache.org/r/69655/diff/1/ Testing --- Tested resulting binaries locally on JDK 8, to stay the same as previously. All the dependency change was either in test code, or in Maven plugin config, or has 'provided' scope. Tested on Travis that build is succesfull in 4 JVMs - oraclejdk8/oraclejdk11/openjdk10/openjdk11: https://travis-ci.org/gzsombor/ranger/builds/474559352 Thanks, Zsombor Gegesy
[jira] [Updated] (RANGER-2317) Enable compilation on JDK11
[ https://issues.apache.org/jira/browse/RANGER-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2317: --- Attachment: RANGER-2317.patch > Enable compilation on JDK11 > --- > > Key: RANGER-2317 > URL: https://issues.apache.org/jira/browse/RANGER-2317 > Project: Ranger > Issue Type: Improvement > Components: admin, plugins > Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: master > > Attachments: RANGER-2317.patch > > > Currently, Ranger can be compiled only with JDK 8, however JDK 11 is the > current LTS release for Java, it is essential to support it. As a first step, > we need to ensure that Ranger can be compiled on JDK 11. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (RANGER-2317) Enable compilation on JDK11
Zsombor Gegesy created RANGER-2317: -- Summary: Enable compilation on JDK11 Key: RANGER-2317 URL: https://issues.apache.org/jira/browse/RANGER-2317 Project: Ranger Issue Type: Improvement Components: admin, plugins Reporter: Zsombor Gegesy Assignee: Zsombor Gegesy Fix For: master Currently, Ranger can be compiled only with JDK 8, however JDK 11 is the current LTS release for Java, it is essential to support it. As a first step, we need to ensure that Ranger can be compiled on JDK 11. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 68620: RANGER-2206: Add Policy Validity Period and Override as search filters and add distinguishing icons in policy listing page
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68620/#review211592 --- Converting always between dates and strings is not the best solution, I think. It would be much simpler, if RangerValiditySchedule would contain an Instant for startTime and endTime, and the caller service would similarly put an Instant or a ZonedDateTime into the SearchFilter and this filtering could be done on comparing Instants. - Zsombor Gegesy On Sept. 4, 2018, 2:35 p.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68620/ > --- > > (Updated Sept. 4, 2018, 2:35 p.m.) > > > Review request for ranger, Gautam Borad, Mehul Parikh, and Velmurugan > Periasamy. > > > Bugs: RANGER-2206 > https://issues.apache.org/jira/browse/RANGER-2206 > > > Repository: ranger > > > Description > --- > > – Add policy validity period as search filter in policy listing page. Both > start date and end date (along with timezone). Search should show policies > that have validity period specified and meet that condition. > – Add policy override as search filter in policy listing page. Search should > show only policies defined as override. > – Add indication in policy listing page to distinguish policies with validity > period and override policies. Add icons for visual clue. > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/store/ServiceStore.java > 9924cb4 > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java > 32af4cd > security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java > 7b0fd87 > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > 59741d3 > security-admin/src/main/webapp/scripts/modules/globalize/message/en.js > 99cbf55 > security-admin/src/main/webapp/scripts/utils/XAEnums.js ea80545 > security-admin/src/main/webapp/scripts/utils/XAUtils.js d9366a1 > security-admin/src/main/webapp/scripts/views/common/CustomSubgrid.js > ad89a13 > > security-admin/src/main/webapp/scripts/views/policies/RangerPolicyTableLayout.js > 1150838 > security-admin/src/main/webapp/styles/xa.css c601d54 > > > Diff: https://reviews.apache.org/r/68620/diff/1/ > > > Testing > --- > > 1.User is able to perform search through Start Time, End Time,Timezone and > Override. > 2.Verified that there is indication available for Override policy > 3.Verified that policy table shows results as per the search criteria along > with those policies which does not have 4.validity schedule. > 5.Verified that there is icon available next to the Policy ID, upon clicking > on that it display details of all Validity periods with respect to that > policy. > 6.Verified search functionality through Curl command as well. > > > Thanks, > > Nitin Galave > >
Re: Review Request 69067: RANGER-2255 JavaScript tests for UI loading
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69067/#review211591 --- Any particular reason, why these tests are added to the 'src/main/webapp/tests' folder, instead under 'src/test/javascript'? This way, these js files are added to the webapp, isn't it? - Zsombor Gegesy On Oct. 22, 2018, 8:34 p.m., Csaba Koncz wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69067/ > --- > > (Updated Oct. 22, 2018, 8:34 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2255 > https://issues.apache.org/jira/browse/RANGER-2255 > > > Repository: ranger > > > Description > --- > > Added JavaScript testing infrastructure (Karma, Mocha, PhantomJS) and a basic > test to verify that both the initial uncompressed JavaScript module tree and > the minified version can be loaded by a browser. > > There are two test (Karma) configurations that correspond to the two module > loading methods. > > Later on more sophisticated tests can be added that mock backend services and > do "real" business logic testing. > > > Diffs > - > > security-admin/pom.xml 7f81e6873 > security-admin/src/main/webapp/index.html 4eb97353b > security-admin/src/main/webapp/minify.build.js 4b94bb159 > security-admin/src/main/webapp/package-lock.json f7a72833f > security-admin/src/main/webapp/package.json 9f2274445 > security-admin/src/main/webapp/scripts/Init.js 52b49e5fb > security-admin/src/main/webapp/scripts/Main.js 941414618 > security-admin/src/main/webapp/tests/karma-common.conf.js PRE-CREATION > security-admin/src/main/webapp/tests/karma-dev.conf.js PRE-CREATION > security-admin/src/main/webapp/tests/karma-prd.conf.js PRE-CREATION > security-admin/src/main/webapp/tests/test-main.js PRE-CREATION > security-admin/src/main/webapp/tests/tests/main-test.js PRE-CREATION > > > Diff: https://reviews.apache.org/r/69067/diff/3/ > > > Testing > --- > > Executed mvn clean verify. The output contained the following: > > [INFO] --- frontend-maven-plugin:1.6:npm (npm run karma dev) @ > security-admin-web --- > [INFO] Running 'npm run karma -- start > security-admin-web-2.0.0-SNAPSHOT/tests/karma-dev.conf.js' in > /Users/ckoncz/work/git/hwx/ranger/security-admin/target > [INFO] > [INFO] > security-admin@2.0.0 karma > /Users/ckoncz/work/git/hwx/ranger/security-admin/target > [INFO] > karma "start" > "security-admin-web-2.0.0-SNAPSHOT/tests/karma-dev.conf.js" > [INFO] > [INFO] 17 10 2018 21:32:26.799:WARN [watcher]: Pattern > "/Users/ckoncz/work/git/hwx/ranger/security-admin/target/security-admin-web-2.0.0-SNAPSHOT/node_modules/chai/chai.js" > does not match any file. > [INFO] 17 10 2018 21:32:26.802:WARN [watcher]: Pattern > "/Users/ckoncz/work/git/hwx/ranger/security-admin/target/security-admin-web-2.0.0-SNAPSHOT/node_modules/karma-requirejs/lib/adapter.js" > does not match any file. > [INFO] 17 10 2018 21:32:26.944:INFO [karma]: Karma v3.0.0 server started at > http://0.0.0.0:9876/ > [INFO] 17 10 2018 21:32:26.944:INFO [launcher]: Launching browser PhantomJS > with unlimited concurrency > [INFO] 17 10 2018 21:32:26.949:INFO [launcher]: Starting browser PhantomJS > [INFO] 17 10 2018 21:32:27.792:INFO [PhantomJS 2.1.1 (Mac OS X 0.0.0)]: > Connected on socket PKDEs28ja09HLFyL with id 27321101 >LOG: 'just before loading Main' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) >LOG: 'initialize a Communicator' >LOG: 'Initialize a Region Manager' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) > [INFO] 17 10 2018 21:32:28.788:WARN [web-server]: 404: > /service/users/profile?_=1539804747867 > [INFO] 17 10 2018 21:32:28.827:WARN [web-server]: 404: /images/avatar.png >LOG: 'initialize a Controller Controller' > [INFO] PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 > secs) >LOG: 'initialized a TopNav ItemView' >LOG: 'initialized a ProfileBar ItemView' >LOG: 'initialized a Footer ItemView' >LOG: 'Error in service/plugins/checksso REST call404' >LOG: 'initialized a ErrorView ItemView' >LOG: 'mainModule.success=false' >PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 1 of 1 SUCCESS (0.966 secs > / 0.958 secs) &g
Re: Review Request 68685: RANGER-2216: Ranger Audit UI lacks the feature to search the audits using Policy Id
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68685/#review211590 --- Ship it! Ship It! - Zsombor Gegesy On Sept. 11, 2018, 1:29 p.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68685/ > --- > > (Updated Sept. 11, 2018, 1:29 p.m.) > > > Review request for ranger, Gautam Borad, Mehul Parikh, Pradeep Agrawal, and > Velmurugan Periasamy. > > > Bugs: RANGER-2216 > https://issues.apache.org/jira/browse/RANGER-2216 > > > Repository: ranger > > > Description > --- > > Currently, we do not have a feature which answers the question "who made this > change to this policy" without scrolling through the 100s of pages of audit. > A search by Policy Id > > > Diffs > - > > security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js 697d33d > > > Diff: https://reviews.apache.org/r/68685/diff/1/ > > > Testing > --- > > Verified that user is able to perform search through Policy ID on > Audits>>Access Ta > > > Thanks, > > Nitin Galave > >
Review Request 69524: RANGER-2307 - native authentication improvements
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69524/ --- Review request for ranger. Bugs: RANGER-2307 https://issues.apache.org/jira/browse/RANGER-2307 Repository: ranger Description --- Currently credValidator and pamCredValidator don't handle well configuration problems - when the user doesn't have permission to read /etc/shadow or access pam, or when the shadow file is not filled properly. This could cause 'core dumps', and hard to fix deployment issues. With this change, at least, it doesn't segfaults, when the crypt function returns null, and it shows the underlying error message in its response. Diffs - unixauthnative/src/main/c/credValidator.c e426bdd2f unixauthpam/src/main/c/pamCredValidator.c 60d38aebd Diff: https://reviews.apache.org/r/69524/diff/1/ Testing --- Tryied to call both method on live cluster - as root / non-root, with wrong username, missing password from shadow file, etc. Thanks, Zsombor Gegesy
[jira] [Updated] (RANGER-2307) Native code can segfault or return misleading error messages
[ https://issues.apache.org/jira/browse/RANGER-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2307: --- Attachment: RANGER-2307-native-fixes.patch > Native code can segfault or return misleading error messages > > > Key: RANGER-2307 > URL: https://issues.apache.org/jira/browse/RANGER-2307 > Project: Ranger > Issue Type: Bug > Components: usersync >Affects Versions: 1.2.0 >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Labels: native, pam, unix > Attachments: RANGER-2307-native-fixes.patch > > > Currently credValidator and pamCredValidator don't handle well configuration > problems - when the user doesn't have permission to read /etc/shadow or > access pam, or when the shadow file is not filled properly. This could cause > 'core dumps', and hard to fix deployment issues -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (RANGER-2307) Native code can segfault or return misleading error messages
Zsombor Gegesy created RANGER-2307: -- Summary: Native code can segfault or return misleading error messages Key: RANGER-2307 URL: https://issues.apache.org/jira/browse/RANGER-2307 Project: Ranger Issue Type: Bug Components: usersync Affects Versions: 1.2.0 Reporter: Zsombor Gegesy Assignee: Zsombor Gegesy Currently credValidator and pamCredValidator don't handle well configuration problems - when the user doesn't have permission to read /etc/shadow or access pam, or when the shadow file is not filled properly. This could cause 'core dumps', and hard to fix deployment issues -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 68881: RANGER-2203: Review and update database schema for ranger policies to minimize database queries/updates
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68881/#review209757 --- security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java Lines 219 (patched) <https://reviews.apache.org/r/68881/#comment294321> If policy can be null - which I assume based on the first line - this will cause an NPE. If policy can't be null, then the conditional on the first line is redundant, and here, you can write 'if (policyId == null)' security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyRetriever.java Lines 596 (patched) <https://reviews.apache.org/r/68881/#comment294322> This getAllPolicyItems is the same as in the PolicyRefUpdater - can you move to a common place, and use it from there? For example, it can be added as RangerPolicy.getAllPolicyItems() This list of list of RangerPolicyItem is a bit complicated, and I can't see where this structure is needed, as all the callers are just iterating over the RangerPolicyItems. Why can't you just return a List ? security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefConditionDao.java Lines 41 (patched) <https://reviews.apache.org/r/68881/#comment294323> You can return Collections.emptyList() in every negative case. - Zsombor Gegesy On Oct. 18, 2018, 4:47 p.m., Abhay Kulkarni wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68881/ > --- > > (Updated Oct. 18, 2018, 4:47 p.m.) > > > Review request for ranger, Madhan Neethiraj, Mehul Parikh, Nikhil P, Pradeep > Agrawal, and Velmurugan Periasamy. > > > Bugs: RANGER-2186, RANGER-2195 and RANGER-2219 > https://issues.apache.org/jira/browse/RANGER-2186 > https://issues.apache.org/jira/browse/RANGER-2195 > https://issues.apache.org/jira/browse/RANGER-2219 > > > Repository: ranger > > > Description > --- > > Notes: > > This is a composite patch for updating database schema for Ranger policy and > tag related objects. > > Notes: > > This patch is more complete than the one posted earlier > (https://reviews.apache.org/r/68876/). Database scripts to drop foreign-keys > are missing for some database types. They need to be added. > > This patch also subsumes back-ports of RANGER-2186 and RANGER-2195 from > master to ranger-0.7 branch. Porting those JIRA separately involved fixing > conflicts in many common files. > > Description: > > Currently, ranger policies are fully normalized and stored in a multiple > Relational database tables. There is a performance overhead incurred when > retrieving a ranger policy, as multiple database accesses are required to > fully reconstruct it. This is significant when there are large ranger > policies (that is, the number of resources addressed by the policy is large), > and/or when there is a large number of ranger policies in an installation. > > This Jira tracks alternate design of database schema, where a policy is > stored in a de-normalized way, in its entirely, in one database table > (preferably as a JSON string). > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/authorization/utils/JsonUtils.java > PRE-CREATION > > agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractServiceStore.java > a75ca59e8 > agents-common/src/main/java/org/apache/ranger/plugin/store/TagStore.java > 5918b1292 > embeddedwebserver/scripts/ranger-admin-services.sh 19dab2a9d > security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql > PRE-CREATION > security-admin/db/mysql/patches/036-denormalize-tag-tables.sql PRE-CREATION > security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql > PRE-CREATION > security-admin/db/oracle/patches/036-denormalize-tag-tables.sql > PRE-CREATION > security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql > PRE-CREATION > security-admin/db/postgres/patches/036-denormalize-tag-tables.sql > PRE-CREATION > security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql > PRE-CREATION > security-admin/db/sqlanywhere/patches/036-denormalize-tag-tables.sql > PRE-CREATION > security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql > bf73c34ea > security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql > PRE-CREATION > security-admin/db/sqlserver/patches/036-denormalize-tag-tables.sql > PRE-CREATION > security-admin
Re: Review Request 67782: RANGER-2144 - refactor DAO creation
/apache/ranger/rest/TestUserREST.java 3c3051b39 security-admin/src/test/java/org/apache/ranger/service/TestRangerServiceDefService.java a36b19879 security-admin/src/test/java/org/apache/ranger/service/TestRangerServiceDefServiceBase.java 74f1dd0c4 security-admin/src/test/java/org/apache/ranger/service/TestRangerServiceService.java 163e4169f security-admin/src/test/java/org/apache/ranger/service/TestRangerServiceServiceBase.java d90e978a1 security-admin/src/test/java/org/apache/ranger/service/TestRangerServiceWithAssignedIdService.java c5221cda5 security-admin/src/test/java/org/apache/ranger/service/TestRangerTagDefService.java 8f19ffd78 security-admin/src/test/java/org/apache/ranger/service/TestRangerTagResourceMapService.java 27ec8e140 security-admin/src/test/java/org/apache/ranger/service/TestRangerTagService.java 1175989e2 security-admin/src/test/java/org/apache/ranger/service/TestUserService.java c48739b94 Diff: https://reviews.apache.org/r/67782/diff/3/ Changes: https://reviews.apache.org/r/67782/diff/2-3/ Testing --- Tested with unit tests and on a live cluster. Thanks, Zsombor Gegesy
[jira] [Updated] (RANGER-2144) Refactor DAO usage
[ https://issues.apache.org/jira/browse/RANGER-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2144: --- Attachment: RANGER-2144-3.patch > Refactor DAO usage > -- > > Key: RANGER-2144 > URL: https://issues.apache.org/jira/browse/RANGER-2144 > Project: Ranger > Issue Type: Improvement > Components: admin >Affects Versions: 1.0.0 >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Labels: code-cleanup > Attachments: RANGER-2144-3.patch > > > Currently, RangerDaoManagerBase is used to construct new DAO instances, > instead of relying on Spring to provide one for it. This would reduce the > code to write and run, and make it less interdependent, and simplifies the > tests, as less code would be needed to mock. > As RangerDaoManagerBase is used everywhere, and to avoid having huge > patches, it would be better do it in smaller steps. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (RANGER-2144) Refactor DAO usage
[ https://issues.apache.org/jira/browse/RANGER-2144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated RANGER-2144: --- Attachment: (was: RANGER-2144.patch) > Refactor DAO usage > -- > > Key: RANGER-2144 > URL: https://issues.apache.org/jira/browse/RANGER-2144 > Project: Ranger > Issue Type: Improvement > Components: admin >Affects Versions: 1.0.0 >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Labels: code-cleanup > Attachments: RANGER-2144-3.patch > > > Currently, RangerDaoManagerBase is used to construct new DAO instances, > instead of relying on Spring to provide one for it. This would reduce the > code to write and run, and make it less interdependent, and simplifies the > tests, as less code would be needed to mock. > As RangerDaoManagerBase is used everywhere, and to avoid having huge > patches, it would be better do it in smaller steps. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 68128: RANGER-2170:Ranger supports plugin to enable, monitor and manage Elasticsearch
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68128/#review207073 --- plugin-elasticsearch/src/main/java/org/apache/ranger/authorization/elasticsearch/authorizer/RangerElasticsearchAuthorizer.java Lines 99 (patched) <https://reviews.apache.org/r/68128/#comment290256> I don't get it, why this clientIPAddress is used. This is not a client IP address, but the current ElasticServer's address. This could confuse the users. Why not leaving just empty, if ES won't provide it automatically? plugin-elasticsearch/src/main/java/org/apache/ranger/services/elasticsearch/client/ElasticsearchClient.java Lines 139 (patched) <https://reviews.apache.org/r/68128/#comment290258> I guess, joptsimple.internal.Strings.EMPTY is just "". I don't think, it is good idea to depend on a library just for this - and this library is not directly included in the pom. ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/RangerElasticserachPlugin.java Lines 50 (patched) <https://reviews.apache.org/r/68128/#comment290259> RangerElasticserachPlugin -> RangerElasticsearchPlugin ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java Lines 76 (patched) <https://reviews.apache.org/r/68128/#comment290262> Isn't it possible to get the user's IP address from action.remoteAddress()? I'm not too familiar with ES inner workings. ranger-elasticsearch-plugin-shim/src/main/java/org/apache/ranger/authorization/elasticsearch/plugin/action/filter/RangerSecurityActionFilter.java Lines 88 (patched) <https://reviews.apache.org/r/68128/#comment290261> Isn't it a security problem, if no user name is there, the plugin allows everything? - Zsombor Gegesy On Aug. 1, 2018, 9:27 a.m., Qiang Zhang wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68128/ > --- > > (Updated Aug. 1, 2018, 9:27 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, > Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy. > > > Bugs: RANGER-2170 > https://issues.apache.org/jira/browse/RANGER-2170 > > > Repository: ranger > > > Description > --- > > Elasticsearch is a distributed, RESTful search and analytics engine capable > of solving a growing number of use cases. > Like Apache Solr, it is also an index server based on Lucence. > Ranger supports plugin to enable, monitor and manage Elasticsearch, > to control index security of Elasticsearch. > > As there is X-Pack plugin for the Elasticsearch, but it is not free. > X-Pack is an Elastic Stack extension that bundles security, alerting, > monitoring, reporting, > and graph capabilities into one easy-to-install package. > We refer to the Indices Privileges design of X-Pack, > by keeping the permissions consistent, > to make user use ranger Elasticsearch plugin easily. > Reference X-Pack Indices Privileges: > https://www.elastic.co/guide/en/x-pack/current/security-privileges.html > > Here we develop Ranger Elasticsearch plugin, based on Elasticsearch version > 6.2.2. > Elasticsearch 6.2.2 was released in February 20, 2018, reference > release-notes: > https://www.elastic.co/guide/en/elasticsearch/reference/6.2/release-notes-6.2.2.html > Not like other system, Elasticsearch has no basic authentication, > it uses X-pack plugin to support basic authentication, > role-based access control, SSL/TLS encryption, LDAP and so on. > Not like X-pack, our Ranger Elasticsearch plugin is designed to do > authorization, > it is to control index of Elasticsearch without authentication, > this plugin should work with other Elasticsearch plugin to authenticate users. > > > Diffs > - > > agents-common/scripts/enable-agent.sh ce0dc8c > agents-common/src/main/java/org/apache/ranger/plugin/client/BaseClient.java > e654f2b > > agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java > 5e74da8 > > agents-common/src/main/resources/service-defs/ranger-servicedef-elasticsearch.json > PRE-CREATION > plugin-elasticsearch/.gitignore PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-audit-changes.cfg > PRE-CREATION > plugin-elasticsearch/conf/ranger-elasticsearch-audit.xml PRE-CREATION &
Re: Review Request 68286: RANGER-2186: Increment service-specific policy and tag versions after update transaction is committed
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68286/#review207072
---
security-admin/src/main/java/org/apache/ranger/common/db/RangerTransactionSynchronizationAdapter.java
Lines 122 (patched)
<https://reviews.apache.org/r/68286/#comment290255>
Why this complicated is machinery with the thread locals and lists and
registration is needed?
You can easily have the same functionality - running a 'Runnable' object
after a transaction finish, with just like this:
```java
public void executeOnTransactionCommit(Runnable runnable) {
TransactionSynchronizationManager.registerSynchronization(new
TransactionSynchronizationAdapter() {
public void afterCommit() {
runnable.run();
}
}
}
```
- Zsombor Gegesy
On Aug. 10, 2018, 12:58 a.m., Abhay Kulkarni wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68286/
> ---
>
> (Updated Aug. 10, 2018, 12:58 a.m.)
>
>
> Review request for ranger, Madhan Neethiraj, Ramesh Mani, and Velmurugan
> Periasamy.
>
>
> Bugs: RANGER-2186
> https://issues.apache.org/jira/browse/RANGER-2186
>
>
> Repository: ranger
>
>
> Description
> ---
>
> Policy updates to different policies within a service, when successful,
> update the service's policy version. If the update transactions are
> concurrent, and executed on different ranger-admin servers (in HA
> configuration), then it is possible that policy-version of the transaction
> that commits later overwrites policy-version of earlier transaction,
> effectively losing track of the first change.
>
> If policy-version is updated after update to policy is committed, then the
> window of such loss is greatly reduced.
>
> Similar considerations apply for tag updates.
>
>
> Diffs
> -
>
>
> agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractServiceStore.java
> 69ded6dc8
> security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java
> 0773616f9
>
> security-admin/src/main/java/org/apache/ranger/common/db/RangerTransactionSynchronizationAdapter.java
> 2a62fb408
>
> security-admin/src/main/java/org/apache/ranger/db/XXServiceVersionInfoDao.java
> e1003297a
> security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java
> cb496ea8b
>
>
> Diff: https://reviews.apache.org/r/68286/diff/1/
>
>
> Testing
> ---
>
> Passed all unit tests
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 68288: RANGER-2177 Fix browser hangs bug
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68288/#review207071 --- Ship it! Ship It! - Zsombor Gegesy On Aug. 10, 2018, 5:43 a.m., Xing Peng wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68288/ > --- > > (Updated Aug. 10, 2018, 5:43 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O > hEigeartaigh, Gautam Borad, Madhan Neethiraj, Nitin Galave, pengjianhua, > Ramesh Mani, Selvamohan Neethiraj, sam rome, Venkat Ranganathan, Velmurugan > Periasamy, and Qiang Zhang. > > > Bugs: RANGER-2177 > https://issues.apache.org/jira/browse/RANGER-2177 > > > Repository: ranger > > > Description > --- > > We have found one bug while using this validation feature. > Case : > When we specify more than two configuration item with the same name and click > on save button, it shows validation popup but browser hangs after that( i > think some loop in code goes in infinite). > Can you check the above case? > > > Diffs > - > > security-admin/src/main/webapp/scripts/views/service/ServiceForm.js > 726fb5f3d > > > Diff: https://reviews.apache.org/r/68288/diff/1/ > > > Testing > --- > > > Thanks, > > Xing Peng > >
Re: Review Request 68290: RANGER-2185:when user has access to some of Hive databases, execute command 'show databases' will show the access databases
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68290/#review207069 --- I don't think, this is a good solution - Hive specific changes shoudn't be added to the generic plugin code. There is already a HiveAccessType.USE, and a RangerHiveAccessRequest is created if hiveOpType == HiveOperationType.SHOWDATABASES. I would rather see this accessType added to the UI, so users can create policies with database=* and accessType=USE. Adding unit test would be nice too. - Zsombor Gegesy On Aug. 10, 2018, 11:14 a.m., dhomme wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68290/ > --- > > (Updated Aug. 10, 2018, 11:14 a.m.) > > > Review request for ranger. > > > Bugs: RANGER-2185 > https://issues.apache.org/jira/browse/RANGER-2185 > > > Repository: ranger > > > Description > --- > > When a user execute 'show databases;' via hive, ranger hive plugin should > check if the user has any databases to access but not all(*). > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessResource.java > 2ee616a1b > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessResourceImpl.java > 580048627 > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessResourceReadOnly.java > 18bb1f44f > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java > 23d1efa15 > > agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java > 415263eef > > hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java > 3ed79df83 > > > Diff: https://reviews.apache.org/r/68290/diff/3/ > > > Testing > --- > > Tesetd with local env. > > > Thanks, > > dhomme > >
Re: Review Request 67782: RANGER-2144 - refactor DAO creation
/TestRangerTagService.java 1175989e2 security-admin/src/test/java/org/apache/ranger/service/TestUserService.java c48739b94 Diff: https://reviews.apache.org/r/67782/diff/2/ Changes: https://reviews.apache.org/r/67782/diff/1-2/ Testing --- Tested with unit tests and on a live cluster. Thanks, Zsombor Gegesy
Re: Review Request 68083: RANGER-2167 - Upgrade to Apache parent pom version 20
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68083/#review206730 --- Ship it! Ship It! - Zsombor Gegesy On July 27, 2018, 3:28 p.m., Colm O hEigeartaigh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68083/ > --- > > (Updated July 27, 2018, 3:28 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2167 > https://issues.apache.org/jira/browse/RANGER-2167 > > > Repository: ranger > > > Description > --- > > This task is to upgrade to the Apache parent pom version 20. This > necessitates a PMD upgrade and quite a few fixes are required as a result. > One notable change is to switch the KMS Keystore implementation to use > ConcurrentHashMap instead of HashTable. > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerValiditySchedule.java > 55f787d23 > > agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerValidator.java > ed5aa8d41 > > agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerValidityScheduleValidator.java > f34d124d6 > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceACLs.java > 34098fab6 > > agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefaultPolicyEvaluator.java > e16148d4b > > agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerValidityScheduleEvaluator.java > b48ff3b68 > > agents-common/src/test/java/org/apache/ranger/plugin/model/validation/TestRangerValidator.java > f9b342832 > dev-support/ranger-pmd-ruleset.xml f03cda371 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java > cd5a8c2a8 > > plugin-kafka/src/test/java/org/apache/ranger/authorization/kafka/authorizer/KafkaRangerAuthorizerGSSTest.java > 262447857 > > plugin-kms/src/main/java/org/apache/ranger/authorization/kms/authorizer/RangerKmsAuthorizer.java > c3d75a14d > > plugin-kms/src/main/java/org/apache/ranger/services/kms/client/KMSClient.java > af0ac71f0 > > plugin-nifi/src/main/java/org/apache/ranger/services/nifi/client/NiFiConnectionMgr.java > 739bef62c > pom.xml 8d7ea131e > > ranger-tools/src/test/java/org/apache/ranger/policyengine/RangerPolicyEnginePerformanceTest.java > 11af0a84d > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > e4449df2e > > security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java > b4a3f9355 > > security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKrbFilter.java > ca0d17eb6 > > security-admin/src/main/java/org/apache/ranger/service/RangerPolicyService.java > a3ff825dd > > security-admin/src/main/java/org/apache/ranger/service/RangerServiceService.java > df3fdb57c > security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java > b0b011022 > > > Diff: https://reviews.apache.org/r/68083/diff/1/ > > > Testing > --- > > > Thanks, > > Colm O hEigeartaigh > >
[jira] [Commented] (RANGER-1951) build problems with the saveVersion.py script
[ https://issues.apache.org/jira/browse/RANGER-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559651#comment-16559651 ] Zsombor Gegesy commented on RANGER-1951: Patch merged to [master|https://github.com/apache/ranger/commit/4308c8aecbd7c537469475e6f456c0ab043ea931] > build problems with the saveVersion.py script > - > > Key: RANGER-1951 > URL: https://issues.apache.org/jira/browse/RANGER-1951 > Project: Ranger > Issue Type: Bug > Components: build-infra >Affects Versions: 0.7.1 >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 2.0.0 > > Attachments: RANGER-1951-1.patch, RANGER-1951-2.patch > > > Currently the saveVersion.py has the following problems: > * it doesn't work with python3 due to 'inconsistent whitespace usage' and > because in python3 the byte array is different from a string > * The checksum is generated from all the java source files from > ranger-util/target, which contains at most one java file - a previously > generated ranger-util/target/gen/org/apache/ranger/common/package-info.java > * -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (RANGER-1951) build problems with the saveVersion.py script
[ https://issues.apache.org/jira/browse/RANGER-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy resolved RANGER-1951. Resolution: Fixed > build problems with the saveVersion.py script > - > > Key: RANGER-1951 > URL: https://issues.apache.org/jira/browse/RANGER-1951 > Project: Ranger > Issue Type: Bug > Components: build-infra >Affects Versions: 0.7.1 >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 2.0.0 > > Attachments: RANGER-1951-1.patch, RANGER-1951-2.patch > > > Currently the saveVersion.py has the following problems: > * it doesn't work with python3 due to 'inconsistent whitespace usage' and > because in python3 the byte array is different from a string > * The checksum is generated from all the java source files from > ranger-util/target, which contains at most one java file - a previously > generated ranger-util/target/gen/org/apache/ranger/common/package-info.java > * -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 68077: RANGER-2166: A ClassNotFound exception is thrown with atlasrest as a tag source
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68077/#review206541 --- Ship it! Ship It! - Zsombor Gegesy On July 27, 2018, 1:37 a.m., Abhay Kulkarni wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68077/ > --- > > (Updated July 27, 2018, 1:37 a.m.) > > > Review request for ranger, Madhan Neethiraj and Velmurugan Periasamy. > > > Bugs: RANGER-2166 > https://issues.apache.org/jira/browse/RANGER-2166 > > > Repository: ranger > > > Description > --- > > tagsync throws ClassNotFound exception when using REST API to download tags > from Atlas. Missing libraries containing required classes are added. > > Note that this requires fix to Atlas Client. (Addressed by ATLAS-2794) > > > Diffs > - > > src/main/assembly/tagsync.xml bc6e28b94 > > > Diff: https://reviews.apache.org/r/68077/diff/1/ > > > Testing > --- > > Ran tagsync with atlasrest and atlas as tag sources. Ensured that tags are > downloaded from Atlas and delivered to Ranger. Also ensured that kafka events > from Atlas are received and uploaded to Ranger. > > > Thanks, > > Abhay Kulkarni > >
Re: Review Request 68042: 'Email Address' search is not working properly along with other filter in user listing page, userRoles filters also needs to be improved.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68042/#review206540 --- Ship it! Ship It! - Zsombor Gegesy On July 27, 2018, 7:32 a.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68042/ > --- > > (Updated July 27, 2018, 7:32 a.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2160 > https://issues.apache.org/jira/browse/RANGER-2160 > > > Repository: ranger > > > Description > --- > > 'Email Address' search is not working properly along with other filter in > user listing page,userRoles filters also needs to be improved. > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java ea60e99 > security-admin/src/main/java/org/apache/ranger/common/RangerConstants.java > 35b9d0a > security-admin/src/main/java/org/apache/ranger/common/SearchUtil.java > 6f28e7b > > security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java > 7df0640 > security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 79f347d > security-admin/src/test/java/org/apache/ranger/rest/TestXUserREST.java > e564e28 > > > Diff: https://reviews.apache.org/r/68042/diff/2/ > > > Testing > --- > > Tested if email id filter & user roles along with other filters are working > properly. > > > Thanks, > > Nikhil P > >
Re: Review Request 68042: 'Email Address' search is not working properly along with other filter in user listing page, userRoles filters also needs to be improved.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68042/#review206496 --- security-admin/src/main/java/org/apache/ranger/common/RangerConstants.java Lines 175 (patched) <https://reviews.apache.org/r/68042/#comment289474> This could be "final", and maybe Set, as only the "contains" method is used. - Zsombor Gegesy On July 25, 2018, 2:46 p.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68042/ > --- > > (Updated July 25, 2018, 2:46 p.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2160 > https://issues.apache.org/jira/browse/RANGER-2160 > > > Repository: ranger > > > Description > --- > > 'Email Address' search is not working properly along with other filter in > user listing page,userRoles filters also needs to be improved. > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 7f5eab7 > security-admin/src/main/java/org/apache/ranger/common/RangerConstants.java > 35b9d0a > security-admin/src/main/java/org/apache/ranger/common/SearchUtil.java > 6f28e7b > > security-admin/src/main/java/org/apache/ranger/patch/cliutil/RoleBasedUserSearchUtil.java > 7df0640 > security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 79f347d > security-admin/src/test/java/org/apache/ranger/rest/TestXUserREST.java > e564e28 > > > Diff: https://reviews.apache.org/r/68042/diff/1/ > > > Testing > --- > > Tested if email id filter & user roles along with other filters are working > properly. > > > Thanks, > > Nikhil P > >
Re: Review Request 67835: RANGER-2150: Unit test coverage for XUserMgr and UserMgr class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67835/#review205842 --- Ship it! Ship It! - Zsombor Gegesy On júl. 7, 2018, 5:45 de, Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/67835/ > --- > > (Updated júl. 7, 2018, 5:45 de) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Selvamohan Neethiraj, > Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2150 > https://issues.apache.org/jira/browse/RANGER-2150 > > > Repository: ranger > > > Description > --- > > **Problem Statement:** Unit test coverage for XUserMgr and UserMgr class need > to be increased. > > Currently Unit Test coverage of XUserMgr is 33.5% and UserMgr class is 54.9% > Total No. of test cases in TestXUserMgr class are: 37 > Total No. of test cases in TestUserMgr class are: 23 > > **Proposed Solution:** > Proposed patch contains lot of additional unit test cases and modification in > existing unit test cases. > Statistics after the patch are: > Test coverage of XUserMgr is 90.2% and UserMgr class is 85.3% > Total No. of test cases in TestXUserMgr class are: 100 > Total No. of test cases in TestUserMgr class are: 41 > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java 6d94e4f > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 7f5eab7 > security-admin/src/test/java/org/apache/ranger/biz/TestUserMgr.java 10453a5 > security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java > 39c60a9 > > > Diff: https://reviews.apache.org/r/67835/diff/2/ > > > Testing > --- > > Tested on my local environment by using command : > > mvn clean compile package assembly:assembly install > > [INFO] Reactor Summary: > [INFO] > [INFO] ranger . SUCCESS [ 9.399 > s] > [INFO] Jdbc SQL Connector . SUCCESS [ 2.039 > s] > [INFO] Credential Support . SUCCESS [ 0.792 > s] > [INFO] Audit Component SUCCESS [ 2.001 > s] > [INFO] Common library for Plugins . SUCCESS [ 3.299 > s] > [INFO] Installer Support Component SUCCESS [ 0.398 > s] > [INFO] Credential Builder . SUCCESS [ 1.066 > s] > [INFO] Embedded Web Server Invoker SUCCESS [ 1.243 > s] > [INFO] Key Management Service . SUCCESS [ 2.776 > s] > [INFO] ranger-plugin-classloader .. SUCCESS [ 0.463 > s] > [INFO] HBase Security Plugin Shim . SUCCESS [ 2.522 > s] > [INFO] HBase Security Plugin .. SUCCESS [ 1.758 > s] > [INFO] Hdfs Security Plugin ... SUCCESS [ 1.822 > s] > [INFO] Hive Security Plugin ... SUCCESS [ 7.668 > s] > [INFO] Knox Security Plugin Shim .. SUCCESS [ 1.452 > s] > [INFO] Knox Security Plugin ... SUCCESS [ 4.083 > s] > [INFO] Storm Security Plugin .. SUCCESS [ 1.046 > s] > [INFO] YARN Security Plugin ... SUCCESS [ 0.924 > s] > [INFO] Ranger Util SUCCESS [ 7.301 > s] > [INFO] Unix Authentication Client . SUCCESS [ 1.258 > s] > [INFO] Security Admin Web Application . SUCCESS [01:31 > min] > [INFO] KAFKA Security Plugin .. SUCCESS [ 4.827 > s] > [INFO] SOLR Security Plugin ... SUCCESS [ 5.235 > s] > [INFO] NiFi Security Plugin ... SUCCESS [ 0.950 > s] > [INFO] Unix User Group Synchronizer ... SUCCESS [03:34 > min] > [INFO] Ldap Config Check Tool . SUCCESS [ 2.540 > s] > [INFO] Unix Authentication Service SUCCESS [ 4.010 > s] > [INFO] KMS Security Plugin SUCCESS [ 5.922 > s] > [INFO] Tag Synchronizer ... SUCCESS [ 12.373 > s] > [INFO
Re: Review Request 67835: RANGER-2150: Unit test coverage for XUserMgr and UserMgr class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/67835/#review205764 --- security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java Line 621 (original), 621 (patched) <https://reviews.apache.org/r/67835/#comment288646> No need to make it public, the test class is in the same package, so package protected would be fine. security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java Line 813 (original), 813 (patched) <https://reviews.apache.org/r/67835/#comment288648> No need to make it public, the test class is in the same package, so package protected would be fine. security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java Line 1186 (original), 1186 (patched) <https://reviews.apache.org/r/67835/#comment288649> No need to make it public, the test class is in the same package, so package protected would be fine. security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java Line 328 (original), 328 (patched) <https://reviews.apache.org/r/67835/#comment288650> No need to make it public, the test class is in the same package, so package protected would be fine. security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java Lines 637 (patched) <https://reviews.apache.org/r/67835/#comment288651> The vxGroupUserList = new VXGroupUserList() assignment is unnecessary, as it is reassigned in the next line security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java Line 875 (original), 870 (patched) <https://reviews.apache.org/r/67835/#comment288652> No need to make it public, the test class is in the same package, so package protected would be fine. security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java Line 1270 (original) <https://reviews.apache.org/r/67835/#comment288653> returnList is assigned in the next lines - no need to declare to be null. security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java Line 2258 (original), 2251 (patched) <https://reviews.apache.org/r/67835/#comment288655> No need to make it public, the test class is in the same package, so package protected would be fine. security-admin/src/test/java/org/apache/ranger/biz/TestUserMgr.java Lines 191 (patched) <https://reviews.apache.org/r/67835/#comment288659> You can annotate this method with @After and JUnit will call it automatically after every test method, no need to call it by hand. - Zsombor Gegesy On July 5, 2018, 8:48 p.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/67835/ > --- > > (Updated July 5, 2018, 8:48 p.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, Selvamohan Neethiraj, > Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2150 > https://issues.apache.org/jira/browse/RANGER-2150 > > > Repository: ranger > > > Description > --- > > **Problem Statement:** Unit test coverage for XUserMgr and UserMgr class need > to be increased. > > Currently Unit Test coverage of XUserMgr is 33.5% and UserMgr class is 54.9% > Total No. of test cases in TestXUserMgr class are: 37 > Total No. of test cases in TestUserMgr class are: 23 > > **Proposed Solution:** > Proposed patch contains lot of additional unit test cases and modification in > existing unit test cases. > Statistics after the patch are: > Test coverage of XUserMgr is 90.2% and UserMgr class is 85.3% > Total No. of test cases in TestXUserMgr class are: 100 > Total No. of test cases in TestUserMgr class are: 41 > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java 6d94e4f > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 7f5eab7 > security-admin/src/test/java/org/apache/ranger/biz/TestUserMgr.java 10453a5 > security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java > 39c60a9 > > > Diff: https://reviews.apache.org/r/67835/diff/1/ > > > Testing > --- > > Tested on my local environment by using command : > > mvn clean compile package assembly:assembly install > > [INFO] Reactor Summary: > [INFO] > [INFO] ranger . SUCCESS [ 9.399 > s] > [INFO] Jdbc SQL Connector . SUCCESS [ 2.039 > s] > [INFO] Credential Support . SUCCESS [ 0.792 > s] >
