Re: Review Request 66085: RANGER-2019: Handle upgrade scenario to rename the old ATLAS service def and use the new service def

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66085/#review199382
---




security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java
Lines 37 (patched)


Consider using already existing 
EmbeddedServiceDefsUtil.EMBEDDED_SERVICEDEF_ATLAS_NAME, instead of defining new 
constant SERVICEDBSTORE_SERVICEDEFBYNAME_ATLAS_NAME.



security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java
Lines 90 (patched)


Why is it critical to bail out when "serviceId != 11"? If the intention is 
to avoid rename on multiple runs of this patch, then I would suggest to check 
for resource-names and access-types to determine if the service-def is older 
version or not. If the service-def has all resources and accessTypes listed 
below, then it is an older service-def and should be renamed:

 - resources: entity, type, operation, taxonomy, term
 - accessTypes: read, create, update, delete, all



security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java
Lines 91 (patched)


add a info level log here. 

  LOG.info(EMBEDDED_SERVICEDEF_ATLAS_NAME + ": service-def not found. No 
patching is needed");



security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java
Lines 95 (patched)


Consider renaming service-def first. The suffix used to rename the 
service-def should be used to rename the services as well.

String  serviceDefName = EMBEDDED_SERVICEDEF_ATLAS_NAME;
XXServiceDefDao serviceDefDao  = daoMgr.getXXServiceDef();
XXServiceDefserviceDef = serviceDefDao.findByName(serviceDefName);

if (serviceDef == null) {
  LOG.info(serviceDefName + ": service-def not found. No patching is 
needed");
  
  return;
}

String suffix = null;

for (int i = 1; true; i++) {
  suffix = ".v" + i;

  if (serviceDefDao.findByName(serviceDefName + suffix) == null) {
break;
  }
}

String serviceDefNewName = serviceDefName + suffix;

LOG.info("Renaming service-def " + serviceDefName + " as " + 
serviceDefNewName);

serviceDef.setName(serviceDefNewName);

serviceDefDao.update(serviceDef);

LOG.info("Renamed service-def " + serviceDefName + " as " + 
serviceDefNewName);

XXServiceDaoserviceDao = daoMgr.getXXService();
List services   = 
serviceDao.findByServiceDefId(serviceDef.getId());

if (CollectionUtils.isNotEmpty(services)) {
  for (XXService service : services) {
String serviceName= service.getName();
String serviceNewName = serviceName + suffix;

LOG.info("Renaming service " + serviceName + " as " + serviceNewName);

if (serviceDao.findByName(serviceNewName) != null) {
  LOG.warn("Another service named " + serviceNewName + " already 
exists. Not renaming " + serviceName);
  
  continue;
}

service.setName(serviceNewName);

serviceDao.update(service);

LOG.info("Renamed service " + serviceName + " as " + serviceNewName);
  }
}



security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java
Lines 103 (patched)


When would this be false? Should that be considered as a patch failure 
(instead of ignoring it)? Atleast a warning should be logged.



security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java
Lines 104 (patched)


"Renamed service " + service.getName() + " to " updateServiceName



security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java
Lines 111 (patched)


When would this be false? I think failing to rename should be considered as 
a patch failure.


- Madhan Neethiraj


On March 17, 2018, 9:33 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66085/
> ---
> 
> (Updated March 17, 2018, 9:33 p.m.)
> 
> 
> Review request for ranger, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan 
> Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-2019
> https://issues.apache.org/jira/browse/RANGER-2019
> 
> 
> 

Re: Review Request 66102: tagsync updates to handle name-services in federated hdfs

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66102/#review199381
---


Fix it, then Ship it!





tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHdfsResourceMapper.java
Lines 64 (patched)


It will be helpful to see sample configurations, as shown below, here and 
in ranger-tagsync-site.xml:

## service-name for HDFS in 'cl1' cluster, that service nameService 'ns1'

ranger.tagsync.atlas.hdfs.instance.cl1.nameservice.ns1.ranger.service=cl1_hdfs_ns1

## service-name for HDFS in 'cl2' cluster, with no specific nameService
ranger.ragsync.atlas.hdfs.instance.cl2.ranger.service=cl2_hdfs


- Madhan Neethiraj


On March 15, 2018, 10:15 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66102/
> ---
> 
> (Updated March 15, 2018, 10:15 p.m.)
> 
> 
> Review request for ranger, Apoorv Naik, Madhan Neethiraj, Vishal Suvagia, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2024
> https://issues.apache.org/jira/browse/RANGER-2024
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> HDFS entity contained in entity notification includes a name-service-id in 
> case a federated hdfs file/directory is associated with a classification(i.e. 
> tag). tagsync needs to be able to handle such entities and map them correctly 
> to appropriate service-resource in Ranger.
> 
> 
> Diffs
> -
> 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHdfsResourceMapper.java
>  378542cd8 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasResourceMapper.java
>  5d067a5b4 
>   
> tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHdfsResourceMapper.java
>  627690405 
> 
> 
> Diff: https://reviews.apache.org/r/66102/diff/1/
> 
> 
> Testing
> ---
> 
> Developed and passed unit tests.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Review Request 66085: RANGER-2019: Handle upgrade scenario to rename the old ATLAS service def and use the new service def

[Pradeep Agrawal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66085/
---

Review request for ranger, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan 
Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan 
Periasamy.


Bugs: RANGER-2019
https://issues.apache.org/jira/browse/RANGER-2019


Repository: ranger


Description
---

**Problem Statement:** Based on this RANGER-2004 commit, Atlas service def is 
updated. During the upgrade of existing env, old Atlas service def should be 
renamed so that new service def can be used.

**Proposed Solution:** Proposed Java patch can address the atlas service def 
rename fix. During migration to latest Ranger, this Java patch should do the 
following:
1. Check for service-def named 'atlas'. If it is not found - nothing more to do.
2. Rename all service instances of type 'atlas', with addition of suffix "-v1". 
Existing service named 'cl1_atlas' will become 'cl1_atlas-v1'. If 
'cl1_atlas-v1' already exists, append an count like 'cl1_atlas-v1.1', 
'cl1_atlas-v1.2'
3. After all service instances are renamed, rename service-def as 'atlas-v1'.

**Note:** In Upgrade case, as the old atlas service def id can't be used to 
create the new atlas service def so I am proposing change to update the atlas 
servicedef id from 11 to 15.


Diffs
-

  agents-common/src/main/resources/service-defs/ranger-servicedef-atlas.json 
2891129 
  
security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/66085/diff/1/


Testing
---

Tested for fresh and upgrade case.


Thanks,

Pradeep Agrawal

Re: Atlas policies to filter access

[Madhan Neethiraj]

Colm,

Authorizations are in place for following operations on entity:
 - create/update/delete/get
 - add/update/remove of classifications

Requirements on filtering of search results based on authorizations needs to be 
discussed further. If you have usecases, can you please file an Atlas JIRA?

Madhan



On 3/15/18, 6:19 AM, "Colm O hEigeartaigh"  wrote:

I was able to get past the problems in the previous mail. I'm trying to use
the v2 API but authorization doesn't seem to kick in:

curl -v -u username:password
http://localhost:21000/api/atlas/v2/search/basic?typeName=hdfs_path

This call succeeds without the Ranger plugin being called. In
'conf/application.properties' I have "atlas.authorizer.impl = RANGER" and
there are no obvious errors in the logs. Is the Atlas authorization
framework integrated with the newer REST API yet?

Colm.

On Tue, Mar 13, 2018 at 5:49 PM, Colm O hEigeartaigh 
wrote:

> Thanks Madhan. Just to clarify - ATLAS-2459 is not yet applied, so do I
> have to apply this manually to get this to work?
>
> When trying to install the current Ranger 1.1.0-SNAPSHOT plugin with the
> latest Atlas SNAPSHOT distribution I see an error in application.log:
>
> java.lang.NoClassDefFoundError: org/codehaus/jackson/jaxrs/
> JacksonJsonProvider
> at org.apache.ranger.plugin.util.RangerRESTClient.buildClient(
> RangerRESTClient.java:209)
>
> I'm wondering if there is a conflict between the jackson-jaxrs-1.9.13.jar
> in the Atlas plugin lib and the version of Jackson used in Atlas?
>
> Colm.
>
> On Mon, Mar 12, 2018 at 9:14 PM, Madhan Neethiraj 
> wrote:
>
>> Colm,
>>
>> Perhaps you are using the Atlas service-def from Ranger master, against
>> Atlas from branch-0.8 (or from master before ATLAS-2459)? Earlier Atlas
>> versions use a different authorization model, which don't allow access
>> controls at instance/type levels. Please try with Atlas from master 
branch.
>>
>> Hope this helps.
>>
>> Madhan
>>
>>
>>
>>
>> On 3/12/18, 11:16 AM, "Colm O hEigeartaigh"  wrote:
>>
>> Hi all,
>>
>> I'm using the Ranger plugin to secure access to Atlas. How can I
>> create a
>> policy in Ranger to allow a user access to a subset of the entities?
>> So for
>> example, I want to allow "alice" to "read" all entities that have a
>> given
>> type. I created an authorization policy of "type" "Table", but I get
>> the
>> following error:
>>
>> curl -u alice:password "http://localhost:21000/api/at
>> las/entities?type=Table
>> "
>> Error 403 {AuthorizationError:You are not
>> authorized for READ on [ENTITY] : *}
>>
>> How can I allow authorization for a subset of the entities? I guess I
>> need
>> an authorization policy for "Entity" but it's not clear what values
>> apart
>> from "*" are supported here?
>>
>> Colm.
>>
>>
>> --
>> Colm O hEigeartaigh
>>
>> Talend Community Coder
>> http://coders.talend.com
>>
>>
>>
>>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
>



-- 
Colm O hEigeartaigh

Talend Community Coder
http://coders.talend.com

Re: Review Request 66105: tagsync updates to handle hbase namespace entity notification

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66105/#review199378
---


Fix it, then Ship it!





tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHbaseResourceMapper.java
Lines 73 (patched)


tblName => nameSpaceName


- Madhan Neethiraj


On March 15, 2018, 10:25 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66105/
> ---
> 
> (Updated March 15, 2018, 10:25 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Ramesh Mani.
> 
> 
> Bugs: RANGER-2025
> https://issues.apache.org/jira/browse/RANGER-2025
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Updated tagsync to handle notification about a Hbase namespace is associated 
> with a classification (tag).
> 
> 
> Diffs
> -
> 
>   
> tagsync/src/main/java/org/apache/ranger/tagsync/source/atlas/AtlasHbaseResourceMapper.java
>  33e804a20 
>   
> tagsync/src/test/java/org/apache/ranger/tagsync/process/TestHbaseResourceMapper.java
>  499247c41 
> 
> 
> Diff: https://reviews.apache.org/r/66105/diff/1/
> 
> 
> Testing
> ---
> 
> Developed and passed unit tests.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 66107: Evaluate grantor's group membership in the plugin for grant/revoke request

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66107/#review199376
---


Ship it!




Ship It!

- Madhan Neethiraj


On March 15, 2018, 11:07 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66107/
> ---
> 
> (Updated March 15, 2018, 11:07 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-2027
> https://issues.apache.org/jira/browse/RANGER-2027
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Instead of computing group membership in the ranger-admin, it is better to 
> compute it in the plugin using component's user identity manager and relayed 
> to ranger-admin in the grant/revoke request. Ranger-Admin's user identify 
> manager may be used to compute group membership only as a fallback for 
> backward compatibility.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/util/GrantRevokeRequest.java
>  0c5b2d80b 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
>  12b675b45 
>   
> hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java
>  2c2a51866 
>   security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 
> dad8a975e 
> 
> 
> Diff: https://reviews.apache.org/r/66107/diff/1/
> 
> 
> Testing
> ---
> 
> Passed all unit tests
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 66106: Update Hbase plugin to handle default namespace

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66106/#review199375
---


Fix it, then Ship it!





hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/AuthorizationSession.java
Line 188 (original), 188 (patched)


Consider replacing strings "table", "column-family" and "column" with 
following constants introduced in this patch:
  
  RangerHBaseResource.KEY_TABLE
  RangerHBaseResource.KEY_COLUMN_FAMILY
  RangerHBaseResource.KEY_COLUMNS



hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerHbaseResource.java
Lines 31 (patched)


Consider renaming "RangerHbaseResource" to "RangerHBaseResource".



hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerHbaseResource.java
Lines 36 (patched)


Please update for indentation (tabs/spaces?)



hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerHbaseResource.java
Lines 52 (patched)


Consider the following rewrite, to make it a little easier to read:

public void setValue(String key, Object value) {
  // special handling for tables in 'default' namespace
  if (StringUtils.equals(key, KEY_TABLE)) {
if (value instanceof String) {
  String tableName = (String) value;

  if (!tableName.contains(NAMESPACE_SEPARATOR)) {
List tableNames = new ArrayList<>(2);

tableNames.add(tableName);
tableNames.add(DEFAULT_NAMESPACE + tableName);

value = tableNames;
  } else if (StringUtil.startsWith(tableName, DEFAULT_NAMESPACE) {
List tableNames = new ArrayList<>(2);

tableNames.add(tableName.substring(DEFAULT_NAMESPACE.length());
tableNames.add(tableName);

value = tableNames;
  }
}
  }
  
  super.setValue(key, value);
}


- Madhan Neethiraj


On March 16, 2018, 11:08 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66106/
> ---
> 
> (Updated March 16, 2018, 11:08 p.m.)
> 
> 
> Review request for ranger and Madhan Neethiraj.
> 
> 
> Bugs: RANGER-2026
> https://issues.apache.org/jira/browse/RANGER-2026
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> HBase supports a "default" namespace in which Hbase tables are implicitly 
> present when namespace is not explicitly specified. Ranger Hbase policies 
> with table-resource specified with or without a "default" namespace qualifier 
> should match if a table in default namespace is accessed with or without 
> "default" namespace qualifier.
> 
> 
> Diffs
> -
> 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/AuthorizationSession.java
>  1349aef9e 
>   
> hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerHbaseResource.java
>  PRE-CREATION 
>   
> hbase-agent/src/test/java/org/apache/ranger/authorization/hbase/HBaseRangerAuthorizationTest.java
>  665640f07 
>   
> hbase-agent/src/test/java/org/apache/ranger/authorization/hbase/TestPolicyEngine.java
>  9f0e5ac87 
>   hbase-agent/src/test/resources/hbase-policies-tag.json 61728c800 
>   hbase-agent/src/test/resources/policyengine/test_policyengine_hbase.json 
> f563c2850 
> 
> 
> Diff: https://reviews.apache.org/r/66106/diff/2/
> 
> 
> Testing
> ---
> 
> Passed all unit tests.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: [VOTE] Apache Ranger Release 1.0.0-rc1

[Selvamohan Neethiraj]

+1

Validated Signature
Validated Checksum
Build the source - successfully

Thanks,
Selva-


> On Mar 14, 2018, at 9:27 PM, Sailaja Polavarapu  
> wrote:
> 
> Rangers:
> Thank you for your contribution to Apache Ranger community. Apache Ranger 
> 1.0.0 release candidate #1 is now available for a vote within dev community. 
> Links to release artifacts are given below. I kindly request all of the 
> Rangers (dev & PMC members) to review and vote on this release.
> 
> Git tag for the release:
> 
> https://github.com/apache/ranger/tree/release-1.0.0-rc1 (last commit id:  
> c79aad4362b6f806da47462ec84e35108ba8eb1f)
> 
> 
> Sources for the release:
> https://dist.apache.org/repos/dist/dev/ranger/1.0.0-rc1/apache-ranger-1.0.0.tar.gz
> 
> 
> Source release verification:
> 
> PGP Signature:
> 
> https://dist.apache.org/repos/dist/dev/ranger/1.0.0-rc1/apache-ranger-1.0.0.tar.gz.asc
> 
> 
> MD5/SHA Hash:
> 
> https://dist.apache.org/repos/dist/dev/ranger/1.0.0-rc1/apache-ranger-1.0.0.tar.gz.mds
> 
> 
> Keys to verify the signature of the release artifact are available at:
> 
> https://dist.apache.org/repos/dist/release/ranger/KEYS
> 
> 
> Release Notes:
> 
>   https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75975356
> 
> 
> Build verification steps can be found at:
> 
>   http://ranger.apache.org/quick_start_guide.html
> 
> 
> The vote will be open for at least 72 hours or until necessary number of 
> votes are reached.
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
> 
> Here is my +1
> 
> Thanks,
> Sailaja.
> 



signature.asc
Description: Message signed with OpenPGP