[jira] [Assigned] (RANGER-2325) Implement ranger plugin for Ozone

[Sailaja Polavarapu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-2325:
--

Assignee: Sailaja Polavarapu

> Implement ranger plugin for Ozone
> -
>
> Key: RANGER-2325
> URL: https://issues.apache.org/jira/browse/RANGER-2325
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins, Ranger
>Reporter: Sailaja Polavarapu
>Assignee: Sailaja Polavarapu
>Priority: Major
> Fix For: 2.0.0
>
>
> Ozone is an object store in HDFS. Implement ranger plugin for authorization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (RANGER-2325) Implement ranger plugin for Ozone

[Sailaja Polavarapu (JIRA)]

Sailaja Polavarapu created RANGER-2325:
--

 Summary: Implement ranger plugin for Ozone
 Key: RANGER-2325
 URL: https://issues.apache.org/jira/browse/RANGER-2325
 Project: Ranger
  Issue Type: New Feature
  Components: plugins, Ranger
Reporter: Sailaja Polavarapu
 Fix For: 2.0.0


Ozone is an object store in HDFS. Implement ranger plugin for authorization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Question on Ranger policy?

[Don Bosco Durai]

Ramesh, this looks good. 

Thanks

Bosco


On 1/23/19, 12:57 PM, "Ramesh Mani"  wrote:

Bosco,

There were some efforts done on this. Please check this out
https://github.com/jjmeyer0/ranger-client

This might help too.

Thanks,
Ramesh

On 1/23/19, 11:20 AM, "Don Bosco Durai"  wrote:

>I got a similar request from someone else also. I feel, lot of them might
>have already wrote their custom code to manage Ranger policies. We can
>see if someone wants to contribute it back to the community.
>
>Reetika, can you can create a JIRA to track this?
>
>Thanks
>
>Bosco
>
>
>On 1/22/19, 3:37 PM, "Ramesh Mani"  wrote:
>
>Hi Reetika Agrawal,
>
>Apache Ranger only has REST APIs for the policy operations.
>
>Thanks,
>Ramesh
>
>
>On 1/19/19, 9:11 PM, "reetika agrawal" 
>wrote:
>
>>Hi,
>>Is there any ranger java client available to get all the ranger
>policies
>>instead of using API calls?
>>
>>-- 
>>Thanks,
>>Reetika Agrawal
>
>
>
>

Re: Question on Ranger policy?

[Ramesh Mani]

Bosco,

There were some efforts done on this. Please check this out
https://github.com/jjmeyer0/ranger-client

This might help too.

Thanks,
Ramesh

On 1/23/19, 11:20 AM, "Don Bosco Durai"  wrote:

>I got a similar request from someone else also. I feel, lot of them might
>have already wrote their custom code to manage Ranger policies. We can
>see if someone wants to contribute it back to the community.
>
>Reetika, can you can create a JIRA to track this?
>
>Thanks
>
>Bosco
>
>
>On 1/22/19, 3:37 PM, "Ramesh Mani"  wrote:
>
>Hi Reetika Agrawal,
>
>Apache Ranger only has REST APIs for the policy operations.
>
>Thanks,
>Ramesh
>
>
>On 1/19/19, 9:11 PM, "reetika agrawal" 
>wrote:
>
>>Hi,
>>Is there any ranger java client available to get all the ranger
>policies
>>instead of using API calls?
>>
>>-- 
>>Thanks,
>>Reetika Agrawal
>
>
>
>

Re: Question on Ranger policy?

[Don Bosco Durai]

Zsombor

 

This looks good. Do you which jar will have the client library? And does it 
generate the JavaDoc for the Java REST API?

 

Thanks

 

Bosco

 

 

From: "Zs." 
Reply-To: 
Date: Wednesday, January 23, 2019 at 12:40 PM
To: ranger 
Cc: "u...@ranger.apache.org" 
Subject: Re: Question on Ranger policy?

 

There is an Enunciate plugin configured, which generates client libraries 
during build time - unfortunately, it's not published anywhere, you have to 
build Ranger by yourself, and I guess, it's not heavily used.

 

Regards,

 Zsombor

 

On Wed, Jan 23, 2019 at 8:20 PM Don Bosco Durai  wrote:

I got a similar request from someone else also. I feel, lot of them might have 
already wrote their custom code to manage Ranger policies. We can see if 
someone wants to contribute it back to the community.

Reetika, can you can create a JIRA to track this?

Thanks

Bosco


On 1/22/19, 3:37 PM, "Ramesh Mani"  wrote:

Hi Reetika Agrawal,

Apache Ranger only has REST APIs for the policy operations.

Thanks,
Ramesh


On 1/19/19, 9:11 PM, "reetika agrawal" 
wrote:

>Hi,
>Is there any ranger java client available to get all the ranger policies
>instead of using API calls?
>
>-- 
>Thanks,
>Reetika Agrawal

Re: Question on Ranger policy?

[Zs.]

There is an Enunciate  plugin
configured, which generates client libraries during build time -
unfortunately, it's not published anywhere, you have to build Ranger by
yourself, and I guess, it's not heavily used.

Regards,
 Zsombor

On Wed, Jan 23, 2019 at 8:20 PM Don Bosco Durai  wrote:

> I got a similar request from someone else also. I feel, lot of them might
> have already wrote their custom code to manage Ranger policies. We can see
> if someone wants to contribute it back to the community.
>
> Reetika, can you can create a JIRA to track this?
>
> Thanks
>
> Bosco
>
>
> On 1/22/19, 3:37 PM, "Ramesh Mani"  wrote:
>
> Hi Reetika Agrawal,
>
> Apache Ranger only has REST APIs for the policy operations.
>
> Thanks,
> Ramesh
>
>
> On 1/19/19, 9:11 PM, "reetika agrawal" 
> wrote:
>
> >Hi,
> >Is there any ranger java client available to get all the ranger
> policies
> >instead of using API calls?
> >
> >--
> >Thanks,
> >Reetika Agrawal
>
>
>
>
>

Re: Review Request 69703: RANGER-2232: Security Zones feature in Apache Ranger

[Abhay Kulkarni]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69703/
---

(Updated Jan. 23, 2019, 7:39 p.m.)


Review request for ranger, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin 
Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan 
Periasamy.


Changes
---

Fixed Security Zone validation code and merged with master branch


Bugs: RANGER-2232
https://issues.apache.org/jira/browse/RANGER-2232


Repository: ranger


Description
---

This is to introduce a new abstraction in Apache Ranger that would allow 
carving/bucketing of resources in a service into multiple zones, for better 
administration of security policies. This would enable multiple administrators 
to setup security policies for a service – based on the zones to which they 
have been granted administration rights. 

For example, let us consider 2 security zones ‘finance’ and ‘sales’:

Security zone ‘finance’ includes all contents in Hive database named ‘finance’ 
Security zone ‘sales’ includes all contents in ‘sales’ database 
Set of users and groups are designated as administrators each zone 
Users are allowed to setup policies only in zones in which they are 
administrators 
Policies defined in a zone are applicable only for resources of the zone
A zone can be extended to include resource from multiple services like HDFS, 
Hive, HBase, Kafka, .., allowing administrators of a zone to setup policies for 
resources owned by their organization across multiple services.
Audit logs will include name of the zone in which the accessed resource 
resides. Only users having appropriate permissions on the security zone can 
view its audit logs.


Diffs (updated)
-

  
agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java
 329e2f0b7 
  agents-audit/src/main/java/org/apache/ranger/audit/model/AuthzAuditEvent.java 
8d71851e8 
  
agents-audit/src/main/java/org/apache/ranger/audit/provider/solr/SolrAuditProvider.java
 26633fd6e 
  
agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerDefaultAuditHandler.java
 b8da19215 
  
agents-common/src/main/java/org/apache/ranger/plugin/errors/ValidationErrorCode.java
 9b9ccd112 
  agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicy.java 
c2185a7f1 
  
agents-common/src/main/java/org/apache/ranger/plugin/model/RangerPolicyResourceSignature.java
 b56b8dd4b 
  
agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java
 ddedf3e17 
  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidator.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerValidator.java
 51324b093 
  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerZoneResourceMatcher.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequestReadOnly.java
 891749d03 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessResult.java
 8e7844f5d 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngine.java
 e6c0e5a94 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
 ab26d41d6 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyRepository.java
 f64e773ac 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyresourcematcher/RangerDefaultPolicyResourceMatcher.java
 c1b29d3fa 
  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
 b898d292c 
  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
 7221f6b15 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractPredicateUtil.java
 7446df604 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/SecurityZonePredicateUtil.java
 PRE-CREATION 
  
agents-common/src/main/java/org/apache/ranger/plugin/store/SecurityZoneStore.java
 PRE-CREATION 
  agents-common/src/main/java/org/apache/ranger/plugin/store/ServiceStore.java 
9924cb4c4 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/GrantRevokeRequest.java
 f4fe58993 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTUtils.java 
efb27aafa 
  agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java 
33f82dd34 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/ServicePolicies.java 
1ae3fc387 
  
agents-common/src/test/java/org/apache/ranger/plugin/model/TestRangerPolicyResourceSignature.java
 38c425dc6 
  
agents-common/src/test/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidatorTest.java
 PRE-CREATION 
  
hbase-agent/src/main/

Re: Question on Ranger policy?

[Don Bosco Durai]

I got a similar request from someone else also. I feel, lot of them might have 
already wrote their custom code to manage Ranger policies. We can see if 
someone wants to contribute it back to the community.

Reetika, can you can create a JIRA to track this?

Thanks

Bosco


On 1/22/19, 3:37 PM, "Ramesh Mani"  wrote:

Hi Reetika Agrawal,

Apache Ranger only has REST APIs for the policy operations.

Thanks,
Ramesh


On 1/19/19, 9:11 PM, "reetika agrawal" 
wrote:

>Hi,
>Is there any ranger java client available to get all the ranger policies
>instead of using API calls?
>
>-- 
>Thanks,
>Reetika Agrawal

[jira] [Commented] (RANGER-2324) Bootstrapping Solr in Ranger service start-up

[Don Bosco Durai (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750238#comment-16750238
 ] 

Don Bosco Durai commented on RANGER-2324:
-

[~bpatel] can we have some more detail on what you are planning to do here? 
Thanks

> Bootstrapping Solr in Ranger service start-up
> -
>
> Key: RANGER-2324
> URL: https://issues.apache.org/jira/browse/RANGER-2324
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: bhavik patel
>Assignee: bhavik patel
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69730: RANGER-2320: Make db schema patches script idempotent for all DB Flavors

[Velmurugan Periasamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69730/#review212239
---


Ship it!




Ship It!

- Velmurugan Periasamy


On Jan. 23, 2019, 8:54 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69730/
> ---
> 
> (Updated Jan. 23, 2019, 8:54 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2320
> https://issues.apache.org/jira/browse/RANGER-2320
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-2291 covers changes only in optimized DB schema script, existing DB 
> patches script should be reviewed and if required these scripts also should 
> be made idempotent.
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/patches/009-updated_schema.sql 14ec1d872 
>   security-admin/db/mysql/patches/013-permissionmodel.sql 1c6bcfd13 
>   security-admin/db/mysql/patches/020-datamask-policy.sql 87c0a183a 
>   security-admin/db/mysql/patches/030-policy-labels-schema.sql 33af2a858 
>   security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql 
> 84db52604 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> 98c45b05d 
>   security-admin/db/postgres/patches/020-datamask-policy.sql 795721c99 
>   security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql 
> 61c7986e4 
>   security-admin/db/sqlserver/patches/020-datamask-policy.sql c50d2b83d 
>   security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql 
> bd646d695 
> 
> 
> Diff: https://reviews.apache.org/r/69730/diff/2/
> 
> 
> Testing
> ---
> 
> Tested the upgrade case from ranger-0.7 to ranger-2.0 for mysql, postgres and 
> sqlserver db.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code

[Velmurugan Periasamy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69677/#review212238
---


Ship it!




Ship It!

- Velmurugan Periasamy


On Jan. 22, 2019, 1:47 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69677/
> ---
> 
> (Updated Jan. 22, 2019, 1:47 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2287
> https://issues.apache.org/jira/browse/RANGER-2287
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** There are lot of repeated code in db_setup.py which 
> can be removed which shall help developers to make any changes in db_setup.py 
> in future.
> 
> **Proposed Solution:** Proposed patch shall remove the db setup methods of 
> each db flavor and shall use a single method for a specific work for each db 
> flavor. Based on the db flavor, config values shall be populated and handled 
> in the code after this patch.
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/db_setup.py f1223b38c 
> 
> 
> Diff: https://reviews.apache.org/r/69677/diff/3/
> 
> 
> Testing
> ---
> 
> **Use Cases covered for all the db flavors:**
> *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of 
> ranger admin.
> *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same 
> db config on Ranger 2.0 installation config and run the setup.sh. Ranger was 
> upgraded successfully.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Created] (RANGER-2324) Bootstrapping Solr in Ranger service start-up

[bhavik patel (JIRA)]

bhavik patel created RANGER-2324:


 Summary: Bootstrapping Solr in Ranger service start-up
 Key: RANGER-2324
 URL: https://issues.apache.org/jira/browse/RANGER-2324
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: bhavik patel
Assignee: bhavik patel






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (RANGER-2234) Cannot add or update a child row,a foreign key constraint fails when installing ranger-admin

[Haihui Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haihui Xu reassigned RANGER-2234:
-

Assignee: Akash Pawale  (was: Haihui Xu)

> Cannot add or update a child row,a foreign key constraint fails when 
> installing ranger-admin
> 
>
> Key: RANGER-2234
> URL: https://issues.apache.org/jira/browse/RANGER-2234
> Project: Ranger
>  Issue Type: Bug
>  Components: admin, Ranger
>Affects Versions: 1.1.0
>Reporter: Haihui Xu
>Assignee: Akash Pawale
>Priority: Major
>  Labels: patch
> Fix For: 2.0.0
>
> Attachments: RANGER-2234-02.patch, RANGER-2234-03.patch, 
> RANGER-2234_Cannot add or update a child row,a foreign key constraint fails 
> when installing ranger-admin.patch
>
>
> Installing ranger-admin use mysql as the database,execute setup.sh, in 
> progress something happend, the error logs are as flowing:
> 2018-09-26 17:17:08,539 [I] Table xa_access_audit does not exist in database 
> ranger
> 2018-09-26 17:17:08,539 [I] Importing db schema to database ranger from file: 
> ranger_core_db_mysql.sql
> 2018-09-26 17:17:08,540 [JISQL] /home/ranger/jdk1.8.0_121/bin/java -cp 
> /usr/share/java/mysql-connector-java.jar:/home/ranger/ranger-1.1.0-admin/jisql/lib/*
>  org.apache.util.sql.Jisql -driver mysqlconj -cstring 
> jdbc:mysql://10.139.16.75/ranger -u 'root' -p '' -noheader -trim -c 
> \; -input 
> /home/ranger/ranger-1.1.0-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> Error executing: INSERT INTO 
> x_portal_user_role(create_time,update_time,added_by_id,upd_by_id,user_id,user_role,status)
>  VALUES (UTC_TIMESTAMP(),UTC_TIMESTAMP(),NULL,NULL,2,'ROLE_SYS_ADMIN',1); 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`))
> SQLException : SQL state: 23000 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`)) ErrorCode: 1452
> 2018-09-26 17:22:20,882 [E] ranger_core_db_mysql.sql file import failed!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (RANGER-2234) Cannot add or update a child row,a foreign key constraint fails when installing ranger-admin

[Haihui Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haihui Xu reassigned RANGER-2234:
-

Assignee: Haihui Xu  (was: Akash Pawale)

> Cannot add or update a child row,a foreign key constraint fails when 
> installing ranger-admin
> 
>
> Key: RANGER-2234
> URL: https://issues.apache.org/jira/browse/RANGER-2234
> Project: Ranger
>  Issue Type: Bug
>  Components: admin, Ranger
>Affects Versions: 1.1.0
>Reporter: Haihui Xu
>Assignee: Haihui Xu
>Priority: Major
>  Labels: patch
> Fix For: 2.0.0
>
> Attachments: RANGER-2234-02.patch, RANGER-2234-03.patch, 
> RANGER-2234_Cannot add or update a child row,a foreign key constraint fails 
> when installing ranger-admin.patch
>
>
> Installing ranger-admin use mysql as the database,execute setup.sh, in 
> progress something happend, the error logs are as flowing:
> 2018-09-26 17:17:08,539 [I] Table xa_access_audit does not exist in database 
> ranger
> 2018-09-26 17:17:08,539 [I] Importing db schema to database ranger from file: 
> ranger_core_db_mysql.sql
> 2018-09-26 17:17:08,540 [JISQL] /home/ranger/jdk1.8.0_121/bin/java -cp 
> /usr/share/java/mysql-connector-java.jar:/home/ranger/ranger-1.1.0-admin/jisql/lib/*
>  org.apache.util.sql.Jisql -driver mysqlconj -cstring 
> jdbc:mysql://10.139.16.75/ranger -u 'root' -p '' -noheader -trim -c 
> \; -input 
> /home/ranger/ranger-1.1.0-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> Error executing: INSERT INTO 
> x_portal_user_role(create_time,update_time,added_by_id,upd_by_id,user_id,user_role,status)
>  VALUES (UTC_TIMESTAMP(),UTC_TIMESTAMP(),NULL,NULL,2,'ROLE_SYS_ADMIN',1); 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`))
> SQLException : SQL state: 23000 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`)) ErrorCode: 1452
> 2018-09-26 17:22:20,882 [E] ranger_core_db_mysql.sql file import failed!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2287) Improve and optimize db_setup.py file code

[Zsombor Gegesy (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749768#comment-16749768
 ] 

Zsombor Gegesy commented on RANGER-2287:


Awesome, you removed nearly 4000 lines of code, I'm impressed !

> Improve and optimize db_setup.py file code
> --
>
> Key: RANGER-2287
> URL: https://issues.apache.org/jira/browse/RANGER-2287
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: 
> 0001-RANGER-2287-Improve-and-optimize-db_setup.py-file-co.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code

[Zsombor Gegesy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69677/#review212231
---


Ship it!




Ship It!

- Zsombor Gegesy


On Jan. 22, 2019, 1:47 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69677/
> ---
> 
> (Updated Jan. 22, 2019, 1:47 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2287
> https://issues.apache.org/jira/browse/RANGER-2287
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** There are lot of repeated code in db_setup.py which 
> can be removed which shall help developers to make any changes in db_setup.py 
> in future.
> 
> **Proposed Solution:** Proposed patch shall remove the db setup methods of 
> each db flavor and shall use a single method for a specific work for each db 
> flavor. Based on the db flavor, config values shall be populated and handled 
> in the code after this patch.
> 
> 
> Diffs
> -
> 
>   security-admin/scripts/db_setup.py f1223b38c 
> 
> 
> Diff: https://reviews.apache.org/r/69677/diff/3/
> 
> 
> Testing
> ---
> 
> **Use Cases covered for all the db flavors:**
> *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of 
> ranger admin.
> *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same 
> db config on Ranger 2.0 installation config and run the setup.sh. Ranger was 
> upgraded successfully.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Commented] (RANGER-2234) Cannot add or update a child row,a foreign key constraint fails when installing ranger-admin

[Pradeep Agrawal (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749685#comment-16749685
 ] 

Pradeep Agrawal commented on RANGER-2234:
-

Patch committed to master branch : 
https://github.com/apache/ranger/commit/14d20f88ad012ed5054dc81583be7419b553688c

> Cannot add or update a child row,a foreign key constraint fails when 
> installing ranger-admin
> 
>
> Key: RANGER-2234
> URL: https://issues.apache.org/jira/browse/RANGER-2234
> Project: Ranger
>  Issue Type: Bug
>  Components: admin, Ranger
>Affects Versions: 1.1.0
>Reporter: Haihui Xu
>Assignee: Akash Pawale
>Priority: Major
>  Labels: patch
> Fix For: 2.0.0
>
> Attachments: RANGER-2234-02.patch, RANGER-2234-03.patch, 
> RANGER-2234_Cannot add or update a child row,a foreign key constraint fails 
> when installing ranger-admin.patch
>
>
> Installing ranger-admin use mysql as the database,execute setup.sh, in 
> progress something happend, the error logs are as flowing:
> 2018-09-26 17:17:08,539 [I] Table xa_access_audit does not exist in database 
> ranger
> 2018-09-26 17:17:08,539 [I] Importing db schema to database ranger from file: 
> ranger_core_db_mysql.sql
> 2018-09-26 17:17:08,540 [JISQL] /home/ranger/jdk1.8.0_121/bin/java -cp 
> /usr/share/java/mysql-connector-java.jar:/home/ranger/ranger-1.1.0-admin/jisql/lib/*
>  org.apache.util.sql.Jisql -driver mysqlconj -cstring 
> jdbc:mysql://10.139.16.75/ranger -u 'root' -p '' -noheader -trim -c 
> \; -input 
> /home/ranger/ranger-1.1.0-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> Error executing: INSERT INTO 
> x_portal_user_role(create_time,update_time,added_by_id,upd_by_id,user_id,user_role,status)
>  VALUES (UTC_TIMESTAMP(),UTC_TIMESTAMP(),NULL,NULL,2,'ROLE_SYS_ADMIN',1); 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`))
> SQLException : SQL state: 23000 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`)) ErrorCode: 1452
> 2018-09-26 17:22:20,882 [E] ranger_core_db_mysql.sql file import failed!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (RANGER-2234) Cannot add or update a child row,a foreign key constraint fails when installing ranger-admin

[Pradeep Agrawal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pradeep Agrawal resolved RANGER-2234.
-
Resolution: Fixed

> Cannot add or update a child row,a foreign key constraint fails when 
> installing ranger-admin
> 
>
> Key: RANGER-2234
> URL: https://issues.apache.org/jira/browse/RANGER-2234
> Project: Ranger
>  Issue Type: Bug
>  Components: admin, Ranger
>Affects Versions: 1.1.0
>Reporter: Haihui Xu
>Assignee: Akash Pawale
>Priority: Major
>  Labels: patch
> Fix For: 2.0.0
>
> Attachments: RANGER-2234-02.patch, RANGER-2234-03.patch, 
> RANGER-2234_Cannot add or update a child row,a foreign key constraint fails 
> when installing ranger-admin.patch
>
>
> Installing ranger-admin use mysql as the database,execute setup.sh, in 
> progress something happend, the error logs are as flowing:
> 2018-09-26 17:17:08,539 [I] Table xa_access_audit does not exist in database 
> ranger
> 2018-09-26 17:17:08,539 [I] Importing db schema to database ranger from file: 
> ranger_core_db_mysql.sql
> 2018-09-26 17:17:08,540 [JISQL] /home/ranger/jdk1.8.0_121/bin/java -cp 
> /usr/share/java/mysql-connector-java.jar:/home/ranger/ranger-1.1.0-admin/jisql/lib/*
>  org.apache.util.sql.Jisql -driver mysqlconj -cstring 
> jdbc:mysql://10.139.16.75/ranger -u 'root' -p '' -noheader -trim -c 
> \; -input 
> /home/ranger/ranger-1.1.0-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> Error executing: INSERT INTO 
> x_portal_user_role(create_time,update_time,added_by_id,upd_by_id,user_id,user_role,status)
>  VALUES (UTC_TIMESTAMP(),UTC_TIMESTAMP(),NULL,NULL,2,'ROLE_SYS_ADMIN',1); 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`))
> SQLException : SQL state: 23000 
> com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
> Cannot add or update a child row: a foreign key constraint fails 
> (`ranger`.`x_portal_user_role`, CONSTRAINT `x_portal_user_role_FK_user_id` 
> FOREIGN KEY (`user_id`) REFERENCES `x_portal_user` (`id`)) ErrorCode: 1452
> 2018-09-26 17:22:20,882 [E] ranger_core_db_mysql.sql file import failed!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69730: RANGER-2320: Make db schema patches script idempotent for all DB Flavors

[Pradeep Agrawal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69730/
---

(Updated Jan. 23, 2019, 8:54 a.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
Nikhil P, Ramesh Mani, and Velmurugan Periasamy.


Changes
---

updated patch after syncing the local ranger repo with the latest ranger master.


Bugs: RANGER-2320
https://issues.apache.org/jira/browse/RANGER-2320


Repository: ranger


Description
---

RANGER-2291 covers changes only in optimized DB schema script, existing DB 
patches script should be reviewed and if required these scripts also should be 
made idempotent.


Diffs (updated)
-

  security-admin/db/mysql/patches/009-updated_schema.sql 14ec1d872 
  security-admin/db/mysql/patches/013-permissionmodel.sql 1c6bcfd13 
  security-admin/db/mysql/patches/020-datamask-policy.sql 87c0a183a 
  security-admin/db/mysql/patches/030-policy-labels-schema.sql 33af2a858 
  security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql 84db52604 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
98c45b05d 
  security-admin/db/postgres/patches/020-datamask-policy.sql 795721c99 
  security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql 
61c7986e4 
  security-admin/db/sqlserver/patches/020-datamask-policy.sql c50d2b83d 
  security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql 
bd646d695 


Diff: https://reviews.apache.org/r/69730/diff/2/

Changes: https://reviews.apache.org/r/69730/diff/1-2/


Testing
---

Tested the upgrade case from ranger-0.7 to ranger-2.0 for mysql, postgres and 
sqlserver db.


Thanks,

Pradeep Agrawal