Re: Review Request 74122: RANGER-3914: Change sync_source column's datatype from varchar to text

2022-09-15 Thread Pradeep Agrawal 


> On Sept. 15, 2022, 5:41 p.m., Madhan Neethiraj wrote:
> > security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> > Line 120 (original), 120 (patched)
> > 
> >
> > Pradeep - while at this, I suggest similar change for other columns as 
> > well: notes, other_attributes.

Thanks for suggestion. Though there is no problem with those attributes yet i 
would also prefer to change them (at least for fresh install because existing 
install will not be having any issue with that).


- Pradeep


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74122/#review224674
---


On Sept. 15, 2022, 5:08 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74122/
> ---
> 
> (Updated Sept. 15, 2022, 5:08 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Nikhil P, 
> Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3914
> https://issues.apache.org/jira/browse/RANGER-3914
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:**
> Ranger install or upgrade may fail with below error when mysql engine having 
> database charset set to utfmb4.
> 
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: 
> Row size too large. The maximum row size for the used table type, not 
> counting BLOBs, is 65535. 
> This includes storage overhead, check the manual. You have to change some 
> columns to TEXT or BLOBs
> 
> SQLException : SQL state: 42000 
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Row size too 
> large. The maximum row size for the used table type, not counting BLOBs, is 
> 65535. This includes storage overhead, check the manual. You have to change 
> some columns to TEXT or BLOBs ErrorCode: 1118
> 
> **Proposed solution:**
> Since BLOBs are not counted in the calculation of row size we can change 
> sync_source column type to Blob/text/clob type.
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 309c4196b 
>   
> security-admin/db/mysql/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  5a40a89e0 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> 1af0a04ac 
>   
> security-admin/db/oracle/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  ed8dbaf7f 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> baa6288d2 
>   
> security-admin/db/postgres/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  68117a123 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  5e8070ba2 
>   
> security-admin/db/sqlanywhere/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  335459058 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> 8b5833eae 
>   
> security-admin/db/sqlserver/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  3b8a8d83a 
> 
> 
> Diff: https://reviews.apache.org/r/74122/diff/1/
> 
> 
> Testing
> ---
> 
> Tested Ranger install and upgrade with this patch which was failing earlier.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Comment Edited] (RANGER-3847) [Ranger] : Http status & Error message is not correct for /xaudit/trx_log

2022-09-15 Thread Ramachandran (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605131#comment-17605131
 ] 

Ramachandran edited comment on RANGER-3847 at 9/16/22 3:18 AM:
---

In Ranger rest API if you pass an Invalid attribute type in the JSON request 
body,  it throws a 404 error instead of 400 (Bad Request). It seems a bug where 
we need to handle the JsonMappingException by adding Exception Mapper globally 
to handle incorrect JSON format or Invalid attribute type in the JSON request 
body otherwise our Rest API will throw a 404 error instead of 400 error cc >> 
[~mad...@apache.org]  [~abhayk]  [~rmani] .

 


was (Author: JIRAUSER295265):
In Ranger rest API if you pass an Invalid attribute type in the JSON request 
body,  it throws a 404 error instead of 400 (Bad Request). It seems a bug where 
we need to handle the JsonMappingException by adding Exception Mapper cc >> 
[~mad...@apache.org] 

 

> [Ranger] : Http status & Error message is not correct for /xaudit/trx_log
> -
>
> Key: RANGER-3847
> URL: https://issues.apache.org/jira/browse/RANGER-3847
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Anupam Rai
>Priority: Minor
>
> [Ranger] : Http status & Error message is not correct for /xaudit/trx_log
> Request with bad request : Passing String values
> {code:java}
> curl --location --request POST 'https://-/service/xaudit/trx_log' \
> --header 'Authorization: Basic X==' \
> --header 'Accept: application/json' \
> --header 'Content-Type: application/json' \
> --data-raw '{
>   "objectClassType" : "abcdef",
>   "objectId" : "abcdef"
> }' {code}
> Response : 404 Not Found
> Request with proper request : Passing int values
> {code:java}
> curl --location --request POST 
> 'https:///service/xaudit/trx_log' \
> --header 'Authorization: Basic =' \
> --header 'Accept: application/json' \
> --header 'Content-Type: application/json' \
> --data-raw '{
>   "objectClassType" : "123456",
>   "objectId" : "1234567"
> }' {code}
> Response : ok 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74122: RANGER-3914: Change sync_source column's datatype from varchar to text

2022-09-15 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74122/#review224674
---


Fix it, then Ship it!





security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
Line 120 (original), 120 (patched)


Pradeep - while at this, I suggest similar change for other columns as 
well: notes, other_attributes.


- Madhan Neethiraj


On Sept. 15, 2022, 5:08 p.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74122/
> ---
> 
> (Updated Sept. 15, 2022, 5:08 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Nikhil P, 
> Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3914
> https://issues.apache.org/jira/browse/RANGER-3914
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:**
> Ranger install or upgrade may fail with below error when mysql engine having 
> database charset set to utfmb4.
> 
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: 
> Row size too large. The maximum row size for the used table type, not 
> counting BLOBs, is 65535. 
> This includes storage overhead, check the manual. You have to change some 
> columns to TEXT or BLOBs
> 
> SQLException : SQL state: 42000 
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Row size too 
> large. The maximum row size for the used table type, not counting BLOBs, is 
> 65535. This includes storage overhead, check the manual. You have to change 
> some columns to TEXT or BLOBs ErrorCode: 1118
> 
> **Proposed solution:**
> Since BLOBs are not counted in the calculation of row size we can change 
> sync_source column type to Blob/text/clob type.
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 309c4196b 
>   
> security-admin/db/mysql/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  5a40a89e0 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> 1af0a04ac 
>   
> security-admin/db/oracle/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  ed8dbaf7f 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> baa6288d2 
>   
> security-admin/db/postgres/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  68117a123 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  5e8070ba2 
>   
> security-admin/db/sqlanywhere/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  335459058 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> 8b5833eae 
>   
> security-admin/db/sqlserver/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  3b8a8d83a 
> 
> 
> Diff: https://reviews.apache.org/r/74122/diff/1/
> 
> 
> Testing
> ---
> 
> Tested Ranger install and upgrade with this patch which was failing earlier.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Commented] (RANGER-2928) [Ranger Zone REST API] Resources data is missing in XML format

2022-09-15 Thread Ramachandran (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605453#comment-17605453
 ] 

Ramachandran commented on RANGER-2928:
--

[~madhan]  I believe Most probably everyone uses JSON format only. We can not 
guarantee that all the rest API works with XML support due to 
marshalling/unmarshalling not happening properly.

Better we keep only application/json and remove application/xml support from 
the code to make it consistent.

[~Shukla] Are there compelling usecases to support XML format?
 
 

> [Ranger Zone REST API] Resources data is missing in XML format
> --
>
> Key: RANGER-2928
> URL: https://issues.apache.org/jira/browse/RANGER-2928
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Abhishek Shukla
>Priority: Minor
>  Labels: ranger
> Attachments: RangerSecurityZoneServiceJAXBDemo.zip
>
>
> Observed that the following Ranger Zone API is missing resources in response 
> section for XML format.
> https://:6182/service/public/v2/api/zones/
> {noformat}
> 
> 
> 
> 2
> true
> Admin
> Admin
> 2020-07-28T10:15:44Z
> 2020-07-28T10:17:52Z
> test_zone_1
> 
> 
> cm_ozone
> 
> 
> 
> 
> 
> cm_hdfs
> 
> 
> 
> 
> 
> admin
> public
> 
> 
> {noformat}
>  
> While the same is present for JSON response.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Review Request 74122: RANGER-3914: Change sync_source column's datatype from varchar to text

2022-09-15 Thread Pradeep Agrawal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74122/
---

Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Nikhil P, Pradeep 
Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan 
Periasamy.


Bugs: RANGER-3914
https://issues.apache.org/jira/browse/RANGER-3914


Repository: ranger


Description
---

**Problem Statement:**
Ranger install or upgrade may fail with below error when mysql engine having 
database charset set to utfmb4.

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: 
Row size too large. The maximum row size for the used table type, not counting 
BLOBs, is 65535. 
This includes storage overhead, check the manual. You have to change some 
columns to TEXT or BLOBs

SQLException : SQL state: 42000 
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Row size too large. 
The maximum row size for the used table type, not counting BLOBs, is 65535. 
This includes storage overhead, check the manual. You have to change some 
columns to TEXT or BLOBs ErrorCode: 1118

**Proposed solution:**
Since BLOBs are not counted in the calculation of row size we can change 
sync_source column type to Blob/text/clob type.


Diffs
-

  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 309c4196b 
  
security-admin/db/mysql/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
 5a40a89e0 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
1af0a04ac 
  
security-admin/db/oracle/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
 ed8dbaf7f 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
baa6288d2 
  
security-admin/db/postgres/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
 68117a123 
  
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 
5e8070ba2 
  
security-admin/db/sqlanywhere/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
 335459058 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
8b5833eae 
  
security-admin/db/sqlserver/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
 3b8a8d83a 


Diff: https://reviews.apache.org/r/74122/diff/1/


Testing
---

Tested Ranger install and upgrade with this patch which was failing earlier.


Thanks,

Pradeep Agrawal

[jira] [Commented] (RANGER-2928) [Ranger Zone REST API] Resources data is missing in XML format

2022-09-15 Thread Madhan Neethiraj (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605449#comment-17605449
 ] 

Madhan Neethiraj commented on RANGER-2928:
--

[~ramackri]  - given JSON is the only format used by Ranger plugins and servers 
for REST APIs, I suggest to deprecate/remove support for XML format. I would 
recommend to not add any more dependencies (or code) to support XML format.

Are there compelling usecases to support XML format?

> [Ranger Zone REST API] Resources data is missing in XML format
> --
>
> Key: RANGER-2928
> URL: https://issues.apache.org/jira/browse/RANGER-2928
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Abhishek Shukla
>Priority: Minor
>  Labels: ranger
> Attachments: RangerSecurityZoneServiceJAXBDemo.zip
>
>
> Observed that the following Ranger Zone API is missing resources in response 
> section for XML format.
> https://:6182/service/public/v2/api/zones/
> {noformat}
> 
> 
> 
> 2
> true
> Admin
> Admin
> 2020-07-28T10:15:44Z
> 2020-07-28T10:17:52Z
> test_zone_1
> 
> 
> cm_ozone
> 
> 
> 
> 
> 
> cm_hdfs
> 
> 
> 
> 
> 
> admin
> public
> 
> 
> {noformat}
>  
> While the same is present for JSON response.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (RANGER-3914) Change sync_source column's datatype from varchar to text

2022-09-15 Thread Pradeep Agrawal (Jira) 
Pradeep Agrawal created RANGER-3914:
---

 Summary: Change sync_source column's datatype from varchar to text
 Key: RANGER-3914
 URL: https://issues.apache.org/jira/browse/RANGER-3914
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Affects Versions: 2.3.0, 2.2.0
Reporter: Pradeep Agrawal
Assignee: Pradeep Agrawal
 Fix For: 3.0.0, 2.4.0


Ranger install on mysql engine with utf8mb4 may hit the below issue due to 
sync_source column in x_portal_user table. 

[https://dev.mysql.com/doc/refman/8.0/en/column-count-limit.html#:~:text=The%20internal%20representation%20of%20a,the%20rest%20of%20the%20row].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (RANGER-2928) [Ranger Zone REST API] Resources data is missing in XML format

2022-09-15 Thread Ramachandran (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605437#comment-17605437
 ] 

Ramachandran edited comment on RANGER-2928 at 9/15/22 4:19 PM:
---

 
{code:java}
[
  {
    "id": 2,
    "isEnabled": true,
    "createdBy": "Admin",
    "updatedBy": "Admin",
    "createTime": 1663240837000,
    "updateTime": 1663244581000,
    "name": "test",
    "services": {
      "test": {
        "resources": [
          {
            "database": [
              "db",
              "db2"
            ],
            "column": [
              "col1",
              "col2"
            ],
            "table": [
              "table1",
              "table2"
            ]
          }
        ]
      }
    },
    "tagServices": [
      "test"
    ],
    "adminUsers": [
      "admin",
      "keyadmin"
    ],
    "adminUserGroups": [
      "public",
      "test"
    ],
    "auditUsers": [
      "admin",
      "keyadmin"
    ],
    "auditUserGroups": [
      "public",
      "test"
    ],
    "description": "test"
  }
] 
 {code}
 

The above mentioned json response populating resources correctly.

While using content-type is application/xml , we are facing issues 

During marshalling, the below is not getting mapped 
List>> resources;  

[https://github.com/apache/ranger/blob/master/agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java#L135]

It seems we need to add XmlAdapter to convert between unmappable and mappable 
objects

Refer:
[http://blog.bdoughan.com/2010/07/xmladapter-jaxbs-secret-weapon.html]

I did a POC for this  via a standalone java project and attached it here 
[^RangerSecurityZoneServiceJAXBDemo.zip] cc >> [~mad...@apache.org]  [~abhayk]  
[~rmani]


was (Author: JIRAUSER295265):
 
{code:java}
[
  {
    "id": 2,
    "isEnabled": true,
    "createdBy": "Admin",
    "updatedBy": "Admin",
    "createTime": 1663240837000,
    "updateTime": 1663244581000,
    "name": "test",
    "services": {
      "test": {
        "resources": [
          {
            "database": [
              "db",
              "db2"
            ],
            "column": [
              "col1",
              "col2"
            ],
            "table": [
              "table1",
              "table2"
            ]
          }
        ]
      }
    },
    "tagServices": [
      "test"
    ],
    "adminUsers": [
      "admin",
      "keyadmin"
    ],
    "adminUserGroups": [
      "public",
      "test"
    ],
    "auditUsers": [
      "admin",
      "keyadmin"
    ],
    "auditUserGroups": [
      "public",
      "test"
    ],
    "description": "test"
  }
] 
 {code}
 

The above mentioned json response populating resources correct.

While using content-type is application/xml , we are facing issues 

During marshalling, the below is not getting mapped 
List>> resources;  

[https://github.com/apache/ranger/blob/master/agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java#L135]

It seems we need to add XmlAdapter to convert between unmappable and mappable 
objects

Refer:
[http://blog.bdoughan.com/2010/07/xmladapter-jaxbs-secret-weapon.html]

I did a POC for this  via a standalone java project and attached it here 
[^RangerSecurityZoneServiceJAXBDemo.zip] cc >> [~mad...@apache.org]  [~abhayk]  
[~rmani]

> [Ranger Zone REST API] Resources data is missing in XML format
> --
>
> Key: RANGER-2928
> URL: https://issues.apache.org/jira/browse/RANGER-2928
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Abhishek Shukla
>Priority: Minor
>  Labels: ranger
> Attachments: RangerSecurityZoneServiceJAXBDemo.zip
>
>
> Observed that the following Ranger Zone API is missing resources in response 
> section for XML format.
> https://:6182/service/public/v2/api/zones/
> {noformat}
> 
> 
> 
> 2
> true
> Admin
> Admin
> 2020-07-28T10:15:44Z
> 2020-07-28T10:17:52Z
> test_zone_1
> 
> 
> cm_ozone
> 
> 
> 
> 
> 
> cm_hdfs
> 
> 
> 
> 
> 
> admin
> public
> 
> 
> {noformat}
>  
> While the same is present for JSON response.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-2928) [Ranger Zone REST API] Resources data is missing in XML format

2022-09-15 Thread Ramachandran (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605437#comment-17605437
 ] 

Ramachandran commented on RANGER-2928:
--

 
{code:java}
[
  {
    "id": 2,
    "isEnabled": true,
    "createdBy": "Admin",
    "updatedBy": "Admin",
    "createTime": 1663240837000,
    "updateTime": 1663244581000,
    "name": "test",
    "services": {
      "test": {
        "resources": [
          {
            "database": [
              "db",
              "db2"
            ],
            "column": [
              "col1",
              "col2"
            ],
            "table": [
              "table1",
              "table2"
            ]
          }
        ]
      }
    },
    "tagServices": [
      "test"
    ],
    "adminUsers": [
      "admin",
      "keyadmin"
    ],
    "adminUserGroups": [
      "public",
      "test"
    ],
    "auditUsers": [
      "admin",
      "keyadmin"
    ],
    "auditUserGroups": [
      "public",
      "test"
    ],
    "description": "test"
  }
] 
 {code}
 

The above mentioned json response populating resources correct.

While using content-type is application/xml , we are facing issues 

During marshalling, the below is not getting mapped 
List>> resources;  

[https://github.com/apache/ranger/blob/master/agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java#L135]

It seems we need to add XmlAdapter to convert between unmappable and mappable 
objects

Refer:
[http://blog.bdoughan.com/2010/07/xmladapter-jaxbs-secret-weapon.html]

I did a POC for this  via a standalone java project and attached it here 
[^RangerSecurityZoneServiceJAXBDemo.zip] cc >> [~mad...@apache.org]  [~abhayk]  
[~rmani]

> [Ranger Zone REST API] Resources data is missing in XML format
> --
>
> Key: RANGER-2928
> URL: https://issues.apache.org/jira/browse/RANGER-2928
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Abhishek Shukla
>Priority: Minor
>  Labels: ranger
> Attachments: RangerSecurityZoneServiceJAXBDemo.zip
>
>
> Observed that the following Ranger Zone API is missing resources in response 
> section for XML format.
> https://:6182/service/public/v2/api/zones/
> {noformat}
> 
> 
> 
> 2
> true
> Admin
> Admin
> 2020-07-28T10:15:44Z
> 2020-07-28T10:17:52Z
> test_zone_1
> 
> 
> cm_ozone
> 
> 
> 
> 
> 
> cm_hdfs
> 
> 
> 
> 
> 
> admin
> public
> 
> 
> {noformat}
>  
> While the same is present for JSON response.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-2928) [Ranger Zone REST API] Resources data is missing in XML format

2022-09-15 Thread Ramachandran (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramachandran updated RANGER-2928:
-
Attachment: RangerSecurityZoneServiceJAXBDemo.zip

> [Ranger Zone REST API] Resources data is missing in XML format
> --
>
> Key: RANGER-2928
> URL: https://issues.apache.org/jira/browse/RANGER-2928
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Abhishek Shukla
>Priority: Minor
>  Labels: ranger
> Attachments: RangerSecurityZoneServiceJAXBDemo.zip
>
>
> Observed that the following Ranger Zone API is missing resources in response 
> section for XML format.
> https://:6182/service/public/v2/api/zones/
> {noformat}
> 
> 
> 
> 2
> true
> Admin
> Admin
> 2020-07-28T10:15:44Z
> 2020-07-28T10:17:52Z
> test_zone_1
> 
> 
> cm_ozone
> 
> 
> 
> 
> 
> cm_hdfs
> 
> 
> 
> 
> 
> admin
> public
> 
> 
> {noformat}
>  
> While the same is present for JSON response.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (RANGER-3719) Can not create mysql table with charset utf8mb4.

2022-09-15 Thread Pradeep Agrawal (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pradeep Agrawal reassigned RANGER-3719:
---

Assignee: Pradeep Agrawal

> Can not create mysql table with charset utf8mb4.
> 
>
> Key: RANGER-3719
> URL: https://issues.apache.org/jira/browse/RANGER-3719
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments: 0001-Add-UnitTest-for-mysql-create-database.patch
>
>
> Mysql have a limitation - "{*}The maximum row size for the used table type, 
> not counting BLOBs, is 65535"{*}
> Our mysql table use too many "VARCHAR(4000)", if mysql charset is "utf8mb4", 
> it will exceed the limitation of mysql.
>  
> For example
> {code:java}
> ]# mysql
> MariaDB [(none)]> create database ranger_utf8mb4 default charset utf8mb4;
> Query OK, 1 row affected (0.000 sec)
> MariaDB [(none)]> use ranger_utf8mb4
> Database changed
> MariaDB [ranger_utf8mb4]> source optimized/current/ranger_core_db_mysql.sql
> ...
> ERROR 1118 (42000) at line 104 in file: 
> 'optimized/current/ranger_core_db_mysql.sql': Row size too large. The maximum 
> row size for the used table type, not counting BLOBs, is 65535. This includes 
> storage overhead, check the manual. You have to change some columns to TEXT 
> or BLOBs
> ...
>  104 CREATE TABLE `x_portal_user` (
>  105   `id` bigint(20) NOT NULL AUTO_INCREMENT,
>  106   `create_time` datetime DEFAULT NULL,
>  107   `update_time` datetime DEFAULT NULL,
>  108   `added_by_id` bigint(20) DEFAULT NULL,
>  109   `upd_by_id` bigint(20) DEFAULT NULL,
>  110   `first_name` varchar(1022) DEFAULT NULL,
>  111   `last_name` varchar(1022) DEFAULT NULL,
>  112   `pub_scr_name` varchar(2048) DEFAULT NULL,
>  113   `login_id` varchar(767) DEFAULT NULL,
>  114   `password` varchar(512) NOT NULL,
>  115   `email` varchar(512) DEFAULT NULL,
>  116   `status` int(11) NOT NULL DEFAULT '0',
>  117   `user_src` int(11) NOT NULL DEFAULT '0',
>  118   `notes` varchar(4000) DEFAULT NULL,
>  119   `other_attributes` varchar(4000) DEFAULT NULL,
>  120   `sync_source` varchar(4000) DEFAULT NULL,
>  121   PRIMARY KEY (`id`),
>  122   UNIQUE KEY `x_portal_user_UK_login_id` (`login_id`),
>  123   UNIQUE KEY `x_portal_user_UK_email` (`email`),
>  124   KEY `x_portal_user_FK_added_by_id` (`added_by_id`),
>  125   KEY `x_portal_user_FK_upd_by_id` (`upd_by_id`),
>  126   KEY `x_portal_user_cr_time` (`create_time`),
>  127   KEY `x_portal_user_up_time` (`update_time`),
>  128   KEY `x_portal_user_name` (`first_name`(767)),
>  129   KEY `x_portal_user_email` (`email`),
>  130   CONSTRAINT `x_portal_user_FK_added_by_id` FOREIGN KEY (`added_by_id`) 
> REFERENCES `x_portal_user` (`id`),
>  131   CONSTRAINT `x_portal_user_FK_upd_by_id` FOREIGN KEY (`upd_by_id`) 
> REFERENCES `x_portal_user` (`id`)
>  132 ) ROW_FORMAT=DYNAMIC;
> {code}
> My suggestion is to either change all fields that are not indexed to TEXT, or 
> fix the character set of 'create database' to utf8mb3.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (RANGER-3913) Reduce number of calls to FilenameUtils.wildcardMatch() when evaluating resource matching

2022-09-15 Thread Abhay Kulkarni (Jira) 
Abhay Kulkarni created RANGER-3913:
--

 Summary: Reduce number of calls to FilenameUtils.wildcardMatch() 
when evaluating resource matching
 Key: RANGER-3913
 URL: https://issues.apache.org/jira/browse/RANGER-3913
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: Abhay Kulkarni
Assignee: Abhay Kulkarni


When evaluating resource matching for a path-type resource in a recursive 
policy containing wildcards, several calls may be made to Hadoop library 
function FilenameUtils.wildcardMatch(). These calls may be expensive and 
minimizing such calls may help improve policy evaluation performance.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-2778) Create Ranger Service Def for S3

2022-09-15 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-2778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605246#comment-17605246
 ] 

Bhavik Patel commented on RANGER-2778:
--

Anyone has already completed this task as a part of RANGER-1300 ?

> Create Ranger Service Def for S3
> 
>
> Key: RANGER-2778
> URL: https://issues.apache.org/jira/browse/RANGER-2778
> Project: Ranger
>  Issue Type: Task
>  Components: plugins
>Reporter: Bosco
>Assignee: Bosco
>Priority: Major
>
> Create a Ranger ServiceDef for Ranger Plugin which can be implemented  by 
> different providers



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-1300) S3 support

2022-09-15 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605244#comment-17605244
 ] 

Bhavik Patel commented on RANGER-1300:
--

[~syedatifakhtar] [~bolke]  are you working on this?

> S3 support
> --
>
> Key: RANGER-1300
> URL: https://issues.apache.org/jira/browse/RANGER-1300
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins
>Reporter: Jose
>Priority: Major
> Attachments: ranger-servicedef-aws-s3.json
>
>
> As more and more people are deploying hadoop into AWS and as S3 is used in 
> lots of application. It'd be nice to have S3 support built into Ranger.
> It's not a trivial task. Right now Ranger Storage support (only hdfs) runs 
> directly in the Namenode



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-3009) Query with column mask fail with ParseException if column name has special char

2022-09-15 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605238#comment-17605238
 ] 

Bhavik Patel commented on RANGER-3009:
--

Anyone is observing this issue?

> Query with column mask fail with ParseException if column name has special 
> char
> ---
>
> Key: RANGER-3009
> URL: https://issues.apache.org/jira/browse/RANGER-3009
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.1.0
>Reporter: Rajkumar Singh
>Priority: Major
>
> hive query with column masking failed with ParseException
> Table DDL
> {code:java}
> CREATE TABLE `emp`( `id` string, `name#` string);
> {code}
> The following query failed
> {code:java}
> select `emp`.`id`, `emp`.`name#` from (SELECT `id`, 
> CAST(mask_show_first_n(name#, 4, 'x', 'x', 'x', -1, '1') AS string) AS 
> `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM 
> `default`.`emp` )`emp`;
> {code}
> Error: Error while compiling statement: FAILED: ParseException line 1:79 
> character '#' not supported here (state=42000,code=4)
> quoting manually helped 
> {code:java}
> select `emp`.`id`, `emp`.`name#` from (SELECT `id`, 
> CAST(mask_show_first_n(`name#`, 4, 'x', 'x', 'x', -1, '1') AS string) AS 
> `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM 
> `default`.`emp` )`emp`;
> {code}
> manual query change will not work for Ranger authorizer as following query
> {code:java}
> select * from emp;
> {code}
> will be rewritten to 
> {code:java}
> select `emp`.`id`, `emp`.`name#` from (SELECT `id`, 
> CAST(mask_show_first_n(name#, 4, 'x', 'x', 'x', -1, '1') AS string) AS 
> `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM 
> `default`.`emp` )`emp`;
> {code}
> Ranger apply the transformer for column 
> RangerHiveAuthorizer#applyRowFilterAndColumnMasking so we should consider the 
> enclosing the column names in the back-ticks to make it work
> https://github.com/apache/ranger/blob/master/hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java#L1332



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-3627) Implement Spark extension

2022-09-15 Thread Bhavik Patel (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605236#comment-17605236
 ] 

Bhavik Patel commented on RANGER-3627:
--

It's already implemented as a part of 
https://issues.apache.org/jira/browse/RANGER-2128.

Can you please validate the review request if it works fine than I will merge 
into master branch.

> Implement Spark extension
> -
>
> Key: RANGER-3627
> URL: https://issues.apache.org/jira/browse/RANGER-3627
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>Reporter: Jakub Leś
>Priority: Major
>
> Hi 
> I would like to implement Ranger Spark SQL. Is this ok ?
>  
> Best regards,
> Jakub
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-3847) [Ranger] : Http status & Error message is not correct for /xaudit/trx_log

2022-09-15 Thread Anupam Rai (Jira) 


 [ 
https://issues.apache.org/jira/browse/RANGER-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anupam Rai updated RANGER-3847:
---
Description: 
[Ranger] : Http status & Error message is not correct for /xaudit/trx_log

Request with bad request : Passing String values
{code:java}
curl --location --request POST 'https://-/service/xaudit/trx_log' \
--header 'Authorization: Basic X==' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data-raw '{
  "objectClassType" : "abcdef",
  "objectId" : "abcdef"
}' {code}
Response : 404 Not Found

Request with proper request : Passing int values
{code:java}
curl --location --request POST 
'https:///service/xaudit/trx_log' \
--header 'Authorization: Basic =' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data-raw '{
  "objectClassType" : "123456",
  "objectId" : "1234567"
}' {code}
Response : ok 

 

  was:
[Ranger] : Http status & Error message is not correct for /xaudit/trx_log

Request with bad request : Passing String values
{code:java}
curl --location --request POST 
'https://quasar-daqgbo-1.quasar-daqgbo.root.hwx.site:6182/service/xaudit/trx_log'
 \
--header 'Authorization: Basic aHJ0X3FhOlBhc3N3b3JkQDEyMw==' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data-raw '{
  "objectClassType" : "abcdef",
  "objectId" : "abcdef"
}' {code}
Response : 404 Not Found

Request with proper request : Passing int values
{code:java}
curl --location --request POST 
'https://quasar-daqgbo-1.quasar-daqgbo.root.hwx.site:6182/service/xaudit/trx_log'
 \
--header 'Authorization: Basic aHJ0X3FhOlBhc3N3b3JkQDEyMw==' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data-raw '{
  "objectClassType" : "123456",
  "objectId" : "1234567"
}' {code}
Response : ok 

 


> [Ranger] : Http status & Error message is not correct for /xaudit/trx_log
> -
>
> Key: RANGER-3847
> URL: https://issues.apache.org/jira/browse/RANGER-3847
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Anupam Rai
>Priority: Minor
>
> [Ranger] : Http status & Error message is not correct for /xaudit/trx_log
> Request with bad request : Passing String values
> {code:java}
> curl --location --request POST 'https://-/service/xaudit/trx_log' \
> --header 'Authorization: Basic X==' \
> --header 'Accept: application/json' \
> --header 'Content-Type: application/json' \
> --data-raw '{
>   "objectClassType" : "abcdef",
>   "objectId" : "abcdef"
> }' {code}
> Response : 404 Not Found
> Request with proper request : Passing int values
> {code:java}
> curl --location --request POST 
> 'https:///service/xaudit/trx_log' \
> --header 'Authorization: Basic =' \
> --header 'Accept: application/json' \
> --header 'Content-Type: application/json' \
> --data-raw '{
>   "objectClassType" : "123456",
>   "objectId" : "1234567"
> }' {code}
> Response : ok 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-3885) User REST API /users/firstnames returns only null with status code 204

2022-09-15 Thread Abhishek (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605133#comment-17605133
 ] 

Abhishek commented on RANGER-3885:
--

Thanks for the patch [~ramackri] 

> User REST API /users/firstnames returns only null with status code 204
> --
>
> Key: RANGER-3885
> URL: https://issues.apache.org/jira/browse/RANGER-3885
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Abhishek
>Assignee: Ramachandran
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: 
> 0001-RANGER-3885-User-REST-API-users-firstnames-returns-o.patch
>
>
> According to the documentation, the API 
> [resource_UserREST_suggestUserFirstName_GET|https://ranger.apache.org/apidocs/resource_UserREST.html#resource_UserREST_suggestUserFirstName_GET]
>  should return status code 200 when a request is made, but the API always 
> returns a status code 204 No Content.
> From the code, it is clear that the API call just returns null without 
> performing any operations.
> {code:java|bgColor=#f4f5f7}
> @GET
>   @Path("/firstnames")
>   @Produces({ "application/json", "application/xml" })
>   public String suggestUserFirstName(@QueryParam("letters") String 
> letters,
>   @Context HttpServletRequest req) {
>   return null;
>   }
> {code}
> If this API is used by the customers, then this API has to be fixed, 
> otherwise this API has to be removed from the code file and from
> the documentation



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-3847) [Ranger] : Http status & Error message is not correct for /xaudit/trx_log

2022-09-15 Thread Ramachandran (Jira) 


[ 
https://issues.apache.org/jira/browse/RANGER-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605131#comment-17605131
 ] 

Ramachandran commented on RANGER-3847:
--

In Ranger rest API if you pass an Invalid attribute type in the JSON request 
body,  it throws a 404 error instead of 400 (Bad Request). It seems a bug where 
we need to handle the JsonMappingException by adding Exception Mapper cc >> 
[~mad...@apache.org] 

 

> [Ranger] : Http status & Error message is not correct for /xaudit/trx_log
> -
>
> Key: RANGER-3847
> URL: https://issues.apache.org/jira/browse/RANGER-3847
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Anupam Rai
>Priority: Minor
>
> [Ranger] : Http status & Error message is not correct for /xaudit/trx_log
> Request with bad request : Passing String values
> {code:java}
> curl --location --request POST 
> 'https://quasar-daqgbo-1.quasar-daqgbo.root.hwx.site:6182/service/xaudit/trx_log'
>  \
> --header 'Authorization: Basic aHJ0X3FhOlBhc3N3b3JkQDEyMw==' \
> --header 'Accept: application/json' \
> --header 'Content-Type: application/json' \
> --data-raw '{
>   "objectClassType" : "abcdef",
>   "objectId" : "abcdef"
> }' {code}
> Response : 404 Not Found
> Request with proper request : Passing int values
> {code:java}
> curl --location --request POST 
> 'https://quasar-daqgbo-1.quasar-daqgbo.root.hwx.site:6182/service/xaudit/trx_log'
>  \
> --header 'Authorization: Basic aHJ0X3FhOlBhc3N3b3JkQDEyMw==' \
> --header 'Accept: application/json' \
> --header 'Content-Type: application/json' \
> --data-raw '{
>   "objectClassType" : "123456",
>   "objectId" : "1234567"
> }' {code}
> Response : ok 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)