[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123545#comment-17123545 ] RickyMa commented on RANGER-2789: - Thanks. It's closed. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Fix For: 2.1.0 > > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120805#comment-17120805 ] RickyMa commented on RANGER-2789: - [~pradeep] : Finally... I clicked on 'Update Diff'. All is done.:D Please check it again. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120767#comment-17120767 ] Pradeep Agrawal commented on RANGER-2789: - [~RickyMa] : i still see old patch there > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120586#comment-17120586 ] RickyMa commented on RANGER-2789: - [~pradeep] : My bad. I uploaded the newest patch last time, but I forgot to publish it... When you said 'published the changes', all I was thinking about 'updating the pull request' Now I've really published the review request. But I'm new to this, so I'm not sure if I'm doing this right. Could you please check it again to see if you can see the patch right now? > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120475#comment-17120475 ] Pradeep Agrawal commented on RANGER-2789: - [~RickyMa] : You need to upload your new patch in the Review board here: [https://reviews.apache.org/r/72346/] > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120158#comment-17120158 ] RickyMa commented on RANGER-2789: - [~pradeep] : Yes. I have published. Here is the link: [https://github.com/apache/ranger/pull/61/commits/a16e64ee24fd5d3c260696ea540c60578d67f1bb] > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120122#comment-17120122 ] Pradeep Agrawal commented on RANGER-2789: - [~RickyMa] : I can see the new patch in Review request, have you published the changes after uploading the patch ? > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119672#comment-17119672 ] RickyMa commented on RANGER-2789: - [~pradeep] : Sorry. It's done. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119493#comment-17119493 ] Pradeep Agrawal commented on RANGER-2789: - [~RickyMa] : please upload the new patch in the Review board also.. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119386#comment-17119386 ] RickyMa commented on RANGER-2789: - [~pradeep] : new patch uploaded~ > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119348#comment-17119348 ] RickyMa commented on RANGER-2789: - [~pradeep] : It's all done. I've retested it. Please have a look. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17119258#comment-17119258 ] Pradeep Agrawal commented on RANGER-2789: - [~RickyMa] : Madhan has given few review comment on your patch, can you address them and restest the patch. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[
https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118960#comment-17118960
]
RickyMa commented on RANGER-2789:
-
[~pradeep] I increased the number of users to 10,000 and changed the pageSize
to 10,000. Here is the result:
time curl -s -o /dev/null -X GET -u test:test -H 'Accept:application/json'
'[http://10.1.0.121:6080/service/xusers/users/?pageSize=1=0'|http://10.1.0.121:6080/service/xusers/users/?pageSize=1=0%27]
{code:java}
real 7m10.208s
user 0m0.010s
sys 0m0.017s
{code}
time curl -s -o /dev/null -X GET -u test:test -H 'Accept:application/json'
'http://10.1.0.121:6080/service/xusers/users/?pageSize=1=0'
{code:java}
real 0m3.115s
user 0m0.006s
sys 0m0.014s
{code}
As you can see, the perfermance increased a lot.
> GET API service/xusers/users turns very slow when there are more than 1000
> users
>
>
> Key: RANGER-2789
> URL: https://issues.apache.org/jira/browse/RANGER-2789
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.0.0
> Environment: hdp 2.2
>Reporter: RickyMa
>Priority: Minor
> Labels: optimization, performance, ranger, rangeradmin, slow
> Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We have more than 1000 users in production environment.
> When calling API 'service/xusers/users?pageSize=1000=0' using HTTP
> GET method, Ranger Admin takes 10+ minutes to response.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118935#comment-17118935 ] RickyMa commented on RANGER-2789: - Hey [~pradeep], I retest it. I created 1000 users in my own environment and found out that it's true that it only took seconds to respond. But it really took 10+ minutes for Ranger Admin to respond in our customer's environment. Different VMs, different network conditions, different databases or some other reasons might have caused this and created this time gap.(I guess it is mainly because of the databases. Because after my customers applied this patch, the response time really reduced a lot.) With the increasing number of users, it may take longer for Ranger Admin to respond(Because there will be more database queries). But my patch just greatly reduced the reponse time as you can see. And I think this patch will especially help the people with poor performance databases. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115341#comment-17115341 ] Pradeep Agrawal commented on RANGER-2789: - [~RickyMa] : Can you test the current performance in the master branch code. I don't see performance issue with 1000 users in the current master code. After the first request your code is returning the result faster, but without your patch also ranger is not taking time in minutes. Here is what i have tried: *Without your patch :* 1) created 1000 users in ranger 2) called : curl -v -w "@temp.txt" -u admin:ranger1234 -H "Accept: application/json" -H "Content-Type:application/json" -X GET '[http://10.21.23.66:6080/service/xusers/users?pageSize=1000=0'|http://10.21.23.66:6080/service/xusers/users?pageSize=1000=0%27] First request : 9 seconds second request : 8 seconds third request : 8 seconds *With your patch :* 1) created 1000 users in ranger 2) called : curl -v -w "@temp.txt" -u admin:ranger1234 -H "Accept: application/json" -H "Content-Type:application/json" -X GET '[http://10.21.23.66:6080/service/xusers/users?pageSize=1000=0'|http://10.21.23.66:6080/service/xusers/users?pageSize=1000=0%27] First request : 8 seconds second request : 1 second third request : 1 second > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17115092#comment-17115092 ] RickyMa commented on RANGER-2789: - Hi [~pradeep], it's been a while, please let me know of any progress on this issue. Thank you. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080611#comment-17080611 ] RickyMa commented on RANGER-2789: - [~pradeep] Thanks for your reply. :) > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080468#comment-17080468 ] Pradeep Agrawal commented on RANGER-2789: - [~RickyMa] Good finding and thanks for the fix. Currently I don't have environment to test this patch. I shall update you on this soon. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 20m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080436#comment-17080436 ] RickyMa commented on RANGER-2789: - Review request URL：[https://reviews.apache.org/r/72346/] > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 10m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080429#comment-17080429 ] RickyMa commented on RANGER-2789: - I've already uploaded the patch. By applying this patch, the response time has decreased from 10+ minutes to 30 seconds in our production environment. The problem of this issue is that the original code will query the database for every one of the 1000+ users: [https://github.com/apache/ranger/blob/master/security-admin/src/main/java/org/apache/ranger/service/XUserServiceBase.java#L84] !image-2020-04-10-19-37-31-570.png! The main ideas of the uploaded patch is: # Find all users in the database at one time. # Use HashMap as cache to store all the database info in memory. # Any follow-up db operation can just use the cache. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: RickyMa >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > Attachments: RANGER-2789.patch, image-2020-04-10-19-37-31-570.png > > Time Spent: 10m > Remaining Estimate: 0h > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2789) GET API service/xusers/users turns very slow when there are more than 1000 users
[ https://issues.apache.org/jira/browse/RANGER-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080372#comment-17080372 ] MaHaoxiang commented on RANGER-2789: We found the problem, later I will upload the patch attachment. > GET API service/xusers/users turns very slow when there are more than 1000 > users > > > Key: RANGER-2789 > URL: https://issues.apache.org/jira/browse/RANGER-2789 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 > Environment: hdp 2.2 >Reporter: MaHaoxiang >Priority: Minor > Labels: optimization, performance, ranger, rangeradmin, slow > > We have more than 1000 users in production environment. > When calling API 'service/xusers/users?pageSize=1000=0' using HTTP > GET method, Ranger Admin takes 10+ minutes to response. -- This message was sent by Atlassian Jira (v8.3.4#803005)
