vyommani opened a new pull request, #1265: URL: https://github.com/apache/ranger/pull/1265
…artup ## What changes were proposed in this pull request? RangerDefaultRequestProcessor.preProcess() calls RangerAuthContext.getRolesForUserAndGroups() for every request, and each call builds a new HashSet of all the user's roles. In a bulk isAccessAllowed(Collection) call, all N requests are kept until the call returns, so memory is N × R. For Kafka, a metadata request becomes one authorize() call with an action per topic. With the SRM principal in ~6,000 roles and ~2,400 topics, that's ~600 MB of role sets per call, which caused broker OOM during SRM startup. Introduce RangerBatchEvalContext, created per isAccessAllowed(Collection) call and attached to each request's context only while evaluatePolicies() runs. It's removed in a finally block. preProcess() computes the normalized user, groups and roles once per distinct (user, groups) pair, after name transformation, email-to-username and user-store groups, and shares one read-only set across the requests. Authorization results are unchanged. The single-request path is unchanged, and no Kafka plugin change is needed. Role-set memory per bulk call drops from N × R to R, and getRolesForUserAndGroups() runs once per distinct (user, groups) pair instead of once per request. ## How was this patch tested? Processor-level: sharing, call counts, different users and groups, pre-set and empty roles, null user, normalization on. Plugin-level: bulk results equal single-request results; the context is cleared, including when evaluation throws. Kafka-level: all actions in one authorize() call share one role set. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
