Hi IS TEAM, we are getting this error continuously even for 20+ concurrent users when testing asset crud operation in ES 2.1.0 in clustered setup. It is a two nodes cluster fronted with nginx and we have enabled sticky sessions. We have JDBC user store (mysql DB) shared with both nodes.
Problem here is sometimes cacheEntry in line [1] becomes null when concurrency level is increased. We are able to overcome this by changing the code as [2]. Can you please look into this. 1. https://github.com/wso2/carbon-kernel/blob/4.4.x/core/org.wso2.carbon.user.core/src/main/java/org/wso2/carbon/user/core/authorization/PermissionTree.java#L951-L952 2. synchronized (this) { cacheEntry = (GhostResource<TreeNode>) permissionCache.get(cacheKey); if (*cacheEntry == null ||* cacheEntry.getResource() == null) { updatePermissionTreeFromDB(); * if (cacheEntry == null) { cacheEntry = new GhostResource<TreeNode>(root); permissionCache.put(cacheKey, cacheEntry); } else { cacheEntry.setResource(root); }* if (log.isDebugEnabled()) { log.debug("Set resource to true"); } } } Thanks Danesh On Thu, Jun 23, 2016 at 11:40 AM, Dilini Gunatilake <[email protected]> wrote: > Hi All, > > I have also faced a similar issue[1] to the issue mentioned by Danesh in > [2] above when we are doing a long running test for ES 210. Were you able > to find a root cause for this issue? > > [1] https://wso2.org/jira/browse/STORE-1352 > > Thanks. > > Regards, > Dilini > > On Wed, Jun 1, 2016 at 6:04 PM, Kasun Bandara <[email protected]> wrote: > >> Hi Thushara, >> >> Seems like the root cause of the issue originates from the below >> stack-trace. >> >> *JDBCAuthorizationManager.java:1378)* >> * at >> org.wso2.carbon.user.core.authorization.JDBCAuthorizationManager.isUserAuthorized(JDBCAuthorizationManager.java:168)* >> >> As you said in the offline discussion this occurs when the number of >> concurrent users increase from 80 to 100 right?. I think if you can debug >> the *isUserAuthorized() *method in *JDBCAuthorizationManager *class*, *you >> can the trace out the exact reason behind this concurrency issue*.* >> (debug it in both the occasions : 80 Users and 100 Users) >> >> Thanks, >> Kasun. >> >> On Wed, Jun 1, 2016 at 11:18 AM, Thushara Ranawaka <[email protected]> >> wrote: >> >>> Hi Everyone, >>> >>> This is continuously reproducible for 100 concurrent users in a single >>> node G-Reg setup. >>> Any idea of why this is happen. >>> >>> TID: [-1234] [] [2016-05-30 13:12:07,771] ERROR >>> {org.wso2.carbon.user.core.common.AbstractUserStoreManager} - Error >>> occurred while accessing Java Security Manager Privilege Block >>> {org.wso2.carbon.user.core.common.AbstractUserStoreManager} >>> TID: [-1234] [] [2016-05-30 13:13:00,324] ERROR >>> {org.wso2.carbon.user.core.authorization.JDBCAuthorizationManager} - >>> Error occurred while accessing Java Security Manager Privilege Block >>> {org.wso2.carbon.user.core.authorization.JDBCAuthorizationManager} >>> TID: [-1234] [] [2016-05-30 13:13:03,183] ERROR {rxt-permissions} - >>> org.mozilla.javascript.WrappedException: Wrapped >>> org.wso2.carbon.user.core.UserStoreException: Error occurred while >>> accessing Java Security Manager Privilege Block (eval code#1(eval)#86) >>> at org.mozilla.javascript.Context.throwAsScriptRuntimeEx( >>> Context.java:1754) >>> at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:148) >>> at org.mozilla.javascript.NativeJavaMethod.call( >>> NativeJavaMethod.java:225) >>> at org.mozilla.javascript.optimizer.OptRuntime.callN(OptRuntime.java:52) >>> at org.jaggeryjs.rhino.<rxt>.scripts.permissions.c0._c_ >>> anonymous_52(<rxt>/scripts/permissions/permissions.js:594) >>> at org.jaggeryjs.rhino.<rxt>.scripts.permissions.c0.call(< >>> rxt>/scripts/permissions/permissions.js) >>> at org.mozilla.javascript.optimizer.OptRuntime.callName( >>> OptRuntime.java:63) >>> at org.jaggeryjs.rhino.<rxt>.scripts.permissions.c0._c_ >>> anonymous_53(<rxt>/scripts/permissions/permissions.js:637) >>> at org.jaggeryjs.rhino.<rxt>.scripts.permissions.c0.call(< >>> rxt>/scripts/permissions/permissions.js) >>> at org.mozilla.javascript.optimizer.OptRuntime.callName( >>> OptRuntime.java:63) >>> at org.jaggeryjs.rhino.<rxt>.scripts.permissions.c0._c_ >>> anonymous_54(<rxt>/scripts/permissions/permissions.js:658) >>> at org.jaggeryjs.rhino.<rxt>.scripts.permissions.c0.call(< >>> rxt>/scripts/permissions/permissions.js) >>> at org.mozilla.javascript.optimizer.OptRuntime.callName( >>> OptRuntime.java:63) >>> at org.jaggeryjs.rhino.<rxt>.scripts.permissions.c0._c_ >>> anonymous_60(<rxt>/scripts/permissions/permissions.js:751) >>> at org.jaggeryjs.rhino.<rxt>.scripts.permissions.c0.call(< >>> rxt>/scripts/permissions/permissions.js) >>> at org.mozilla.javascript.optimizer.OptRuntime.callN(OptRuntime.java:52) >>> at org.jaggeryjs.rhino.store.modules.c5._c_anonymous_8(/ >>> store/modules/page-decorators.js:200) >>> at org.jaggeryjs.rhino.store.modules.c5.call(/store/ >>> modules/page-decorators.js) >>> at org.mozilla.javascript.Interpreter.interpretLoop( >>> Interpreter.java:1473) >>> >>> >>> >>> >>> Caused by: org.wso2.carbon.user.core.UserStoreException: Error occurred >>> while accessing Java Security Manager Privilege Block >>> at org.wso2.carbon.user.core.authorization.JDBCAuthorizationManager. >>> callSecure(JDBCAuthorizationManager.java:1378) >>> at org.wso2.carbon.user.core.authorization.JDBCAuthorizationManager. >>> isUserAuthorized(JDBCAuthorizationManager.java:168) >>> at sun.reflect.GeneratedMethodAccessor195.invoke(Unknown Source) >>> >>> Thanks, >>> Thushara. >>> >>> On Thu, Mar 31, 2016 at 9:27 AM, Danesh Kuruppu <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> I am currently doing GREG performance testing in clustered setup. It is >>>> a two nodes cluster fronted with nginx and we have enabled sticky sessions. >>>> We have JDBC user store (mysql DB) shared with both nodes. >>>> >>>> User store exception[1][2] occurred while performing asset crud >>>> operation in publisher for 20+ concurrent users. >>>> >>>> Any idea of why this is happen. >>>> Thanks >>>> Danesh >>>> >>>> [1] >>>> Caused by: org.wso2.carbon.user.core.UserStoreException: Error >>>> occurred while accessing Java Security Manager Privilege Block >>>> at org.wso2.carbon.user.core.authorization. >>>> JDBCAuthorizationManager.callSecure(JDBCAuthorizationManager.java:1378) >>>> at org.wso2.carbon.user.core.authorization. >>>> JDBCAuthorizationManager.isUserAuthorized( >>>> JDBCAuthorizationManager.java:168) >>>> at sun.reflect.GeneratedMethodAccessor169.invoke(Unknown Source) >>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke( >>>> DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126) >>>> ... 99 more >>>> Caused by: java.security.PrivilegedActionException: java.lang.reflect. >>>> InvocationTargetException >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at org.wso2.carbon.user.core.authorization. >>>> JDBCAuthorizationManager.callSecure(JDBCAuthorizationManager.java:1363) >>>> ... 104 more >>>> Caused by: java.lang.reflect.InvocationTargetException >>>> at sun.reflect.GeneratedMethodAccessor247.invoke(Unknown Source) >>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke( >>>> DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> at org.wso2.carbon.user.core.authorization. >>>> JDBCAuthorizationManager$2.run(JDBCAuthorizationManager.java:1366) >>>> ... 106 more >>>> >>>> [2] >>>> Caused by: org.wso2.carbon.user.core.UserStoreException: Error >>>> occurred while accessing Java Security Manager Privilege Block >>>> at org.wso2.carbon.user.core.authorization. >>>> JDBCAuthorizationManager.callSecure(JDBCAuthorizationManager.java:1378) >>>> at org.wso2.carbon.user.core.authorization. >>>> JDBCAuthorizationManager.getAllowedRolesForResource( >>>> JDBCAuthorizationManager.java:312) >>>> at org.wso2.carbon.registry.core.jdbc.realm. >>>> RegistryAuthorizationManager.getAllowedRolesForResource( >>>> RegistryAuthorizationManager.java:233) >>>> at org.wso2.carbon.registry.indexing.indexer.IndexDocumentCreator. >>>> addAllowedRoles(IndexDocumentCreator.java:345) >>>> ... 8 more >>>> Caused by: java.security.PrivilegedActionException: java.lang.reflect. >>>> InvocationTargetException >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at org.wso2.carbon.user.core.authorization. >>>> JDBCAuthorizationManager.callSecure(JDBCAuthorizationManager.java:1363) >>>> ... 11 more >>>> Caused by: java.lang.reflect.InvocationTargetException >>>> at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source) >>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke( >>>> DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> at org.wso2.carbon.user.core.authorization. >>>> JDBCAuthorizationManager$2.run(JDBCAuthorizationManager.java:1366) >>>> ... 13 more >>>> Caused by: java.lang.NullPointerException >>>> at org.wso2.carbon.user.core.authorization.PermissionTree. >>>> updatePermissionTree(PermissionTree.java:952) >>>> at org.wso2.carbon.user.core.authorization. >>>> JDBCAuthorizationManager.getAllowedRolesForResource( >>>> JDBCAuthorizationManager.java:317) >>>> ... 17 more >>>> -- >>>> >>>> Danesh Kuruppu >>>> Software Engineer >>>> WSO2 Inc, >>>> Mobile: +94 (77) 1690552 >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> [email protected] >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> Thanks, >>> Thushara Kasun Ranawaka >>> Software Engineer >>> WSO2 Inc.; <http://www.wso2.com> >>> lean.enterprise.middleware >>> Mobile : *+94 (0) 773438949 <%2B94%20%280%29%20773438949>* >>> *[email protected] <[email protected]>* >>> >> >> >> >> -- >> Kasun Bandara >> *Software Engineer* >> Mobile : +94 (0) 718 338 360 >> <%2B94%20%280%29%20773%20451194> >> [email protected] <[email protected]> >> >> _______________________________________________ >> Dev mailing list >> [email protected] >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > > *Dilini GunatilakeSoftware Engineer - QA Team* > Mobile : +94 (0) 771 162518 > [email protected] > -- *Danesh Kuruppu* Senior Software Engineer | WSO2 Email: [email protected] Mobile: +94 (77) 1690552 Web: WSO2 Inc <https://wso2.com/signature>
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
