[jira] [Commented] (RANGER-2727) Ghost policy occurs in plugin when creating and getting policies concurrently
[
https://issues.apache.org/jira/browse/RANGER-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034193#comment-17034193
]
Harshal Chavan commented on RANGER-2727:
[~stigahuang] - Tried on apache master, was not able to reproduce the issue
with 7000 rounds. But one thing i observed that when i stop the script
aftersome and do not refresh the page and try to delete the policy then i get
the issue '*org.apache.ranger.plugin.model.RangerPolicy :Data Not Found for
given Id*' which is correct because the policy is deleted but page is not
refreshed, and when i refresh the page the policy wont be there.
> Ghost policy occurs in plugin when creating and getting policies concurrently
> -
>
> Key: RANGER-2727
> URL: https://issues.apache.org/jira/browse/RANGER-2727
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: create_delete_policy.py
>
>
> Attached a python script to reproduce this issue. It creates and deletes 10
> column masking policies repeatedly. Run it and refresh column masking
> policies in Hive service in Ranger Admin UI. Some policies won't be deleted
> after several rounds.
> Delete them manually in Ranger Admin UI will get an error as
> org.apache.ranger.plugin.model.RangerPolicy :Data Not Found for given Id
> Can't neither view them nor edit them. The web UI will hang. Ranger logs
> reveal the same error:
> {code}
> 2020-02-10 03:08:05,597 [http-bio-6080-exec-2] INFO
> org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:63) - Request
> failed. loginId=admin, logMessage=org.apache.ranger.plugin.model.RangerPolicy
> :Data Not Found for given Id
> javax.ws.rs.WebApplicationException
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:301)
> at
> org.apache.ranger.service.RangerBaseModelService.read(RangerBaseModelService.java:240)
> at
> org.apache.ranger.biz.ServiceDBStore.getPolicy(ServiceDBStore.java:2171)
> at org.apache.ranger.rest.ServiceREST.getPolicy(ServiceREST.java:1829)
> at
> org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
> at
> org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$315c4133.getPolicy()
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.applic
[jira] [Commented] (RANGER-2705) Group sync does does not parse DNs properly
[
https://issues.apache.org/jira/browse/RANGER-2705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034174#comment-17034174
]
Sailaja Polavarapu commented on RANGER-2705:
[~larsfrancke],
I agree there is an issue in the getShortUserName(). But if you notice, the
username returned from this method is only used when groupSearchFirstEnabled:
true, userSearchEnabled: false. Is this your configuration?
[https://github.com/apache/ranger/blob/master/ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapDeltaUserGroupBuilder.java#L770]
Also, if ranger sync the users with comma character properly like (Francke\,
Lars), how are these users configured in Hadoop for authorization with ranger?
Can you provide some details on what is the use case?
BTW - I completely agree that there is an issue with getShortUserName() and
will work on the fix. But trying to understand how these users are used in
hadoop as commas are generally not allowed for the unix users.
> Group sync does does not parse DNs properly
> ---
>
> Key: RANGER-2705
> URL: https://issues.apache.org/jira/browse/RANGER-2705
> Project: Ranger
> Issue Type: Bug
> Components: usersync
>Reporter: Lars Francke
>Priority: Major
>
> When we have enabled user & group search
> ({{ranger.usersync.group.search.first.enabled}} = false) we expect Ranger to
> get the groups and its members and compare them to what already exists.
> Our DN/CN looks like this:
> {code:java}
> CN=Francke\, Lars,OU=bla bla.
> {code}
> Our CN contains a comma but the {{getShortUserName}} method in
> {{LdapDeltaUserGroupBuilder}} has this piece of code:
> {code:java}
> StringTokenizer stc = new StringTokenizer(longUserName, ",");
> String firstToken = stc.nextToken();{code}
> The intention is that it gets the "{{CN=Francke\, Lars}}" part (the first
> part of the comma-separated DN) but that doesn't work if that contains a
> comma itself. It is escaped but Ranger just splits at the comma. That's
> definitely a bug. It should use the {{LdapName}} class instead and/or parse
> according to the RFC 2253 but maybe even that is wrong what it really should
> probably do is the same as user sync?
> This way we currently cannot use (incremental) group sync at all because if
> we do we don't get any groups at all as the user search doesn't take its own
> groups when group sync is also enabled (this was another surprise).
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (RANGER-2727) Ghost policy occurs in plugin when creating and getting policies concurrently
[
https://issues.apache.org/jira/browse/RANGER-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17034162#comment-17034162
]
Pradeep Agrawal commented on RANGER-2727:
-
[~stigahuang] : Can you provide more details about ranger branch/version or
last commit.
> Ghost policy occurs in plugin when creating and getting policies concurrently
> -
>
> Key: RANGER-2727
> URL: https://issues.apache.org/jira/browse/RANGER-2727
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: create_delete_policy.py
>
>
> Attached a python script to reproduce this issue. It creates and deletes 10
> column masking policies repeatedly. Run it and refresh column masking
> policies in Hive service in Ranger Admin UI. Some policies won't be deleted
> after several rounds.
> Delete them manually in Ranger Admin UI will get an error as
> org.apache.ranger.plugin.model.RangerPolicy :Data Not Found for given Id
> Can't neither view them nor edit them. The web UI will hang. Ranger logs
> reveal the same error:
> {code}
> 2020-02-10 03:08:05,597 [http-bio-6080-exec-2] INFO
> org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:63) - Request
> failed. loginId=admin, logMessage=org.apache.ranger.plugin.model.RangerPolicy
> :Data Not Found for given Id
> javax.ws.rs.WebApplicationException
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:301)
> at
> org.apache.ranger.service.RangerBaseModelService.read(RangerBaseModelService.java:240)
> at
> org.apache.ranger.biz.ServiceDBStore.getPolicy(ServiceDBStore.java:2171)
> at org.apache.ranger.rest.ServiceREST.getPolicy(ServiceREST.java:1829)
> at
> org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
> at
> org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$315c4133.getPolicy()
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.app
Re: Review Request 72085: RANGER-2721: Fix build instructions in README.txt and Docker build script
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72085/#review219536 --- Ship it! Ship It! - Pradeep Agrawal On Feb. 5, 2020, 6:28 p.m., Andrew Luo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72085/ > --- > > (Updated Feb. 5, 2020, 6:28 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2721 > https://issues.apache.org/jira/browse/RANGER-2721 > > > Repository: ranger > > > Description > --- > > README.md still shows the command "mvn clean compile package install > assembly:assembly" in the instructions. However, if you run this command, it > fails due to a conflict between package and assembly:assembly with the error > message "Failed to execute goal > org.apache.maven.plugins:maven-assembly-plugin:2.6:single (default) on > project ranger-distro: Failed to create assembly: Error creating assembly > archive admin: Problem creating TAR: entry > 'ranger-2.1.0-SNAPSHOT-admin/ews/webapp/META-INF/MANIFEST.MF' closed at '0' > before the '136' bytes specified in the header were written". This is > because after commit > https://github.com/apache/ranger/commit/e06abc00667b3935ad86a146be73853fed807eda, > the assembly:assembly phase is no longer needed (part of package now) > This is a trivial fix to change the README.txt to update the instructions. > > > Diffs > - > > README.txt c1a2f1b96 > build_ranger_using_docker.sh 17c8ed0ca > > > Diff: https://reviews.apache.org/r/72085/diff/1/ > > > Testing > --- > > > Thanks, > > Andrew Luo > >
Review Request 72106: RANGER-2718: Utility to update user and group names in stored policy json
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72106/ --- Review request for ranger. Bugs: RANGER-2718 https://issues.apache.org/jira/browse/RANGER-2718 Repository: ranger Description --- **Problem Statement:** When The case conversion property to syncs users/groups is changed then in the ranger admin user/group names get updated but the user/group is not updated in policy json. **Proposed solution:** A standalone utility can be added to update the users and groups name in the policy json as they(existing user/groups) appear in ranger db. **Note:** There could be few other issues while syncing users with diffrent case but the below ones are not covered in this patch. 1) user name case was changed in x_user table but not in the x_portal_user table. 2) user name case may not change if user is added in the security zone. 5) Behaviour might be different in different DB flavours. 6) user/group name in service config may not change Diffs - security-admin/scripts/updateUserAndGroupNamesInJson.py PRE-CREATION security-admin/src/main/java/org/apache/ranger/patch/cliutil/UpdateUserAndGroupNamesInJson.java PRE-CREATION Diff: https://reviews.apache.org/r/72106/diff/1/ Testing --- applied the patch in the master branch and built it. untar the generated ranger--admin.tar.gz file. updated install.properties and run the setup.sh with old ranger db name, username and related credentials run the command 'python updateUserAndGroupNamesInJson.py' Utility should update the user and groups name in the policy json as they(existing user/groups) appears in UI Thanks, Pradeep Agrawal
[jira] [Resolved] (RANGER-2699) JVM metrics for Ranger usersync and Ranger tagsync
[ https://issues.apache.org/jira/browse/RANGER-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-2699. - Resolution: Fixed [https://github.com/apache/ranger/commit/1eec98e4739e47f139776c1ec0ccee57ecb7dd73] > JVM metrics for Ranger usersync and Ranger tagsync > -- > > Key: RANGER-2699 > URL: https://issues.apache.org/jira/browse/RANGER-2699 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 2.1.0 >Reporter: Mehul Parikh >Assignee: Dineshkumar Yadav >Priority: Major > Fix For: 2.1.0 > > > Record JVM metrics for usersync and tagsync. Regularly save metric info on a > json file and keep updating the same based on jvm resources utilized by the > process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (RANGER-2699) JVM metrics for Ranger usersync and Ranger tagsync
[ https://issues.apache.org/jira/browse/RANGER-2699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dineshkumar Yadav reassigned RANGER-2699: - Assignee: Dineshkumar Yadav (was: Mehul Parikh) > JVM metrics for Ranger usersync and Ranger tagsync > -- > > Key: RANGER-2699 > URL: https://issues.apache.org/jira/browse/RANGER-2699 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 2.1.0 >Reporter: Mehul Parikh >Assignee: Dineshkumar Yadav >Priority: Major > Fix For: 2.1.0 > > > Record JVM metrics for usersync and tagsync. Regularly save metric info on a > json file and keep updating the same based on jvm resources utilized by the > process. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 71993: RANGER-2699 : JVM metrics for Ranger usersync and Ranger tagsync
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71993/#review219535 --- Ship it! Ship It! - Pradeep Agrawal On Feb. 5, 2020, 2:49 p.m., Dineshkumar Yadav wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71993/ > --- > > (Updated Feb. 5, 2020, 2:49 p.m.) > > > Review request for ranger, Ankita Sinha, Gautam Borad, Mehul Parikh, and > Pradeep Agrawal. > > > Bugs: RANGER-2699 > https://issues.apache.org/jira/browse/RANGER-2699 > > > Repository: ranger > > > Description > --- > > Added metrics information in usersync and tagsync services > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerMetricsUtil.java > PRE-CREATION > tagsync/conf/templates/installprop2xml.properties 510cd6f44 > tagsync/conf/templates/ranger-tagsync-template.xml b8bfbf510 > tagsync/scripts/install.properties be33cc274 > tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSyncConfig.java > c4173da68 > > tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSyncMetricsProducer.java > PRE-CREATION > > tagsync/src/main/java/org/apache/ranger/tagsync/process/TagSynchronizer.java > 8806c7469 > tagsync/src/main/resources/ranger-tagsync-site.xml 0b9ef8492 > > ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java > a041345cf > > ugsync/src/main/java/org/apache/ranger/usergroupsync/UserSyncMetricsProducer.java > PRE-CREATION > unixauthservice/scripts/install.properties d4c651308 > unixauthservice/scripts/templates/installprop2xml.properties fa342fbff > unixauthservice/scripts/templates/ranger-ugsync-template.xml 0cacc954b > > unixauthservice/src/main/java/org/apache/ranger/authentication/UnixAuthenticationService.java > 1ee5e21ba > > > Diff: https://reviews.apache.org/r/71993/diff/11/ > > > Testing > --- > > Validated metrics which are getting generated at configured interval when > usersync and tagsync services start. > I have kept below parameter configurable > 1. Metrics Name (Default value ranger_tagsync_metric.json and > ranger_usersync_metric.json) > 2. Metrics Directory (must have write permission and default value is logdir) > 3. frequency at metrics get updated (default value: 10 sec) > 4. metrics enabled or disabled (default is disabled) > Checked for ranger_tagsync_metric.json and ranger_usersync_metric.json being > regularly updated based on JVM resources utilization. Validated for usersync > as well as tagsync process. > > > Thanks, > > Dineshkumar Yadav > >
Re: Review Request 72085: RANGER-2721: Fix build instructions in README.txt and Docker build script
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72085/#review219534 --- Ship it! Ship It! - Velmurugan Periasamy On Feb. 5, 2020, 6:28 p.m., Andrew Luo wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72085/ > --- > > (Updated Feb. 5, 2020, 6:28 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2721 > https://issues.apache.org/jira/browse/RANGER-2721 > > > Repository: ranger > > > Description > --- > > README.md still shows the command "mvn clean compile package install > assembly:assembly" in the instructions. However, if you run this command, it > fails due to a conflict between package and assembly:assembly with the error > message "Failed to execute goal > org.apache.maven.plugins:maven-assembly-plugin:2.6:single (default) on > project ranger-distro: Failed to create assembly: Error creating assembly > archive admin: Problem creating TAR: entry > 'ranger-2.1.0-SNAPSHOT-admin/ews/webapp/META-INF/MANIFEST.MF' closed at '0' > before the '136' bytes specified in the header were written". This is > because after commit > https://github.com/apache/ranger/commit/e06abc00667b3935ad86a146be73853fed807eda, > the assembly:assembly phase is no longer needed (part of package now) > This is a trivial fix to change the README.txt to update the instructions. > > > Diffs > - > > README.txt c1a2f1b96 > build_ranger_using_docker.sh 17c8ed0ca > > > Diff: https://reviews.apache.org/r/72085/diff/1/ > > > Testing > --- > > > Thanks, > > Andrew Luo > >
[jira] [Commented] (RANGER-2727) Ghost policy occurs in plugin when creating and getting policies concurrently
[
https://issues.apache.org/jira/browse/RANGER-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033613#comment-17033613
]
Velmurugan Periasamy commented on RANGER-2727:
--
[~stigahuang] - I have added you as contributor and assigned the jira to you.
> Ghost policy occurs in plugin when creating and getting policies concurrently
> -
>
> Key: RANGER-2727
> URL: https://issues.apache.org/jira/browse/RANGER-2727
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: create_delete_policy.py
>
>
> Attached a python script to reproduce this issue. It creates and deletes 10
> column masking policies repeatedly. Run it and refresh column masking
> policies in Hive service in Ranger Admin UI. Some policies won't be deleted
> after several rounds.
> Delete them manually in Ranger Admin UI will get an error as
> org.apache.ranger.plugin.model.RangerPolicy :Data Not Found for given Id
> Can't neither view them nor edit them. The web UI will hang. Ranger logs
> reveal the same error:
> {code}
> 2020-02-10 03:08:05,597 [http-bio-6080-exec-2] INFO
> org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:63) - Request
> failed. loginId=admin, logMessage=org.apache.ranger.plugin.model.RangerPolicy
> :Data Not Found for given Id
> javax.ws.rs.WebApplicationException
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:301)
> at
> org.apache.ranger.service.RangerBaseModelService.read(RangerBaseModelService.java:240)
> at
> org.apache.ranger.biz.ServiceDBStore.getPolicy(ServiceDBStore.java:2171)
> at org.apache.ranger.rest.ServiceREST.getPolicy(ServiceREST.java:1829)
> at
> org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
> at
> org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$315c4133.getPolicy()
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.appl
[jira] [Assigned] (RANGER-2727) Ghost policy occurs in plugin when creating and getting policies concurrently
[
https://issues.apache.org/jira/browse/RANGER-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Velmurugan Periasamy reassigned RANGER-2727:
Assignee: Quanlong Huang
> Ghost policy occurs in plugin when creating and getting policies concurrently
> -
>
> Key: RANGER-2727
> URL: https://issues.apache.org/jira/browse/RANGER-2727
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: create_delete_policy.py
>
>
> Attached a python script to reproduce this issue. It creates and deletes 10
> column masking policies repeatedly. Run it and refresh column masking
> policies in Hive service in Ranger Admin UI. Some policies won't be deleted
> after several rounds.
> Delete them manually in Ranger Admin UI will get an error as
> org.apache.ranger.plugin.model.RangerPolicy :Data Not Found for given Id
> Can't neither view them nor edit them. The web UI will hang. Ranger logs
> reveal the same error:
> {code}
> 2020-02-10 03:08:05,597 [http-bio-6080-exec-2] INFO
> org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:63) - Request
> failed. loginId=admin, logMessage=org.apache.ranger.plugin.model.RangerPolicy
> :Data Not Found for given Id
> javax.ws.rs.WebApplicationException
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:301)
> at
> org.apache.ranger.service.RangerBaseModelService.read(RangerBaseModelService.java:240)
> at
> org.apache.ranger.biz.ServiceDBStore.getPolicy(ServiceDBStore.java:2171)
> at org.apache.ranger.rest.ServiceREST.getPolicy(ServiceREST.java:1829)
> at
> org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
> at
> org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$315c4133.getPolicy()
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey
Re: Please add me to the developer group
Thanks for your interest in contributing to Ranger. I have added you as contributor. Welcome to Ranger community. On Mon, Feb 10, 2020 at 12:44 AM Quanlong Huang wrote: Hi, Could anyone add me to the developer group and assign RANGER-2727 to me? I can't assign it to myself. My JIRA usename name is stigahuang. Thanks, Quanlong
[jira] [Resolved] (RANGER-2596) Ranger isn't compiling
[ https://issues.apache.org/jira/browse/RANGER-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-2596. - Resolution: Cannot Reproduce Unable to reproduce this. if this is still happening to you please provide more details and reopen the jira. > Ranger isn't compiling > --- > > Key: RANGER-2596 > URL: https://issues.apache.org/jira/browse/RANGER-2596 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.0.0 >Reporter: Chethan UK >Priority: Major > > I did following > > git clone -b ranger-2.0 https://github.com/apache/ranger.git ranger > cd ranger > export MAVEN_OPTS="-Xmx512M" > mvn clean compile package assembly:assembly > But error is > ``` > [*INFO*] *Reactor Summary for ranger 2.0.1-SNAPSHOT:* > [*INFO*] > [*INFO*] ranger . *FAILURE* [ > 41.124 s] > [*INFO*] Jdbc SQL Connector . *SUCCESS* [ > 27.781 s] > [*INFO*] Credential Support . *SUCCESS* > [06:28 min] > [*INFO*] Audit Component *SUCCESS* > [02:30 min] > [*INFO*] Common library for Plugins . *SUCCESS* > [05:47 min] > [*INFO*] Installer Support Component *SUCCESS* [ > 0.863 s] > [*INFO*] Credential Builder . *SUCCESS* [ > 15.159 s] > [*INFO*] Embedded Web Server Invoker *SUCCESS* [ > 32.248 s] > [*INFO*] Key Management Service . *SUCCESS* > [02:54 min] > [*INFO*] ranger-plugin-classloader .. *SUCCESS* [ > 1.910 s] > [*INFO*] HBase Security Plugin Shim . *SUCCESS* > [09:14 min] > [*INFO*] HBase Security Plugin .. *SUCCESS* > [02:55 min] > [*INFO*] Hdfs Security Plugin ... *SUCCESS* [ > 25.848 s] > [*INFO*] Hive Security Plugin ... *SUCCESS* > [17:44 min] > [*INFO*] Knox Security Plugin Shim .. *SUCCESS* > [03:03 min] > [*INFO*] Knox Security Plugin ... *SUCCESS* > [10:18 min] > [*INFO*] Storm Security Plugin .. *SUCCESS* > [01:03 min] > [*INFO*] YARN Security Plugin ... *SUCCESS* [ > 2.659 s] > [*INFO*] Ozone Security Plugin .. *SUCCESS* > [03:51 min] > [*INFO*] Ranger Util *SUCCESS* [ > 5.512 s] > [*INFO*] Unix Authentication Client . *SUCCESS* [ > 9.539 s] > [*INFO*] Security Admin Web Application . *SUCCESS* > [07:36 min] > [*INFO*] KAFKA Security Plugin .. *SUCCESS* > [01:46 min] > [*INFO*] SOLR Security Plugin ... *SUCCESS* > [05:24 min] > [*INFO*] NiFi Security Plugin ... *SUCCESS* [ > 4.675 s] > [*INFO*] NiFi Registry Security Plugin .. *SUCCESS* [ > 4.684 s] > [*INFO*] Unix User Group Synchronizer ... *SUCCESS* > [05:04 min] > [*INFO*] Ldap Config Check Tool . *SUCCESS* [ > 9.450 s] > [*INFO*] Unix Authentication Service *SUCCESS* [ > 1.908 s] > [*INFO*] KMS Security Plugin *SUCCESS* [ > 8.385 s] > [*INFO*] Tag Synchronizer ... *SUCCESS* > [01:32 min] > [*INFO*] Hdfs Security Plugin Shim .. *SUCCESS* [ > 1.964 s] > [*INFO*] Hive Security Plugin Shim .. *SUCCESS* [ > 28.175 s] > [*INFO*] YARN Security Plugin Shim .. *SUCCESS* [ > 1.739 s] > [*INFO*] OZONE Security Plugin Shim . *SUCCESS* [ > 2.754 s] > [*INFO*] Storm Security Plugin shim . *SUCCESS* [ > 7.328 s] > [*INFO*] KAFKA Security Plugin Shim . *SUCCESS* [ > 1.822 s] > [*INFO*] SOLR Security Plugin Shim .. *SUCCESS* [ > 14.409 s] > [*INFO*] Atlas Security Plugin Shim . *SUCCESS* [ > 13.283 s] > [*INFO*] KMS Security Plugin Shim ... *SUCCESS* [ > 2.357 s] > [*INFO*] ranger-examples *SUCCESS* [ > 0.044 s] > [*INFO*] Ranger Examples - Conditions and ContextEnrichers .. *SUCCESS* [ > 4.404 s] > [*INFO*] Ranger Examples - SampleApp *SUCCESS* [ > 0.738 s] > [*INFO*] Ranger Examples - Ranger Plugin for SampleApp .. *SUCCESS* [ > 2.023 s] > [*INFO*] Ranger Tools ... *SUCCESS* [ > 25.620 s] > [*INFO*] Atlas Securi
[jira] [Resolved] (RANGER-2593) Error during Ranger Installations
[
https://issues.apache.org/jira/browse/RANGER-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal resolved RANGER-2593.
-
Resolution: Cannot Reproduce
Unable to reproduce with python 2.7.
> Error during Ranger Installations
> -
>
> Key: RANGER-2593
> URL: https://issues.apache.org/jira/browse/RANGER-2593
> Project: Ranger
> Issue Type: Bug
> Components: admin, build-infra, Ranger
>Reporter: Chethan UK
>Priority: Critical
>
> Unable to install Apache Ranger
>
> RANGER_INSTALL_DIR='/usr/lib/ranger'
>
> pushd "${RANGER_INSTALL_DIR}/ranger-admin" && ./set_globals.sh && ./setup.sh
> ```bash
> Creating new core 'ranger_audits' using command:
> [http://localhost:8983/solr/admin/cores?action=CREATE&name=ranger_audits&instanceDir=ranger_audits]
>
> {
> "responseHeader":
> { "status":0, "QTime":2614}
> ,
> "core":"ranger_audits"}
>
>
> + pushd /usr/lib/ranger/ranger-admin
> /usr/lib/ranger/ranger-admin /usr/lib/ranger
> + ./set_globals.sh
> [2019/09/26 10:58:04]: [I] Soft linking /etc/ranger/admin/conf to
> ews/webapp/WEB-INF/classes/conf
> + ./setup.sh
> 2019-09-26 10:58:04,652 - Running Ranger PolicyManager Web
> Application Install Script -
> 2019-09-26 10:58:04,654 [I] uname=Linux
> 2019-09-26 10:58:04,657 [I] hostname=cluster-ranger-m
> 2019-09-26 10:58:04,663 [I] DB_FLAVOR=MYSQL
> 2019-09-26 10:58:04,666 [I] Audit source=solr
> 2019-09-26 10:58:04,670 [I] Checking distribution name..
> 2019-09-26 10:58:04,679 [I] Found distribution : PRETTY_NAME="Debian
> 2019-09-26 10:58:04,680 [I] check if command
> /usr/lib/jvm/java-8-openjdk-amd64/bin/java exists
> 2019-09-26 10:58:04,682 [I] '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'
> command found
> ./setup.sh: line 325: bc: command not found
> ./setup.sh: line 325: [: -eq: unary operator expected
> 2019-09-26 10:58:04,756 [I] Checking MYSQL CONNECTOR FILE :
> /usr/share/java/mysql-connector-java.jar
> 2019-09-26 10:58:04,757 [I] MYSQL CONNECTOR FILE :
> /usr/share/java/mysql-connector-java.jar file found
> 2019-09-26 10:58:04,760 [I] Setting up UNIX user : ranger and group: ranger
> 2019-09-26 10:58:04,766 [I] the ranger user already exists and belongs to
> group ranger
> 2019-09-26 10:58:04,768 [I] Setting up UNIX user : ranger and group: ranger
> DONE
> 2019-09-26 10:58:04,769 [I] Setting up installation files and directory
> 2019-09-26 10:58:04,774 [I] Creating
> /usr/lib/ranger/ranger-admin/ews/webapp/WEB-INF/classes/lib
> 2019-09-26 10:58:04,778 [I] Setting up init.d
> 2019-09-26 10:58:04,782 [I] Creating script S88ranger-admin/K90ranger-admin
> in /etc/rc2.d directory
> 2019-09-26 10:58:04,786 [I] Creating script S88ranger-admin/K90ranger-admin
> in /etc/rc3.d directory
> 2019-09-26 10:58:04,798 [I] Setting up installation files and directory DONE
> 2019-09-26 10:58:04,821 [I] /usr/lib/ranger/ranger-admin/ews/webapp folder
> found
> 2019-09-26 10:58:04,823 [I]
> db/mysql/optimized/current/ranger_core_db_mysql.sql file found
> 2019-09-26 10:58:04,825 [I] Copying MYSQL Connector to
> /usr/lib/ranger/ranger-admin/ews/webapp/WEB-INF/lib
> 2019-09-26 10:58:04,830 [I] Copying MYSQL Connector to
> /usr/lib/ranger/ranger-admin/ews/webapp/WEB-INF/lib DONE
> 2019-09-26 10:58:04,832 [I] check if command python exists
> 2019-09-26 10:58:04,833 [I] 'python' command found
> File "db_setup.py", line 192
> def is_new_install(xa_db_host, db_user, db_password, db_name):
> ^
> TabError: inconsistent use of tabs and spaces in indentation
> 2019-09-26 10:58:04,856 [E] validatePassword(). Password for rangertagsync
> user cannot be blank
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Review Request 72100: RANGER-2725 Improve the Checkstyle integration from RANGER-2588
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72100/#review219533 --- Ship it! Ship It! - Pradeep Agrawal On Feb. 7, 2020, 3:52 p.m., Lars Francke wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72100/ > --- > > (Updated Feb. 7, 2020, 3:52 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2725 > https://issues.apache.org/jira/browse/RANGER-2725 > > > Repository: ranger > > > Description > --- > > RANGER-2588 Added a basic checkstyle configuration. > > This moves the versions to newer ones (the original ones were from 2015) and > adds a bunch of rules. I went through the Sun and Google code styles as well > as the new proposed OpenJDK ones and picked a bunch of rules that make sense > to me. > > > Diffs > - > > dev-support/checkstyle-java-header.txt eaa97d791 > dev-support/checkstyle-suppressions.xml 0415e48f0 > dev-support/checkstyle.xml 4d7f13491 > pom.xml 0ffbb19a7 > > > Diff: https://reviews.apache.org/r/72100/diff/2/ > > > Testing > --- > > > Thanks, > > Lars Francke > >
[jira] [Commented] (RANGER-2725) Improve the Checkstyle integration from RANGER-2588
[ https://issues.apache.org/jira/browse/RANGER-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033594#comment-17033594 ] Lars Francke commented on RANGER-2725: -- Thanks for committing! What do you mean with "separate patch for each rule"? Do you really want me to create ~30 or so Jiras each enabling a single rule? > Improve the Checkstyle integration from RANGER-2588 > --- > > Key: RANGER-2725 > URL: https://issues.apache.org/jira/browse/RANGER-2725 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Lars Francke >Assignee: Lars Francke >Priority: Minor > Attachments: > 0001-RANGER-2725-Improve-the-Checkstyle-integration-from-.patch, > RANGER-2725.1.patch, RANGER-2725.2.patch > > > RANGER-2588 Added a basic checkstyle configuration. > This moves the versions to newer ones (the original ones were from 2015) and > adds a bunch of rules. I went through the Sun and Google code styles as well > as the new proposed OpenJDK ones and picked a bunch of rules that make sense > to me. > I avoided adding rules that validate names of anything (e.g. classes and so > on). > This will generate a lot of warnings but...well...that just needs cleaning up > :) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (RANGER-2725) Improve the Checkstyle integration from RANGER-2588
[ https://issues.apache.org/jira/browse/RANGER-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17033487#comment-17033487 ] Pradeep Agrawal commented on RANGER-2725: - [~larsfrancke] I have disabled the rules and have committed your change. later we can create separate patch for each rules and track it. > Improve the Checkstyle integration from RANGER-2588 > --- > > Key: RANGER-2725 > URL: https://issues.apache.org/jira/browse/RANGER-2725 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Lars Francke >Assignee: Lars Francke >Priority: Minor > Attachments: > 0001-RANGER-2725-Improve-the-Checkstyle-integration-from-.patch, > RANGER-2725.1.patch, RANGER-2725.2.patch > > > RANGER-2588 Added a basic checkstyle configuration. > This moves the versions to newer ones (the original ones were from 2015) and > adds a bunch of rules. I went through the Sun and Google code styles as well > as the new proposed OpenJDK ones and picked a bunch of rules that make sense > to me. > I avoided adding rules that validate names of anything (e.g. classes and so > on). > This will generate a lot of warnings but...well...that just needs cleaning up > :) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (RANGER-2725) Improve the Checkstyle integration from RANGER-2588
[ https://issues.apache.org/jira/browse/RANGER-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-2725: Attachment: 0001-RANGER-2725-Improve-the-Checkstyle-integration-from-.patch > Improve the Checkstyle integration from RANGER-2588 > --- > > Key: RANGER-2725 > URL: https://issues.apache.org/jira/browse/RANGER-2725 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Lars Francke >Assignee: Lars Francke >Priority: Minor > Attachments: > 0001-RANGER-2725-Improve-the-Checkstyle-integration-from-.patch, > RANGER-2725.1.patch, RANGER-2725.2.patch > > > RANGER-2588 Added a basic checkstyle configuration. > This moves the versions to newer ones (the original ones were from 2015) and > adds a bunch of rules. I went through the Sun and Google code styles as well > as the new proposed OpenJDK ones and picked a bunch of rules that make sense > to me. > I avoided adding rules that validate names of anything (e.g. classes and so > on). > This will generate a lot of warnings but...well...that just needs cleaning up > :) -- This message was sent by Atlassian Jira (v8.3.4#803005)
