[jira] [Created] (RANGER-3561) Upgrade Storm version to 1.2.4
Pradeep Agrawal created RANGER-3561: --- Summary: Upgrade Storm version to 1.2.4 Key: RANGER-3561 URL: https://issues.apache.org/jira/browse/RANGER-3561 Project: Ranger Issue Type: Wish Components: Ranger Affects Versions: 3.0.0 Reporter: Pradeep Agrawal Assignee: Pradeep Agrawal Fix For: 3.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3560) Upgrade kylin version to 2.6.6
[ https://issues.apache.org/jira/browse/RANGER-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3560: Attachment: 0001-RANGER-3560-Upgrade-kylin-version-to-2.6.6.patch > Upgrade kylin version to 2.6.6 > -- > > Key: RANGER-3560 > URL: https://issues.apache.org/jira/browse/RANGER-3560 > Project: Ranger > Issue Type: Wish > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: 0001-RANGER-3560-Upgrade-kylin-version-to-2.6.6.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3561) Upgrade Storm version to 1.2.4
[ https://issues.apache.org/jira/browse/RANGER-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3561: Attachment: 0001-RANGER-3561-Upgrade-Storm-version-to-1.2.4.patch > Upgrade Storm version to 1.2.4 > -- > > Key: RANGER-3561 > URL: https://issues.apache.org/jira/browse/RANGER-3561 > Project: Ranger > Issue Type: Wish > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: 0001-RANGER-3561-Upgrade-Storm-version-to-1.2.4.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3540) Add support to read audit logs from Amazon CloudWatch
[ https://issues.apache.org/jira/browse/RANGER-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3540: Attachment: RANGER-3540-5.patch > Add support to read audit logs from Amazon CloudWatch > - > > Key: RANGER-3540 > URL: https://issues.apache.org/jira/browse/RANGER-3540 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3540-Add-support-to-read-audit-logs-from-Amaz-1.patch, > 0001-RANGER-3540-Add-support-to-read-audit-logs-from-Amaz.patch, > RANGER-3540-3.patch, RANGER-3540-5.patch > > > Add support to read audit logs from Amazon CloudWatch and display the content > in Ranger UI. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3539) Add jacoco-maven-plugin for code coverage
[ https://issues.apache.org/jira/browse/RANGER-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464408#comment-17464408 ] Pradeep Agrawal commented on RANGER-3539: - [~bpatel] : Can you review this ? > Add jacoco-maven-plugin for code coverage > - > > Key: RANGER-3539 > URL: https://issues.apache.org/jira/browse/RANGER-3539 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3539-Add-jacoco-maven-plugin-for-code-coverag.patch > > > References : > # [https://www.baeldung.com/jacoco] > # [https://www.baeldung.com/sonarqube-jacoco-code-coverage] > # [https://mkyong.com/maven/maven-jacoco-code-coverage-example/] > # > [https://medium.com/backend-habit/generate-codecoverage-report-with-jacoco-and-sonarqube-ed15c4045885] > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3540) Add support to read audit logs from Amazon CloudWatch
[ https://issues.apache.org/jira/browse/RANGER-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3540: Attachment: RANGER-3540-6.patch > Add support to read audit logs from Amazon CloudWatch > - > > Key: RANGER-3540 > URL: https://issues.apache.org/jira/browse/RANGER-3540 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3540-Add-support-to-read-audit-logs-from-Amaz-1.patch, > 0001-RANGER-3540-Add-support-to-read-audit-logs-from-Amaz.patch, > RANGER-3540-3.patch, RANGER-3540-5.patch, RANGER-3540-6.patch > > > Add support to read audit logs from Amazon CloudWatch and display the content > in Ranger UI. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3539) Add jacoco-maven-plugin for code coverage
[ https://issues.apache.org/jira/browse/RANGER-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464497#comment-17464497 ] Pradeep Agrawal commented on RANGER-3539: - 1) Yes, jacoco need to be installed but its just to view the details in report format. This is optional and for those who want to use it. The link i had given has the steps to install jacoco. it will not take more than 5 minutes. Other alternate approach for developers is to use *EclEmma* eclipse plugin to get the coverage of each java file. EclEmma tool also uses jacoco libraries internally. 2) If you refer the given links for installation, it seems jacoco installation could be specific to every one's environment so not sure it will be a good idea to plug it with build process, causing a ranger build process to be a blocker if its does not fit to any environment. Proposed plugin does not force build process to have jacoco installed. Previous commands (mvn clean install) will continue working without referring to jacoco plugin. As of now i don't want to make it mandatory unless we have some decent level of coverage. 3) As i mentioned above no need to make it part of build process and its a standard mvn command for jacoco plugin and easily available on the net, so no need to remember. 4) Yes, But currently overall coverage is too low, we can not force anything as of now. Yes, it can be addressed separately but people need to contribute for missing test cases to increase the coverage to a decent level. 5) Use EclEmma eclipse plugin for eclipse IDE, we can find same for other IDE's. Refer [https://www.eclemma.org/] > Add jacoco-maven-plugin for code coverage > - > > Key: RANGER-3539 > URL: https://issues.apache.org/jira/browse/RANGER-3539 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3539-Add-jacoco-maven-plugin-for-code-coverag.patch > > > References : > # [https://www.baeldung.com/jacoco] > # [https://www.baeldung.com/sonarqube-jacoco-code-coverage] > # [https://mkyong.com/maven/maven-jacoco-code-coverage-example/] > # > [https://medium.com/backend-habit/generate-codecoverage-report-with-jacoco-and-sonarqube-ed15c4045885] > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-2994) ranger release 2.1.0 cannot compile due to broken Kylin plugin dependency
[ https://issues.apache.org/jira/browse/RANGER-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17464620#comment-17464620 ] Pradeep Agrawal commented on RANGER-2994: - [~sneethiraj] : Can you review RANGER-3560 patch (https://reviews.apache.org/r/73775) > ranger release 2.1.0 cannot compile due to broken Kylin plugin dependency > - > > Key: RANGER-2994 > URL: https://issues.apache.org/jira/browse/RANGER-2994 > Project: Ranger > Issue Type: Bug > Components: plugins >Affects Versions: 2.1.0 >Reporter: Georgi Ivanov >Assignee: Selvamohan Neethiraj >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.1 > > Attachments: > 0001-RANGER-2994-kylin-library-version-is-updated-to-a-li.patch > > > Ranger 2.1.0 release cannot compile due to broken kylin maven dependency. > the kylin version is set to 2.6.4 in the main project pom.xml > > 2.6.4 > > plugin-kylin/pom.xml > ranger-kylin-plugin-shim/pom.xml > both reference kylin-server-base which has dependency on kylin-datasource-sdk > which in turn has dependency on calcite-linq4j-1.16.0-kylin-r2.jar which is > not available. > > mvn org.apache.maven.plugins:maven-dependency-plugin:3.1.2:copy > -Dartifact=org.apache.calcite:calcite-linq4j:jar:1.16.0-kylin-r2 > -DoutputDirectory=./ > fails > and > [https://mvnrepository.com/artifact/org.apache.kylin/kylin-datasource-sdk/2.6.4] > > shows the broken dependency. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3540) Add support to read audit logs from Amazon CloudWatch
[ https://issues.apache.org/jira/browse/RANGER-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17465277#comment-17465277 ] Pradeep Agrawal commented on RANGER-3540: - Thanks [~yInnovation] for reviewing and guiding on this. I am going to commit the changes. > Add support to read audit logs from Amazon CloudWatch > - > > Key: RANGER-3540 > URL: https://issues.apache.org/jira/browse/RANGER-3540 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3540-Add-support-to-read-audit-logs-from-Amaz-1.patch, > 0001-RANGER-3540-Add-support-to-read-audit-logs-from-Amaz.patch, > RANGER-3540-3.patch, RANGER-3540-5.patch, RANGER-3540-6.patch > > > Add support to read audit logs from Amazon CloudWatch and display the content > in Ranger UI. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3290) ArrayIndexOutOfBoundsException if solr is down
[
https://issues.apache.org/jira/browse/RANGER-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17465937#comment-17465937
]
Pradeep Agrawal commented on RANGER-3290:
-
https://github.com/apache/ranger/commit/bbbc23c005432c715d741e6ea61a9aaddc569845
> ArrayIndexOutOfBoundsException if solr is down
> --
>
> Key: RANGER-3290
> URL: https://issues.apache.org/jira/browse/RANGER-3290
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Reporter: Andras Salamon
>Priority: Minor
> Attachments: RANGER-3290-01.patch, RANGER-3290-02.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> [SolrCollectionBootstrapper|https://github.com/apache/ranger/blob/master/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java]
> uploads a config to solr and creates a collection.
> If solr is down (there are no live servers) during the collection creation
> phase, we got a meaningful error message (from catalina.out):
> {noformat}May 19, 2021 4:37:44 PM
> org.apache.ranger.server.tomcat.SolrCollectionBootstrapper getCollections
> SEVERE: getCollections() operation failed :
> org.apache.solr.client.solrj.SolrServerException: No live SolrServers
> available to handle this request{noformat}
> On the other hand, if solr is down during the config upload phase the error
> message is not too useful:
> {noformat}May 20, 2021 8:07:59 AM
> org.apache.ranger.server.tomcat.SolrCollectionBootstrapper uploadConfiguration
> SEVERE: Error while uploading configs :
> java.lang.ArrayIndexOutOfBoundsException: 0{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (RANGER-3290) ArrayIndexOutOfBoundsException if solr is down
[
https://issues.apache.org/jira/browse/RANGER-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal resolved RANGER-3290.
-
Fix Version/s: 3.0.0
Resolution: Fixed
> ArrayIndexOutOfBoundsException if solr is down
> --
>
> Key: RANGER-3290
> URL: https://issues.apache.org/jira/browse/RANGER-3290
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Reporter: Andras Salamon
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: RANGER-3290-01.patch, RANGER-3290-02.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> [SolrCollectionBootstrapper|https://github.com/apache/ranger/blob/master/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java]
> uploads a config to solr and creates a collection.
> If solr is down (there are no live servers) during the collection creation
> phase, we got a meaningful error message (from catalina.out):
> {noformat}May 19, 2021 4:37:44 PM
> org.apache.ranger.server.tomcat.SolrCollectionBootstrapper getCollections
> SEVERE: getCollections() operation failed :
> org.apache.solr.client.solrj.SolrServerException: No live SolrServers
> available to handle this request{noformat}
> On the other hand, if solr is down during the config upload phase the error
> message is not too useful:
> {noformat}May 20, 2021 8:07:59 AM
> org.apache.ranger.server.tomcat.SolrCollectionBootstrapper uploadConfiguration
> SEVERE: Error while uploading configs :
> java.lang.ArrayIndexOutOfBoundsException: 0{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3290) ArrayIndexOutOfBoundsException if solr is down
[
https://issues.apache.org/jira/browse/RANGER-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17466117#comment-17466117
]
Pradeep Agrawal commented on RANGER-3290:
-
[~asalamon74] : Please close the Review request.
> ArrayIndexOutOfBoundsException if solr is down
> --
>
> Key: RANGER-3290
> URL: https://issues.apache.org/jira/browse/RANGER-3290
> Project: Ranger
> Issue Type: Improvement
> Components: audit
>Reporter: Andras Salamon
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: RANGER-3290-01.patch, RANGER-3290-02.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> [SolrCollectionBootstrapper|https://github.com/apache/ranger/blob/master/embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java]
> uploads a config to solr and creates a collection.
> If solr is down (there are no live servers) during the collection creation
> phase, we got a meaningful error message (from catalina.out):
> {noformat}May 19, 2021 4:37:44 PM
> org.apache.ranger.server.tomcat.SolrCollectionBootstrapper getCollections
> SEVERE: getCollections() operation failed :
> org.apache.solr.client.solrj.SolrServerException: No live SolrServers
> available to handle this request{noformat}
> On the other hand, if solr is down during the config upload phase the error
> message is not too useful:
> {noformat}May 20, 2021 8:07:59 AM
> org.apache.ranger.server.tomcat.SolrCollectionBootstrapper uploadConfiguration
> SEVERE: Error while uploading configs :
> java.lang.ArrayIndexOutOfBoundsException: 0{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3550) support for using user/tag attributes in row-filter expressions and conditions
[
https://issues.apache.org/jira/browse/RANGER-3550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17466397#comment-17466397
]
Pradeep Agrawal commented on RANGER-3550:
-
[~madhan] : Please close the RR [https://reviews.apache.org/r/73759/]
> support for using user/tag attributes in row-filter expressions and conditions
> --
>
> Key: RANGER-3550
> URL: https://issues.apache.org/jira/browse/RANGER-3550
> Project: Ranger
> Issue Type: Improvement
> Components: plugins
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Enhancing row-filtering (introduced in Ranger 0.6.0 - RANGER-908) to enable
> use of user attributes in filter expressions can help address a wider set of
> use cases, including the following:
> # restrict users to see only records of the department they belong to:
> {code:java}
> dept = '${{USER.dept}}'{code}
> # restrict users to see only records assigned to them:
> {code:java}
> assignee = '${{USER._name}}'{code}
>
> In addition, it will be useful to be able to refer user/tag attributes in
> condition expressions, as shown in following examples:
> # allow access only for full-time users:
> {code:java}
> ${{USER.employeeType == 'full-time'}}{code}
> # allow access only if VISIBILITY tag has attribute type set to public:
> {code:java}
> ${{TAGS.VISIBILITY.level == 'public'}}{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (RANGER-3576) service creation is failing intermittently due to MySQLIntegrityConstraintViolationException
Pradeep Agrawal created RANGER-3576:
---
Summary: service creation is failing intermittently due to
MySQLIntegrityConstraintViolationException
Key: RANGER-3576
URL: https://issues.apache.org/jira/browse/RANGER-3576
Project: Ranger
Issue Type: Bug
Components: Ranger
Affects Versions: 2.2.0, 2.1.0, 2.0.0, 3.0.0, 2.3.0
Reporter: Pradeep Agrawal
Assignee: Pradeep Agrawal
Fix For: 3.0.0, 2.3.0
Parallel service creation with the same users in the service config or default
policies may fail with following error.
{code:java}
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
violates unique constraint "x_user_uk_user_name"
Detail: Key (user_name)=(rangerlookup) already exists.
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:312)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
at
org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:153)
at
org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:119)
at
com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:410)
at
org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898)
... 144 more
2022-01-04 20:15:45,422 ERROR org.apache.ranger.biz.XUserMgr: Error creating
user: rangerlookup
javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse
Persistence Services - 2.7.7.v20200504-69f2c2b80d):
org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key
value violates unique constraint "x_user_uk_user_name"
Detail: Key (user_name)=(rangerlookup) already exists.
Error Code: 0
Call: INSERT INTO x_user (ID, ADDED_BY_ID, CREATE_TIME, CRED_STORE_ID, DESCR,
IS_VISIBLE, USER_NAME, OTHER_ATTRIBUTES, STATUS, UPDATE_TIME, UPD_BY_ID) VALUES
(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
bind => [11 parameters bound]
at
org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:980)
at jdk.internal.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:301)
at com.sun.proxy.$Proxy26.flush(Unknown Source)
at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:90)
at
org.apache.ranger.service.AbstractBaseResourceService.createResource(AbstractBaseResourceService.java:258)
at
org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2535)
at
org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at
org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:283)
at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
at
org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$886cf35.createServiceConfigUser()
at
org.apache.ranger.biz.ServiceDBStore.createDefaultPolicyUsersAndGroups(ServiceDBStore.java:3321)
at
org.apache.ranger.biz.ServiceDBStore.populateDefaultPolicies(ServiceDBStore.java:3236)
at
org.apache.ranger.biz.ServiceDBStore.createDefaultPolicies(ServiceDBStore.java:3139)
at
org.apache.ranger.biz.ServiceDBStore.createService(ServiceDBStore.java:1545)
at org.apache.ranger.rest.ServiceREST.createService(ServiceREST.java:767)
{code}
Note: Issue is intermittent and difficult to reproduce manually.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (RANGER-3576) service creation is failing intermittently due to DB unique key constraint violation
[
https://issues.apache.org/jira/browse/RANGER-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3576:
Summary: service creation is failing intermittently due to DB unique key
constraint violation (was: service creation is failing intermittently due to
MySQLIntegrityConstraintViolationException)
> service creation is failing intermittently due to DB unique key constraint
> violation
>
>
> Key: RANGER-3576
> URL: https://issues.apache.org/jira/browse/RANGER-3576
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.0.0, 2.1.0, 3.0.0, 2.2.0, 2.3.0
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Parallel service creation with the same users in the service config or
> default policies may fail with following error.
> {code:java}
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:312)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:153)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:119)
> at
> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:410)
> at
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898)
> ... 144 more
> 2022-01-04 20:15:45,422 ERROR org.apache.ranger.biz.XUserMgr: Error creating
> user: rangerlookup
> javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse
> Persistence Services - 2.7.7.v20200504-69f2c2b80d):
> org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key
> value violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> Error Code: 0
> Call: INSERT INTO x_user (ID, ADDED_BY_ID, CREATE_TIME, CRED_STORE_ID, DESCR,
> IS_VISIBLE, USER_NAME, OTHER_ATTRIBUTES, STATUS, UPDATE_TIME, UPD_BY_ID)
> VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
> bind => [11 parameters bound]
> at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:980)
> at jdk.internal.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:301)
> at com.sun.proxy.$Proxy26.flush(Unknown Source)
> at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:90)
> at
> org.apache.ranger.service.AbstractBaseResourceService.createResource(AbstractBaseResourceService.java:258)
> at
> org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2535)
> at
> org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:283)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
> at
> org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$886cf35.createServiceConfigUser()
> at
> org.apache.ranger.biz.ServiceDBStore.createDefaultPolicyUsersA
[jira] [Updated] (RANGER-3576) service creation is failing intermittently due to DB unique key constraint violation
[
https://issues.apache.org/jira/browse/RANGER-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3576:
Attachment: 0001-RANGER-3576-service-creation-is-failing-intermittent.patch
> service creation is failing intermittently due to DB unique key constraint
> violation
>
>
> Key: RANGER-3576
> URL: https://issues.apache.org/jira/browse/RANGER-3576
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.0.0, 2.1.0, 3.0.0, 2.2.0, 2.3.0
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3576-service-creation-is-failing-intermittent.patch
>
>
> Parallel service creation with the same users in the service config or
> default policies may fail with following error.
> {code:java}
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:312)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:153)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:119)
> at
> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:410)
> at
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898)
> ... 144 more
> 2022-01-04 20:15:45,422 ERROR org.apache.ranger.biz.XUserMgr: Error creating
> user: rangerlookup
> javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse
> Persistence Services - 2.7.7.v20200504-69f2c2b80d):
> org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key
> value violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> Error Code: 0
> Call: INSERT INTO x_user (ID, ADDED_BY_ID, CREATE_TIME, CRED_STORE_ID, DESCR,
> IS_VISIBLE, USER_NAME, OTHER_ATTRIBUTES, STATUS, UPDATE_TIME, UPD_BY_ID)
> VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
> bind => [11 parameters bound]
> at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:980)
> at jdk.internal.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:301)
> at com.sun.proxy.$Proxy26.flush(Unknown Source)
> at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:90)
> at
> org.apache.ranger.service.AbstractBaseResourceService.createResource(AbstractBaseResourceService.java:258)
> at
> org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2535)
> at
> org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:283)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
> at
> org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$886cf35.createServiceConfigUser()
> at
> org.apache.ranger.biz.ServiceDBStore.createDefaultPolicyUsersAndGroups(ServiceDBStore.java:
[jira] [Updated] (RANGER-3576) service creation is failing intermittently due to DB unique key constraint violation
[
https://issues.apache.org/jira/browse/RANGER-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3576:
Attachment:
0001-RANGER-3576-service-creation-is-failing-intermittent-1.patch
> service creation is failing intermittently due to DB unique key constraint
> violation
>
>
> Key: RANGER-3576
> URL: https://issues.apache.org/jira/browse/RANGER-3576
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.0.0, 2.1.0, 3.0.0, 2.2.0, 2.3.0
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3576-service-creation-is-failing-intermittent-1.patch
>
>
> Parallel service creation with the same users in the service config or
> default policies may fail with following error.
> {code:java}
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:312)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:153)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:119)
> at
> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:410)
> at
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898)
> ... 144 more
> 2022-01-04 20:15:45,422 ERROR org.apache.ranger.biz.XUserMgr: Error creating
> user: rangerlookup
> javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse
> Persistence Services - 2.7.7.v20200504-69f2c2b80d):
> org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key
> value violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> Error Code: 0
> Call: INSERT INTO x_user (ID, ADDED_BY_ID, CREATE_TIME, CRED_STORE_ID, DESCR,
> IS_VISIBLE, USER_NAME, OTHER_ATTRIBUTES, STATUS, UPDATE_TIME, UPD_BY_ID)
> VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
> bind => [11 parameters bound]
> at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:980)
> at jdk.internal.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:301)
> at com.sun.proxy.$Proxy26.flush(Unknown Source)
> at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:90)
> at
> org.apache.ranger.service.AbstractBaseResourceService.createResource(AbstractBaseResourceService.java:258)
> at
> org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2535)
> at
> org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:283)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
> at
> org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$886cf35.createServiceConfigUser()
> at
> org.apache.ranger.biz.ServiceDBStore.createDefaultPolicyUsersAndGroups(ServiceDBStore.
[jira] [Updated] (RANGER-3576) service creation is failing intermittently due to DB unique key constraint violation
[
https://issues.apache.org/jira/browse/RANGER-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3576:
Attachment: (was:
0001-RANGER-3576-service-creation-is-failing-intermittent.patch)
> service creation is failing intermittently due to DB unique key constraint
> violation
>
>
> Key: RANGER-3576
> URL: https://issues.apache.org/jira/browse/RANGER-3576
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.0.0, 2.1.0, 3.0.0, 2.2.0, 2.3.0
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3576-service-creation-is-failing-intermittent-1.patch
>
>
> Parallel service creation with the same users in the service config or
> default policies may fail with following error.
> {code:java}
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:312)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:153)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:119)
> at
> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:410)
> at
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898)
> ... 144 more
> 2022-01-04 20:15:45,422 ERROR org.apache.ranger.biz.XUserMgr: Error creating
> user: rangerlookup
> javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse
> Persistence Services - 2.7.7.v20200504-69f2c2b80d):
> org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key
> value violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> Error Code: 0
> Call: INSERT INTO x_user (ID, ADDED_BY_ID, CREATE_TIME, CRED_STORE_ID, DESCR,
> IS_VISIBLE, USER_NAME, OTHER_ATTRIBUTES, STATUS, UPDATE_TIME, UPD_BY_ID)
> VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
> bind => [11 parameters bound]
> at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:980)
> at jdk.internal.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:301)
> at com.sun.proxy.$Proxy26.flush(Unknown Source)
> at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:90)
> at
> org.apache.ranger.service.AbstractBaseResourceService.createResource(AbstractBaseResourceService.java:258)
> at
> org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2535)
> at
> org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:283)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
> at
> org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$886cf35.createServiceConfigUser()
> at
> org.apache.ranger.biz.ServiceDBStore.createDefaultPolicyUsersAndGroups(Servic
[jira] [Assigned] (RANGER-2999) Ranger build is failing if you do not have already downloaded jar under ~/.m2 for calcite-linq4j-1.16.0-kylin-r2
[ https://issues.apache.org/jira/browse/RANGER-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal reassigned RANGER-2999: --- Assignee: Pradeep Agrawal > Ranger build is failing if you do not have already downloaded jar under ~/.m2 > for calcite-linq4j-1.16.0-kylin-r2 > > > Key: RANGER-2999 > URL: https://issues.apache.org/jira/browse/RANGER-2999 > Project: Ranger > Issue Type: Bug > Components: build-infra >Reporter: Selvamohan Neethiraj >Assignee: Pradeep Agrawal >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > 21:07:17 [ERROR] Failed to execute goal on project ranger-kylin-plugin: Could > not resolve dependencies for project > org.apache.ranger:ranger-kylin-plugin:jar:3.0.0-SNAPSHOT: Failed to collect > dependencies at org.apache.kylin:kylin-server-base:jar:2.6.6 -> > org.apache.kylin:kylin-query:jar:2.6.6 -> > org.apache.kylin:kylin-datasource-sdk:jar:2.6.6 -> > org.apache.calcite:calcite-linq4j:jar:1.16.0-kylin-r2: Failed to read > artifact descriptor for > org.apache.calcite:calcite-linq4j:jar:1.16.0-kylin-r2: Could not transfer > artifact org.apache.calcite:calcite-linq4j:pom:1.16.0-kylin-r2 from/to > spring-snapshots (http://repo.spring.io/libs-snapshot): Authorization failed > for > http://repo.spring.io/libs-snapshot/org/apache/calcite/calcite-linq4j/1.16.0-kylin-r2/calcite-linq4j-1.16.0-kylin-r2.pom > 403 Forbidden -> [Help 1] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (RANGER-2999) Ranger build is failing if you do not have already downloaded jar under ~/.m2 for calcite-linq4j-1.16.0-kylin-r2
[ https://issues.apache.org/jira/browse/RANGER-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-2999. - Fix Version/s: 3.0.0 Resolution: Duplicate > Ranger build is failing if you do not have already downloaded jar under ~/.m2 > for calcite-linq4j-1.16.0-kylin-r2 > > > Key: RANGER-2999 > URL: https://issues.apache.org/jira/browse/RANGER-2999 > Project: Ranger > Issue Type: Bug > Components: build-infra >Reporter: Selvamohan Neethiraj >Assignee: Pradeep Agrawal >Priority: Critical > Fix For: 3.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 21:07:17 [ERROR] Failed to execute goal on project ranger-kylin-plugin: Could > not resolve dependencies for project > org.apache.ranger:ranger-kylin-plugin:jar:3.0.0-SNAPSHOT: Failed to collect > dependencies at org.apache.kylin:kylin-server-base:jar:2.6.6 -> > org.apache.kylin:kylin-query:jar:2.6.6 -> > org.apache.kylin:kylin-datasource-sdk:jar:2.6.6 -> > org.apache.calcite:calcite-linq4j:jar:1.16.0-kylin-r2: Failed to read > artifact descriptor for > org.apache.calcite:calcite-linq4j:jar:1.16.0-kylin-r2: Could not transfer > artifact org.apache.calcite:calcite-linq4j:pom:1.16.0-kylin-r2 from/to > spring-snapshots (http://repo.spring.io/libs-snapshot): Authorization failed > for > http://repo.spring.io/libs-snapshot/org/apache/calcite/calcite-linq4j/1.16.0-kylin-r2/calcite-linq4j-1.16.0-kylin-r2.pom > 403 Forbidden -> [Help 1] -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3576) service creation is failing intermittently due to DB unique key constraint violation
[
https://issues.apache.org/jira/browse/RANGER-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3576:
Attachment: 0001-RANGER-3576-service-creation-is-failing-intermittent.patch
> service creation is failing intermittently due to DB unique key constraint
> violation
>
>
> Key: RANGER-3576
> URL: https://issues.apache.org/jira/browse/RANGER-3576
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 2.0.0, 2.1.0, 3.0.0, 2.2.0, 2.3.0
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3576-service-creation-is-failing-intermittent-1.patch,
> 0001-RANGER-3576-service-creation-is-failing-intermittent.patch
>
>
> Parallel service creation with the same users in the service config or
> default policies may fail with following error.
> {code:java}
> Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
> violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2532)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2267)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:312)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:153)
> at
> org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:119)
> at
> com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:410)
> at
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898)
> ... 144 more
> 2022-01-04 20:15:45,422 ERROR org.apache.ranger.biz.XUserMgr: Error creating
> user: rangerlookup
> javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse
> Persistence Services - 2.7.7.v20200504-69f2c2b80d):
> org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key
> value violates unique constraint "x_user_uk_user_name"
> Detail: Key (user_name)=(rangerlookup) already exists.
> Error Code: 0
> Call: INSERT INTO x_user (ID, ADDED_BY_ID, CREATE_TIME, CRED_STORE_ID, DESCR,
> IS_VISIBLE, USER_NAME, OTHER_ATTRIBUTES, STATUS, UPDATE_TIME, UPD_BY_ID)
> VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
> bind => [11 parameters bound]
> at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:980)
> at jdk.internal.reflect.GeneratedMethodAccessor91.invoke(Unknown Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:301)
> at com.sun.proxy.$Proxy26.flush(Unknown Source)
> at org.apache.ranger.common.db.BaseDao.create(BaseDao.java:90)
> at
> org.apache.ranger.service.AbstractBaseResourceService.createResource(AbstractBaseResourceService.java:258)
> at
> org.apache.ranger.biz.XUserMgr.createServiceConfigUser(XUserMgr.java:2535)
> at
> org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:283)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
> at
> org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$886cf35.createServiceConfigUser()
> at
> org.apache.ranger.biz
[jira] [Updated] (RANGER-3498) RANGER : Remove log4j1 dependencies.
[ https://issues.apache.org/jira/browse/RANGER-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3498: Attachment: slf4j_patch1.diff > RANGER : Remove log4j1 dependencies. > > > Key: RANGER-3498 > URL: https://issues.apache.org/jira/browse/RANGER-3498 > Project: Ranger > Issue Type: Task > Components: Ranger >Reporter: Mateen N Mansoori >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 3.0.0 > > Attachments: slf4j_patch1.diff > > > Remove log4j1 dependencies. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3498) RANGER : Remove log4j1 dependencies.
[ https://issues.apache.org/jira/browse/RANGER-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17474533#comment-17474533 ] Pradeep Agrawal commented on RANGER-3498: - [~madhan] : I needed more changes to get the logs for ranger-admin, ranger-usersync, ranger-tagsync modules. Please see if they need to be added in your patch. please refer [^slf4j_patch1.diff] . I have not tried the plugins or other module logs as i don't have the environment currently but will update here soon if anything need to be checked. > RANGER : Remove log4j1 dependencies. > > > Key: RANGER-3498 > URL: https://issues.apache.org/jira/browse/RANGER-3498 > Project: Ranger > Issue Type: Task > Components: Ranger >Reporter: Mateen N Mansoori >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 3.0.0 > > Attachments: slf4j_patch1.diff > > > Remove log4j1 dependencies. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (RANGER-3589) Ranger java patches failing due to admin privilege checks.
[ https://issues.apache.org/jira/browse/RANGER-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-3589. - Resolution: Fixed https://github.com/apache/ranger/commit/65798673112e037d2f884f78eaeb05588dbf0659 > Ranger java patches failing due to admin privilege checks. > -- > > Key: RANGER-3589 > URL: https://issues.apache.org/jira/browse/RANGER-3589 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Mateen N Mansoori >Priority: Major > > Ranger java patch will fail if using ServiceDBStore's updatePolicy(...) > method. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (RANGER-3589) Ranger java patches failing due to admin privilege checks.
[ https://issues.apache.org/jira/browse/RANGER-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal reassigned RANGER-3589: --- Assignee: Mateen Mansoori > Ranger java patches failing due to admin privilege checks. > -- > > Key: RANGER-3589 > URL: https://issues.apache.org/jira/browse/RANGER-3589 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Mateen N Mansoori >Assignee: Mateen Mansoori >Priority: Major > > Ranger java patch will fail if using ServiceDBStore's updatePolicy(...) > method. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (RANGER-3558) Remove Ranger dependency on log4j 1.X
[ https://issues.apache.org/jira/browse/RANGER-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-3558. - Fix Version/s: 3.0.0 Resolution: Duplicate > Remove Ranger dependency on log4j 1.X > - > > Key: RANGER-3558 > URL: https://issues.apache.org/jira/browse/RANGER-3558 > Project: Ranger > Issue Type: Wish > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > > if possible lets remove Ranger dependency on log4j 1.X (1.2.17) -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Reopened] (RANGER-3589) Ranger java patches failing due to admin privilege checks.
[ https://issues.apache.org/jira/browse/RANGER-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal reopened RANGER-3589: - > Ranger java patches failing due to admin privilege checks. > -- > > Key: RANGER-3589 > URL: https://issues.apache.org/jira/browse/RANGER-3589 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Mateen N Mansoori >Assignee: Mateen Mansoori >Priority: Major > > Ranger java patch will fail if using ServiceDBStore's updatePolicy(...) > method. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3589) Ranger java patches failing due to admin privilege checks.
[ https://issues.apache.org/jira/browse/RANGER-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17478342#comment-17478342 ] Pradeep Agrawal commented on RANGER-3589: - Reopening this issue to handle it better way(if possible) > Ranger java patches failing due to admin privilege checks. > -- > > Key: RANGER-3589 > URL: https://issues.apache.org/jira/browse/RANGER-3589 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Mateen N Mansoori >Assignee: Mateen Mansoori >Priority: Major > > Ranger java patch will fail if using ServiceDBStore's updatePolicy(...) > method. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (RANGER-3589) Ranger java patches failing due to admin privilege checks.
[ https://issues.apache.org/jira/browse/RANGER-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-3589. - Fix Version/s: 3.0.0 Resolution: Fixed https://github.com/apache/ranger/commit/bfc9c544107dd6d2a052dc8941503de335f930b6 > Ranger java patches failing due to admin privilege checks. > -- > > Key: RANGER-3589 > URL: https://issues.apache.org/jira/browse/RANGER-3589 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Mateen N Mansoori >Assignee: Mateen Mansoori >Priority: Major > Fix For: 3.0.0 > > > Ranger java patch will fail if using ServiceDBStore's updatePolicy(...) > method. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3579) Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832
[ https://issues.apache.org/jira/browse/RANGER-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479320#comment-17479320 ] Pradeep Agrawal commented on RANGER-3579: - [~brahmareddy] : check this commit https://github.com/apache/ranger/commit/71809108fd106b664b6f9d53e0efd86d4c5cd039 > Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832 > -- > > Key: RANGER-3579 > URL: https://issues.apache.org/jira/browse/RANGER-3579 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Brahma Reddy Battula >Priority: Major > Attachments: RANGER-3579.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3579) Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832
[ https://issues.apache.org/jira/browse/RANGER-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479322#comment-17479322 ] Pradeep Agrawal commented on RANGER-3579: - for 2.3 branch : https://reviews.apache.org/r/73806/ > Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832 > -- > > Key: RANGER-3579 > URL: https://issues.apache.org/jira/browse/RANGER-3579 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Brahma Reddy Battula >Priority: Major > Attachments: RANGER-3579.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (RANGER-3591) Upgrade protobuf-java to 3.19.3
Pradeep Agrawal created RANGER-3591: --- Summary: Upgrade protobuf-java to 3.19.3 Key: RANGER-3591 URL: https://issues.apache.org/jira/browse/RANGER-3591 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Pradeep Agrawal Assignee: Pradeep Agrawal Fix For: 3.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (RANGER-3592) Upgrade Spring framework to 5.3.15
Pradeep Agrawal created RANGER-3592: --- Summary: Upgrade Spring framework to 5.3.15 Key: RANGER-3592 URL: https://issues.apache.org/jira/browse/RANGER-3592 Project: Ranger Issue Type: Bug Components: Ranger Reporter: Pradeep Agrawal Assignee: Pradeep Agrawal Fix For: 3.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3592) Upgrade Spring framework to 5.3.15
[ https://issues.apache.org/jira/browse/RANGER-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3592: Attachment: 0002-RANGER-3592-Upgrade-Spring-framework-to-5.3.15.patch > Upgrade Spring framework to 5.3.15 > -- > > Key: RANGER-3592 > URL: https://issues.apache.org/jira/browse/RANGER-3592 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: 0002-RANGER-3592-Upgrade-Spring-framework-to-5.3.15.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3591) Upgrade protobuf-java to 3.19.3
[ https://issues.apache.org/jira/browse/RANGER-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3591: Attachment: 0001-RANGER-3591-Upgrade-protobuf-java-to-3.19.3.patch > Upgrade protobuf-java to 3.19.3 > --- > > Key: RANGER-3591 > URL: https://issues.apache.org/jira/browse/RANGER-3591 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: 0001-RANGER-3591-Upgrade-protobuf-java-to-3.19.3.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3579) Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832
[ https://issues.apache.org/jira/browse/RANGER-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480097#comment-17480097 ] Pradeep Agrawal commented on RANGER-3579: - [~bpatel] : No more log4j changes please, try to exclude and remove usage if you find any references. There are few things still need to be sorted out on top of madhan's patch. > Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832 > -- > > Key: RANGER-3579 > URL: https://issues.apache.org/jira/browse/RANGER-3579 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Brahma Reddy Battula >Priority: Major > Attachments: RANGER-3579.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3595) Tar of KMS contains rubbish files
[
https://issues.apache.org/jira/browse/RANGER-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480176#comment-17480176
]
Pradeep Agrawal commented on RANGER-3595:
-
[~kirbyzhou] : Related to the problem 1 you mentioned above. why you think that
kms.xml entry is the problem for kms while similar entry in admin-web.xml is
not the problem for security-admin.
Reference:
[https://github.com/apache/ranger/blob/master/distro/src/main/assembly/admin-web.xml#L26]
> Tar of KMS contains rubbish files
> -
>
> Key: RANGER-3595
> URL: https://issues.apache.org/jira/browse/RANGER-3595
> Project: Ranger
> Issue Type: Improvement
> Components: kms
>Reporter: kirby zhou
>Priority: Major
>
> There are lots of .class files under ews/webapp/. They wont be loaded by any
> classpath. And they are duplicated against files inside
> ews/webapp/lib/ranger-kms-3.0.0-SNAPSHOT.jar.
> It seems dirty and may cause some security problem.
> {code:bash}
> #] tar tf target/ranger-3.0.0-SNAPSHOT-kms.tar.gz ranger-3.0.0-SNAPSHOT-kms/
> | egrep 'ews/webapp/org' | head
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/biz/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/dao/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/entity/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/crypto/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/crypto/key/
> //代码占位符
> {code}
> * The reason is that:
> distro/src/main/assembly/kms.xml
>
> {code:java}
>
> true
>
> org.apache.ranger:ranger-kms
>
>
> ews/webapp
> false
> true
>
> {code}
> Why ?
>
> The secret is in kms/scripts/setup.sh:
>
> {code:java}
> setup_kms(){
> #copying ranger kms provider
> oldP=${PWD}
> cd $PWD/ews/webapp
> log "[I] Adding ranger kms provider as services in hadoop-common jar"
> for f in lib/hadoop-common*.jar
> do
> ${JAVA_HOME}/bin/jar -uf ${f}
> META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory
> chown ${unix_user}:${unix_group} ${f}
> done
> cd ${oldP}
> }
> {code}
>
>
> The code above is VERY VERY DIRTY!
> It hacks into hadoop-common.jar., Overwrite resource
> "META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory". Ensure
> the following code can load
> 'org.apache.hadoop.crypto.key.RangerKeyStoreProvider$Factory' by
> 'META-INF/.../KeyProviderFactory'.
>
>
> {code:java}
> // org.apache.hadoop.crypto.key: KeyProviderFactory.java
> private static final ServiceLoader serviceLoader =
> ServiceLoader.load(KeyProviderFactory.class,
> KeyProviderFactory.class.getClassLoader());
> {code}
>
>
> But this is unnecessary.
> ServiceLoader will read all resources with the same name using the
> ClassLoader of KeyProviderFactory. We just need to put a jar contains that
> property side by side of hadoop-common.jar ( ews/webapp/lib/ ). And
> ranger-kms-3.0.0-SNAPSHOT.jar already here.
> {code:java}
> % tar tf ../target/ranger-*-kms.tar.gz | egrep 'kms[^/]*\.jar|hadoop-common'
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/lib/ranger-kms-3.0.0-SNAPSHOT.jar
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/lib/hadoop-common-3.3.0.jar
> ...
> % tar tf target/ranger-kms-3.0.0-SNAPSHOT.jar | fgrep ProviderFactory
> META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3587) Failed to apply 046*.sql patch
[ https://issues.apache.org/jira/browse/RANGER-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480654#comment-17480654 ] Pradeep Agrawal commented on RANGER-3587: - [~bpatel] I tested upgrade from 2.0 to 2.1 and 3.0, using mysql connector jar 5.1.25 on centos mysql (Server version: 5.7.35 MySQL Community Server (GPL)). The issue is not reproducible therefore you need to check your env. > Failed to apply 046*.sql patch > -- > > Key: RANGER-3587 > URL: https://issues.apache.org/jira/browse/RANGER-3587 > Project: Ranger > Issue Type: Bug > Components: admin >Affects Versions: 2.1.0 >Reporter: Bhavik Patel >Priority: Blocker > > [JISQL] /usr/java/latest/bin/java -cp > /opt/tusker/ranger-admin/ranger-2.1.0-1-admin/ews/lib/mysql-connector-java-5.1.25.jar:/opt/tusker/ranger-admin/ranger-2.1.0-1-admin/jisql/lib/* > org.apache.util.sql.Jisql -driver mysqlconj -cstring > jdbc:mysql://localhost.visa.com/ranger_admin -u 'rangeradmin' -p '' > -noheader -trim -c \; -input > /opt/tusker/ranger-admin/ranger-2.1.0-1-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql > Error executing: CREATE FUNCTION `getXportalUIdByLoginId`(input_val > VARCHAR(100)) RETURNS int(11) BEGIN DECLARE myid INT; SELECT x_portal_user.id > into myid FROM x_portal_user WHERE x_portal_user.login_id = input_val; RETURN > myid; END > java.sql.SQLException: This function has none of DETERMINISTIC, NO SQL, or > READS SQL DATA in its declaration and binary logging is enabled (you *might* > want to use the less safe log_bin_trust_function_creators variable) > SQLException : SQL state: HY000 java.sql.SQLException: This function has none > of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary > logging is enabled (you *might* want to use the less safe > log_bin_trust_function_creators variable) ErrorCode: 1418 > > Note: mysql version is 5.7.35 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (RANGER-3587) Failed to apply 046*.sql patch
[ https://issues.apache.org/jira/browse/RANGER-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-3587. - Assignee: Pradeep Agrawal Resolution: Cannot Reproduce > Failed to apply 046*.sql patch > -- > > Key: RANGER-3587 > URL: https://issues.apache.org/jira/browse/RANGER-3587 > Project: Ranger > Issue Type: Bug > Components: admin >Affects Versions: 2.1.0 >Reporter: Bhavik Patel >Assignee: Pradeep Agrawal >Priority: Blocker > > [JISQL] /usr/java/latest/bin/java -cp > /opt/tusker/ranger-admin/ranger-2.1.0-1-admin/ews/lib/mysql-connector-java-5.1.25.jar:/opt/tusker/ranger-admin/ranger-2.1.0-1-admin/jisql/lib/* > org.apache.util.sql.Jisql -driver mysqlconj -cstring > jdbc:mysql://localhost.visa.com/ranger_admin -u 'rangeradmin' -p '' > -noheader -trim -c \; -input > /opt/tusker/ranger-admin/ranger-2.1.0-1-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql > Error executing: CREATE FUNCTION `getXportalUIdByLoginId`(input_val > VARCHAR(100)) RETURNS int(11) BEGIN DECLARE myid INT; SELECT x_portal_user.id > into myid FROM x_portal_user WHERE x_portal_user.login_id = input_val; RETURN > myid; END > java.sql.SQLException: This function has none of DETERMINISTIC, NO SQL, or > READS SQL DATA in its declaration and binary logging is enabled (you *might* > want to use the less safe log_bin_trust_function_creators variable) > SQLException : SQL state: HY000 java.sql.SQLException: This function has none > of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary > logging is enabled (you *might* want to use the less safe > log_bin_trust_function_creators variable) ErrorCode: 1418 > > Note: mysql version is 5.7.35 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (RANGER-3594) mysql setup scripts failed with Master/Slave mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal resolved RANGER-3594.
-
Assignee: Pradeep Agrawal
Resolution: Cannot Reproduce
[~kirbyzhou] : Unable to reproduce the issue, if you have all the steps, along
with source of all required libraries and installation env. please mention
here.
Please check your env again.
> mysql setup scripts failed with Master/Slave mysql
> --
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments:
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> master/slave mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
>
> When setup with Master/Slave MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with Master/Slave requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with Master/Slave requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ).
>
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3595) Tar of KMS contains rubbish files
[
https://issues.apache.org/jira/browse/RANGER-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480808#comment-17480808
]
Pradeep Agrawal commented on RANGER-3595:
-
[~kirbyzhou] : Can't blame anyone for this, this is very old code, may be it
was designed like this for certain reason. I would request others to review and
provide their opinions.
> Tar of KMS contains rubbish files
> -
>
> Key: RANGER-3595
> URL: https://issues.apache.org/jira/browse/RANGER-3595
> Project: Ranger
> Issue Type: Improvement
> Components: kms
>Reporter: kirby zhou
>Priority: Major
>
> There are lots of .class files under ews/webapp/. They wont be loaded by any
> classpath. And they are duplicated against files inside
> ews/webapp/lib/ranger-kms-3.0.0-SNAPSHOT.jar.
> It seems dirty and may cause some security problem.
> {code:bash}
> #] tar tf target/ranger-3.0.0-SNAPSHOT-kms.tar.gz ranger-3.0.0-SNAPSHOT-kms/
> | egrep 'ews/webapp/org' | head
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/biz/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/dao/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/entity/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/crypto/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/crypto/key/
> //代码占位符
> {code}
> * The reason is that:
> distro/src/main/assembly/kms.xml
>
> {code:java}
>
> true
>
> org.apache.ranger:ranger-kms
>
>
> ews/webapp
> false
> true
>
> {code}
> Why ?
>
> The secret is in kms/scripts/setup.sh:
>
> {code:java}
> setup_kms(){
> #copying ranger kms provider
> oldP=${PWD}
> cd $PWD/ews/webapp
> log "[I] Adding ranger kms provider as services in hadoop-common jar"
> for f in lib/hadoop-common*.jar
> do
> ${JAVA_HOME}/bin/jar -uf ${f}
> META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory
> chown ${unix_user}:${unix_group} ${f}
> done
> cd ${oldP}
> }
> {code}
>
>
> The code above is VERY VERY DIRTY!
> It hacks into hadoop-common.jar., Overwrite resource
> "META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory". Ensure
> the following code can load
> 'org.apache.hadoop.crypto.key.RangerKeyStoreProvider$Factory' by
> 'META-INF/.../KeyProviderFactory'.
>
>
> {code:java}
> // org.apache.hadoop.crypto.key: KeyProviderFactory.java
> private static final ServiceLoader serviceLoader =
> ServiceLoader.load(KeyProviderFactory.class,
> KeyProviderFactory.class.getClassLoader());
> {code}
>
>
> But this is unnecessary.
> ServiceLoader will read all resources with the same name using the
> ClassLoader of KeyProviderFactory. We just need to put a jar contains that
> property side by side of hadoop-common.jar ( ews/webapp/lib/ ). And
> ranger-kms-3.0.0-SNAPSHOT.jar already here.
> {code:java}
> % tar tf ../target/ranger-*-kms.tar.gz | egrep 'kms[^/]*\.jar|hadoop-common'
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/lib/ranger-kms-3.0.0-SNAPSHOT.jar
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/lib/hadoop-common-3.3.0.jar
> ...
> % tar tf target/ranger-kms-3.0.0-SNAPSHOT.jar | fgrep ProviderFactory
> META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3594) mysql setup scripts failed with Master/Slave mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480851#comment-17480851
]
Pradeep Agrawal commented on RANGER-3594:
-
[~kirbyzhou] : I am trying with default installation of mysql, anything beyond
default config "details" should be mentioned in Jira and User need to handle
such env. specific constraints as per the underlying database env. Developers
may not have everything in their pc so "steps to reproduce" should be added in
the Jira if the issue is not reproducible with default settings.
> mysql setup scripts failed with Master/Slave mysql
> --
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments:
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> master/slave mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
>
> When setup with Master/Slave MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with Master/Slave requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with Master/Slave requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ).
>
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3594) mysql setup scripts failed with binlog-enabled mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480856#comment-17480856
]
Pradeep Agrawal commented on RANGER-3594:
-
before ranger installation you have to run below statement in mysql, after
ranger installation you can reset it.
{code:java}
SET GLOBAL log_bin_trust_function_creators = 1; {code}
> mysql setup scripts failed with binlog-enabled mysql
>
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments:
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> binlog-enabled mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
> When setup with binlog-enabled MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with binlog requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with binlog requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ), and it seems too danger to grant SUPER to $db_user. Maybe we can let
> db_setup.py runs with $db_root_user instead of $db_user, or DO NOT use store
> procedure any more to avoid such problems.
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3594) mysql setup scripts failed with binlog-enabled mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480859#comment-17480859
]
Pradeep Agrawal commented on RANGER-3594:
-
References :
# [https://dev.mysql.com/doc/refman/8.0/en/stored-programs-logging.html]
#
[https://mariadb.com/kb/en/would-some-one-tell-me-what-is-wrong-with-this-function/+comments/2730]
# [https://aws.amazon.com/premiumsupport/knowledge-center/rds-mysql-functions/]
#
[https://docs.microsoft.com/en-us/azure/mysql/howto-troubleshoot-common-errors]
#
[https://community.cloudera.com/t5/Support-Questions/Ranger-service-is-failing-to-install/td-p/310115]
> mysql setup scripts failed with binlog-enabled mysql
>
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments:
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> binlog-enabled mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
> When setup with binlog-enabled MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with binlog requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with binlog requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ), and it seems too danger to grant SUPER to $db_user. Maybe we can let
> db_setup.py runs with $db_root_user instead of $db_user, or DO NOT use store
> procedure any more to avoid such problems.
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3594) mysql setup scripts failed with binlog-enabled mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480873#comment-17480873
]
Pradeep Agrawal commented on RANGER-3594:
-
[~kirbyzhou] : yes, will try to change the function to procedure.
> mysql setup scripts failed with binlog-enabled mysql
>
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments:
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> binlog-enabled mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
> When setup with binlog-enabled MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with binlog requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with binlog requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ), and it seems too danger to grant SUPER to $db_user. Maybe we can let
> db_setup.py runs with $db_root_user instead of $db_user, or DO NOT use store
> procedure any more to avoid such problems.
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3573) Add vim in docker base image
[ https://issues.apache.org/jira/browse/RANGER-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481530#comment-17481530 ] Pradeep Agrawal commented on RANGER-3573: - Please close the RR : https://reviews.apache.org/r/73785/ > Add vim in docker base image > > > Key: RANGER-3573 > URL: https://issues.apache.org/jira/browse/RANGER-3573 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Abhishek Kumar >Assignee: Abhishek Kumar >Priority: Minor > > add vim in base image. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Reopened] (RANGER-3590) User with Auditor role in security zone can change a policy's name and description
[ https://issues.apache.org/jira/browse/RANGER-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal reopened RANGER-3590: - > User with Auditor role in security zone can change a policy's name and > description > -- > > Key: RANGER-3590 > URL: https://issues.apache.org/jira/browse/RANGER-3590 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Dineshkumar Yadav >Assignee: Dineshkumar Yadav >Priority: Major > Fix For: 3.0.0 > > > h3. Reproduction > h3. Precondition > # User hrt_2, and hrt_3 have roles User in Ranger. > # Create a security zone with name "test_security_zone" and with: > Admin users: hrt_2 > Auditor Users: hrt_3 > Resource Services: cm_hive, and for database test_db > # Login as hrt_2, and create a hive policy named "test_security_zone_policy" > with arbitrary content. > h4. Test steps > # Login as hrt_3 and try to create a new hive policy > "new_test_security_zone_policy" with arbitrary content. > # As hrt_3, try to change the name or description of > "test_security_zone_policy". > # As hrt_3, try to change the resource, or permissions of > "test_security_zone_policy" (e.g. add another database, or add a new user to > Allow Conditions) > h4. Expected behavior > # Creation of new policy should be denied for hrt_3. > # Update of already existing policy's name or description should be denied > for hrt_3. > # Update of resources, permissions should be denied for hrt_3. > h4. Actual behavior > # Creation of new policy is denied as expected. > # Update succeeds. > # Trying to update resources or permission results in access denied, as > expected. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3590) User with Auditor role in security zone can change a policy's name and description
[ https://issues.apache.org/jira/browse/RANGER-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17481867#comment-17481867 ] Pradeep Agrawal commented on RANGER-3590: - revert commit : https://github.com/apache/ranger/commit/e5c7ee70239be8e6a1df877deac3dded4ab7fc29 > User with Auditor role in security zone can change a policy's name and > description > -- > > Key: RANGER-3590 > URL: https://issues.apache.org/jira/browse/RANGER-3590 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Dineshkumar Yadav >Assignee: Dineshkumar Yadav >Priority: Major > Fix For: 3.0.0 > > > h3. Reproduction > h3. Precondition > # User hrt_2, and hrt_3 have roles User in Ranger. > # Create a security zone with name "test_security_zone" and with: > Admin users: hrt_2 > Auditor Users: hrt_3 > Resource Services: cm_hive, and for database test_db > # Login as hrt_2, and create a hive policy named "test_security_zone_policy" > with arbitrary content. > h4. Test steps > # Login as hrt_3 and try to create a new hive policy > "new_test_security_zone_policy" with arbitrary content. > # As hrt_3, try to change the name or description of > "test_security_zone_policy". > # As hrt_3, try to change the resource, or permissions of > "test_security_zone_policy" (e.g. add another database, or add a new user to > Allow Conditions) > h4. Expected behavior > # Creation of new policy should be denied for hrt_3. > # Update of already existing policy's name or description should be denied > for hrt_3. > # Update of resources, permissions should be denied for hrt_3. > h4. Actual behavior > # Creation of new policy is denied as expected. > # Update succeeds. > # Trying to update resources or permission results in access denied, as > expected. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3498) RANGER : Remove log4j1 dependencies.
[ https://issues.apache.org/jira/browse/RANGER-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482221#comment-17482221 ] Pradeep Agrawal commented on RANGER-3498: - Commit links : # [https://github.com/apache/ranger/commit/71809108fd106b664b6f9d53e0efd86d4c5cd039] # [https://github.com/apache/ranger/commit/d7d58ef548b02347f33253973ecb22cf9b24df1e] # https://github.com/apache/ranger/commit/ba999ed35d28226a74965bca16b6efc9c46b5df2 > RANGER : Remove log4j1 dependencies. > > > Key: RANGER-3498 > URL: https://issues.apache.org/jira/browse/RANGER-3498 > Project: Ranger > Issue Type: Task > Components: Ranger >Reporter: Mateen N Mansoori >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: slf4j_patch1.diff > > > Remove log4j1 dependencies. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (RANGER-3581) setup.sh can not "CREATE FUNCTION" on MySQL with Master/Slave profile.
[
https://issues.apache.org/jira/browse/RANGER-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal reassigned RANGER-3581:
---
Fix Version/s: (was: 3.0.0)
(was: 2.2.0)
(was: 2.3.0)
Assignee: Pradeep Agrawal
Resolution: Duplicate
> setup.sh can not "CREATE FUNCTION" on MySQL with Master/Slave profile.
> --
>
> Key: RANGER-3581
> URL: https://issues.apache.org/jira/browse/RANGER-3581
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments:
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> "CREATE FUNCTION" in MySQL requires extra characteristics informaion.
> Otherwise, it refused to create it with Master/Slave profile.
> * {{CONTAINS SQL}} indicates that the routine does not contain statements
> that read or write data. This is the default if none of these characteristics
> is given explicitly. Examples of such statements are {{SET @x = 1}} or {{{}DO
> RELEASE_LOCK('abc'){}}}, which execute but neither read nor write data.
> * {{NO SQL}} indicates that the routine contains no SQL statements.
> * {{READS SQL DATA}} indicates that the routine contains statements that
> read data (for example,
> [{{SELECT}}|https://dev.mysql.com/doc/refman/8.0/en/select.html]), but not
> statements that write data.
> * {{MODIFIES SQL DATA}} indicates that the routine contains statements that
> may write data (for example,
> [{{INSERT}}|https://dev.mysql.com/doc/refman/8.0/en/insert.html] or
> [{{DELETE}}|https://dev.mysql.com/doc/refman/8.0/en/delete.html]).
> Our "getXportalUIdByLoginId" and "getModulesIdByName" is type of "{{{}READS
> SQL DATA{}}}".
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3596) Ranger versions 1.1.0, 2.0.0, and 2.1.0 rely on LOG4j1.x. Is it affected by vulnerability CVE-2022-23302/23305/23307?
[ https://issues.apache.org/jira/browse/RANGER-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482229#comment-17482229 ] Pradeep Agrawal commented on RANGER-3596: - Changes are not allowed in released branches, please add changes in your local/forked branches and continue. > Ranger versions 1.1.0, 2.0.0, and 2.1.0 rely on LOG4j1.x. Is it affected by > vulnerability CVE-2022-23302/23305/23307? > - > > Key: RANGER-3596 > URL: https://issues.apache.org/jira/browse/RANGER-3596 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 1.1.0, 2.0.0, 2.1.0 >Reporter: Yuanzhe Geng >Priority: Major > > Ranger versions 1.1.0, 2.0.0, and 2.1.0 rely on LOG4j1.x. Is it affected by > vulnerability CVE-2022-23302/23305/23307? -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (RANGER-3596) Ranger versions 1.1.0, 2.0.0, and 2.1.0 rely on LOG4j1.x. Is it affected by vulnerability CVE-2022-23302/23305/23307?
[ https://issues.apache.org/jira/browse/RANGER-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-3596. - Assignee: Pradeep Agrawal Resolution: Won't Fix > Ranger versions 1.1.0, 2.0.0, and 2.1.0 rely on LOG4j1.x. Is it affected by > vulnerability CVE-2022-23302/23305/23307? > - > > Key: RANGER-3596 > URL: https://issues.apache.org/jira/browse/RANGER-3596 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 1.1.0, 2.0.0, 2.1.0 >Reporter: Yuanzhe Geng >Assignee: Pradeep Agrawal >Priority: Major > > Ranger versions 1.1.0, 2.0.0, and 2.1.0 rely on LOG4j1.x. Is it affected by > vulnerability CVE-2022-23302/23305/23307? -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (RANGER-3579) Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832
[ https://issues.apache.org/jira/browse/RANGER-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal reassigned RANGER-3579: --- Assignee: Pradeep Agrawal > Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832 > -- > > Key: RANGER-3579 > URL: https://issues.apache.org/jira/browse/RANGER-3579 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Brahma Reddy Battula >Assignee: Pradeep Agrawal >Priority: Major > Attachments: RANGER-3579.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (RANGER-3600) Add fallback support in RangerPolicy
Pradeep Agrawal created RANGER-3600: --- Summary: Add fallback support in RangerPolicy Key: RANGER-3600 URL: https://issues.apache.org/jira/browse/RANGER-3600 Project: Ranger Issue Type: Improvement Components: Ranger Reporter: Pradeep Agrawal Assignee: Pradeep Agrawal Fix For: 3.0.0 -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3498) RANGER : Remove log4j1 dependencies.
[ https://issues.apache.org/jira/browse/RANGER-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3498: Attachment: 0001-RANGER-3498-RANGER-Remove-log4j1-dependencies.patch > RANGER : Remove log4j1 dependencies. > > > Key: RANGER-3498 > URL: https://issues.apache.org/jira/browse/RANGER-3498 > Project: Ranger > Issue Type: Task > Components: Ranger >Reporter: Mateen N Mansoori >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-RANGER-3498-RANGER-Remove-log4j1-dependencies.patch, slf4j_patch1.diff > > > Remove log4j1 dependencies. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3498) RANGER : Remove log4j1 dependencies.
[ https://issues.apache.org/jira/browse/RANGER-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482245#comment-17482245 ] Pradeep Agrawal commented on RANGER-3498: - Mateen's KMS patch for 2.3 branch : [^0001-RANGER-3498-RANGER-Remove-log4j1-dependencies.patch] Commit link : https://github.com/apache/ranger/commit/54d491cdee6f2704b7862e45c03317fc8536bf68 > RANGER : Remove log4j1 dependencies. > > > Key: RANGER-3498 > URL: https://issues.apache.org/jira/browse/RANGER-3498 > Project: Ranger > Issue Type: Task > Components: Ranger >Reporter: Mateen N Mansoori >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-RANGER-3498-RANGER-Remove-log4j1-dependencies.patch, slf4j_patch1.diff > > > Remove log4j1 dependencies. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3579) Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832
[ https://issues.apache.org/jira/browse/RANGER-3579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482256#comment-17482256 ] Pradeep Agrawal commented on RANGER-3579: - commit link master branch : [https://github.com/apache/ranger/commit/885d12ddd59eaa1401ef7d9ee528553ad836b958] commit link 2.3 branch : https://github.com/apache/ranger/commit/28032bf3b29ca48187352a1ae774a51d8505ce7d > Upgrade Log4j2 to 2.17.1 due to CVE-2021-44832 > -- > > Key: RANGER-3579 > URL: https://issues.apache.org/jira/browse/RANGER-3579 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Brahma Reddy Battula >Assignee: Pradeep Agrawal >Priority: Major > Attachments: RANGER-3579.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3560) Upgrade kylin version to 2.6.6
[ https://issues.apache.org/jira/browse/RANGER-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482257#comment-17482257 ] Pradeep Agrawal commented on RANGER-3560: - 2.3 branch commit link : https://github.com/apache/ranger/commit/0198a1ccf45efb50e8d5c5a5f71034a9e2d93929 > Upgrade kylin version to 2.6.6 > -- > > Key: RANGER-3560 > URL: https://issues.apache.org/jira/browse/RANGER-3560 > Project: Ranger > Issue Type: Wish > Components: Ranger >Affects Versions: 3.0.0 >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: 0001-RANGER-3560-Upgrade-kylin-version-to-2.6.6.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3594) mysql setup scripts failed with binlog-enabled mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483175#comment-17483175
]
Pradeep Agrawal commented on RANGER-3594:
-
[~bpatel] No, but please test and let me know.
> mysql setup scripts failed with binlog-enabled mysql
>
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments:
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> binlog-enabled mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
> When setup with binlog-enabled MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with binlog requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with binlog requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ), and it seems too danger to grant SUPER to $db_user. Maybe we can let
> db_setup.py runs with $db_root_user instead of $db_user, or DO NOT use store
> procedure any more to avoid such problems.
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (RANGER-3594) mysql setup scripts failed with binlog-enabled mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3594:
Attachment: 0001-RANGER-3594-Ranger-setup-fails-for-mariadb-mysql-whe.patch
> mysql setup scripts failed with binlog-enabled mysql
>
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Attachments:
> 0001-RANGER-3594-Ranger-setup-fails-for-mariadb-mysql-whe.patch,
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> binlog-enabled mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
> When setup with binlog-enabled MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with binlog requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with binlog requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ), and it seems too danger to grant SUPER to $db_user. Maybe we can let
> db_setup.py runs with $db_root_user instead of $db_user, or DO NOT use store
> procedure any more to avoid such problems.
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (RANGER-3594) mysql setup scripts failed with binlog-enabled mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3594:
Fix Version/s: 3.0.0
2.3.0
> mysql setup scripts failed with binlog-enabled mysql
>
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3594-Ranger-setup-fails-for-mariadb-mysql-whe.patch,
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> binlog-enabled mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
> When setup with binlog-enabled MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with binlog requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with binlog requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ), and it seems too danger to grant SUPER to $db_user. Maybe we can let
> db_setup.py runs with $db_root_user instead of $db_user, or DO NOT use store
> procedure any more to avoid such problems.
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3594) mysql setup scripts failed with binlog-enabled mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485581#comment-17485581
]
Pradeep Agrawal commented on RANGER-3594:
-
[~bpatel] / [~kirbyzhou] : is this good to go ?
> mysql setup scripts failed with binlog-enabled mysql
>
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3594-Ranger-setup-fails-for-mariadb-mysql-whe.patch,
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> binlog-enabled mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
> When setup with binlog-enabled MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with binlog requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with binlog requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ), and it seems too danger to grant SUPER to $db_user. Maybe we can let
> db_setup.py runs with $db_root_user instead of $db_user, or DO NOT use store
> procedure any more to avoid such problems.
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (RANGER-3594) mysql setup scripts failed with binlog-enabled mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3594:
Component/s: Ranger
(was: admin)
> mysql setup scripts failed with binlog-enabled mysql
>
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3594-Ranger-setup-fails-for-mariadb-mysql-whe.patch,
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> binlog-enabled mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
> When setup with binlog-enabled MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with binlog requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with binlog requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ), and it seems too danger to grant SUPER to $db_user. Maybe we can let
> db_setup.py runs with $db_root_user instead of $db_user, or DO NOT use store
> procedure any more to avoid such problems.
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3594) mysql setup scripts failed with binlog-enabled mysql
[
https://issues.apache.org/jira/browse/RANGER-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485611#comment-17485611
]
Pradeep Agrawal commented on RANGER-3594:
-
2.3 branch commit :
https://github.com/apache/ranger/commit/a435f34005c0f8d4beea6d122e5118f847df61b2
> mysql setup scripts failed with binlog-enabled mysql
>
>
> Key: RANGER-3594
> URL: https://issues.apache.org/jira/browse/RANGER-3594
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 3.0.0, 2.2.0, 2.3.0
>Reporter: kirby zhou
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3594-Ranger-setup-fails-for-mariadb-mysql-whe.patch,
> 0001-add-FUNCTION-description-for-mysql-master-slave.patch
>
>
> There are some sql scripts which create functions in mysql, failed with
> binlog-enabled mysql.
> * security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
> * security-admin/db/mysql/patches/013-permissionmodel.sql
> * security-admin/db/mysql/patches/037-create-security-zone-schema.sql
> *
> security-admin/db/mysql/patches/046-insert-statename-in-x-ranger-global-state.sql
> Codes like:
>
> {code:java}
> DELIMITER $$
> DROP FUNCTION if exists getXportalUIdByLoginId$$
> CREATE FUNCTION `getXportalUIdByLoginId`(input_val VARCHAR(100)) RETURNS
> int(11)
> BEGIN DECLARE myid INT; SELECT x_portal_user.id into myid FROM x_portal_user
> WHERE x_portal_user.login_id = input_val;
> RETURN myid;
> END $$
> DELIMITER ;
> DELIMITER $$
> DROP FUNCTION if exists getModulesIdByName$$
> CREATE FUNCTION `getModulesIdByName`(input_val VARCHAR(100)) RETURNS int(11)
> BEGIN DECLARE myid INT; SELECT x_modules_master.id into myid FROM
> x_modules_master
> WHERE x_modules_master.module = input_val;
> RETURN myid;
> END $$ {code}
>
> When setup with binlog-enabled MySQL database, it will cause failure.
> Because of 2 problem.
> 1. CREATE FUNCTION with binlog requires some "characteristic" flag.
> Otherwise, error:
> {code:java}
> This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its
> declaration and binary.{code}
> getXportalUIdByLoginId and getModulesIdByName are both read-only, so we can
> put 'READS SQL DATA' here.
>
> 2. CREATE FUNCTION with binlog requires SUPER privilege ON *.*
> Otherwise, error:
>
> {code:java}
> You do not have the SUPER privilege and binary logging is enabled (you might
> want to use the less safe log_bin_trust_function_creators variable){code}
>
> But our dba_setup.py do not grant SUPER to $db_user ( default is rangeradmin
> ), and it seems too danger to grant SUPER to $db_user. Maybe we can let
> db_setup.py runs with $db_root_user instead of $db_user, or DO NOT use store
> procedure any more to avoid such problems.
>
>
>
> There are lots of sql contains the same function, which one should I patch it?
> It seems that
> "security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql" is
> called by setup.sh, but what about others?
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (RANGER-3601) Add a switch on the access_log fils that are generated by EmbeddedServer
[ https://issues.apache.org/jira/browse/RANGER-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485853#comment-17485853 ] Pradeep Agrawal edited comment on RANGER-3601 at 2/2/22, 2:46 PM: -- [~LinZW] Can you provide few sample entries which are repeating ? was (Author: pradeep.agrawal): Can you provide few sample entries which are repeating ? > Add a switch on the access_log fils that are generated by EmbeddedServer > > > Key: RANGER-3601 > URL: https://issues.apache.org/jira/browse/RANGER-3601 > Project: Ranger > Issue Type: Improvement > Components: kms >Affects Versions: 2.1.0 >Reporter: LinZhongwei >Priority: Minor > > Hi > We are using 2 rangerKMS instances to protect our data in our cluster. But > we find that too much access log files are generated by embedded server. I > find that the switch of access log config is hardcoded in the source code. > Can you add a switch property such as 'ranger.accesslog.enabled' to let users > to decide whether to enable access logging? > Because the size of access log are about 50GB everyday.. We want to decrease > the IO on the server. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3601) Add a switch on the access_log fils that are generated by EmbeddedServer
[ https://issues.apache.org/jira/browse/RANGER-3601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17485853#comment-17485853 ] Pradeep Agrawal commented on RANGER-3601: - Can you provide few sample entries which are repeating ? > Add a switch on the access_log fils that are generated by EmbeddedServer > > > Key: RANGER-3601 > URL: https://issues.apache.org/jira/browse/RANGER-3601 > Project: Ranger > Issue Type: Improvement > Components: kms >Affects Versions: 2.1.0 >Reporter: LinZhongwei >Priority: Minor > > Hi > We are using 2 rangerKMS instances to protect our data in our cluster. But > we find that too much access log files are generated by embedded server. I > find that the switch of access log config is hardcoded in the source code. > Can you add a switch property such as 'ranger.accesslog.enabled' to let users > to decide whether to enable access logging? > Because the size of access log are about 50GB everyday.. We want to decrease > the IO on the server. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (RANGER-3542) Invalid HTTPS Check
[
https://issues.apache.org/jira/browse/RANGER-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal reassigned RANGER-3542:
---
Assignee: Pradeep Agrawal
> Invalid HTTPS Check
> ---
>
> Key: RANGER-3542
> URL: https://issues.apache.org/jira/browse/RANGER-3542
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Reporter: David Mollitor
>Assignee: Pradeep Agrawal
>Priority: Minor
>
> [https://github.com/apache/ranger/blob/0258fcf7ab25473b056fffc103840806c18fdcad/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java#L243]
>
> {code:java|title=RangerRESTClient.java}
> mIsSSL = StringUtils.containsIgnoreCase(mUrl, "https");
> {code}
> This can trigger inadvertently if the host name just happens to have "https"
> in the name. Better/safer to use Java URL to parse {{mUrl}} and look at the
> protocol explicitly.
> For example: {{http://my.serverhttps.com}} would trigger as an ssl enabled
> endpoint.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (RANGER-3542) Invalid HTTPS Check
[
https://issues.apache.org/jira/browse/RANGER-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3542:
Attachment: 0001-RANGER-3542-Fix-invalid-HTTPS-check.patch
> Invalid HTTPS Check
> ---
>
> Key: RANGER-3542
> URL: https://issues.apache.org/jira/browse/RANGER-3542
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Reporter: David Mollitor
>Assignee: Pradeep Agrawal
>Priority: Minor
> Attachments: 0001-RANGER-3542-Fix-invalid-HTTPS-check.patch
>
>
> [https://github.com/apache/ranger/blob/0258fcf7ab25473b056fffc103840806c18fdcad/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java#L243]
>
> {code:java|title=RangerRESTClient.java}
> mIsSSL = StringUtils.containsIgnoreCase(mUrl, "https");
> {code}
> This can trigger inadvertently if the host name just happens to have "https"
> in the name. Better/safer to use Java URL to parse {{mUrl}} and look at the
> protocol explicitly.
> For example: {{http://my.serverhttps.com}} would trigger as an ssl enabled
> endpoint.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3542) Invalid HTTPS Check
[
https://issues.apache.org/jira/browse/RANGER-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487967#comment-17487967
]
Pradeep Agrawal commented on RANGER-3542:
-
master branch :
[https://github.com/apache/ranger/commit/26070383c6300da91926ed77e128d35c9808056c]
2.3-branch:
https://github.com/apache/ranger/commit/5e24f09f1a54ac5e07079758d3fc45a4bf16677d
> Invalid HTTPS Check
> ---
>
> Key: RANGER-3542
> URL: https://issues.apache.org/jira/browse/RANGER-3542
> Project: Ranger
> Issue Type: Bug
> Components: plugins
>Reporter: David Mollitor
>Assignee: Pradeep Agrawal
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 0001-RANGER-3542-Fix-invalid-HTTPS-check.patch
>
>
> [https://github.com/apache/ranger/blob/0258fcf7ab25473b056fffc103840806c18fdcad/agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java#L243]
>
> {code:java|title=RangerRESTClient.java}
> mIsSSL = StringUtils.containsIgnoreCase(mUrl, "https");
> {code}
> This can trigger inadvertently if the host name just happens to have "https"
> in the name. Better/safer to use Java URL to parse {{mUrl}} and look at the
> protocol explicitly.
> For example: {{http://my.serverhttps.com}} would trigger as an ssl enabled
> endpoint.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (RANGER-3600) Ranger service tags import request failure
[ https://issues.apache.org/jira/browse/RANGER-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3600: Summary: Ranger service tags import request failure (was: Add fallback support in RangerPolicy) > Ranger service tags import request failure > -- > > Key: RANGER-3600 > URL: https://issues.apache.org/jira/browse/RANGER-3600 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3600) Ranger service tags import request failure
[ https://issues.apache.org/jira/browse/RANGER-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3600: Description: Ranger service tag import request may fail if RangerServiceResource objects of ServiceTags objects does not have ranger service name attribute values. > Ranger service tags import request failure > -- > > Key: RANGER-3600 > URL: https://issues.apache.org/jira/browse/RANGER-3600 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > > Ranger service tag import request may fail if RangerServiceResource objects > of ServiceTags objects does not have ranger service name attribute values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3600) Ranger service tags import request failure
[ https://issues.apache.org/jira/browse/RANGER-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3600: Attachment: 0001-RANGER-3600-Ranger-service-tags-import-request-failu.patch > Ranger service tags import request failure > -- > > Key: RANGER-3600 > URL: https://issues.apache.org/jira/browse/RANGER-3600 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3600-Ranger-service-tags-import-request-failu.patch > > > Ranger service tag import request may fail if RangerServiceResource objects > of ServiceTags objects does not have ranger service name attribute values. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (RANGER-3653) Replace aws java sdk bom dependencies with bundled dependencies
Pradeep Agrawal created RANGER-3653: --- Summary: Replace aws java sdk bom dependencies with bundled dependencies Key: RANGER-3653 URL: https://issues.apache.org/jira/browse/RANGER-3653 Project: Ranger Issue Type: Wish Components: Ranger Reporter: Pradeep Agrawal Assignee: Pradeep Agrawal Fix For: 3.0.0 1) Replace aws java sdk bom dependencies with bundled dependencies 2) Improve StringUtils class dependencies. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3653) Replace aws java sdk bom dependencies with bundled dependencies
[ https://issues.apache.org/jira/browse/RANGER-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3653: Attachment: 0001-RANGER-3653-Replace-aws-java-sdk-bom-dependencies-wi.patch > Replace aws java sdk bom dependencies with bundled dependencies > --- > > Key: RANGER-3653 > URL: https://issues.apache.org/jira/browse/RANGER-3653 > Project: Ranger > Issue Type: Wish > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3653-Replace-aws-java-sdk-bom-dependencies-wi.patch > > > 1) Replace aws java sdk bom dependencies with bundled dependencies > 2) Improve StringUtils class dependencies. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3653) Replace aws java sdk bom dependencies with bundled dependencies
[ https://issues.apache.org/jira/browse/RANGER-3653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17502077#comment-17502077 ] Pradeep Agrawal commented on RANGER-3653: - Commit link : https://github.com/apache/ranger/commit/4eb8401215549c169f7fb9726eb21527e3f4e151 > Replace aws java sdk bom dependencies with bundled dependencies > --- > > Key: RANGER-3653 > URL: https://issues.apache.org/jira/browse/RANGER-3653 > Project: Ranger > Issue Type: Wish > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3653-Replace-aws-java-sdk-bom-dependencies-wi.patch > > > 1) Replace aws java sdk bom dependencies with bundled dependencies > 2) Improve StringUtils class dependencies. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Reopened] (RANGER-3485) db setup scripts should not convert the db user to lowercase during setup
[ https://issues.apache.org/jira/browse/RANGER-3485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal reopened RANGER-3485: - > db setup scripts should not convert the db user to lowercase during setup > -- > > Key: RANGER-3485 > URL: https://issues.apache.org/jira/browse/RANGER-3485 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3485) db setup scripts should not convert the db user to lowercase during setup
[ https://issues.apache.org/jira/browse/RANGER-3485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3485: Fix Version/s: 3.0.0 > db setup scripts should not convert the db user to lowercase during setup > -- > > Key: RANGER-3485 > URL: https://issues.apache.org/jira/browse/RANGER-3485 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3485-db-setup-scripts-should-not-convert-the-.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (RANGER-3485) db setup scripts should not convert the db user to lowercase during setup
[ https://issues.apache.org/jira/browse/RANGER-3485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal updated RANGER-3485: Attachment: 0001-RANGER-3485-db-setup-scripts-should-not-convert-the-.patch > db setup scripts should not convert the db user to lowercase during setup > -- > > Key: RANGER-3485 > URL: https://issues.apache.org/jira/browse/RANGER-3485 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3485-db-setup-scripts-should-not-convert-the-.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3485) db setup scripts should not convert the db user to lowercase during setup
[ https://issues.apache.org/jira/browse/RANGER-3485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17515689#comment-17515689 ] Pradeep Agrawal commented on RANGER-3485: - [~bpatel] : Can you review the RR. https://reviews.apache.org/r/73919/ > db setup scripts should not convert the db user to lowercase during setup > -- > > Key: RANGER-3485 > URL: https://issues.apache.org/jira/browse/RANGER-3485 > Project: Ranger > Issue Type: Bug > Components: Ranger >Reporter: Pradeep Agrawal >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 3.0.0 > > Attachments: > 0001-RANGER-3485-db-setup-scripts-should-not-convert-the-.patch > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (RANGER-3691) Upgrade spring to 5.3.18 CVE-2022-22965
[ https://issues.apache.org/jira/browse/RANGER-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-3691. - Fix Version/s: 3.0.0 Assignee: kirby zhou Resolution: Fixed Commit link : https://github.com/apache/ranger/commit/9ffa882f731a3d13c6d0bc0791b2363fb9289672 > Upgrade spring to 5.3.18 CVE-2022-22965 > --- > > Key: RANGER-3691 > URL: https://issues.apache.org/jira/browse/RANGER-3691 > Project: Ranger > Issue Type: Bug > Components: admin, kms >Reporter: kirby zhou >Assignee: kirby zhou >Priority: Blocker > Fix For: 3.0.0 > > > [https://tanzu.vmware.com/security/cve-2022-22965|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-22965] > [https://github.com/spring-projects/spring-framework/releases] > > Spring has a new 0day Remote-Code-Execution problem, related to spring-beans > and JDK9+ > Fixed at spring 5.3.18 / 5.2.20 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3691) Upgrade spring to 5.3.18 CVE-2022-22965
[ https://issues.apache.org/jira/browse/RANGER-3691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517200#comment-17517200 ] Pradeep Agrawal commented on RANGER-3691: - [~kirbyzhou] : Please close the RR : [https://reviews.apache.org/r/73924]/ > Upgrade spring to 5.3.18 CVE-2022-22965 > --- > > Key: RANGER-3691 > URL: https://issues.apache.org/jira/browse/RANGER-3691 > Project: Ranger > Issue Type: Bug > Components: admin, kms >Reporter: kirby zhou >Assignee: kirby zhou >Priority: Blocker > Fix For: 3.0.0 > > > [https://tanzu.vmware.com/security/cve-2022-22965|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-22965] > [https://github.com/spring-projects/spring-framework/releases] > > Spring has a new 0day Remote-Code-Execution problem, related to spring-beans > and JDK9+ > Fixed at spring 5.3.18 / 5.2.20 > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Reopened] (RANGER-3632) Improve ranger logs, RENAME_ON_ROTATE and others
[
https://issues.apache.org/jira/browse/RANGER-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal reopened RANGER-3632:
-
> Improve ranger logs, RENAME_ON_ROTATE and others
> -
>
> Key: RANGER-3632
> URL: https://issues.apache.org/jira/browse/RANGER-3632
> Project: Ranger
> Issue Type: Improvement
> Components: admin, kms
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Currently, the filename of the access-log in use has a timestamp as the
> suffix. This brings trouble to some log monitoring and analysis programs,
> such as "tail -f access-log"
> Need to add an option to enable tomcat's RenameOnRotate capability to fix the
> file name of access-log.
>
> {code:java}
> // in EmbeddedServer::start()
> valve.setRenameOnRotate(
> EmbeddedServerUtil.getConfig(ACCESS_LOG_RENAME_ON_ROTATE, false);
> );{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3632) Improve ranger logs, RENAME_ON_ROTATE and others
[
https://issues.apache.org/jira/browse/RANGER-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517275#comment-17517275
]
Pradeep Agrawal commented on RANGER-3632:
-
This commit is causing test/build failure, hence reopening this :
[https://github.com/apache/ranger/commit/db99f639017bc9bbd71a7c5772adc1545ca83ec0]
{code:java}
Tests run: 57, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.296 sec - in
org.apache.ranger.rest.TestServiceREST
12:53:57.125 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - ==>
JVMShutdownHook.run()
12:53:57.125 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - JVMShutdownHook:
Signalling async audit cleanup to start.
12:53:57.125 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.provider.AuditProviderFactory -
RangerAsyncAuditCleanup: Starting cleanup
12:53:57.125 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - JVMShutdownHook:
Waiting up to 30 seconds for audit cleanup to finish.
12:53:57.125 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.queue.AuditAsyncQueue - Stop called. name=test.async
12:53:57.125 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.provider.AuditProviderFactory -
RangerAsyncAuditCleanup: Done cleanup
12:53:57.125 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.provider.AuditProviderFactory -
RangerAsyncAuditCleanup: Waiting to audit cleanup start signal
12:53:57.126 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - JVMShutdownHook: Audit
cleanup finished after 1 milli seconds
12:53:57.126 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - JVMShutdownHook:
Interrupting ranger async audit cleanup thread
12:53:57.126 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - <==
JVMShutdownHook.run()
12:53:57.126 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - ==>
JVMShutdownHook.run()
12:53:57.126 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - JVMShutdownHook:
Signalling async audit cleanup to start.
12:53:57.126 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - JVMShutdownHook:
Waiting up to 30 seconds for audit cleanup to finish.
12:53:57.126 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.provider.AuditProviderFactory -
RangerAsyncAuditCleanup: Interrupted while waiting for audit startCleanup
signal! Exiting the thread...
java.lang.InterruptedException: null
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:312)
at
org.apache.ranger.audit.provider.AuditProviderFactory$RangerAsyncAuditCleanup.run(AuditProviderFactory.java:501)
at java.lang.Thread.run(Thread.java:748)
12:53:57.126 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.provider.AuditProviderFactory -
RangerAsyncAuditCleanup: Starting cleanup
12:53:57.126 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.queue.AuditAsyncQueue - Stop called. name=test.async
12:53:57.126 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.provider.AuditProviderFactory -
RangerAsyncAuditCleanup: Done cleanup
12:53:57.126 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.provider.AuditProviderFactory -
RangerAsyncAuditCleanup: Waiting to audit cleanup start signal
12:53:57.126 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - JVMShutdownHook: Audit
cleanup finished after 0 milli seconds
12:53:57.126 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - JVMShutdownHook:
Interrupting ranger async audit cleanup thread
12:53:57.126 [shutdown-hook-0] INFO
org.apache.ranger.audit.provider.AuditProviderFactory - <==
JVMShutdownHook.run()
12:53:57.126 [Ranger async Audit cleanup] INFO
org.apache.ranger.audit.provider.AuditProviderFactory -
RangerAsyncAuditCleanup: Interrupted while waiting for audit startCleanup
signal! Exiting the thread...
java.lang.InterruptedException: null
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:998)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:312)
at
org.apache.ranger.audit.provider.AuditProviderFactory$RangerAsyncAuditCleanup.run(AuditProviderFactory.java:501)
at java.lang.Thread.run(Thread.java:748)
12:53:57.126 [Thread-2] DEBUG org.apa
[jira] [Commented] (RANGER-3632) Improve ranger logs, RENAME_ON_ROTATE and others
[
https://issues.apache.org/jira/browse/RANGER-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517286#comment-17517286
]
Pradeep Agrawal commented on RANGER-3632:
-
[~kirbyzhou] I have reverted commit :
[https://github.com/apache/ranger/commit/db99f639017bc9bbd71a7c5772adc1545ca83ec0]
check previous message for build failure details.
> Improve ranger logs, RENAME_ON_ROTATE and others
> -
>
> Key: RANGER-3632
> URL: https://issues.apache.org/jira/browse/RANGER-3632
> Project: Ranger
> Issue Type: Improvement
> Components: admin, kms
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Currently, the filename of the access-log in use has a timestamp as the
> suffix. This brings trouble to some log monitoring and analysis programs,
> such as "tail -f access-log"
> Need to add an option to enable tomcat's RenameOnRotate capability to fix the
> file name of access-log.
>
> {code:java}
> // in EmbeddedServer::start()
> valve.setRenameOnRotate(
> EmbeddedServerUtil.getConfig(ACCESS_LOG_RENAME_ON_ROTATE, false);
> );{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3686) Docker setup to run Ranger with MySQL database
[ https://issues.apache.org/jira/browse/RANGER-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517856#comment-17517856 ] Pradeep Agrawal commented on RANGER-3686: - [~madhan] Thanks, Please close the RR also. > Docker setup to run Ranger with MySQL database > -- > > Key: RANGER-3686 > URL: https://issues.apache.org/jira/browse/RANGER-3686 > Project: Ranger > Issue Type: Improvement > Components: Ranger >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: RANGER-3686.patch > > > Current Docker setup for Apache Ranger run with Postgres database. Enhancing > this to supporting runing Ranger with MySQL will help validate issues > specific to MySQL (like RANGER-3681). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3681) Ranger Database deadlock when createPolicy is running parallel
[
https://issues.apache.org/jira/browse/RANGER-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517857#comment-17517857
]
Pradeep Agrawal commented on RANGER-3681:
-
[~Xuze Yang] : Please close the RR https://reviews.apache.org/r/73913/
> Ranger Database deadlock when createPolicy is running parallel
> --
>
> Key: RANGER-3681
> URL: https://issues.apache.org/jira/browse/RANGER-3681
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 2.1.0
> Environment: ranger-version: 2.1.0
> mysql-verison: 5.7.34-log
> mysql-engine: InnoDB
> mysql-isolation-level: REPEATABLE-READ
>Reporter: Xuze Yang
>Assignee: Xuze Yang
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 0001-RANGER-3681-Ranger Database deadlock when
> createPolicy is running parallel.patch, 0002-RANGER-3681-Ranger Database
> deadlock when createPolicy is running parallel.patch,
> 0003-RANGER-3681-Ranger-Database-deadlock-when-createPolicy-is-running-parallel.patch
>
>
> h1. Error
> ranger-log：
> {noformat}
> 2022-03-21 20:14:29,685 [http-bio-6080-exec-13] ERROR
> org.apache.ranger.rest.ServiceREST (ServiceREST.java:1709) -
> createPolicy(RangerPolicy={id={null} guid={null} isEnabled={true}
> createdBy={null} updatedBy={null} createTime={null} updateTime={null}
> version={1} service={default-Hive}
> name={dcp-desensitize_a162c40cdc0140b1848b98415575be6c-1647864869626}
> policyType={0} policyPriority={0} description={}
> resourceSignature={4f15e3de95c81650ad869cb93a8c47a132bbec54bdf5de8c01f5075c19754cd7}
> isAuditEnabled={true} serviceType={null}
> resources={database={RangerPolicyResource={values={dcp } isExcludes={false}
> isRecursive={false} }} column={RangerPolicyResource={values={* }
> isExcludes={false} isRecursive={false} }}
> table={RangerPolicyResource={values={desensitize_a162c40cdc0140b1848b98415575be6c
> } isExcludes={false} isRecursive={false} }} } policyLabels={Consoler }
> policyConditions={}
> policyItems={RangerPolicyItem={accessTypes={RangerPolicyItemAccess={type={all}
> isAllowed={true} }} users={tangbiao2 } groups={} roles={} conditions={}
> delegateAdmin={false} }} denyPolicyItems={} allowExceptions={}
> denyExceptions={} dataMaskPolicyItems={} rowFilterPolicyItems={} options={}
> validitySchedules={, zoneName=null, isDenyAllElse={false} }}) failed
> javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse
> Persistence Services - 2.5.2.v20140319-9ad6abd):
> org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception:
> com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException: Deadlock
> found when trying to get lock; try restarting transaction
> Error Code: 1213
> Call: INSERT INTO x_policy_ref_resource (ADDED_BY_ID, CREATE_TIME, policy_id,
> resource_def_id, resource_name, UPDATE_TIME, UPD_BY_ID) VALUES (?, ?, ?, ?,
> ?, ?, ?)
> bind => [7 parameters bound]
> Query: ValueReadQuery(name="x_policy_ref_resource_SEQ" sql="SELECT
> LAST_INSERT_ID()")
> at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush(EntityManagerImpl.java:868)
> at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:301)
> at com.sun.proxy.$Proxy30.flush(Unknown Source)
> at org.apache.ranger.common.db.BaseDao.batchCreate(BaseDao.java:102)
> ...{noformat}
> mysql-deadlock：
> {noformat}
>
> LATEST DETECTED DEADLOCK
>
> 2022-03-21 09:47:22 0x7ff3a4859700
> *** (1) TRANSACTION:
> TRANSACTION 7036760, ACTIVE 0 sec inserting
> mysql tables in use 1, locked 1
> LOCK WAIT 23 lock struct(s), heap size 1136, 12 row lock(s), undo log entries
> 2
> MySQL thread id 27293, OS thread handle 140684415063808, query id 383930
> 192.168.0.76 DHCloudBG update
> INSERT INTO x_policy_ref_resource (ADDED_BY_ID, CREATE_TIME, policy_id,
> resource_def_id, resource_name, UPDATE_TIME, UPD_BY_ID) VALUES (1,
> '2022-03-20 12:47:22.666', 13921, 5, 'database', '2022-03-20 12:47:22.681', 1)
> *** (1) WAITING FOR THIS LOCK TO BE GRANTED:
> RECORD LOCKS space id 531 page no 4 n bits 376 index
> x_policy_ref_res_UK_polId_resDefId of table `ranger`.`x_policy_ref_resource`
> trx id 7036760 lock_mode X insert intention waiting
> Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits > 0
> 0: len 8; hex 73757072656d756d; asc supremum;;
> *** (2) TRANSACTION:
> TRANSACTI
[jira] [Commented] (RANGER-3667) Improve feedback in policy creation UI when resource does not exist
[
https://issues.apache.org/jira/browse/RANGER-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517858#comment-17517858
]
Pradeep Agrawal commented on RANGER-3667:
-
Seems patch is committed :
[https://github.com/apache/ranger/commit/d8fc10a36c9b23e807f8c8deaecec9bc0ae7ed05]
If nothing is pending please close this and related RR
https://reviews.apache.org/r/73903
> Improve feedback in policy creation UI when resource does not exist
> ---
>
> Key: RANGER-3667
> URL: https://issues.apache.org/jira/browse/RANGER-3667
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Reporter: Dhaval Rajpara
>Assignee: Dhaval Rajpara
>Priority: Major
> Attachments: 0001-RANGER-3667.patch, 0002-RANGER-3667.patch
>
>
> 1. In Ranger's policy creation UI, when a resource (e.g. Hive database or
> table name) is entered in the Policy Details, the autocomplete feature will
> proactively present a dropdown of possible matches to known resource names
> pulled from the service, which the user can then select to populate the
> fields in the policy.
> 2. If there is only one match to an existing resource name, then only that
> single name will be presented in the dropdown.
> 3. If there are no matches, then the text already entered into the field will
> be presented in the autocomplete dropdown. This behavior is exactly the same
> as (2) whether the resource exists or not.
> 4. While there are some use cases where a policy may need to be created prior
> to creating the actual resource itself, there is no validation or feedback in
> the UI to indicate if a resource name already exists. In the case of a simple
> typo error, this lack of feedback can result in the creation of invalid
> policies that are then difficult to isolate and fix.
> This request is to include some additional feedback ("not found" message or
> similar) in the UI, to indicate when a resource does not exist. This would
> also assist in identifying communication issues between Ranger and the
> backend services.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (RANGER-3667) Improve feedback in policy creation UI when resource does not exist
[
https://issues.apache.org/jira/browse/RANGER-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3667:
Fix Version/s: 3.0.0
2.3.0
> Improve feedback in policy creation UI when resource does not exist
> ---
>
> Key: RANGER-3667
> URL: https://issues.apache.org/jira/browse/RANGER-3667
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
>Reporter: Dhaval Rajpara
>Assignee: Dhaval Rajpara
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 0001-RANGER-3667.patch, 0002-RANGER-3667.patch
>
>
> 1. In Ranger's policy creation UI, when a resource (e.g. Hive database or
> table name) is entered in the Policy Details, the autocomplete feature will
> proactively present a dropdown of possible matches to known resource names
> pulled from the service, which the user can then select to populate the
> fields in the policy.
> 2. If there is only one match to an existing resource name, then only that
> single name will be presented in the dropdown.
> 3. If there are no matches, then the text already entered into the field will
> be presented in the autocomplete dropdown. This behavior is exactly the same
> as (2) whether the resource exists or not.
> 4. While there are some use cases where a policy may need to be created prior
> to creating the actual resource itself, there is no validation or feedback in
> the UI to indicate if a resource name already exists. In the case of a simple
> typo error, this lack of feedback can result in the creation of invalid
> policies that are then difficult to isolate and fix.
> This request is to include some additional feedback ("not found" message or
> similar) in the UI, to indicate when a resource does not exist. This would
> also assist in identifying communication issues between Ranger and the
> backend services.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (RANGER-3690) Fix NullPointerException in java patch 054
[ https://issues.apache.org/jira/browse/RANGER-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-3690. - Fix Version/s: 3.0.0 Resolution: Fixed > Fix NullPointerException in java patch 054 > -- > > Key: RANGER-3690 > URL: https://issues.apache.org/jira/browse/RANGER-3690 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.2.0 >Reporter: Abhishek Kumar >Assignee: Abhishek Kumar >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Null pointer exception is seen on line 87 in the java patch > PatchForSyncSourceUpdate_J10054 when a ranger upgrade takes place. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3690) Fix NullPointerException in java patch 054
[ https://issues.apache.org/jira/browse/RANGER-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517869#comment-17517869 ] Pradeep Agrawal commented on RANGER-3690: - Patch committed : master branch: https://github.com/apache/ranger/commit/8662fb128ca43a2c5b4e0c507a216e2e769de108 2.3 branch : https://github.com/apache/ranger/commit/7fc670bdfad96c04268d24abdd7cf6d065d65819 > Fix NullPointerException in java patch 054 > -- > > Key: RANGER-3690 > URL: https://issues.apache.org/jira/browse/RANGER-3690 > Project: Ranger > Issue Type: Bug > Components: Ranger >Affects Versions: 2.2.0 >Reporter: Abhishek Kumar >Assignee: Abhishek Kumar >Priority: Major > Fix For: 2.3.0 > > > Null pointer exception is seen on line 87 in the java patch > PatchForSyncSourceUpdate_J10054 when a ranger upgrade takes place. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (RANGER-3692) Ranger cannot connect to the DB when the DB is outaged for a long time
[
https://issues.apache.org/jira/browse/RANGER-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518570#comment-17518570
]
Pradeep Agrawal commented on RANGER-3692:
-
[~zilong zhu] : please close RR https://reviews.apache.org/r/73927/
> Ranger cannot connect to the DB when the DB is outaged for a long time
> --
>
> Key: RANGER-3692
> URL: https://issues.apache.org/jira/browse/RANGER-3692
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 2.1.0
>Reporter: Zilong Zhu
>Assignee: Zilong Zhu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3692-Ranger-cannot-connect-to-the-DB-when-the.patch,
> 0002-RANGER-3692-Ranger-cannot-connect-to-the-DB-when-the.patch
>
>
> We had a database problem where the database was offline for more than a
> week. However ranger connot connect to the DB.
> {code:java}
> Internal Exception: java.sql.SQLException: Connections could not be acquired
> from the underlying database!
> [C3P0PooledConnectionPoolManager[identityToken->1hgf80qaljdycrokead8h|73c6299]-HelperThread-#0]
> WARN com.mchange.v2.log.slf4j.Slf4jMLog$Slf4jMLogger$WarnLogger
> (Slf4jMLog.java:223) -
> com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7179549 --
> Acquisition Attempt Failed!!! Clearing pending acquires. While trying to
> acquire a needed new resource, we failed to succeed more than the maximum
> number of allowed acquisition attempts (30). Last acquisition attempt
> exception:
> com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link
> failure
> [C3P0PooledConnectionPoolManager[identityToken->1hgf80qaljdycrokead8h|73c6299]-HelperThread-#0]
> WARN com.mchange.v2.log.slf4j.Slf4jMLog$Slf4jMLogger$WarnLogger
> (Slf4jMLog.java:220) - Having failed to acquire a resource,
> com.mchange.v2.resourcepool.BasicResourcePool@5efb2b9 is interrupting all
> Threads waiting on a resource to check out. Will try again in response to new
> client requests. {code}
> {code:java}
> Internal Exception: java.sql.SQLException: An SQLException was provoked by
> the following failure: com.mchange.v2.resourcepool.ResourcePoolException: A
> ResourcePool cannot acquire a new resource -- the factory or source appears
> to be down.
> {code}
> I found out that this is a bug in c3p0 0.9.5.3. This bug was resolved in
> 0.9.5.4. So I suggest to upgrade the version of c3p0 to 0.9.5.4.
> [Force kill acquires by rscadrde · Pull Request #91 · swaldman/c3p0 ·
> GitHub|https://github.com/swaldman/c3p0/pull/91]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (RANGER-3692) Ranger cannot connect to the DB when the DB is outaged for a long time
[
https://issues.apache.org/jira/browse/RANGER-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal resolved RANGER-3692.
-
Fix Version/s: 3.0.0
2.3.0
Resolution: Fixed
commit link master branch :
[https://github.com/apache/ranger/commit/a93571daedaa45ffc98869410148cf1c55a90eaf]
2.3 branch :
https://github.com/apache/ranger/commit/d9d080dea207319147190f2a8875effca95d4d73
> Ranger cannot connect to the DB when the DB is outaged for a long time
> --
>
> Key: RANGER-3692
> URL: https://issues.apache.org/jira/browse/RANGER-3692
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 2.1.0
>Reporter: Zilong Zhu
>Assignee: Zilong Zhu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3692-Ranger-cannot-connect-to-the-DB-when-the.patch,
> 0002-RANGER-3692-Ranger-cannot-connect-to-the-DB-when-the.patch
>
>
> We had a database problem where the database was offline for more than a
> week. However ranger connot connect to the DB.
> {code:java}
> Internal Exception: java.sql.SQLException: Connections could not be acquired
> from the underlying database!
> [C3P0PooledConnectionPoolManager[identityToken->1hgf80qaljdycrokead8h|73c6299]-HelperThread-#0]
> WARN com.mchange.v2.log.slf4j.Slf4jMLog$Slf4jMLogger$WarnLogger
> (Slf4jMLog.java:223) -
> com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7179549 --
> Acquisition Attempt Failed!!! Clearing pending acquires. While trying to
> acquire a needed new resource, we failed to succeed more than the maximum
> number of allowed acquisition attempts (30). Last acquisition attempt
> exception:
> com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link
> failure
> [C3P0PooledConnectionPoolManager[identityToken->1hgf80qaljdycrokead8h|73c6299]-HelperThread-#0]
> WARN com.mchange.v2.log.slf4j.Slf4jMLog$Slf4jMLogger$WarnLogger
> (Slf4jMLog.java:220) - Having failed to acquire a resource,
> com.mchange.v2.resourcepool.BasicResourcePool@5efb2b9 is interrupting all
> Threads waiting on a resource to check out. Will try again in response to new
> client requests. {code}
> {code:java}
> Internal Exception: java.sql.SQLException: An SQLException was provoked by
> the following failure: com.mchange.v2.resourcepool.ResourcePoolException: A
> ResourcePool cannot acquire a new resource -- the factory or source appears
> to be down.
> {code}
> I found out that this is a bug in c3p0 0.9.5.3. This bug was resolved in
> 0.9.5.4. So I suggest to upgrade the version of c3p0 to 0.9.5.4.
> [Force kill acquires by rscadrde · Pull Request #91 · swaldman/c3p0 ·
> GitHub|https://github.com/swaldman/c3p0/pull/91]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3611) Uncatched NullPointerException when missing lastKnownVersion in ServiceREST::getServicePoliciesIfUpdated
[
https://issues.apache.org/jira/browse/RANGER-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17518573#comment-17518573
]
Pradeep Agrawal commented on RANGER-3611:
-
[~kirbyzhou] : Please close the RR. https://reviews.apache.org/r/73835/
> Uncatched NullPointerException when missing lastKnownVersion in
> ServiceREST::getServicePoliciesIfUpdated
>
>
> Key: RANGER-3611
> URL: https://issues.apache.org/jira/browse/RANGER-3611
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 2.2.0, 2.3.0
>Reporter: kirby zhou
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> A simple Rest API call by CURL will cause uncatched NullPointerException in
> logs.
> Actual:
>
> {code:java}
> ]% curl -v http://localhost:6080/service/plugins/policies/download/hdfsdev
> ...
> < HTTP/1.1 404 Not Found
> ...
> No Message here
> * Closing connection 0 {code}
>
> And logs in catalina.out
> {code:java}
> EVERE: Servlet.service() for servlet [REST Service] in context with path []
> threw exception
> java.lang.NullPointerException
> at
> org.apache.ranger.rest.ServiceREST.getServicePoliciesIfUpdated(ServiceREST.java:3054)
> at
> org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692)
> at
> org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$43bccb60.getServicePoliciesIfUpdated()
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
> at
> org.apa
[jira] [Resolved] (RANGER-3611) Uncatched NullPointerException when missing lastKnownVersion in ServiceREST::getServicePoliciesIfUpdated
[
https://issues.apache.org/jira/browse/RANGER-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal resolved RANGER-3611.
-
Fix Version/s: 3.0.0
2.3.0
Resolution: Fixed
Master branch commit link :
https://github.com/apache/ranger/commit/49ac30ab5bdc8465156a89783b30c960b7499682
2.3 branch commit link :
https://github.com/apache/ranger/commit/88a375f3cb5ef4cc110494cd2582cb2f760e3a9d
> Uncatched NullPointerException when missing lastKnownVersion in
> ServiceREST::getServicePoliciesIfUpdated
>
>
> Key: RANGER-3611
> URL: https://issues.apache.org/jira/browse/RANGER-3611
> Project: Ranger
> Issue Type: Bug
> Components: admin
>Affects Versions: 2.2.0, 2.3.0
>Reporter: kirby zhou
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
>
> A simple Rest API call by CURL will cause uncatched NullPointerException in
> logs.
> Actual:
>
> {code:java}
> ]% curl -v http://localhost:6080/service/plugins/policies/download/hdfsdev
> ...
> < HTTP/1.1 404 Not Found
> ...
> No Message here
> * Closing connection 0 {code}
>
> And logs in catalina.out
> {code:java}
> EVERE: Servlet.service() for servlet [REST Service] in context with path []
> threw exception
> java.lang.NullPointerException
> at
> org.apache.ranger.rest.ServiceREST.getServicePoliciesIfUpdated(ServiceREST.java:3054)
> at
> org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692)
> at
> org.apache.ranger.rest.ServiceREST$$EnhancerBySpringCGLIB$$43bccb60.getServicePoliciesIfUpdated()
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at
> com.sun.jersey.sp
[jira] [Resolved] (RANGER-3689) Ranger : ranger-2.3 Port missing commits.
[ https://issues.apache.org/jira/browse/RANGER-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pradeep Agrawal resolved RANGER-3689. - Assignee: Pradeep Agrawal Resolution: Done > Ranger : ranger-2.3 Port missing commits. > - > > Key: RANGER-3689 > URL: https://issues.apache.org/jira/browse/RANGER-3689 > Project: Ranger > Issue Type: Task > Components: Ranger >Affects Versions: 2.3.0 >Reporter: Mateen N Mansoori >Assignee: Pradeep Agrawal >Priority: Major > Fix For: 2.3.0 > > > The ranger-2.3 branch is approximately 70 commits behind the master branch, > using this ticket to track the porting of missing commits. > List of commits which are missing from ranger-2.3 : > |1|[RANGER-3435: Add unique index on guid and service id column of > x_poli…|https://github.com/apache/ranger/commit/ec7e57e284a50773f8106a748b117692e9a81105]| > |2|[RANGER-3439: REST api to get or delete ranger policy based on guid > an…|https://github.com/apache/ranger/commit/db9f9a488e99092b9c0dba125dcb5e5efa2ad9a6]| > |3|[RANGER-3433: Null Dereference in ServiceREST getPolicyByName > method|https://github.com/apache/ranger/commit/1639896aac695057971c7b73a0b91265b3c3e772]| > |4|[RANGER-3023: Permission tab takes longer time to load with large > numb…|https://github.com/apache/ranger/commit/5732060da4c4c88c1fba3c89fa5369ea56c9b942] > …| > |5|[RANGER-3509: updateRoles() REST API updated to permit > role-admins|https://github.com/apache/ranger/commit/e2566827e963afe8b939f4f1a22ccab22716ba04]| > |6|[RANGER-3505: modified code to ignore case while validating a user > for…|https://github.com/apache/ranger/commit/5ca622fedeb0db6738ebe4a7628ccdbcc7d22fbd]| > |7|[RANGER-3510 : Ranger upgrade spring framework version to > 5.3.12|https://github.com/apache/ranger/commit/63aeb5285c3259e6669f88ffbe4192aedd833733]| > |8|[RANGER-3504 : Create framework to execute DB patch dependent on Java > …|https://github.com/apache/ranger/commit/dc6dc621fc99f1dbff355c2e2ac00472155a0baf]| > |9|[RANGER-3516 : J10045 patch is taking more time during > upgrade|https://github.com/apache/ranger/commit/8068996e42d79a8c0d9bb56b77bb4ec82bfe4113]| > |10|[RANGER-3519: Provide an option to optimize space needed by Trie > objects|https://github.com/apache/ranger/commit/71888f243d38ae7cff5e0406c7d54a386d269664]| > |11|[RANGER-3519: Provide an option to optimize space needed by Trie objects > - > part2|https://github.com/apache/ranger/commit/5852efde1cba728ad580231ad02145ea72861186]| > |12|[RANGER-3439: Add rest api to get or delete ranger policy based on guid - > part2|https://github.com/apache/ranger/commit/000e6351ee4628979a20e2b72ac6f226e6dd1c0e]| > |13|[RANGER-3507:Handle trailing slash in the ranger Hive URL policy > autho…|https://github.com/apache/ranger/commit/d8f674d3fab849aee7daf8e49a21856fdee82c34]| > |14|[RANGER-3514: Java patch to update sync source on > upgrades|https://github.com/apache/ranger/commit/5fb097fda8c51dc9fe671e4105e8b8a7fb5697cd]| > |15|[RANGER-3515: Enhance Ranger Java client SSL config to be configured > u…|https://github.com/apache/ranger/commit/b56aa63a9e1b2020e208c170642a96f5d62cd892]| > |16|[RANGER-3522: Improve Tagsync authentication error > reporting|https://github.com/apache/ranger/commit/3f82858760e01ed186a2b3055c95b9cdd343db4b]| > |17|[RANGER-3522: Improve Tagsync authentication error reporting - > Part-2|https://github.com/apache/ranger/commit/03f6d3f18f8576d710928be4b148143b8a9f8d91]| > |18|[RANGER-3493: Add unique index on service and resource_signature > colum…|https://github.com/apache/ranger/commit/de8f5e197fb93fcb924f7a59a88013b99bd1194b]| > |19|[RANGER-3511: Create Java patch to update policy resource-signature > to…|https://github.com/apache/ranger/commit/4fdb3af5fc21f43ab22b2fb4d0e411b500460cbc]| > |20|[RANGER-3490: Make policy resource signature is unique in a > service|https://github.com/apache/ranger/commit/856571c4348e31725498c0922338339c76ebba02]| > |21|[RANGER-3276 Remove duplicate code from > buildks|https://github.com/apache/ranger/commit/3045345f3dea4fa44cc522df7b171d6fb3bd5303]| > |22|[RANGER-3518: Limit the query size stored in Audit > logs|https://github.com/apache/ranger/commit/a7b527bbd0df8ba86eee7b3fdc65b470bbbc17fa]| > |23|[RANGER-3528 : Ranger Group creation audit is not shown during > service…|https://github.com/apache/ranger/commit/bb9b3cd14d5ebdb5381ca4a03db27b469c2277e1]| > |24|[RANGER-3468: Fixed an issue where inactivity timeout request is not > h…|https://github.com/apache/ranger/commit/6678ef77438d1289e0ade0cc2e7652a6bd836621]| > |25|[RANGER-3438: Optimized code to extract GroupPrincipals from javax > Sub…|https://github.com/apache/ranger/commit/84cdf593423f03c3082db3baee9bb89149205b5a]| > |26|[RANGER-3435: Add unique index on guid, service and zone_
[jira] [Commented] (RANGER-3632) Improve ranger logs, RENAME_ON_ROTATE and others
[
https://issues.apache.org/jira/browse/RANGER-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17519292#comment-17519292
]
Pradeep Agrawal commented on RANGER-3632:
-
ranger-2.3 only.
> Improve ranger logs, RENAME_ON_ROTATE and others
> -
>
> Key: RANGER-3632
> URL: https://issues.apache.org/jira/browse/RANGER-3632
> Project: Ranger
> Issue Type: Improvement
> Components: admin, kms
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Currently, the filename of the access-log in use has a timestamp as the
> suffix. This brings trouble to some log monitoring and analysis programs,
> such as "tail -f access-log"
> Need to add an option to enable tomcat's RenameOnRotate capability to fix the
> file name of access-log.
>
> {code:java}
> // in EmbeddedServer::start()
> valve.setRenameOnRotate(
> EmbeddedServerUtil.getConfig(ACCESS_LOG_RENAME_ON_ROTATE, false);
> );{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (RANGER-3704) remove semicolon from c3P0 preferredTestQuery
Pradeep Agrawal created RANGER-3704:
---
Summary: remove semicolon from c3P0 preferredTestQuery
Key: RANGER-3704
URL: https://issues.apache.org/jira/browse/RANGER-3704
Project: Ranger
Issue Type: Wish
Components: Ranger
Reporter: Pradeep Agrawal
Assignee: Pradeep Agrawal
Fix For: 3.0.0, 2.3.0
Error message:
{code:java}
java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:446)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1054)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:213)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:37)
at
oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:733)
at
oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:904)
at
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1082)
at
oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1276)
at
oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:366)
at
com.mchange.v2.c3p0.impl.NewProxyStatement.executeQuery(NewProxyStatement.java:220)
at
com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:286)
at
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:510)
at
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.finerLoggingTestPooledConnection(C3P0PooledConnectionPool.java:452)
at
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.finerLoggingTestPooledConnection(C3P0PooledConnectionPool.java:444)
at
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:434)
at
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:2211)
at
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
Caused by: Error : 933, Position : 18, Sql = select 1 from dual;, OriginalSql =
select 1 from dual;, Error Msg = ORA-00933: SQL command not properly ended
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:498)
... 20 more
2022-04-11 12:38:45,538
[C3P0PooledConnectionPoolManager[identityToken->1br54owao6u0q6i172od24|1f0ea3d4]-HelperThread-#2]
DEBUG com.mchange.v2.log.slf4j.Slf4jMLog$Slf4jMLogger$DebugLogger
(Slf4jMLog.java:207) - com.mchange.v2.c3p0.impl.NewPooledConnection@5f00a319
handling a throwable. {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (RANGER-3704) remove semicolon from c3P0 preferredTestQuery
[
https://issues.apache.org/jira/browse/RANGER-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-3704:
Attachment: 0001-RANGER-3704-remove-semicolon-from-c3P0-preferredTest.patch
> remove semicolon from c3P0 preferredTestQuery
> -
>
> Key: RANGER-3704
> URL: https://issues.apache.org/jira/browse/RANGER-3704
> Project: Ranger
> Issue Type: Wish
> Components: Ranger
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3704-remove-semicolon-from-c3P0-preferredTest.patch
>
>
> Error message:
> {code:java}
> java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended
> at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
> at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:446)
> at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1054)
> at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
> at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
> at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
> at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:213)
> at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:37)
> at
> oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:733)
> at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:904)
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1082)
> at
> oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1276)
> at
> oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:366)
> at
> com.mchange.v2.c3p0.impl.NewProxyStatement.executeQuery(NewProxyStatement.java:220)
> at
> com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:286)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:510)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.finerLoggingTestPooledConnection(C3P0PooledConnectionPool.java:452)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.finerLoggingTestPooledConnection(C3P0PooledConnectionPool.java:444)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:434)
> at
> com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:2211)
> at
> com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
> Caused by: Error : 933, Position : 18, Sql = select 1 from dual;, OriginalSql
> = select 1 from dual;, Error Msg = ORA-00933: SQL command not properly ended
> at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:498)
> ... 20 more
> 2022-04-11 12:38:45,538
> [C3P0PooledConnectionPoolManager[identityToken->1br54owao6u0q6i172od24|1f0ea3d4]-HelperThread-#2]
> DEBUG com.mchange.v2.log.slf4j.Slf4jMLog$Slf4jMLogger$DebugLogger
> (Slf4jMLog.java:207) - com.mchange.v2.c3p0.impl.NewPooledConnection@5f00a319
> handling a throwable. {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3704) remove semicolon from c3P0 preferredTestQuery
[
https://issues.apache.org/jira/browse/RANGER-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520606#comment-17520606
]
Pradeep Agrawal commented on RANGER-3704:
-
No, I checked with previous versions also.
> remove semicolon from c3P0 preferredTestQuery
> -
>
> Key: RANGER-3704
> URL: https://issues.apache.org/jira/browse/RANGER-3704
> Project: Ranger
> Issue Type: Wish
> Components: Ranger
>Reporter: Pradeep Agrawal
>Assignee: Pradeep Agrawal
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-RANGER-3704-remove-semicolon-from-c3P0-preferredTest.patch
>
>
> Error message:
> {code:java}
> java.sql.SQLSyntaxErrorException: ORA-00933: SQL command not properly ended
> at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
> at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:446)
> at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1054)
> at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
> at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
> at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
> at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:213)
> at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:37)
> at
> oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:733)
> at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:904)
> at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1082)
> at
> oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1276)
> at
> oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:366)
> at
> com.mchange.v2.c3p0.impl.NewProxyStatement.executeQuery(NewProxyStatement.java:220)
> at
> com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:286)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:510)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.finerLoggingTestPooledConnection(C3P0PooledConnectionPool.java:452)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.finerLoggingTestPooledConnection(C3P0PooledConnectionPool.java:444)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:434)
> at
> com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:2211)
> at
> com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
> Caused by: Error : 933, Position : 18, Sql = select 1 from dual;, OriginalSql
> = select 1 from dual;, Error Msg = ORA-00933: SQL command not properly ended
> at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:498)
> ... 20 more
> 2022-04-11 12:38:45,538
> [C3P0PooledConnectionPoolManager[identityToken->1br54owao6u0q6i172od24|1f0ea3d4]-HelperThread-#2]
> DEBUG com.mchange.v2.log.slf4j.Slf4jMLog$Slf4jMLogger$DebugLogger
> (Slf4jMLog.java:207) - com.mchange.v2.c3p0.impl.NewPooledConnection@5f00a319
> handling a throwable. {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (RANGER-3632) Improve ranger logs, RENAME_ON_ROTATE and others
[
https://issues.apache.org/jira/browse/RANGER-3632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17520958#comment-17520958
]
Pradeep Agrawal commented on RANGER-3632:
-
[~kirbyzhou] : Can you attach the patch here in jira which need to be merged ?
> Improve ranger logs, RENAME_ON_ROTATE and others
> -
>
> Key: RANGER-3632
> URL: https://issues.apache.org/jira/browse/RANGER-3632
> Project: Ranger
> Issue Type: Improvement
> Components: admin, kms
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Currently, the filename of the access-log in use has a timestamp as the
> suffix. This brings trouble to some log monitoring and analysis programs,
> such as "tail -f access-log"
> Need to add an option to enable tomcat's RenameOnRotate capability to fix the
> file name of access-log.
>
> {code:java}
> // in EmbeddedServer::start()
> valve.setRenameOnRotate(
> EmbeddedServerUtil.getConfig(ACCESS_LOG_RENAME_ON_ROTATE, false);
> );{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
