[jira] [Created] (RANGER-3986) Upgrade trino guice dependency to 5.1.0
ziyue created RANGER-3986: - Summary: Upgrade trino guice dependency to 5.1.0 Key: RANGER-3986 URL: https://issues.apache.org/jira/browse/RANGER-3986 Project: Ranger Issue Type: Improvement Components: plugins Reporter: ziyue The latest trino SPI 403 is running on Java 17, which guice 4.x doesn't support. we should upgrade the guice version to make ranger-trino-plugin possible to running on JVM 17 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3986) Upgrade trino guice dependency to 5.1.0
[ https://issues.apache.org/jira/browse/RANGER-3986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17638607#comment-17638607 ] ziyue commented on RANGER-3986: --- https://reviews.apache.org/r/74219/ > Upgrade trino guice dependency to 5.1.0 > --- > > Key: RANGER-3986 > URL: https://issues.apache.org/jira/browse/RANGER-3986 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: ziyue >Priority: Major > > The latest trino SPI 403 is running on Java 17, which guice 4.x doesn't > support. we should upgrade the guice version to make ranger-trino-plugin > possible to running on JVM 17 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3985) Trino plugin: Check table name when creating tables
[ https://issues.apache.org/jira/browse/RANGER-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonas Hartwig updated RANGER-3985: -- Description: The ranger rules to create tables in Trino only check schema level on create. They should check by table name as well. It easily get inconsistent, if users or groups are allowed to read, drop and alter certain tables like t__* but may create any. So rules to create all tables should then be catalog/schema/* (was: The ranger rules to create tables in Trino only check data base level on create. They should check by table name as well. It easily get inconsistent, if users or groups are allowed to read, drop and alter certain tables like t__* but may create any. At the moment, the same rule is used to check if a schema can be created for table creation) > Trino plugin: Check table name when creating tables > --- > > Key: RANGER-3985 > URL: https://issues.apache.org/jira/browse/RANGER-3985 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 2.3.0 >Reporter: Jonas Hartwig >Priority: Major > Fix For: 2.4.0 > > > The ranger rules to create tables in Trino only check schema level on create. > They should check by table name as well. It easily get inconsistent, if users > or groups are allowed to read, drop and alter certain tables like t__* > but may create any. So rules to create all tables should then be > catalog/schema/* -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3985) Trino plugin: Check table name when creating tables
[ https://issues.apache.org/jira/browse/RANGER-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonas Hartwig updated RANGER-3985: -- Description: The ranger rules to create tables in Trino currently check schema level to create. If this is set, anyone can create any table/view. There is no way to limit the naming of tables. However e.g. drop, alter rights are granted on table level. So user might create any table, but not remove them. To allow a more strict implementation view/table creation should verify table name as well. In that case the previous behaviour can be created by adding a rule to allow create on catalog/schema/*. was:The ranger rules to create tables in Trino only check schema level on create. They should check by table name as well. It easily get inconsistent, if users or groups are allowed to read, drop and alter certain tables like t__* but may create any. So rules to create all tables should then be catalog/schema/* > Trino plugin: Check table name when creating tables > --- > > Key: RANGER-3985 > URL: https://issues.apache.org/jira/browse/RANGER-3985 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 2.3.0 >Reporter: Jonas Hartwig >Priority: Major > Fix For: 2.4.0 > > > The ranger rules to create tables in Trino currently check schema level to > create. > If this is set, anyone can create any table/view. There is no way to limit > the naming of tables. > However e.g. drop, alter rights are granted on table level. So user might > create any table, but not remove them. > To allow a more strict implementation view/table creation should verify table > name as well. > In that case the previous behaviour can be created by adding a rule to allow > create on catalog/schema/*. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3985) Trino plugin: Check table name when creating tables
[ https://issues.apache.org/jira/browse/RANGER-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17638631#comment-17638631 ] Jonas Hartwig commented on RANGER-3985: --- Here is a proposal: https://github.com/apache/ranger/pull/191 > Trino plugin: Check table name when creating tables > --- > > Key: RANGER-3985 > URL: https://issues.apache.org/jira/browse/RANGER-3985 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 2.3.0 >Reporter: Jonas Hartwig >Priority: Major > Fix For: 2.4.0 > > > The ranger rules to create tables in Trino currently check schema level to > create. > If this is set, anyone can create any table/view. There is no way to limit > the naming of tables. > However e.g. drop, alter rights are granted on table level. So user might > create any table, but not remove them. > To allow a more strict implementation view/table creation should verify table > name as well. > In that case the previous behaviour can be created by adding a rule to allow > create on catalog/schema/*. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Review Request 74208: RANGER-3971: Upgrade HBASE version to 2.4.6
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74208/#review224908 --- Ship it! Ship It! - YiJi Gao On 十一月 23, 2022, 4:20 a.m., bhavik patel wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74208/ > --- > > (Updated 十一月 23, 2022, 4:20 a.m.) > > > Review request for ranger. > > > Bugs: RANGER-3971 > https://issues.apache.org/jira/browse/RANGER-3971 > > > Repository: ranger > > > Description > --- > > Upgrade HBASE version to 2.4.6 > > > Diffs > - > > distro/src/main/assembly/admin-web.xml 9b7475492 > hbase-agent/pom.xml 7d2638c05 > > hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java > 417c9c892 > pom.xml 44eef2a0c > > > Diff: https://reviews.apache.org/r/74208/diff/2/ > > > Testing > --- > > 1. Plugin communication is working. > 2. Hbase service test-connections is working. > 3. executed Junit using mvn > > > Thanks, > > bhavik patel > >
[jira] [Created] (RANGER-3987) Potential risk of OOM
KyrieG created RANGER-3987:
--
Summary: Potential risk of OOM
Key: RANGER-3987
URL: https://issues.apache.org/jira/browse/RANGER-3987
Project: Ranger
Issue Type: Bug
Components: admin
Affects Versions: 2.2.0
Reporter: KyrieG
During each policy loading, the attribute "LastActivationTimeInMillis" is
always set to System.currentTimeMillis(). See loadPolicy():
{code:java}
// from PolicyRefresher.java loadPolicy()
//load policy from PolicyAdmin
ServicePolicies svcPolicies = loadPolicyfromPolicyAdmin();
if (svcPolicies == null) {
//if Policy fetch from Policy Admin Fails, load from cache
if (!policiesSetInPlugin) {
svcPolicies = loadFromCache();
}
}
if (PERF_POLICYENGINE_INIT_LOG.isDebugEnabled()) {
long freeMemory = Runtime.getRuntime().freeMemory();
long totalMemory = Runtime.getRuntime().totalMemory();
PERF_POLICYENGINE_INIT_LOG.debug("In-Use memory: " + (totalMemory -
freeMemory) + ", Free memory:" + freeMemory);
}
if (svcPolicies != null) {
plugIn.setPolicies(svcPolicies);
policiesSetInPlugin = true;
serviceDefSetInPlugin = false;
setLastActivationTimeInMillis(System.currentTimeMillis()); // always updated
during each policy loading
lastKnownVersion = svcPolicies.getPolicyVersion() != null ?
svcPolicies.getPolicyVersion() : -1L;
} else {
if (!policiesSetInPlugin && !serviceDefSetInPlugin) {
plugIn.setPolicies(null);
serviceDefSetInPlugin = true;
}
} {code}
In this case, the column "info" from table "x_plugin_info" would always need to
be updated since it is a json string containing activationTime. See
doCreateOrUpdateXXPluginInfo():
{code:java}
// from AssetMgr, doCreateOrUpdateXXPluginInfo().
if (lastPolicyActivationTime != null && lastPolicyActivationTime > 0 &&
(dbObj.getPolicyActivationTime() == null ||
!dbObj.getPolicyActivationTime().equals(lastPolicyActivationTime))) {
dbObj.setPolicyActivationTime(lastPolicyActivationTime);
needsUpdating = true;
} {code}
Since doCreateOrUpdateXXPluginInfo() is a Runnble committed to
RangerTransactionService. (RangerTransactionSynchronizationAdapter in Ranger
2.3.0 though, the risk might still be there). Also see
doCreateOrUpdateXXPluginInfo():
{code:java}
// code placeholder
commitWork = new Runnable() {
@Override
public void run() {
doCreateOrUpdateXXPluginInfo(pluginInfo, entityType,
isTagVersionResetNeeded, clusterName);
}
};
...
activityLogger.commitAfterTransactionComplete(commitWork);{code}
RangerTransactionService use a thread pool with unlimited work queue,
ScheduledExecutorService, to store extra Runnables.
In our cases, there are 1000+ hive and hbase instances, the ranger admin seems
to be under tremendous pressure becuase every instance would periodically
request policy-downloading API and trigger an update of the table
"x_plugin_info". Since the core thread pool seems to be poor and DB is also
likely under pressure, the work queue is stacking, leaking out JVM Heap and
causing OOM finally.
I think adding more core threads would help, but when the system grow, this
part of code would bring a lot overhead, is there any solution?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [ranger] simonvanderveldt commented on pull request #26: RANGER-2128: Implementation of Ranger Spark SQL plugin
simonvanderveldt commented on PR #26: URL: https://github.com/apache/ranger/pull/26#issuecomment-1327689372 What does this PR do/accomplish that isn't already possible with the existing Hive support? We're currently running Spark Thriftserver (3.2.x) with the kyuubi plugin against Ranger where in Ranger we've defined the service as a Hive service and everything with regards to authentication and authorization seems to be working as expected. The only thing that I've observed that doesn't work is the auto-complete when creating policies via the Ranger UI, I assume this is a slight dialect difference in the response from the Spark Thriftserver vs a real HiveServer2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (RANGER-3988) Trino plugin should differntiate between views and tables
Jonas Hartwig created RANGER-3988: - Summary: Trino plugin should differntiate between views and tables Key: RANGER-3988 URL: https://issues.apache.org/jira/browse/RANGER-3988 Project: Ranger Issue Type: Improvement Components: plugins, Ranger Affects Versions: 2.3.0 Reporter: Jonas Hartwig Fix For: 2.4.0 The Trino plugin only "knows" tables. Views are validated against the same rules "as" tables. Views and tables should be treated separately. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3987) Potential risk of OOM
[
https://issues.apache.org/jira/browse/RANGER-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KyrieG updated RANGER-3987:
---
Description:
During each policy loading, the attribute "LastActivationTimeInMillis" is
always set to System.currentTimeMillis(). See loadPolicy():
{code:java}
// from PolicyRefresher.java loadPolicy()
//load policy from PolicyAdmin
ServicePolicies svcPolicies = loadPolicyfromPolicyAdmin();
if (svcPolicies == null) {
//if Policy fetch from Policy Admin Fails, load from cache
if (!policiesSetInPlugin) {
svcPolicies = loadFromCache();
}
}
if (PERF_POLICYENGINE_INIT_LOG.isDebugEnabled()) {
long freeMemory = Runtime.getRuntime().freeMemory();
long totalMemory = Runtime.getRuntime().totalMemory();
PERF_POLICYENGINE_INIT_LOG.debug("In-Use memory: " + (totalMemory -
freeMemory) + ", Free memory:" + freeMemory);
}
if (svcPolicies != null) {
plugIn.setPolicies(svcPolicies);
policiesSetInPlugin = true;
serviceDefSetInPlugin = false;
setLastActivationTimeInMillis(System.currentTimeMillis()); // always updated
during each policy loading
lastKnownVersion = svcPolicies.getPolicyVersion() != null ?
svcPolicies.getPolicyVersion() : -1L;
} else {
if (!policiesSetInPlugin && !serviceDefSetInPlugin) {
plugIn.setPolicies(null);
serviceDefSetInPlugin = true;
}
} {code}
In this case, the column "info" from table "x_plugin_info" would always need to
be updated since it is a json string containing activationTime. See
doCreateOrUpdateXXPluginInfo():
{code:java}
// from AssetMgr, doCreateOrUpdateXXPluginInfo().
if (lastPolicyActivationTime != null && lastPolicyActivationTime > 0 &&
(dbObj.getPolicyActivationTime() == null ||
!dbObj.getPolicyActivationTime().equals(lastPolicyActivationTime))) {
dbObj.setPolicyActivationTime(lastPolicyActivationTime);
needsUpdating = true;
} {code}
Since doCreateOrUpdateXXPluginInfo() is a Runnble committed to
RangerTransactionService. (RangerTransactionSynchronizationAdapter in Ranger
2.3.0 though, the risk might still be there). Also see
doCreateOrUpdateXXPluginInfo():
{code:java}
// code placeholder
commitWork = new Runnable() {
@Override
public void run() {
doCreateOrUpdateXXPluginInfo(pluginInfo, entityType,
isTagVersionResetNeeded, clusterName);
}
};
...
activityLogger.commitAfterTransactionComplete(commitWork);{code}
RangerTransactionService use a thread pool with unlimited work queue,
ScheduledExecutorService, to store extra Runnables.
In our cases, there are 1000+ hive and hbase instances, the ranger admin seems
to be under tremendous pressure becuase every instance would periodically
request policy-downloading API and trigger an update of the table
"x_plugin_info". Since the core thread pool seems to be poor and DB is also
likely under pressure, the work queue is stacking, leaking out JVM Heap and
causing OOM finally.
I think adding more core threads would help, but when the system grow, this
part of code would bring a lot overhead, is there any solution?
was:
During each policy loading, the attribute "LastActivationTimeInMillis" is
always set to System.currentTimeMillis(). See loadPolicy():
{code:java}
// from PolicyRefresher.java loadPolicy()
//load policy from PolicyAdmin
ServicePolicies svcPolicies = loadPolicyfromPolicyAdmin();
if (svcPolicies == null) {
//if Policy fetch from Policy Admin Fails, load from cache
if (!policiesSetInPlugin) {
svcPolicies = loadFromCache();
}
}
if (PERF_POLICYENGINE_INIT_LOG.isDebugEnabled()) {
long freeMemory = Runtime.getRuntime().freeMemory();
long totalMemory = Runtime.getRuntime().totalMemory();
PERF_POLICYENGINE_INIT_LOG.debug("In-Use memory: " + (totalMemory -
freeMemory) + ", Free memory:" + freeMemory);
}
if (svcPolicies != null) {
plugIn.setPolicies(svcPolicies);
policiesSetInPlugin = true;
serviceDefSetInPlugin = false;
setLastActivationTimeInMillis(System.currentTimeMillis()); // always updated
during each policy loading
lastKnownVersion = svcPolicies.getPolicyVersion() != null ?
svcPolicies.getPolicyVersion() : -1L;
} else {
if (!policiesSetInPlugin && !serviceDefSetInPlugin) {
plugIn.setPolicies(null);
serviceDefSetInPlugin = true;
}
} {code}
In this case, the column "info" from table "x_plugin_info" would always need to
be updated since it is a json string containing activationTime. See
doCreateOrUpdateXXPluginInfo():
{code:java}
// from AssetMgr, doCreateOrUpdateXXPluginInfo().
if (lastPolicyActivationTime != null && lastPolicyActivationTime > 0 &&
(dbObj.getPolicyActivationTime() == null ||
!dbObj.getPolicyActivationTime().equals(lastPolicyActivationTime))) {
dbObj.setPolicyActivationTime(lastPolicyActivationTime);
needsUpdating = true;
} {code}
Since doCreateOrUpdateXXPluginInfo() is a Runnble committed to
RangerTransactionService. (RangerTransactionSy
[jira] [Updated] (RANGER-3987) Potential risk of OOM
[
https://issues.apache.org/jira/browse/RANGER-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KyrieG updated RANGER-3987:
---
Description:
During every policy-loading process of other components, the attribute
"LastActivationTimeInMillis" is always set to System.currentTimeMillis(). See
loadPolicy():
{code:java}
// from PolicyRefresher.java loadPolicy()
//load policy from PolicyAdmin
ServicePolicies svcPolicies = loadPolicyfromPolicyAdmin();
if (svcPolicies == null) {
//if Policy fetch from Policy Admin Fails, load from cache
if (!policiesSetInPlugin) {
svcPolicies = loadFromCache();
}
}
if (PERF_POLICYENGINE_INIT_LOG.isDebugEnabled()) {
long freeMemory = Runtime.getRuntime().freeMemory();
long totalMemory = Runtime.getRuntime().totalMemory();
PERF_POLICYENGINE_INIT_LOG.debug("In-Use memory: " + (totalMemory -
freeMemory) + ", Free memory:" + freeMemory);
}
if (svcPolicies != null) {
plugIn.setPolicies(svcPolicies);
policiesSetInPlugin = true;
serviceDefSetInPlugin = false;
setLastActivationTimeInMillis(System.currentTimeMillis()); // always updated
during each policy loading
lastKnownVersion = svcPolicies.getPolicyVersion() != null ?
svcPolicies.getPolicyVersion() : -1L;
} else {
if (!policiesSetInPlugin && !serviceDefSetInPlugin) {
plugIn.setPolicies(null);
serviceDefSetInPlugin = true;
}
} {code}
In this case, the column "info" from table "x_plugin_info" would always need to
be updated since it is a json string containing activationTime. See
doCreateOrUpdateXXPluginInfo():
{code:java}
// from AssetMgr, doCreateOrUpdateXXPluginInfo().
if (lastPolicyActivationTime != null && lastPolicyActivationTime > 0 &&
(dbObj.getPolicyActivationTime() == null ||
!dbObj.getPolicyActivationTime().equals(lastPolicyActivationTime))) {
dbObj.setPolicyActivationTime(lastPolicyActivationTime);
needsUpdating = true;
} {code}
Since doCreateOrUpdateXXPluginInfo() is a Runnble committed to
RangerTransactionService. (RangerTransactionSynchronizationAdapter in Ranger
2.3.0 though, the risk might still be there). Also see
doCreateOrUpdateXXPluginInfo():
{code:java}
// code placeholder
commitWork = new Runnable() {
@Override
public void run() {
doCreateOrUpdateXXPluginInfo(pluginInfo, entityType,
isTagVersionResetNeeded, clusterName);
}
};
...
activityLogger.commitAfterTransactionComplete(commitWork);{code}
RangerTransactionService use a thread pool with unlimited work queue,
ScheduledExecutorService, to store extra Runnables.
In our cases, there are 1000+ hive and hbase instances, the ranger admin seems
to be under tremendous pressure becuase every instance would periodically
request policy-downloading API and trigger an update of the table
"x_plugin_info". Since the core thread pool seems to be poor and DB is also
likely under pressure, the work queue is stacking, leaking out JVM Heap and
causing OOM finally.
I think adding more core threads would help, but when the system grow, this
part of code would bring a lot overhead, is there any solution?
was:
During each policy loading, the attribute "LastActivationTimeInMillis" is
always set to System.currentTimeMillis(). See loadPolicy():
{code:java}
// from PolicyRefresher.java loadPolicy()
//load policy from PolicyAdmin
ServicePolicies svcPolicies = loadPolicyfromPolicyAdmin();
if (svcPolicies == null) {
//if Policy fetch from Policy Admin Fails, load from cache
if (!policiesSetInPlugin) {
svcPolicies = loadFromCache();
}
}
if (PERF_POLICYENGINE_INIT_LOG.isDebugEnabled()) {
long freeMemory = Runtime.getRuntime().freeMemory();
long totalMemory = Runtime.getRuntime().totalMemory();
PERF_POLICYENGINE_INIT_LOG.debug("In-Use memory: " + (totalMemory -
freeMemory) + ", Free memory:" + freeMemory);
}
if (svcPolicies != null) {
plugIn.setPolicies(svcPolicies);
policiesSetInPlugin = true;
serviceDefSetInPlugin = false;
setLastActivationTimeInMillis(System.currentTimeMillis()); // always updated
during each policy loading
lastKnownVersion = svcPolicies.getPolicyVersion() != null ?
svcPolicies.getPolicyVersion() : -1L;
} else {
if (!policiesSetInPlugin && !serviceDefSetInPlugin) {
plugIn.setPolicies(null);
serviceDefSetInPlugin = true;
}
} {code}
In this case, the column "info" from table "x_plugin_info" would always need to
be updated since it is a json string containing activationTime. See
doCreateOrUpdateXXPluginInfo():
{code:java}
// from AssetMgr, doCreateOrUpdateXXPluginInfo().
if (lastPolicyActivationTime != null && lastPolicyActivationTime > 0 &&
(dbObj.getPolicyActivationTime() == null ||
!dbObj.getPolicyActivationTime().equals(lastPolicyActivationTime))) {
dbObj.setPolicyActivationTime(lastPolicyActivationTime);
needsUpdating = true;
} {code}
Since doCreateOrUpdateXXPluginInfo() is a Runnble committed to
RangerTransactionServi
[jira] [Updated] (RANGER-3985) Trino plugin: Check table name when creating tables
[ https://issues.apache.org/jira/browse/RANGER-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonas Hartwig updated RANGER-3985: -- Description: The ranger rules to create tables in Trino only check data base level on create. They should check by table name as well. It easily get inconsistent, if users or groups are allowed to read, drop and alter certain tables like t__* but may create any. At the moment, the same rule is used to check if a catalog can be created for table creation was:The ranger rules to create tables in Trino only check data base level on create. They should check by table name as well. It easily get inconsistent, if users or groups are allowed to read, drop and alter certain tables like t__* but may create any. > Trino plugin: Check table name when creating tables > --- > > Key: RANGER-3985 > URL: https://issues.apache.org/jira/browse/RANGER-3985 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 2.3.0 >Reporter: Jonas Hartwig >Priority: Major > Fix For: 2.4.0 > > > The ranger rules to create tables in Trino only check data base level on > create. They should check by table name as well. It easily get inconsistent, > if users or groups are allowed to read, drop and alter certain tables like > t__* but may create any. > > At the moment, the same rule is used to check if a catalog can be created for > table creation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3984) Support using TiDB as mysql-db in ranger
[
https://issues.apache.org/jira/browse/RANGER-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kirby zhou updated RANGER-3984:
---
Description:
TiDB is a 95% mysql-compatible NewSQL database. For legal reason, we have to
deploy ranger based on tidb. But TiDB is missing some features, which makes
ranger unable to install properly.
[https://docs.pingcap.com/tidb/stable/mysql-compatibility#unsupported-features]
The biggest problem affecting ranger is missing "Stored procedures and
functions", "Select into".
ranger use Stored procedures in setup scripts to simplify SQL.
Some work is needed to remove the stored procedure.
{code:java}
ERROR 1064 (42000) at line 1595 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 14 near "PROCEDURE if exists
getXportalUIdByLoginId"
ERROR 1064 (42000) at line 1596 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 16 near "PROCEDURE
`getXportalUIdByLoginId`(IN input_val VARCHAR(100), OUT myid BIGINT)
BEGIN
SET myid = 0;
SELECT x_portal_user.id into myid FROM x_portal_user WHERE
x_portal_user.login_id = input_val;
END"
ERROR 1064 (42000) at line 1605 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 14 near "PROCEDURE if exists
getModulesIdByName"
ERROR 1064 (42000) at line 1606 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 16 near "PROCEDURE
`getModulesIdByName`(IN input_val VARCHAR(100), OUT myid BIGINT)
BEGIN
SET myid = 0;
SELECT x_modules_master.id into myid FROM x_modules_master WHERE
x_modules_master.module = input_val;
END"
ERROR 1064 (42000) at line 1679 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 14 near "PROCEDURE if exists
insertRangerPrerequisiteEntries"
ERROR 1064 (42000) at line 1680 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 16 near "PROCEDURE
`insertRangerPrerequisiteEntries`()
BEGIN
DECLARE adminID bigint;
DECLARE keyadminID bigint;
DECLARE rangerusersyncID bigint;
DECLARE rangertagsyncID bigint;
DECLARE moduleIdReports bigint;
DECLARE moduleIdResourceBasedPolicies bigint;
DECLARE moduleIdAudit bigint;
DECLARE moduleIdUG bigint;
DECLARE moduleIdTagBasedPolicies bigint;
DECLARE moduleIdKeyMana
ERROR 8108 (HY000) at line 1757 in file: 'ranger_core_db_mysql.sql':
Unsupported type *ast.CallStmt
{code}
was:
TiDB is a 95% mysql-compatible NewSQL database. For legal reason, we have to
deploy ranger based on tidb. But TiDB is missing some features, which makes
ranger unable to install properly.
[https://docs.pingcap.com/tidb/stable/mysql-compatibility#unsupported-features]
The biggest problem affecting ranger is missing "Stored procedures and
functions".
ranger use Stored procedures in setup scripts to simplify SQL.
Some work is needed to remove the stored procedure.
{code:java}
ERROR 1064 (42000) at line 1595 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 14 near "PROCEDURE if exists
getXportalUIdByLoginId"
ERROR 1064 (42000) at line 1596 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 16 near "PROCEDURE
`getXportalUIdByLoginId`(IN input_val VARCHAR(100), OUT myid BIGINT)
BEGIN
SET myid = 0;
SELECT x_portal_user.id into myid FROM x_portal_user WHERE
x_portal_user.login_id = input_val;
END"
ERROR 1064 (42000) at line 1605 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 14 near "PROCEDURE if exists
getModulesIdByName"
ERROR 1064 (42000) at line 1606 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL syntax; check the manual that corresponds to your TiDB
version for the right syntax to use line 1 column 16 near "PROCEDURE
`getModulesIdByName`(IN input_val VARCHAR(100), OUT myid BIGINT)
BEGIN
SET myid = 0;
SELECT x_modules_master.id into myid FROM x_modules_master WHERE
x_modules_master.module = input_val;
END"
ERROR 1064 (42000) at line 1679 in file: 'ranger_core_db_mysql.sql': You have
an error in your SQL
[jira] [Updated] (RANGER-3985) Trino plugin: Check table name when creating tables
[ https://issues.apache.org/jira/browse/RANGER-3985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonas Hartwig updated RANGER-3985: -- Description: The ranger rules to create tables in Trino only check data base level on create. They should check by table name as well. It easily get inconsistent, if users or groups are allowed to read, drop and alter certain tables like t__* but may create any. At the moment, the same rule is used to check if a schema can be created for table creation was: The ranger rules to create tables in Trino only check data base level on create. They should check by table name as well. It easily get inconsistent, if users or groups are allowed to read, drop and alter certain tables like t__* but may create any. At the moment, the same rule is used to check if a catalog can be created for table creation > Trino plugin: Check table name when creating tables > --- > > Key: RANGER-3985 > URL: https://issues.apache.org/jira/browse/RANGER-3985 > Project: Ranger > Issue Type: Improvement > Components: plugins >Affects Versions: 2.3.0 >Reporter: Jonas Hartwig >Priority: Major > Fix For: 2.4.0 > > > The ranger rules to create tables in Trino only check data base level on > create. They should check by table name as well. It easily get inconsistent, > if users or groups are allowed to read, drop and alter certain tables like > t__* but may create any. > > At the moment, the same rule is used to check if a schema can be created for > table creation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (RANGER-3983) Support getColumnMasks and getRowFilters in Trino SPI 376+
[ https://issues.apache.org/jira/browse/RANGER-3983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17638591#comment-17638591 ] ziyue commented on RANGER-3983: --- https://reviews.apache.org/r/74218/ > Support getColumnMasks and getRowFilters in Trino SPI 376+ > -- > > Key: RANGER-3983 > URL: https://issues.apache.org/jira/browse/RANGER-3983 > Project: Ranger > Issue Type: Improvement > Components: plugins >Reporter: ziyue >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > [https://github.com/trinodb/trino/commit/827de57a50426e804761044d24d96b8877b62b7e] > > The functions `getColumnMask` and `getRowFilter` were deprecated since trino > 376, and were removed in 401. > > So we should adapt to that change in ranger implementation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (RANGER-3984) Support using TiDB as mysql-db in ranger
[
https://issues.apache.org/jira/browse/RANGER-3984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kirby zhou updated RANGER-3984:
---
Attachment: ranger_core_db_tidb.patch
> Support using TiDB as mysql-db in ranger
>
>
> Key: RANGER-3984
> URL: https://issues.apache.org/jira/browse/RANGER-3984
> Project: Ranger
> Issue Type: Improvement
> Components: admin, kms
>Affects Versions: 3.0.0, 2.3.0
>Reporter: kirby zhou
>Priority: Major
> Attachments: ranger_core_db_tidb.patch
>
>
> TiDB is a 95% mysql-compatible NewSQL database. For legal reason, we have to
> deploy ranger based on tidb. But TiDB is missing some features, which makes
> ranger unable to install properly.
> [https://docs.pingcap.com/tidb/stable/mysql-compatibility#unsupported-features]
> The biggest problem affecting ranger is missing "Stored procedures and
> functions", "Select into".
> ranger use Stored procedures in setup scripts to simplify SQL.
> Some work is needed to remove the stored procedure.
>
>
> {code:java}
> ERROR 1064 (42000) at line 1595 in file: 'ranger_core_db_mysql.sql': You have
> an error in your SQL syntax; check the manual that corresponds to your TiDB
> version for the right syntax to use line 1 column 14 near "PROCEDURE if
> exists getXportalUIdByLoginId"
> ERROR 1064 (42000) at line 1596 in file: 'ranger_core_db_mysql.sql': You have
> an error in your SQL syntax; check the manual that corresponds to your TiDB
> version for the right syntax to use line 1 column 16 near "PROCEDURE
> `getXportalUIdByLoginId`(IN input_val VARCHAR(100), OUT myid BIGINT)
> BEGIN
> SET myid = 0;
> SELECT x_portal_user.id into myid FROM x_portal_user WHERE
> x_portal_user.login_id = input_val;
> END"
> ERROR 1064 (42000) at line 1605 in file: 'ranger_core_db_mysql.sql': You have
> an error in your SQL syntax; check the manual that corresponds to your TiDB
> version for the right syntax to use line 1 column 14 near "PROCEDURE if
> exists getModulesIdByName"
> ERROR 1064 (42000) at line 1606 in file: 'ranger_core_db_mysql.sql': You have
> an error in your SQL syntax; check the manual that corresponds to your TiDB
> version for the right syntax to use line 1 column 16 near "PROCEDURE
> `getModulesIdByName`(IN input_val VARCHAR(100), OUT myid BIGINT)
> BEGIN
> SET myid = 0;
> SELECT x_modules_master.id into myid FROM x_modules_master WHERE
> x_modules_master.module = input_val;
> END"
> ERROR 1064 (42000) at line 1679 in file: 'ranger_core_db_mysql.sql': You have
> an error in your SQL syntax; check the manual that corresponds to your TiDB
> version for the right syntax to use line 1 column 14 near "PROCEDURE if
> exists insertRangerPrerequisiteEntries"
> ERROR 1064 (42000) at line 1680 in file: 'ranger_core_db_mysql.sql': You have
> an error in your SQL syntax; check the manual that corresponds to your TiDB
> version for the right syntax to use line 1 column 16 near "PROCEDURE
> `insertRangerPrerequisiteEntries`()
> BEGIN
> DECLARE adminID bigint;
> DECLARE keyadminID bigint;
> DECLARE rangerusersyncID bigint;
> DECLARE rangertagsyncID bigint;
> DECLARE moduleIdReports bigint;
> DECLARE moduleIdResourceBasedPolicies bigint;
> DECLARE moduleIdAudit bigint;
> DECLARE moduleIdUG bigint;
> DECLARE moduleIdTagBasedPolicies bigint;
> DECLARE moduleIdKeyMana
> ERROR 8108 (HY000) at line 1757 in file: 'ranger_core_db_mysql.sql':
> Unsupported type *ast.CallStmt
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
