[
https://issues.apache.org/jira/browse/KNOX-2990?focusedWorklogId=902821&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-902821
]
ASF GitHub Bot logged work on KNOX-2990:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Jan/24 14:35
Start Date: 31/Jan/24 14:35
Worklog Time Spent: 10m
Work Description: moresandeep commented on code in PR #826:
URL: https://github.com/apache/knox/pull/826#discussion_r1472744505
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/factory/TokenStateServiceFactory.java:
##########
@@ -61,17 +62,32 @@ protected Service createService(GatewayServices
gatewayServices, ServiceType ser
service.init(gatewayConfig, options);
} catch (ServiceLifecycleException e) {
LOG.errorInitializingService(implementation, e.getMessage(), e);
- service = new AliasBasedTokenStateService();
- ((AliasBasedTokenStateService)
service).setAliasService(getAliasService(gatewayServices));
+ service = useDerbyDatabaseTokenStateService(gatewayServices,
gatewayConfig, options);
}
+ } else if (matchesImplementation(implementation,
DerbyDBTokenStateService.class, true)) {
+ service = useDerbyDatabaseTokenStateService(gatewayServices,
gatewayConfig, options);
}
- logServiceUsage(isEmptyDefaultImplementation(implementation) ?
AliasBasedTokenStateService.class.getName() : implementation, serviceType);
+ logServiceUsage(service.getClass().getName(), serviceType);
}
return service;
}
+ private Service useDerbyDatabaseTokenStateService(GatewayServices
gatewayServices, GatewayConfig gatewayConfig, Map<String, String> options) {
+ Service service;
+ try {
+ service = new DerbyDBTokenStateService();
+ ((DerbyDBTokenStateService)
service).setAliasService(getAliasService(gatewayServices));
+ ((DerbyDBTokenStateService)
service).setMasterService(getMasterService(gatewayServices));
+ service.init(gatewayConfig, options);
+ } catch (ServiceLifecycleException e) {
+ LOG.errorInitializingService(DerbyDBTokenStateService.class.getName(),
e.getMessage(), e);
+ service = new DefaultTokenStateService();
Review Comment:
If we fail to initialize `DerbyDBTokenStateService` should we just fail so
that the user knows there was an issue? looks like we fall back on
`DefaultTokenStateService` . Basically, should we fail fast? i'd say so.
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/factory/TokenStateServiceFactory.java:
##########
@@ -61,17 +62,32 @@ protected Service createService(GatewayServices
gatewayServices, ServiceType ser
service.init(gatewayConfig, options);
} catch (ServiceLifecycleException e) {
LOG.errorInitializingService(implementation, e.getMessage(), e);
- service = new AliasBasedTokenStateService();
- ((AliasBasedTokenStateService)
service).setAliasService(getAliasService(gatewayServices));
+ service = useDerbyDatabaseTokenStateService(gatewayServices,
gatewayConfig, options);
}
+ } else if (matchesImplementation(implementation,
DerbyDBTokenStateService.class, true)) {
Review Comment:
I am thinking if this is a default impl then in almost all the cases this
will be used. Do you think it will speed up knox startup if we move this to the
top, it might be a sub milli-sec improvement :) so feel free to ignore this
comment.
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/token/impl/DerbyDBTokenStateService.java:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+package org.apache.knox.gateway.services.token.impl;
+
+import static
org.apache.knox.gateway.config.impl.GatewayConfigImpl.GATEWAY_DATABASE_NAME;
+import static
org.apache.knox.gateway.config.impl.GatewayConfigImpl.GATEWAY_DATABASE_TYPE;
+import static
org.apache.knox.gateway.services.security.AliasService.NO_CLUSTER_NAME;
+import static
org.apache.knox.gateway.util.JDBCUtils.DATABASE_PASSWORD_ALIAS_NAME;
+import static org.apache.knox.gateway.util.JDBCUtils.DATABASE_USER_ALIAS_NAME;
+import static org.apache.knox.gateway.util.JDBCUtils.DERBY_DB_TYPE;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.knox.gateway.config.GatewayConfig;
+import org.apache.knox.gateway.services.ServiceLifecycleException;
+import org.apache.knox.gateway.services.security.MasterService;
+import org.apache.knox.gateway.shell.jdbc.derby.DerbyDatabase;
+import org.apache.knox.gateway.util.FileUtils;
+
+public class DerbyDBTokenStateService extends JDBCTokenStateService {
+
+ public static final String DEFAULT_TOKEN_DB_USER_NAME = "knox";
+ public static final String DB_NAME = "tokens";
+
+ private DerbyDatabase derbyDatabase;
+ private Path derbyDatabaseFolder;
+ private MasterService masterService;
+
+ public void setMasterService(MasterService masterService) {
+ this.masterService = masterService;
+ }
+
+ @Override
+ public void init(GatewayConfig config, Map<String, String> options) throws
ServiceLifecycleException {
+ try {
+ derbyDatabaseFolder = Paths.get(config.getGatewaySecurityDir(), DB_NAME);
+ startDerby();
+ ((Configuration) config).set(GATEWAY_DATABASE_TYPE, DERBY_DB_TYPE);
+ ((Configuration) config).set(GATEWAY_DATABASE_NAME,
derbyDatabaseFolder.toString());
+ getAliasService().addAliasForCluster(NO_CLUSTER_NAME,
DATABASE_USER_ALIAS_NAME, getDatabaseUserName());
+ getAliasService().addAliasForCluster(NO_CLUSTER_NAME,
DATABASE_PASSWORD_ALIAS_NAME, getDatabasePassword());
+ super.init(config, options);
+
+ // we need the "x" permission too to be able to browse that folder (600
is not enough)
+ if (Files.exists(derbyDatabaseFolder)) {
+ FileUtils.chmod("700", derbyDatabaseFolder.toFile());
+ }
+ } catch (Exception e) {
+ e.printStackTrace();
+ throw new ServiceLifecycleException("Error while initiating
DerbyDBTokenStateService: " + e, e);
+ }
+ }
+
+ private void startDerby() throws Exception {
Review Comment:
Can we start Derby on a separate async thread? DB init can be done in
parallel.
##########
gateway-server/src/main/java/org/apache/knox/gateway/config/impl/GatewayConfigImpl.java:
##########
@@ -282,6 +282,11 @@ public class GatewayConfigImpl extends Configuration
implements GatewayConfig {
private static final String TOKEN_STATE_SERVER_MANAGED =
GATEWAY_CONFIG_FILE_PREFIX + ".knox.token.exp.server-managed";
private static final String USERS_CAN_SEE_ALL_TOKENS =
GATEWAY_CONFIG_FILE_PREFIX + ".knox.token.management.users.can.see.all.tokens";
+ private static final String SKIP_TOKEN_MIGRATION= GATEWAY_CONFIG_FILE_PREFIX
+ ".knox.token.migration.skip";
Review Comment:
Low Priority: If you are making other changes, it might be better to create
a variable for `knox.token` for readability.
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/token/impl/DerbyDBTokenStateService.java:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+package org.apache.knox.gateway.services.token.impl;
+
+import static
org.apache.knox.gateway.config.impl.GatewayConfigImpl.GATEWAY_DATABASE_NAME;
+import static
org.apache.knox.gateway.config.impl.GatewayConfigImpl.GATEWAY_DATABASE_TYPE;
+import static
org.apache.knox.gateway.services.security.AliasService.NO_CLUSTER_NAME;
+import static
org.apache.knox.gateway.util.JDBCUtils.DATABASE_PASSWORD_ALIAS_NAME;
+import static org.apache.knox.gateway.util.JDBCUtils.DATABASE_USER_ALIAS_NAME;
+import static org.apache.knox.gateway.util.JDBCUtils.DERBY_DB_TYPE;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.knox.gateway.config.GatewayConfig;
+import org.apache.knox.gateway.services.ServiceLifecycleException;
+import org.apache.knox.gateway.services.security.MasterService;
+import org.apache.knox.gateway.shell.jdbc.derby.DerbyDatabase;
+import org.apache.knox.gateway.util.FileUtils;
+
+public class DerbyDBTokenStateService extends JDBCTokenStateService {
+
+ public static final String DEFAULT_TOKEN_DB_USER_NAME = "knox";
+ public static final String DB_NAME = "tokens";
+
+ private DerbyDatabase derbyDatabase;
+ private Path derbyDatabaseFolder;
+ private MasterService masterService;
+
+ public void setMasterService(MasterService masterService) {
+ this.masterService = masterService;
+ }
+
+ @Override
+ public void init(GatewayConfig config, Map<String, String> options) throws
ServiceLifecycleException {
+ try {
+ derbyDatabaseFolder = Paths.get(config.getGatewaySecurityDir(), DB_NAME);
+ startDerby();
+ ((Configuration) config).set(GATEWAY_DATABASE_TYPE, DERBY_DB_TYPE);
+ ((Configuration) config).set(GATEWAY_DATABASE_NAME,
derbyDatabaseFolder.toString());
+ getAliasService().addAliasForCluster(NO_CLUSTER_NAME,
DATABASE_USER_ALIAS_NAME, getDatabaseUserName());
+ getAliasService().addAliasForCluster(NO_CLUSTER_NAME,
DATABASE_PASSWORD_ALIAS_NAME, getDatabasePassword());
+ super.init(config, options);
+
+ // we need the "x" permission too to be able to browse that folder (600
is not enough)
+ if (Files.exists(derbyDatabaseFolder)) {
+ FileUtils.chmod("700", derbyDatabaseFolder.toFile());
+ }
+ } catch (Exception e) {
+ e.printStackTrace();
Review Comment:
need to remove print stacktrace
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/token/impl/AliasBasedTokenStateService.java:
##########
@@ -52,6 +52,8 @@
/**
* A TokenStateService implementation based on the AliasService.
+ *
+ * @deprecated Since 2.1.0
*/
public class AliasBasedTokenStateService extends
AbstractPersistentTokenStateService implements
TokenStatePeristerMonitorListener {
Review Comment:
Low Priority: Do you think we should also put a log entry (INFO) stating
this impl is deprecated?
##########
gateway-server/src/main/java/org/apache/knox/gateway/services/token/impl/JournalBasedTokenStateService.java:
##########
@@ -32,6 +32,9 @@
import java.util.Map;
import java.util.Set;
+/**
+ * @deprecated Since 2.1.0
+ */
public class JournalBasedTokenStateService extends
AbstractPersistentTokenStateService {
Review Comment:
Perhaps put a log entry too about deprecation.
Issue Time Tracking
-------------------
Worklog Id: (was: 902821)
Time Spent: 40m (was: 0.5h)
> TokenStateService implementation cleanup
> ----------------------------------------
>
> Key: KNOX-2990
> URL: https://issues.apache.org/jira/browse/KNOX-2990
> Project: Apache Knox
> Issue Type: Task
> Components: Server
> Affects Versions: 2.0.0, 1.6.0, 1.6.1
> Reporter: Sandor Molnar
> Assignee: Sandor Molnar
> Priority: Critical
> Fix For: 2.1.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> This issue is driven by a [DISCUSS] thread initiated on Knox's DEV mailing
> list [here|https://lists.apache.org/thread/fs9nkl6l45o330ttvgvqxj3jnxt63bcs].
> As a result of that discussion, the following needs to be implemented:
> * deprecate the following TSS implementations:
> ** AliasBasedTokenStateService
> ** ZookeeperTokenStateService
> ** JournalBasedTokenStateService
> * document the deprecation of these TSS implementations in v2.1.0 and
> highlight that they will be removed in the upcoming release (v2.2.0?).
> * implement a DerbyDB storage that will store tokens in
> {{$DATA_DIR/security/tokens}} (encrypted or not, it'll be decided later)
> * make sure appropriate file permissions are set on that folder
> * have the {{homepage}} topology configured with JDBC TSS pointing to this
> DerbyDB storage
> * implement a new KnoxCLI command that migrates existing tokens from
> credential stores to the DerbyDB storage
> * automate this new KnoxCLI command in a way such that it runs when Knox
> Gateway is started, token management is enabled, and DerbyDB storage is
> configured
> * ensure that the previous automated step can be controlled (E.g. in case of
> unforeseen errors it can be turned off)
> * document possible data replication scenarios when, in the case of HA
> deployments, existing tokens from one Knox node should be made available in
> other Knox node(s) and there is no other centralized RDBMS in use
> (PostgreSQL, MySQL for instance)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
