This is an automated email from the ASF dual-hosted git repository. wu-sheng pushed a commit to branch feature/banyandb-config-dump-extension in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 98eb3896ea7a99d1cfdc2cd7dbcbcc770bd9a875 Author: Wu Sheng <[email protected]> AuthorDate: Tue Jun 30 15:26:53 2026 +0800 Surface BanyanDB config in /debugging/config/dump via ConfigDumpExtension SPI The BanyanDB storage config moved to a separate bydb.yml / bydb-topn.yml in 10.2.0, so a BanyanDB deployment showed an empty storage.banyandb block in the /debugging/config/dump admin API while ES/JDBC showed theirs. Add a generic ConfigDumpExtension SPI on ServerStatusService that any module loading configuration from a secondary file can implement. BanyanDB registers an extension that flattens its already-loaded, environment-resolved config into storage.banyandb.* (TopN rules under storage.banyandb.topN.*), merged into the same dump and masked centrally by the existing secret-keyword list. The status module stays storage-agnostic (dependency arrow plugin -> core); the Horizon UI needs no change since it groups by the first dotted segment. Adds UTs for the core merge+mask and the flattener, plus a representative config-dump assertion on the banyandb storage e2e case. --- docs/en/changes/changes.md | 1 + docs/en/debugging/config_dump.md | 24 +++- .../server/core/status/ConfigDumpExtension.java | 43 +++++++ .../server/core/status/ServerStatusService.java | 18 +++ .../core/status/ServerStatusServiceTest.java | 83 +++++++++++++ .../banyandb/BanyanDBConfigDumpExtension.java | 137 +++++++++++++++++++++ .../plugin/banyandb/BanyanDBStorageProvider.java | 6 + .../banyandb/BanyanDBConfigDumpExtensionTest.java | 77 ++++++++++++ test/e2e-v2/cases/storage/banyandb/e2e.yaml | 19 +++ .../storage/expected/config-dump-banyandb.yml | 18 +++ 10 files changed, 425 insertions(+), 1 deletion(-) diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md index 28f0c62c14..86a0094398 100644 --- a/docs/en/changes/changes.md +++ b/docs/en/changes/changes.md @@ -319,6 +319,7 @@ * Fix: `layer-extensions.yml` is now excluded from the `skywalking-oap` jar and shipped to the distribution `config/` directory, so an operator-edited `config/layer-extensions.yml` is no longer shadowed by the empty template bundled in the jar. Because the OAP launch script puts `oap-libs/*.jar` ahead of `config/` on the classpath, `ResourceUtils.read("layer-extensions.yml")` previously always resolved the jar-bundled `layers: []` and silently ignored the operator's file — custom layers [...] * Fix: the v2 MAL compiler now resolves custom layers referenced as `Layer.NAME` in an expression. A custom layer declared through a `layerDefinitions:` block (or `layer-extensions.yml` / the `LayerExtension` SPI) has no generated `Layer.*` static field, so `service(['svc'], Layer.IOT_FLEET)` previously failed code generation because `Layer` has no `IOT_FLEET` field. The compiler now lowers every `Layer.NAME` static-field reference to a runtime `Layer.nameOf("NAME")` registry lookup, so [...] * Fix Envoy ALS rendering for the LAL live-debugger and the persisted log `content`: an Istio metadata-exchange peer in `common_properties.filter_state_objects` (legacy Wasm `wasm.*_peer` = `Any{BytesValue}` wrapping a FlatBuffer, or modern `*_peer` = `Any{Struct}`) is now decoded into the readable peer metadata (pod / namespace / labels) instead of an opaque `jsonformat-failed` envelope or base64. The serialization is hardened so a single un-printable field can no longer blank the whole [...] +* Surface the effective BanyanDB configuration (`bydb.yml` / `bydb-topn.yml`) in the `/debugging/config/dump` admin API. Because the BanyanDB config moved to a separate file in 10.2.0, a BanyanDB deployment previously showed an empty `storage.banyandb` block in the dump; its post-environment-resolution values are now merged into the same response under `storage.banyandb.*` (TopN rules under `storage.banyandb.topN.*`), masked by the same secret-keyword list, via a generic `ConfigDumpExten [...] #### UI * Add Airflow layer dashboards and menu i18n under Workflow Scheduler in Horizon UI (SWIP-7). diff --git a/docs/en/debugging/config_dump.md b/docs/en/debugging/config_dump.md index 753f8dd31d..063a8c96b2 100644 --- a/docs/en/debugging/config_dump.md +++ b/docs/en/debugging/config_dump.md @@ -89,4 +89,26 @@ By default, we mask the config keys through the following configurations. ```yaml # Include the list of keywords to filter configurations including secrets. Separate keywords by a comma. keywords4MaskingSecretsOfConfig: ${SW_DEBUGGING_QUERY_KEYWORDS_FOR_MASKING_SECRETS:user,password,trustStorePass,keyStorePass,token,accessKey,secretKey,authentication} -``` \ No newline at end of file +``` + +## BanyanDB Storage Configurations + +When BanyanDB is the active storage, the OAP loads its configuration from the dedicated +`bydb.yml` and `bydb-topn.yml` files (separated out from `application.yml` since 10.2.0). The +effective, environment-resolved values of these files are included in the same dump under the +`storage.banyandb.*` keys (TopN rules under `storage.banyandb.topN.*`), for example: + +```shell +> curl http://127.0.0.1:17128/debugging/config/dump +... +storage.banyandb.global.targets=127.0.0.1:17912 +storage.banyandb.global.user=****** +storage.banyandb.global.password=****** +storage.banyandb.metricsMinute.ttl=7 +storage.banyandb.topN.endpoint_cpm.endpoint_cpm-service.countersNumber=1000 +... +``` + +These rows reflect what the server actually loaded from `bydb.yml` / `bydb-topn.yml` (after +environment-variable overrides), so editing `storage.banyandb.*` under `application.yml` has no +effect. Secret values are masked using the same keyword list described above. \ No newline at end of file diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/status/ConfigDumpExtension.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/status/ConfigDumpExtension.java new file mode 100644 index 0000000000..ec85397c24 --- /dev/null +++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/status/ConfigDumpExtension.java @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.server.core.status; + +import java.util.Map; + +/** + * Contributes additional effective (post-environment-resolution) configurations to the + * {@code /debugging/config/dump} output. It exists for modules whose runtime configuration is + * loaded from a secondary file outside {@code application.yml} — for example the BanyanDB storage + * plugin loads {@code bydb.yml} / {@code bydb-topn.yml} into its own POJO, which the boot-time + * {@link ServerStatusService#dumpBootingConfigurations(String)} cannot otherwise see. + * + * <p>Implementations register themselves through + * {@link ServerStatusService#registerConfigDumpExtension(ConfigDumpExtension)}. + * + * @since 11.0.0 + */ +public interface ConfigDumpExtension { + /** + * @return the effective configurations as fully-qualified {@code module.provider.key} to value + * pairs, keyed exactly like the boot dump so they interleave with it. Values are returned raw; + * the dump masks secrets centrally by key, so each key should carry its field name as the last + * segment (e.g. {@code storage.banyandb.global.password}). + */ + Map<String, String> dumpConfigurations(); +} diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/status/ServerStatusService.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/status/ServerStatusService.java index db9141e8aa..c97984b263 100644 --- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/status/ServerStatusService.java +++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/status/ServerStatusService.java @@ -48,6 +48,8 @@ public class ServerStatusService implements Service { private List<ServerStatusWatcher> statusWatchers = new CopyOnWriteArrayList<>(); + private final List<ConfigDumpExtension> configDumpExtensions = new CopyOnWriteArrayList<>(); + private List<ApplicationConfiguration.ModuleConfiguration> configurations; public void bootedNow(List<ApplicationConfiguration.ModuleConfiguration> configurations, long uptime) { @@ -81,6 +83,18 @@ public class ServerStatusService implements Service { this.statusWatchers.add(watcher); } + /** + * Register a {@link ConfigDumpExtension} whose effective configurations are merged into the + * {@code /debugging/config/dump} output. Used by modules (e.g. the BanyanDB storage plugin) + * that load configuration from a secondary file outside {@code application.yml}. + * + * @param extension the extension contributing extra effective configurations to the dump + * @since 11.0.0 + */ + public void registerConfigDumpExtension(ConfigDumpExtension extension) { + this.configDumpExtensions.add(extension); + } + /** * @return a complete list of booting configurations with effected values. * @since 9.7.0 @@ -117,6 +131,10 @@ public class ServerStatusService implements Service { ) ); } + for (ConfigDumpExtension extension : configDumpExtensions) { + extension.dumpConfigurations().forEach( + (key, value) -> configList.put(key, maskConfigValue(key, value, keywords))); + } return configList; } diff --git a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/status/ServerStatusServiceTest.java b/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/status/ServerStatusServiceTest.java new file mode 100644 index 0000000000..b49c50c2d3 --- /dev/null +++ b/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/status/ServerStatusServiceTest.java @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.server.core.status; + +import java.lang.reflect.Field; +import java.util.List; +import java.util.Map; +import java.util.Properties; +import org.apache.skywalking.oap.server.core.status.ServerStatusService.ConfigList; +import org.apache.skywalking.oap.server.library.module.ApplicationConfiguration; +import org.apache.skywalking.oap.server.library.module.ModuleManager; +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.mockito.Mockito.mock; + +public class ServerStatusServiceTest { + private static final String KEYWORDS = "user,password"; + + @Test + public void shouldMergeAndMaskConfigDumpExtensions() throws Exception { + ServerStatusService service = new ServerStatusService(mock(ModuleManager.class)); + seedBootingConfigurations(service); + + service.registerConfigDumpExtension(() -> Map.of( + "storage.banyandb.global.targets", "127.0.0.1:17912", + "storage.banyandb.global.user", "admin", + "storage.banyandb.global.password", "s3cret", + "storage.banyandb.metricsMinute.ttl", "7" + )); + + ConfigList dump = service.dumpBootingConfigurations(KEYWORDS); + + // Secrets masked by their field-name keyword, applied centrally to extension rows too. + assertEquals("******", dump.get("storage.banyandb.global.user")); + assertEquals("******", dump.get("storage.banyandb.global.password")); + // Non-secret extension values pass through, merged into the same dump. + assertEquals("127.0.0.1:17912", dump.get("storage.banyandb.global.targets")); + assertEquals("7", dump.get("storage.banyandb.metricsMinute.ttl")); + // The application.yml-derived rows are still present. + assertEquals("12800", dump.get("core.default.restPort")); + } + + @Test + public void shouldNotLeakExtensionsBeforeBoot() { + ServerStatusService service = new ServerStatusService(mock(ModuleManager.class)); + service.registerConfigDumpExtension( + () -> Map.of("storage.banyandb.global.targets", "127.0.0.1:17912")); + + // configurations not set yet (pre-boot) -> dump is empty; extensions are not dumped early. + assertFalse(service.dumpBootingConfigurations(KEYWORDS) + .containsKey("storage.banyandb.global.targets")); + } + + private void seedBootingConfigurations(ServerStatusService service) throws Exception { + ApplicationConfiguration appConfig = new ApplicationConfiguration(); + Properties props = new Properties(); + props.put("restPort", "12800"); + ApplicationConfiguration.ModuleConfiguration core = appConfig.addModule("core"); + core.addProviderConfiguration("default", props); + + Field field = ServerStatusService.class.getDeclaredField("configurations"); + field.setAccessible(true); + field.set(service, List.of(core)); + } +} diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConfigDumpExtension.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConfigDumpExtension.java new file mode 100644 index 0000000000..bc047509b9 --- /dev/null +++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConfigDumpExtension.java @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.server.storage.plugin.banyandb; + +import com.google.common.base.Strings; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import org.apache.skywalking.oap.server.core.status.ConfigDumpExtension; +import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig.Global; +import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig.GroupResource; +import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig.Stage; +import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig.TopN; + +/** + * Flattens the loaded, environment-resolved {@link BanyanDBStorageConfig} (from {@code bydb.yml} + * and {@code bydb-topn.yml}) into {@code /debugging/config/dump} rows. Without this, a BanyanDB + * deployment shows an empty {@code storage.banyandb} block in the dump, because that configuration + * lives in a separate file the boot-time dump does not parse. + * + * <p>Keys read straight from the loaded POJO, never re-read from the file, so the values are the + * effective post-environment-override ones. Group segment names mirror the {@code groups:} keys in + * {@code bydb.yml} so the dump reads like the source file. Secret masking (e.g. {@code global.user} + * / {@code global.password}) is applied centrally by the dump, so this extension returns raw values. + * + * @since 11.0.0 + */ +public class BanyanDBConfigDumpExtension implements ConfigDumpExtension { + private final String prefix; + private final BanyanDBStorageConfig config; + + public BanyanDBConfigDumpExtension(final String prefix, final BanyanDBStorageConfig config) { + this.prefix = prefix; + this.config = config; + } + + @Override + public Map<String, String> dumpConfigurations() { + final Map<String, String> dump = new LinkedHashMap<>(); + flattenGlobal(prefix + ".global", config.getGlobal(), dump); + + // Only the groups populated from bydb.yml are emitted; segment names match its `groups:` keys. + final Map<String, GroupResource> groups = new LinkedHashMap<>(); + groups.put("records", config.getRecordsNormal()); + groups.put("recordsLog", config.getRecordsLog()); + groups.put("trace", config.getTrace()); + groups.put("zipkinTrace", config.getZipkinTrace()); + groups.put("recordsBrowserErrorLog", config.getRecordsBrowserErrorLog()); + groups.put("metricsMinute", config.getMetricsMin()); + groups.put("metricsHour", config.getMetricsHour()); + groups.put("metricsDay", config.getMetricsDay()); + groups.put("metadata", config.getMetadata()); + groups.put("property", config.getProperty()); + groups.forEach((name, group) -> flattenGroup(prefix + "." + name, group, dump)); + + config.getTopNConfigs().forEach((metric, rules) -> + rules.forEach((ruleName, topN) -> + flattenTopN(prefix + ".topN." + metric + "." + ruleName, topN, dump))); + return dump; + } + + private void flattenGlobal(final String p, final Global g, final Map<String, String> dump) { + // getTargets()/getCompatibleServerApiVersions() split the backing String into an array; + // join them back so the dump shows the raw comma-separated form the operator configured. + dump.put(p + ".targets", String.join(",", g.getTargets())); + dump.put(p + ".maxBulkSize", String.valueOf(g.getMaxBulkSize())); + dump.put(p + ".flushInterval", String.valueOf(g.getFlushInterval())); + dump.put(p + ".flushTimeout", String.valueOf(g.getFlushTimeout())); + dump.put(p + ".concurrentWriteThreads", String.valueOf(g.getConcurrentWriteThreads())); + dump.put(p + ".profileTaskQueryMaxSize", String.valueOf(g.getProfileTaskQueryMaxSize())); + dump.put(p + ".asyncProfilerTaskQueryMaxSize", String.valueOf(g.getAsyncProfilerTaskQueryMaxSize())); + dump.put(p + ".pprofTaskQueryMaxSize", String.valueOf(g.getPprofTaskQueryMaxSize())); + dump.put(p + ".resultWindowMaxSize", String.valueOf(g.getResultWindowMaxSize())); + dump.put(p + ".metadataQueryMaxSize", String.valueOf(g.getMetadataQueryMaxSize())); + dump.put(p + ".segmentQueryMaxSize", String.valueOf(g.getSegmentQueryMaxSize())); + dump.put(p + ".profileDataQueryBatchSize", String.valueOf(g.getProfileDataQueryBatchSize())); + dump.put(p + ".cleanupUnusedTopNRules", String.valueOf(g.isCleanupUnusedTopNRules())); + dump.put(p + ".namespace", Strings.nullToEmpty(g.getNamespace())); + dump.put(p + ".compatibleServerApiVersions", String.join(",", g.getCompatibleServerApiVersions())); + dump.put(p + ".user", Strings.nullToEmpty(g.getUser())); + dump.put(p + ".password", Strings.nullToEmpty(g.getPassword())); + dump.put(p + ".sslTrustCAPath", Strings.nullToEmpty(g.getSslTrustCAPath())); + } + + private void flattenGroup(final String p, final GroupResource group, final Map<String, String> dump) { + dump.put(p + ".shardNum", String.valueOf(group.getShardNum())); + dump.put(p + ".segmentInterval", String.valueOf(group.getSegmentInterval())); + dump.put(p + ".ttl", String.valueOf(group.getTtl())); + dump.put(p + ".replicas", String.valueOf(group.getReplicas())); + dump.put(p + ".enableWarmStage", String.valueOf(group.isEnableWarmStage())); + dump.put(p + ".enableColdStage", String.valueOf(group.isEnableColdStage())); + dump.put(p + ".defaultQueryStages", group.getDefaultQueryStages().toString()); + final List<Stage> stages = group.getAdditionalLifecycleStages(); + for (int i = 0; i < stages.size(); i++) { + flattenStage(p + ".additionalLifecycleStages." + i, stages.get(i), dump); + } + } + + private void flattenStage(final String p, final Stage stage, final Map<String, String> dump) { + dump.put(p + ".name", stage.getName() == null ? "" : stage.getName().name()); + dump.put(p + ".nodeSelector", Strings.nullToEmpty(stage.getNodeSelector())); + dump.put(p + ".shardNum", String.valueOf(stage.getShardNum())); + dump.put(p + ".segmentInterval", String.valueOf(stage.getSegmentInterval())); + dump.put(p + ".ttl", String.valueOf(stage.getTtl())); + dump.put(p + ".replicas", String.valueOf(stage.getReplicas())); + dump.put(p + ".close", String.valueOf(stage.isClose())); + } + + private void flattenTopN(final String p, final TopN topN, final Map<String, String> dump) { + dump.put(p + ".lruSizeMinute", String.valueOf(topN.getLruSizeMinute())); + dump.put(p + ".lruSizeHourDay", String.valueOf(topN.getLruSizeHourDay())); + dump.put(p + ".countersNumber", String.valueOf(topN.getCountersNumber())); + dump.put(p + ".sort", topN.getSort() == null ? "" : topN.getSort().name()); + dump.put(p + ".groupByTagNames", + topN.getGroupByTagNames() == null ? "[]" : topN.getGroupByTagNames().toString()); + dump.put(p + ".excludes", topN.getExcludes().stream() + .map(kv -> kv.getKey() + "=" + kv.getValue()) + .collect(Collectors.joining(",", "[", "]"))); + } +} diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java index fd858ab688..dbb2b7f3df 100644 --- a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java +++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBStorageProvider.java @@ -27,6 +27,7 @@ import org.apache.skywalking.banyandb.database.v1.BanyandbDatabase; import org.apache.skywalking.library.banyandb.v1.client.grpc.exception.BanyanDBException; import org.apache.skywalking.oap.server.core.CoreModule; import org.apache.skywalking.oap.server.core.RunningMode; +import org.apache.skywalking.oap.server.core.status.ServerStatusService; import org.apache.skywalking.oap.server.core.storage.IBatchDAO; import org.apache.skywalking.oap.server.core.storage.IHistoryDeleteDAO; import org.apache.skywalking.oap.server.core.storage.StorageBuilderFactory; @@ -244,6 +245,11 @@ public class BanyanDBStorageProvider extends ModuleProvider { this.modelInstaller.start(); getManager().find(CoreModule.NAME).provider().getService(ModelRegistry.class).addModelListener(modelInstaller); + + ServerStatusService serverStatusService = + getManager().find(CoreModule.NAME).provider().getService(ServerStatusService.class); + serverStatusService.registerConfigDumpExtension( + new BanyanDBConfigDumpExtension(StorageModule.NAME + "." + name(), this.config)); } catch (Exception e) { throw new ModuleStartException(e.getMessage(), e); } diff --git a/oap-server/server-storage-plugin/storage-banyandb-plugin/src/test/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConfigDumpExtensionTest.java b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/test/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConfigDumpExtensionTest.java new file mode 100644 index 0000000000..daeef8fac5 --- /dev/null +++ b/oap-server/server-storage-plugin/storage-banyandb-plugin/src/test/java/org/apache/skywalking/oap/server/storage/plugin/banyandb/BanyanDBConfigDumpExtensionTest.java @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.apache.skywalking.oap.server.storage.plugin.banyandb; + +import java.util.Map; +import org.apache.skywalking.oap.server.core.query.type.KeyValue; +import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig.MetricsMin; +import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig.Stage; +import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig.StageName; +import org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig.TopN; +import org.junit.jupiter.api.Test; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +public class BanyanDBConfigDumpExtensionTest { + @Test + public void shouldFlattenLoadedConfig() { + BanyanDBStorageConfig config = new BanyanDBStorageConfig(); + config.getGlobal().setTargets("a:1,b:2"); + config.getGlobal().setUser("admin"); + config.getGlobal().setPassword("admin"); + config.getGlobal().setCompatibleServerApiVersions("0.10"); + config.getGlobal().setNamespace("sw"); + + // A metrics-minute group carrying a warm lifecycle stage. + MetricsMin metricsMin = config.getMetricsMin(); + metricsMin.setShardNum(1); + metricsMin.setTtl(7); + metricsMin.setEnableWarmStage(true); + Stage warm = new Stage(); + warm.setName(StageName.warm); + warm.setTtl(30); + metricsMin.getAdditionalLifecycleStages().add(warm); + + // A TopN rule under bydb-topn.yml. + TopN topN = new TopN(); + topN.setCountersNumber(1000); + topN.setSort(TopN.Sort.des); + topN.getExcludes().add(new KeyValue("a", "b")); + config.getTopNConfigs().put("endpoint_cpm", Map.of("endpoint_cpm-service", topN)); + + Map<String, String> dump = + new BanyanDBConfigDumpExtension("storage.banyandb", config).dumpConfigurations(); + + // global: targets joined back to the raw comma form, not the split array. + assertEquals("a:1,b:2", dump.get("storage.banyandb.global.targets")); + // Raw values; masking is the dump's responsibility, not the extension's. + assertEquals("admin", dump.get("storage.banyandb.global.user")); + assertEquals("admin", dump.get("storage.banyandb.global.password")); + assertEquals("0.10", dump.get("storage.banyandb.global.compatibleServerApiVersions")); + // Group + indexed lifecycle stage (enum -> name()). + assertEquals("7", dump.get("storage.banyandb.metricsMinute.ttl")); + assertEquals("true", dump.get("storage.banyandb.metricsMinute.enableWarmStage")); + assertEquals("warm", dump.get("storage.banyandb.metricsMinute.additionalLifecycleStages.0.name")); + assertEquals("30", dump.get("storage.banyandb.metricsMinute.additionalLifecycleStages.0.ttl")); + // TopN nested under topN.<metric>.<rule>. + assertEquals("1000", dump.get("storage.banyandb.topN.endpoint_cpm.endpoint_cpm-service.countersNumber")); + assertEquals("des", dump.get("storage.banyandb.topN.endpoint_cpm.endpoint_cpm-service.sort")); + assertEquals("[a=b]", dump.get("storage.banyandb.topN.endpoint_cpm.endpoint_cpm-service.excludes")); + } +} diff --git a/test/e2e-v2/cases/storage/banyandb/e2e.yaml b/test/e2e-v2/cases/storage/banyandb/e2e.yaml index c59388a5d5..8c9d9a7ff9 100644 --- a/test/e2e-v2/cases/storage/banyandb/e2e.yaml +++ b/test/e2e-v2/cases/storage/banyandb/e2e.yaml @@ -66,6 +66,25 @@ verify: - query: | swctl --display json --admin-url=http://${oap_host}:${oap_17128} admin config ttl expected: ../expected/ttl-config-banyandb.yml + # The effective bydb.yml / bydb-topn.yml config is surfaced in /debugging/config/dump + # (one representative key per section) and the credentials are masked. Assertions live + # in the query; it prints `config-dump-ok` only when all of them pass. + - query: | + dump=$(curl -s http://${oap_host}:${oap_17128}/debugging/config/dump) + exact() { echo "$dump" | grep -Fqx "$1" || { echo "missing or unexpected: $1"; exit 1; }; } + exists() { echo "$dump" | grep -Fq "$1" || { echo "missing: $1"; exit 1; }; } + exact 'storage.provider=banyandb' + exact 'storage.banyandb.global.user=******' + exact 'storage.banyandb.global.password=******' + exact 'storage.banyandb.global.namespace=sw' + exists 'storage.banyandb.global.targets=' + exact 'storage.banyandb.records.ttl=3' + exact 'storage.banyandb.metricsMinute.ttl=7' + exact 'storage.banyandb.metricsMinute.enableWarmStage=true' + exact 'storage.banyandb.metricsMinute.additionalLifecycleStages.0.name=warm' + exists 'storage.banyandb.topN.endpoint_cpm.endpoint_cpm-service.countersNumber=' + echo config-dump-ok + expected: ../expected/config-dump-banyandb.yml cleanup: on: always collect: diff --git a/test/e2e-v2/cases/storage/expected/config-dump-banyandb.yml b/test/e2e-v2/cases/storage/expected/config-dump-banyandb.yml new file mode 100644 index 0000000000..070a4d3de8 --- /dev/null +++ b/test/e2e-v2/cases/storage/expected/config-dump-banyandb.yml @@ -0,0 +1,18 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# The representative bydb config-dump assertions run inside the query (it prints this +# sentinel only when every picked key is present and the secrets are masked). +config-dump-ok
