This is an automated email from the ASF dual-hosted git repository.
wu-sheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/skywalking.git
The following commit(s) were added to refs/heads/master by this push:
new dcfb07014e Fix custom layers: MAL v2 Layer.NAME compilation +
layer-extensions.yml jar shadowing (#13918)
dcfb07014e is described below
commit dcfb07014ea5b6813aed214ba34e6230678d37a3
Author: Marc Navarro <[email protected]>
AuthorDate: Fri Jun 19 08:36:22 2026 +0200
Fix custom layers: MAL v2 Layer.NAME compilation + layer-extensions.yml jar
shadowing (#13918)
**Bug 1 — MAL v2 compiler cannot compile a custom-layer reference**
A custom layer has no generated `Layer.*` static field — it lives only in
the name registry — so referencing it in a MAL expression failed v2 code
generation. The natural form `service(['svc'], Layer.IOT_FLEET)` parses as the
static-field `enumRef` (`IDENTIFIER DOT IDENTIFIER`, exactly like
`Layer.GENERAL`), and codegen emitted it verbatim as a `Layer.IOT_FLEET` field
access — a field that does not exist on `Layer` — so Javassist failed to
compile the generated source.
Fix: `MALMethodChainCodegen` now lowers every `Layer.NAME` static-field
reference to a runtime `Layer.nameOf("NAME")` registry lookup, scoped to
`Layer` only (via `MALCodegenHelper.LAYER_ENUM_TYPE`). `Layer` is a
registry-backed type, not a Java enum, and `Layer.nameOf` resolves both
built-in and custom layers by name; for a built-in this is behavior-preserving,
because `Layer.nameOf("GENERAL")` returns the same instance as the
`Layer.GENERAL` field. The other `ENUM_FQCN` types (`Dete [...]
Tests (`MALClassGeneratorTest`): a custom-layer field form lowers to
`Layer.nameOf("...")` and compiles; a built-in field form lowers identically; a
non-`Layer` enum (`DetectPoint`) keeps its direct field access. All 1350
bundled-rule execution tests pass — every built-in `Layer.*` reference now
routes through `nameOf` and resolves to the same layer (no rule used a custom
layer before, which is why CI never caught it).
> _Note: an earlier revision of this PR fixed only the explicit
`Layer.nameOf('NAME')` method form via a parser reinterpretation
(`EnumStaticCallArgument`). That was superseded by the field-form lowering
above — the field form is the natural, documented-as-built-in syntax and needs
no grammar/AST special-casing. Neither form had shipped, so there is no
compatibility concern._
**Bug 2 — `layer-extensions.yml` bundled in the jar shadows the operator's
`config/` copy**
`layer-extensions.yml` (operator-managed, documented as living in
`config/`) was packaged inside `skywalking-oap.jar` and not shipped to the
distribution `config/`. The launch script
(`dist-material/bin/oapService.sh:38-41`) prepends every `oap-libs/*.jar` to
`CLASSPATH`, leaving `config/` last, so
`ResourceUtils.read("layer-extensions.yml")` →
`getClassLoader().getResource(...)` deterministically resolved the empty
jar-bundled template (`layers: []`) and silently ignored the operator [...]
Fix: add `layer-extensions.yml` to the `maven-jar-plugin` `<excludes>`
(`oap-server/server-starter/pom.xml`) and to the assembly `<includes>`
(`apm-dist/src/main/assembly/binary.xml`), so it follows the same
exclude-from-jar + copy-to-`config/` packaging as every other operator-editable
config. Verified by rebuilding `skywalking-oap.jar` and confirming the file is
no longer present.
---
apm-dist/src/main/assembly/binary.xml | 1 +
docs/en/changes/changes.md | 2 +
docs/en/concepts-and-designs/mal.md | 7 ++-
.../analyzer/v2/compiler/MALCodegenHelper.java | 9 ++++
.../v2/compiler/MALMethodChainCodegen.java | 16 ++++--
.../v2/compiler/MALClassGeneratorTest.java | 59 ++++++++++++++++++++++
oap-server/server-starter/pom.xml | 1 +
7 files changed, 90 insertions(+), 5 deletions(-)
diff --git a/apm-dist/src/main/assembly/binary.xml
b/apm-dist/src/main/assembly/binary.xml
index b392cb39ef..47451e9559 100644
--- a/apm-dist/src/main/assembly/binary.xml
+++ b/apm-dist/src/main/assembly/binary.xml
@@ -74,6 +74,7 @@
<include>telegraf-rules/*</include>
<include>cilium-rules/*</include>
<include>gen-ai-config.yml</include>
+ <include>layer-extensions.yml</include>
</includes>
<outputDirectory>config</outputDirectory>
</fileSet>
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index 7658d17f1d..19944574a6 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -307,6 +307,8 @@
* Fix: continuous profiling policy validation now rejects a threshold / count
of `0` to match the error messages and rover's `value >= threshold` trigger
semantics (a `0` threshold would always trigger). CPU percent and HTTP error
rate are tightened from `[0-100]` to `(0-100]`.
* Fix wrong BanyanDB resource options in record data.
* Align the default BanyanDB stage `segmentInterval` values so each coarser
stage is an integer multiple of the finer one (`records` cold `3` → `4`,
`metricsMinute` cold `5` → `6`, `metricsHour` warm `7` → `10` and cold `15` →
`20`), keeping hot → warm → cold lifecycle migration on the cheap whole-segment
fast path.
+* Fix: `layer-extensions.yml` is now excluded from the `skywalking-oap` jar
and shipped to the distribution `config/` directory, so an operator-edited
`config/layer-extensions.yml` is no longer shadowed by the empty template
bundled in the jar. Because the OAP launch script puts `oap-libs/*.jar` ahead
of `config/` on the classpath, `ResourceUtils.read("layer-extensions.yml")`
previously always resolved the jar-bundled `layers: []` and silently ignored
the operator's file — custom layers [...]
+* Fix: the v2 MAL compiler now resolves custom layers referenced as
`Layer.NAME` in an expression. A custom layer declared through a
`layerDefinitions:` block (or `layer-extensions.yml` / the `LayerExtension`
SPI) has no generated `Layer.*` static field, so `service(['svc'],
Layer.IOT_FLEET)` previously failed code generation because `Layer` has no
`IOT_FLEET` field. The compiler now lowers every `Layer.NAME` static-field
reference to a runtime `Layer.nameOf("NAME")` registry lookup, so [...]
#### UI
* Add Airflow layer dashboards and menu i18n under Workflow Scheduler in
Horizon UI (SWIP-7).
diff --git a/docs/en/concepts-and-designs/mal.md
b/docs/en/concepts-and-designs/mal.md
index 5d8c92be2e..2387cf6ffa 100644
--- a/docs/en/concepts-and-designs/mal.md
+++ b/docs/en/concepts-and-designs/mal.md
@@ -393,10 +393,15 @@ layerDefinitions:
metricsRules:
- name: device_battery_percentage
exp: iot_device_battery_level.tagAverage(['service'], ['host'])
-expSuffix: instance(['host'], ['service'], Layer.nameOf('IOT_FLEET'))
+expSuffix: instance(['host'], ['service'], Layer.IOT_FLEET)
```
Notes:
+- **Reference a custom layer like a built-in one.** Write `Layer.IOT_FLEET`
exactly as you would a
+ built-in such as `Layer.GENERAL` — the compiler lowers every `Layer.<NAME>`
reference to a runtime
+ registry lookup, so a custom layer needs no generated static field. (Because
the lookup is by name,
+ a misspelled name resolves to `Layer.UNDEFINED` at runtime rather than
failing to compile, so keep
+ the name in `layerDefinitions:` and the expression in sync.)
- **Storage encoding is the ordinal int**, persisted in BanyanDB /
Elasticsearch / JDBC. Every
OAP node that reads or writes a given layer must agree on its `(name,
ordinal)` mapping —
deploy a MAL file with `layerDefinitions:` identically across all nodes.
diff --git
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALCodegenHelper.java
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALCodegenHelper.java
index 948f6a949b..428b230a76 100644
---
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALCodegenHelper.java
+++
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALCodegenHelper.java
@@ -39,6 +39,15 @@ final class MALCodegenHelper {
// ---- Well-known enum types used in MAL expressions ----
+ /**
+ * Short name of the {@code Layer} type as written in MAL expressions.
Unlike the other
+ * {@link #ENUM_FQCN} entries (real Java enums), {@code Layer} is a
registry-backed type
+ * whose custom members have no static field, so a {@code Layer.NAME}
reference is lowered
+ * to a runtime {@code Layer.nameOf("NAME")} lookup in code generation
rather than a
+ * static-field access — see {@code
MALMethodChainCodegen#generateArgument}.
+ */
+ static final String LAYER_ENUM_TYPE = "Layer";
+
static final Map<String, String> ENUM_FQCN;
// ---- Well-known helper classes used inside MAL closures ----
diff --git
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALMethodChainCodegen.java
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALMethodChainCodegen.java
index d707c0a49e..7a2b4bc87c 100644
---
a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALMethodChainCodegen.java
+++
b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALMethodChainCodegen.java
@@ -321,12 +321,20 @@ final class MALMethodChainCodegen {
(MALExpressionModel.EnumRefArgument) arg;
final String fqcn =
MALCodegenHelper.ENUM_FQCN.get(enumRef.getEnumType());
- if (fqcn != null) {
- sb.append(fqcn);
+ final String typeRef = fqcn != null ? fqcn : enumRef.getEnumType();
+ if
(MALCodegenHelper.LAYER_ENUM_TYPE.equals(enumRef.getEnumType())) {
+ // Layer is a registry-backed type, not a Java enum. A custom
layer
+ // (layerDefinitions: / layer-extensions.yml / SPI / runtime
hot-update) has
+ // no Layer.* static field, so every Layer.NAME reference is
lowered to a
+ // runtime Layer.nameOf("NAME") lookup. For a built-in layer
this is
+ // equivalent — Layer.nameOf("GENERAL") returns the same
instance as the
+ // Layer.GENERAL field. Other ENUM_FQCN types are real enums
with no
+ // nameOf(String), so they keep the direct static-field
reference below.
+ sb.append(typeRef).append(".nameOf(\"")
+ .append(enumRef.getEnumValue()).append("\")");
} else {
- sb.append(enumRef.getEnumType());
+ sb.append(typeRef).append('.').append(enumRef.getEnumValue());
}
- sb.append('.').append(enumRef.getEnumValue());
} else if (arg instanceof MALExpressionModel.ExprArgument) {
final MALExpressionModel.Expr innerExpr =
((MALExpressionModel.ExprArgument) arg).getExpr();
diff --git
a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClassGeneratorTest.java
b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClassGeneratorTest.java
index f6594c66e8..785adc3038 100644
---
a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClassGeneratorTest.java
+++
b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClassGeneratorTest.java
@@ -27,6 +27,7 @@ import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
@@ -134,6 +135,64 @@ class MALClassGeneratorTest {
assertTrue(source.contains("getOrDefault"));
}
+ @Test
+ void compileCustomLayerViaStaticFieldForm() throws Exception {
+ // A custom layer may also be referenced with the ordinary
static-field syntax
+ // (Layer.IOT_FLEET), exactly like a built-in. Because the layer has
no generated
+ // static field, codegen must lower it to a runtime
Layer.nameOf("IOT_FLEET") lookup
+ // rather than emitting a Layer.IOT_FLEET field access that cannot
compile.
+ final String source = generator.generateSource(
+ "instance_jvm_cpu.sum(['service']).service(['service'],
Layer.IOT_FLEET)");
+ assertNotNull(source);
+ assertTrue(
+ source.contains(
+
"org.apache.skywalking.oap.server.core.analysis.Layer.nameOf(\"IOT_FLEET\")"),
+ "Custom-layer static-field form should be lowered to
Layer.nameOf(...): " + source);
+ assertNotNull(generator.compile(
+ "test_custom_layer_field_form",
+ "instance_jvm_cpu.sum(['service']).service(['service'],
Layer.IOT_FLEET)"));
+ }
+
+ @Test
+ void builtInLayerStaticFieldFormLoweredToNameOf() throws Exception {
+ // Built-in layers are lowered the same way as custom ones:
Layer.GENERAL becomes
+ // Layer.nameOf("GENERAL"), which returns the identical instance as
the Layer.GENERAL
+ // static field, so the rewrite is behavior-preserving and needs no
built-in/custom
+ // branch in codegen.
+ final String source = generator.generateSource(
+ "instance_jvm_cpu.sum(['service']).service(['service'],
Layer.GENERAL)");
+ assertNotNull(source);
+ assertTrue(
+ source.contains(
+
"org.apache.skywalking.oap.server.core.analysis.Layer.nameOf(\"GENERAL\")"),
+ "Built-in layer static-field form should be lowered to
Layer.nameOf(...): " + source);
+ assertNotNull(generator.compile(
+ "test_builtin_layer_field_form",
+ "instance_jvm_cpu.sum(['service']).service(['service'],
Layer.GENERAL)"));
+ }
+
+ @Test
+ void nonLayerEnumStaticFieldFormKeepsDirectReference() throws Exception {
+ // The Layer.nameOf(...) lowering is scoped to Layer only. Real Java
enums such as
+ // DetectPoint have no nameOf(String), so their static-field
references must stay
+ // direct field accesses or the generated source would not compile.
This expression
+ // also carries a Layer ref, confirming the two are lowered
differently side by side.
+ final String source = generator.generateSource(
+ "service_relation_req.sum(['client', 'server']).serviceRelation("
+ + "DetectPoint.SERVER, ['client'], ['server'], '|',
Layer.GENERAL, 'component')");
+ assertNotNull(source);
+ assertTrue(
+ source.contains("DetectPoint.SERVER"),
+ "Non-Layer enum should keep the direct static-field reference: " +
source);
+ assertFalse(
+ source.contains("DetectPoint.nameOf"),
+ "Non-Layer enum must NOT be lowered to a nameOf(...) lookup: " +
source);
+ assertTrue(
+ source.contains(
+
"org.apache.skywalking.oap.server.core.analysis.Layer.nameOf(\"GENERAL\")"),
+ "Layer ref in the same expression should still be lowered to
nameOf(...): " + source);
+ }
+
@Test
void filterSafeNavCompiles() throws Exception {
final String source = generator.generateFilterSource(
diff --git a/oap-server/server-starter/pom.xml
b/oap-server/server-starter/pom.xml
index 8fef6d229f..70eaf6aec0 100644
--- a/oap-server/server-starter/pom.xml
+++ b/oap-server/server-starter/pom.xml
@@ -377,6 +377,7 @@
<exclude>telegraf-rules/</exclude>
<exclude>cilium-rules/</exclude>
<exclude>gen-ai-config.yml</exclude>
+ <exclude>layer-extensions.yml</exclude>
</excludes>
</configuration>
</plugin>
