This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 45a18f9f9824 CAMEL-25206: camel-yaml-dsl - round-trip test of the YAML
DSL writer and reader over the XML route corpus (#27157)
45a18f9f9824 is described below
commit 45a18f9f9824a24ef06db4523f3c8aa1678acd12
Author: Claus Ibsen <[email protected]>
AuthorDate: Thu Oct 1 07:06:08 2026 +0200
CAMEL-25206: camel-yaml-dsl - round-trip test of the YAML DSL writer and
reader over the XML route corpus (#27157)
The XML and Java DSL writers have round-trip tests over route corpora; the
YAML DSL had none (`XmlToYamlTest` in camel-yaml-io only checks the YAML is not
empty).
`YamlRoundTripTest` (in camel-yaml-dsl-validator, which has the YAML reader
and Camel's `YamlValidator`) reads the XML routes of camel-xml-io and
camel-core's model tests, and for each one:
1. dumps it as YAML (`LwModelToYAMLDumper`);
2. validates that YAML with `YamlValidator`, the checks of `camel validate`;
3. loads it back with the YAML DSL;
4. compares both models dumped as XML. The XML side is prepared the way the
routes loader prepares the route it reads back (a flat `<transacted/>` or
`<saga/>` takes the rest of the route; an `interceptFrom` that does not match
the route is removed).
camel-yaml-io and camel-xml-io are added as test dependencies of
camel-yaml-dsl-validator.
### Fixed
- **Route properties**: the YAML dumper writes `routeProperty`, but the
YAML DSL rejected it (`unsupported field: routeProperty`). The route
deserializer now reads it; the YAML schemas and the catalog copy are
regenerated.
- **XML namespace order**: namespaces read from a YAML `namespace` list
lost their order because `NamespaceAwareExpression.getNamespaceAsMap()`
collected them in a `HashMap`. It is now a `LinkedHashMap`, as the copy
constructor already uses.
- **Kamelet EIP doc**: the XML example put the steps after `<kamelet>`,
where the Java and YAML examples nest them (they run at the kamelet's sink).
The XML example now nests them too.
### Left out
- `kamelet.xml`: a flat `<kamelet/>` runs the steps after it once the
kamelet returns; in YAML they are the kamelet's own (flow mode, as in the Java
DSL).
- Route scoped `onException`, `onCompletion`, `intercept`, `interceptFrom`
and `interceptSendToEndpoint` (5 routes): the YAML DSL has them at the top
level or in a route configuration, not inside a route, so the YAML written for
them does not validate. Adding a route scope to YAML was judged overkill
([CAMEL-25207](https://issues.apache.org/jira/browse/CAMEL-25207), Won't Do).
128 of 133 routes read back.
Tests: camel-yaml-dsl, camel-yaml-dsl-validator (168), camel-core-model,
camel-xml-io, camel-yaml-io, camel-java-io all pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Signed-off-by: Claus Ibsen <[email protected]>
---
.../org/apache/camel/catalog/docs/kamelet-eip.adoc | 5 +-
.../camel/catalog/schemas/camelYamlDsl-model.json | 5 +
.../main/docs/modules/eips/pages/kamelet-eip.adoc | 5 +-
.../model/language/NamespaceAwareExpression.java | 3 +-
.../deserializers/RouteDefinitionDeserializer.java | 6 +
.../camel-yaml-dsl-validator/pom.xml | 10 ++
.../dsl/yaml/validator/YamlRoundTripTest.java | 139 +++++++++++++++++++++
.../resources/schema/camelYamlDsl-canonical.json | 7 ++
.../resources/schema/camelYamlDsl-model.json | 5 +
.../generated/resources/schema/camelYamlDsl.json | 7 ++
10 files changed, 186 insertions(+), 6 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kamelet-eip.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kamelet-eip.adoc
index a241edeb52b6..16c97466223d 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kamelet-eip.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kamelet-eip.adoc
@@ -169,8 +169,9 @@ XML::
----
<route>
<from uri="direct:start"/>
- <kamelet name="foo"/>
- <to uri="mock:result"/>
+ <kamelet name="foo">
+ <to uri="mock:result"/>
+ </kamelet>
</route>
----
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/schemas/camelYamlDsl-model.json
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/schemas/camelYamlDsl-model.json
index 5962bfc01ece..5f661bb07177 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/schemas/camelYamlDsl-model.json
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/schemas/camelYamlDsl-model.json
@@ -7054,6 +7054,11 @@
"type" : "object",
"ref" : "from",
"required" : true
+ }, {
+ "name" : "routeProperty",
+ "type" : "array",
+ "ref" : "property",
+ "description" : "To set metadata as properties on the route."
} ]
},
"routeTemplate" : {
diff --git
a/core/camel-core-engine/src/main/docs/modules/eips/pages/kamelet-eip.adoc
b/core/camel-core-engine/src/main/docs/modules/eips/pages/kamelet-eip.adoc
index a241edeb52b6..16c97466223d 100644
--- a/core/camel-core-engine/src/main/docs/modules/eips/pages/kamelet-eip.adoc
+++ b/core/camel-core-engine/src/main/docs/modules/eips/pages/kamelet-eip.adoc
@@ -169,8 +169,9 @@ XML::
----
<route>
<from uri="direct:start"/>
- <kamelet name="foo"/>
- <to uri="mock:result"/>
+ <kamelet name="foo">
+ <to uri="mock:result"/>
+ </kamelet>
</route>
----
diff --git
a/core/camel-core-model/src/main/java/org/apache/camel/model/language/NamespaceAwareExpression.java
b/core/camel-core-model/src/main/java/org/apache/camel/model/language/NamespaceAwareExpression.java
index fc85ec7a6994..4859617b74fe 100644
---
a/core/camel-core-model/src/main/java/org/apache/camel/model/language/NamespaceAwareExpression.java
+++
b/core/camel-core-model/src/main/java/org/apache/camel/model/language/NamespaceAwareExpression.java
@@ -17,7 +17,6 @@
package org.apache.camel.model.language;
import java.util.ArrayList;
-import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
@@ -104,7 +103,7 @@ public abstract class NamespaceAwareExpression extends
SingleInputTypedExpressio
public Map<String, String> getNamespaceAsMap() {
if (namespaces == null && namespace != null) {
- namespaces = new HashMap<>();
+ namespaces = new LinkedHashMap<>();
}
if (namespace != null) {
for (PropertyDefinition def : namespace) {
diff --git
a/dsl/camel-yaml-dsl/camel-yaml-dsl-deserializers/src/main/java/org/apache/camel/dsl/yaml/deserializers/RouteDefinitionDeserializer.java
b/dsl/camel-yaml-dsl/camel-yaml-dsl-deserializers/src/main/java/org/apache/camel/dsl/yaml/deserializers/RouteDefinitionDeserializer.java
index fa6d7e04cd2d..77e0c2c91f21 100644
---
a/dsl/camel-yaml-dsl/camel-yaml-dsl-deserializers/src/main/java/org/apache/camel/dsl/yaml/deserializers/RouteDefinitionDeserializer.java
+++
b/dsl/camel-yaml-dsl/camel-yaml-dsl-deserializers/src/main/java/org/apache/camel/dsl/yaml/deserializers/RouteDefinitionDeserializer.java
@@ -24,6 +24,7 @@ import org.apache.camel.model.ErrorHandlerDefinition;
import org.apache.camel.model.FromDefinition;
import org.apache.camel.model.InputTypeDefinition;
import org.apache.camel.model.OutputTypeDefinition;
+import org.apache.camel.model.PropertyDefinition;
import org.apache.camel.model.RouteDefinition;
import org.apache.camel.spi.annotations.YamlIn;
import org.apache.camel.spi.annotations.YamlProperty;
@@ -67,6 +68,8 @@ import org.snakeyaml.engine.v2.nodes.NodeTuple;
description = "To control how to shut down the
route."),
@YamlProperty(name = "inputType", type =
"object:org.apache.camel.model.InputTypeDefinition"),
@YamlProperty(name = "outputType", type =
"object:org.apache.camel.model.OutputTypeDefinition"),
+ @YamlProperty(name = "routeProperty", type =
"array:org.apache.camel.model.PropertyDefinition",
+ description = "To set metadata as properties
on the route."),
@YamlProperty(name = "from", type =
"object:org.apache.camel.model.FromDefinition", required = true)
})
public class RouteDefinitionDeserializer extends
YamlDeserializerBase<RouteDefinition> {
@@ -162,6 +165,9 @@ public class RouteDefinitionDeserializer extends
YamlDeserializerBase<RouteDefin
case "outputType":
target.setOutputType(asType(val,
OutputTypeDefinition.class));
break;
+ case "routeProperty":
+ target.setRouteProperties(asFlatList(val,
PropertyDefinition.class));
+ break;
case "from":
val.setProperty(RouteDefinition.class.getName(), target);
target.setInput(asType(val, FromDefinition.class));
diff --git a/dsl/camel-yaml-dsl/camel-yaml-dsl-validator/pom.xml
b/dsl/camel-yaml-dsl/camel-yaml-dsl-validator/pom.xml
index 4392c785b763..ab54795fb6f2 100644
--- a/dsl/camel-yaml-dsl/camel-yaml-dsl-validator/pom.xml
+++ b/dsl/camel-yaml-dsl/camel-yaml-dsl-validator/pom.xml
@@ -64,6 +64,16 @@
<version>${networknt-json-schema-validator-version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-yaml-io</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-xml-io</artifactId>
+ <scope>test</scope>
+ </dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
diff --git
a/dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/YamlRoundTripTest.java
b/dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/YamlRoundTripTest.java
new file mode 100644
index 000000000000..2900bf70bf68
--- /dev/null
+++
b/dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/YamlRoundTripTest.java
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.dsl.yaml.validator;
+
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Stream;
+
+import com.networknt.schema.Error;
+import org.apache.camel.impl.DefaultCamelContext;
+import org.apache.camel.model.ModelCamelContext;
+import org.apache.camel.model.RouteDefinition;
+import org.apache.camel.model.RoutesDefinition;
+import org.apache.camel.support.PluginHelper;
+import org.apache.camel.support.ResourceHelper;
+import org.apache.camel.xml.LwModelToXMLDumper;
+import org.apache.camel.xml.in.ModelParser;
+import org.apache.camel.yaml.LwModelToYAMLDumper;
+import org.junit.jupiter.api.Test;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.junit.jupiter.api.Assumptions.assumeTrue;
+
+/**
+ * The routes of the XML test corpus of camel-xml-io are dumped as YAML and
loaded back with the YAML DSL: both models,
+ * dumped as XML, must be the same, so what the YAML dumper writes is what the
YAML DSL reads (CAMEL-25206). The YAML
+ * must also pass the checks of camel validate.
+ */
+class YamlRoundTripTest {
+
+ /** XML routes of the core modules' tests: camel-xml-io's parser tests and
camel-core's model tests. */
+ private static final List<Path> CORPUS = List.of(
+ Path.of("../../../core/camel-xml-io/src/test/resources"),
+
Path.of("../../../core/camel-core/src/test/resources/org/apache/camel/model"));
+
+ /**
+ * Routes that read back differently, and why. In YAML an EIP with no
steps of its own takes the steps after it, as
+ * in the Java DSL; the XML routes of the corpus keep them apart. The YAML
DSL has no route scoped onException,
+ * onCompletion or interceptors (they go at the top level or in a
routeConfiguration): the YAML written for them
+ * does not validate.
+ */
+ private static final Map<String, String> KNOWN = Map.of(
+ "kamelet.xml", "the steps after kamelet are its own in YAML: they
run at the kamelet's sink",
+ "barOnExceptionRoute.xml", "route scoped onException: YAML has it
at the top level or in a routeConfiguration",
+ "onCompletion.xml", "route scoped onCompletion: YAML has it at the
top level or in a routeConfiguration",
+ "barInterceptorRoute.xml", "route scoped intercept: YAML has it at
the top level or in a routeConfiguration",
+ "interceptFrom.xml", "route scoped interceptFrom: YAML has it at
the top level or in a routeConfiguration",
+ "interceptFromAndSendTo.xml",
+ "route scoped interceptSendToEndpoint: YAML has it at the top
level or in a routeConfiguration");
+
+ private static List<RouteDefinition> routes(Path file) {
+ for (String ns : List.of("http://camel.apache.org/schema/xml-io";,
"http://camel.apache.org/schema/spring";)) {
+ try (InputStream in = Files.newInputStream(file)) {
+ RoutesDefinition routes = new ModelParser(in,
ns).parseRoutesDefinition().orElse(null);
+ if (routes != null && !routes.getRoutes().isEmpty()) {
+ return routes.getRoutes();
+ }
+ } catch (Exception e) {
+ // not a routes file in this namespace
+ }
+ }
+ return List.of();
+ }
+
+ @Test
+ void xmlCorpus() throws Exception {
+ assumeTrue(Files.isDirectory(CORPUS.get(0)), "the core test routes are
in the source tree");
+ List<Path> files = new ArrayList<>();
+ for (Path dir : CORPUS) {
+ try (Stream<Path> s = Files.list(dir)) {
+ files.addAll(s.filter(p ->
p.toString().endsWith(".xml")).sorted().toList());
+ }
+ }
+ List<String> failures = new ArrayList<>();
+ int routes = 0;
+ for (Path file : files) {
+ for (RouteDefinition route : routes(file)) {
+ routes++;
+ String problem = roundTrip(route);
+ if (problem != null &&
!KNOWN.containsKey(file.getFileName().toString())) {
+ failures.add(file.getFileName() + "\n" + problem);
+ }
+ }
+ }
+ assertThat(routes).isGreaterThan(100);
+ assertThat(failures).as("%d of %d routes do not read back:%n%s",
failures.size(), routes,
+ String.join("\n\n", failures)).isEmpty();
+ }
+
+ /** Null when the route reads back as it was, else what went wrong. */
+ private static String roundTrip(RouteDefinition route) throws Exception {
+ String yaml;
+ String xml;
+ try (DefaultCamelContext context = new DefaultCamelContext()) {
+ yaml = new LwModelToYAMLDumper().dumpModelAsYaml(context, route);
+ // prepared as the routes loader prepares the route read back
(transacted and saga take the route,
+ // an interceptFrom that does not match the route is removed)
+ RoutesDefinition routes = new RoutesDefinition();
+ routes.setCamelContext(context);
+ routes.prepareRoute(route);
+ xml = new LwModelToXMLDumper().dumpModelAsXml(context, route);
+ }
+ // the YAML written must pass the checks of camel validate, not only
load
+ List<Error> errors = new YamlValidator().validate(yaml);
+ if (!errors.isEmpty()) {
+ return yaml + "\n--- does not validate: " + errors;
+ }
+ try (DefaultCamelContext context = new DefaultCamelContext()) {
+
PluginHelper.getRoutesLoader(context).loadRoutes(ResourceHelper.fromString("route.yaml",
yaml));
+ List<RouteDefinition> loaded =
context.getCamelContextExtension().getContextPlugin(ModelCamelContext.class)
+ .getRouteDefinitions();
+ if (loaded.size() != 1) {
+ return yaml + "\n--- loads " + loaded.size() + " routes";
+ }
+ String back = new LwModelToXMLDumper().dumpModelAsXml(context,
loaded.get(0));
+ return back.equals(xml) ? null : yaml + "\n--- was ---\n" + xml +
"\n--- read back as ---\n" + back;
+ } catch (Exception e) {
+ return yaml + "\n--- does not load: " + e;
+ }
+ }
+}
diff --git
a/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl-canonical.json
b/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl-canonical.json
index 86b49fb70073..fb088be1f633 100644
---
a/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl-canonical.json
+++
b/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl-canonical.json
@@ -4131,6 +4131,13 @@
"routePolicyRef" : {
"type" : "string"
},
+ "routeProperty" : {
+ "type" : "array",
+ "description" : "To set metadata as properties on the route.",
+ "items" : {
+ "$ref" :
"#/items/definitions/org.apache.camel.model.PropertyDefinition"
+ }
+ },
"shutdownRoute" : {
"type" : "string",
"description" : "To control how to shut down the route.",
diff --git
a/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl-model.json
b/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl-model.json
index 5962bfc01ece..5f661bb07177 100644
---
a/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl-model.json
+++
b/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl-model.json
@@ -7054,6 +7054,11 @@
"type" : "object",
"ref" : "from",
"required" : true
+ }, {
+ "name" : "routeProperty",
+ "type" : "array",
+ "ref" : "property",
+ "description" : "To set metadata as properties on the route."
} ]
},
"routeTemplate" : {
diff --git
a/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl.json
b/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl.json
index 8561e8fbf1b5..2a2ff1970e27 100644
---
a/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl.json
+++
b/dsl/camel-yaml-dsl/camel-yaml-dsl/src/generated/resources/schema/camelYamlDsl.json
@@ -5408,6 +5408,13 @@
"routePolicyRef" : {
"type" : "string"
},
+ "routeProperty" : {
+ "type" : "array",
+ "description" : "To set metadata as properties on the route.",
+ "items" : {
+ "$ref" :
"#/items/definitions/org.apache.camel.model.PropertyDefinition"
+ }
+ },
"shutdownRoute" : {
"type" : "string",
"description" : "To control how to shut down the route.",
