This is an automated email from the ASF dual-hosted git repository.
mattcasters pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new 266d7b4e6d Allow MetaInject to run when all input streams are empty
(#9078)
266d7b4e6d is described below
commit 266d7b4e6db094cfd99941ca014e4d85e194d06f
Author: Romain Manni-Bucau <[email protected]>
AuthorDate: Sat Oct 24 22:02:29 2026 +0200
Allow MetaInject to run when all input streams are empty (#9078)
* Allow MetaInject to run when all input streams are empty
Introduce a ConditionStreamBehavior enum that replaces the legacy
allowEmptyStreamOnExecution boolean. It adds a third ALWAYS mode so the
injected template pipeline can run even when every input stream is empty
(e.g. templates relying on constant mappings alone), while ALL_NON_EMPTY
and AT_LEAST_ONE_NON_EMPTY preserve the previous false/true semantics.
Add a general @OnUnknownDeserializationProperty callback honored by both
the XML and JSON serializers, so legacy files still carrying the old
allow_empty_stream_on_execution tag keep mapping correctly (Y/true to
AT_LEAST_ONE_NON_EMPTY, N/false to ALL_NON_EMPTY), without writing it on
new metadata. The helper returns instance-bound Consumer callbacks built
from Class.getMethods(), so only public methods are discovered and a
public override masks its parent.
New files are serialized with only the condition_stream_behavior key.
The dialog checkbox becomes a dropdown of the three modes, with en/fr/pt_BR
labels, and the user/dev manuals document the option and the callback.
* #9078 Keep legacy MetaInject allow-empty-stream behavior working
Backward-compatible handling of the legacy ALLOW_EMPTY_STREAM_ON_EXECUTION
injection key and allow_empty_stream_on_execution config tag after the move
to ConditionStreamBehavior: remap the old injection key and translate
Y/N/true/false to the enum code, apply the legacy tag only when the new enum
is absent, fix unknown-value parsing in the JSON serializer, add missing
i18n,
and cover the legacy paths with parameterized tests.
* #9078 Resolve the legacy MetaInject allow-empty-stream key at every
lookup site
Address the round-2 review feedback on PR #9078: the legacy
ALLOW_EMPTY_STREAM_ON_EXECUTION injection key is now normalized to
CONDITION_STREAM_BEHAVIOR everywhere the mapping is read, not only during
the injection itself.
- collectDataForOneMapping resolves the key up front so lookupGroupKey and
HopMetadataInjector.isTopLevelInjectionKey see the normalized key; a
legacy mapping stored with target_detail=Y is no longer logged as
TargetKeyNotFound and is injected under CONDITION_STREAM_BEHAVIOR.
- getUnavailableTargetKeys checks the resolved key, so the dialog no
longer offers to delete a legacy mapping that would actually run.
- MetaInjectDialog.findMappingForTargetKey normalizes both keys so the
source of a legacy mapping stays visible in the Inject Metadata tree.
- resolveInjectionKey is widened to protected static so the dialog (same
package) and subclasses can reuse it while staying out of the public API.
Docs: the @OnUnknownDeserializationProperty example in the dev manual now
shows the conditionStreamBehaviorSet guard so the legacy boolean cannot
override an explicit condition_stream_behavior (current enum wins when both
tags are present).
Tests: add coverage for getUnavailableTargetKeys keeping a legacy-key
mapping, and for the target_detail=Y scalar path injecting the enum code.
* #9078 Address round-3 review feedback on the ConditionStreamBehavior
change
- ConditionStreamBehavior.isLegacyBoolean now compares with equalsIgnoreCase
instead of default-locale toLowerCase, so an uppercase legacy TRUE still
maps to AT_LEAST_ONE_NON_EMPTY even under a Turkish (or similar) JVM
locale;
the legacy key constant-injection test gains a mixed-case TRUE case.
- MetaInjectDialog.enterMapping builds its MappingTarget with the normalized
target key (new package-private resolveTargetAttributeKey, delegating to
MetaInject.resolveInjectionKey) instead of the raw saved key, so a mapping
still stored as ALLOW_EMPTY_STREAM_ON_EXECUTION is pre-selected in the
mapping dialog and survives a confirm instead of being dropped.
- The condition-stream-behavior CCombo is now SWT.READ_ONLY so only the enum
descriptions can be selected; a typo or cleared field can no longer
silently
fall back to ALL_NON_EMPTY.
- Test/dialog javadocs no longer narrate the PR review history; they state
the
behavior under test, and the resolveInjectionKey javadoc lists every
lookup
site (now including the enter-mapping target row).
---
.../api/OnUnknownDeserializationProperty.java | 61 ++++
.../apache/hop/metadata/api/UnknownProperty.java | 48 +++
.../serializer/json/JsonMetadataParser.java | 33 ++
.../metadata/serializer/xml/XmlMetadataUtil.java | 41 +++
.../apache/hop/metadata/util/ReflectionUtil.java | 74 +++++
.../unknownproperty/PropertyWithLegacy.java | 65 ++++
.../UnknownDeserializationPropertyTest.java | 266 +++++++++++++++++
.../modules/ROOT/pages/metadata-serialization.adoc | 30 ++
.../ROOT/pages/pipeline/transforms/metainject.adoc | 5 +
integration-tests/mdi/0045-meta-inject-valiate.hpl | 4 +-
integration-tests/mdi/0045-meta-inject.hpl | 8 +-
.../mdi/datasets/golden-meta-inject.csv | 4 +-
integration-tests/mdi/files/meta-inject.xml | 2 +-
.../mdi/metadata/dataset/golden-meta-inject.json | 2 +-
.../unit-test/0045-meta-inject-valiate UNIT.json | 6 +-
.../metainject/ConditionStreamBehavior.java | 135 +++++++++
.../pipeline/transforms/metainject/MetaInject.java | 102 ++++++-
.../transforms/metainject/MetaInjectDialog.java | 65 +++-
.../transforms/metainject/MetaInjectMeta.java | 52 +++-
.../metainject/messages/messages_en_US.properties | 6 +-
.../metainject/messages/messages_fr_FR.properties | 5 +-
.../metainject/messages/messages_pt_BR.properties | 6 +-
.../transforms/metainject/MetaInjectMetaTest.java | 161 +++++++++-
.../transforms/metainject/MetaInjectTest.java | 331 +++++++++++++++++++++
24 files changed, 1465 insertions(+), 47 deletions(-)
diff --git
a/core/src/main/java/org/apache/hop/metadata/api/OnUnknownDeserializationProperty.java
b/core/src/main/java/org/apache/hop/metadata/api/OnUnknownDeserializationProperty.java
new file mode 100644
index 0000000000..de8e258669
--- /dev/null
+++
b/core/src/main/java/org/apache/hop/metadata/api/OnUnknownDeserializationProperty.java
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.metadata.api;
+
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/**
+ * Marks a method that is invoked by the metadata serializers when a property
is encountered during
+ * deserialization that does not match any {@link
HopMetadataProperty}-annotated field.
+ *
+ * <p>This is the counterpart of {@link HopMetadataProperty} and is mainly
used to keep reading
+ * legacy or renamed keys: the annotated method receives an {@link
UnknownProperty} describing the
+ * key and its raw textual value, and can translate that value into a current
field. It is honored
+ * by both the XML and the JSON metadata serializers.
+ *
+ * <h2>Contract</h2>
+ *
+ * <ul>
+ * <li>The annotated method must be {@code public}. Discovery is based on
{@link
+ * Class#getMethods()}, so private and protected methods are ignored.
+ * <li>The method must declare exactly one parameter of type {@link
UnknownProperty}. Any other
+ * signature raises an {@link IllegalArgumentException}.
+ * <li>Several {@code @OnUnknownDeserializationProperty} methods may be
declared; they are all
+ * invoked for each unknown property.
+ * <li>Because discovery goes through {@link Class#getMethods()}, a public
override of an
+ * inherited annotated method replaces it: only the most-derived
declaration is invoked, and
+ * the overridden parent method is not called.
+ * </ul>
+ *
+ * <p>Example - a legacy boolean property replaced by an enum is read into the
new field:
+ *
+ * <pre>{@code
+ * @OnUnknownDeserializationProperty
+ * public void onUnknownDeserializationProperty(UnknownProperty property) {
+ * if ("legacy_flag".equals(property.getName())) {
+ * this.value = Boolean.parseBoolean(property.getValue());
+ * }
+ * }
+ * }</pre>
+ */
+@Retention(RetentionPolicy.RUNTIME)
+@Target(ElementType.METHOD)
+public @interface OnUnknownDeserializationProperty {}
diff --git
a/core/src/main/java/org/apache/hop/metadata/api/UnknownProperty.java
b/core/src/main/java/org/apache/hop/metadata/api/UnknownProperty.java
new file mode 100644
index 0000000000..8909c814b7
--- /dev/null
+++ b/core/src/main/java/org/apache/hop/metadata/api/UnknownProperty.java
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.metadata.api;
+
+/** A property found during deserialization that does not map to any annotated
field. */
+public class UnknownProperty {
+ private final String name;
+ private final String value;
+
+ public UnknownProperty(String name, String value) {
+ this.name = name;
+ this.value = value;
+ }
+
+ /**
+ * @return The name of the unknown property as found in the serialized data.
+ */
+ public String getName() {
+ return name;
+ }
+
+ /**
+ * @return The textual value of the unknown property as found in the
serialized data.
+ */
+ public String getValue() {
+ return value;
+ }
+
+ @Override
+ public String toString() {
+ return "UnknownProperty{name='" + name + "', value='" + value + "'}";
+ }
+}
diff --git
a/core/src/main/java/org/apache/hop/metadata/serializer/json/JsonMetadataParser.java
b/core/src/main/java/org/apache/hop/metadata/serializer/json/JsonMetadataParser.java
index a3c81dd592..9ba8bb3100 100644
---
a/core/src/main/java/org/apache/hop/metadata/serializer/json/JsonMetadataParser.java
+++
b/core/src/main/java/org/apache/hop/metadata/serializer/json/JsonMetadataParser.java
@@ -27,6 +27,7 @@ import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.function.Consumer;
import lombok.Getter;
import lombok.Setter;
import org.apache.commons.lang3.StringUtils;
@@ -38,6 +39,7 @@ import org.apache.hop.metadata.api.IHopMetadata;
import org.apache.hop.metadata.api.IHopMetadataObjectFactory;
import org.apache.hop.metadata.api.IHopMetadataProvider;
import org.apache.hop.metadata.api.IHopMetadataSerializer;
+import org.apache.hop.metadata.api.UnknownProperty;
import org.apache.hop.metadata.util.ReflectionUtil;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
@@ -93,6 +95,8 @@ public class JsonMetadataParser<T extends IHopMetadata> {
// Load all the properties found in the JSON...
//
+ List<Consumer<UnknownProperty>> unknownPropertyCallbacks =
+ ReflectionUtil.onUnknownDeserializationPropertyCallbacks(object);
try {
while (jsonParser.nextToken() != JsonToken.END_OBJECT) {
String key = jsonParser.getCurrentName();
@@ -101,6 +105,35 @@ public class JsonMetadataParser<T extends IHopMetadata> {
// This is a recognized piece of data. We can load this...
//
loadProperty(object, jsonParser, key, field);
+ } else if (!unknownPropertyCallbacks.isEmpty()) {
+ // A key that does not map to any annotated field. Read its raw
value and fire the
+ // callbacks so they can translate a legacy/renamed property into a
current field.
+ //
+ // Advance to the value token. For an unknown structured value
(object or array) we
+ // cannot meaningfully hand it to a scalar legacy callback, so
consume the whole
+ // subtree with skipChildren() and pass null. Scalar values are
passed through
+ // unchanged (a string that literally reads "null" stays "null");
only a real JSON
+ // null becomes a Java null.
+ //
+ JsonToken valueToken = jsonParser.nextToken();
+ String rawValue;
+ if (valueToken == JsonToken.START_OBJECT || valueToken ==
JsonToken.START_ARRAY) {
+ jsonParser.skipChildren();
+ rawValue = null;
+ } else {
+ rawValue = valueToken == JsonToken.VALUE_NULL ? null :
jsonParser.getText();
+ }
+ UnknownProperty unknownProperty = new UnknownProperty(key, rawValue);
+ try {
+ unknownPropertyCallbacks.forEach(callback ->
callback.accept(unknownProperty));
+ } catch (IllegalArgumentException e) {
+ throw new HopException(
+ "Error invoking @OnUnknownDeserializationProperty callback for
key '"
+ + key
+ + "' on class "
+ + objectClass.getName(),
+ e);
+ }
}
}
} catch (Exception e) {
diff --git
a/core/src/main/java/org/apache/hop/metadata/serializer/xml/XmlMetadataUtil.java
b/core/src/main/java/org/apache/hop/metadata/serializer/xml/XmlMetadataUtil.java
index a317883a10..0277c47df2 100644
---
a/core/src/main/java/org/apache/hop/metadata/serializer/xml/XmlMetadataUtil.java
+++
b/core/src/main/java/org/apache/hop/metadata/serializer/xml/XmlMetadataUtil.java
@@ -29,6 +29,7 @@ import java.util.List;
import java.util.Map;
import java.util.MissingResourceException;
import java.util.Set;
+import java.util.function.Consumer;
import org.apache.commons.lang3.StringUtils;
import org.apache.hop.core.Const;
import org.apache.hop.core.encryption.Encr;
@@ -50,6 +51,7 @@ import org.apache.hop.metadata.api.IHopMetadataProvider;
import org.apache.hop.metadata.api.IIntCodeConverter;
import org.apache.hop.metadata.api.IMissingPlugin;
import org.apache.hop.metadata.api.IStringEncoder;
+import org.apache.hop.metadata.api.UnknownProperty;
import org.apache.hop.metadata.util.ReflectionUtil;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
@@ -799,6 +801,7 @@ public class XmlMetadataUtil {
if (parentProperty != null) {
childKeysToIgnore.addAll(Set.of(parentProperty.childKeysToIgnore()));
}
+ Set<String> knownKeys = new HashSet<>();
if (object == null) {
object = createNewObject(parentObject, node, clazz);
@@ -842,6 +845,7 @@ public class XmlMetadataUtil {
if (StringUtils.isEmpty(tag)) {
tag = field.getName();
}
+ knownKeys.add(tag);
Node tagNode;
if (property.inline()) {
tagNode = node;
@@ -851,6 +855,9 @@ public class XmlMetadataUtil {
// Get the groupKey & groupNode
String groupKey = property.groupKey();
+ if (StringUtils.isNotEmpty(groupKey)) {
+ knownKeys.add(groupKey);
+ }
Node groupNode;
if (StringUtils.isEmpty(groupKey)) {
groupNode = node;
@@ -912,12 +919,16 @@ public class XmlMetadataUtil {
// Get the groupKey & groupNode
String groupKey = property.groupKey();
+ if (StringUtils.isNotEmpty(groupKey)) {
+ knownKeys.add(groupKey);
+ }
Node groupNode;
if (StringUtils.isEmpty(groupKey)) {
groupNode = node;
} else {
groupNode = XmlHandler.getSubNode(node, groupKey);
}
+ knownKeys.add(tag);
// Find the single argument of the setter
//
@@ -954,6 +965,36 @@ public class XmlMetadataUtil {
throw new HopXmlException("Error calling setter method on class " +
clazz.getName(), e);
}
+ // Fire the @OnUnknownDeserializationProperty callbacks for any child
element that does not map
+ // to an annotated field or group. This allows translating legacy/renamed
keys into current
+ // fields.
+ //
+ List<Consumer<UnknownProperty>> unknownPropertyCallbacks =
+ ReflectionUtil.onUnknownDeserializationPropertyCallbacks(object);
+ if (!unknownPropertyCallbacks.isEmpty()) {
+ for (Node child = node.getFirstChild(); child != null; child =
child.getNextSibling()) {
+ if (child.getNodeType() != Node.ELEMENT_NODE) {
+ continue;
+ }
+ String childName = child.getNodeName();
+ if (knownKeys.contains(childName)) {
+ continue;
+ }
+ UnknownProperty unknownProperty =
+ new UnknownProperty(childName, XmlHandler.getNodeValue(child));
+ try {
+ unknownPropertyCallbacks.forEach(callback ->
callback.accept(unknownProperty));
+ } catch (IllegalArgumentException e) {
+ throw new HopXmlException(
+ "Error invoking @OnUnknownDeserializationProperty callback for
key '"
+ + childName
+ + "' on class "
+ + clazz.getName(),
+ e);
+ }
+ }
+ }
+
if (object instanceof ILegacyXml legacyXml) {
try {
legacyXml.convertLegacyXml(node, metadataProvider);
diff --git
a/core/src/main/java/org/apache/hop/metadata/util/ReflectionUtil.java
b/core/src/main/java/org/apache/hop/metadata/util/ReflectionUtil.java
index e9d56563a4..bb8474d06e 100644
--- a/core/src/main/java/org/apache/hop/metadata/util/ReflectionUtil.java
+++ b/core/src/main/java/org/apache/hop/metadata/util/ReflectionUtil.java
@@ -23,11 +23,16 @@ import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashSet;
import java.util.List;
+import java.util.Map;
import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.function.Consumer;
import java.util.function.Function;
import org.apache.commons.lang3.StringUtils;
import org.apache.hop.core.exception.HopException;
import org.apache.hop.metadata.api.HopMetadataProperty;
+import org.apache.hop.metadata.api.OnUnknownDeserializationProperty;
+import org.apache.hop.metadata.api.UnknownProperty;
public class ReflectionUtil {
private ReflectionUtil() {}
@@ -277,4 +282,73 @@ public class ReflectionUtil {
return new ArrayList<>(methodSet);
}
+
+ private static final Map<Class<?>, List<Method>> UNKNOWN_PROPERTY_METHODS =
+ new ConcurrentHashMap<>();
+
+ /**
+ * Returns the {@link Consumer}s that translate unknown deserialization
properties, declared with
+ * {@link OnUnknownDeserializationProperty}, bound to the given instance.
+ *
+ * <p>Discovery is based on {@link Class#getMethods()}, so only {@code
public} methods are
+ * considered (private/protected annotated methods are ignored) and a public
override in a subtype
+ * replaces the inherited method (only the most-derived declaration is
invoked). Multiple
+ * annotated methods are all returned, in reflection order.
+ *
+ * <p>Each annotated method must declare exactly one parameter of type
{@link UnknownProperty};
+ * anything else raises an {@link IllegalArgumentException}.
+ *
+ * <p>The validated method list is cached per class; the returned consumers
are fresh but cheap.
+ *
+ * @param instance The object on which the annotated methods are invoked.
+ * @return The list of instance-bound callbacks, possibly empty (never
{@code null}).
+ */
+ public static List<Consumer<UnknownProperty>>
onUnknownDeserializationPropertyCallbacks(
+ Object instance) {
+ List<Method> methods =
+ UNKNOWN_PROPERTY_METHODS.computeIfAbsent(instance.getClass(),
ReflectionUtil::collect);
+ List<Consumer<UnknownProperty>> callbacks = new
ArrayList<>(methods.size());
+ for (Method method : methods) {
+ callbacks.add(toUnknownPropertyCallback(instance, method));
+ }
+ return List.copyOf(callbacks);
+ }
+
+ private static List<Method> collect(Class<?> type) {
+ List<Method> methods = new ArrayList<>();
+ for (Method method : type.getMethods()) {
+ if (!method.isAnnotationPresent(OnUnknownDeserializationProperty.class))
{
+ continue;
+ }
+ Class<?>[] parameterTypes = method.getParameterTypes();
+ if (parameterTypes.length != 1 ||
!UnknownProperty.class.equals(parameterTypes[0])) {
+ throw new IllegalArgumentException(
+ "@OnUnknownDeserializationProperty method "
+ + type.getName()
+ + "."
+ + method.getName()
+ + " must have exactly one parameter of type "
+ + UnknownProperty.class.getName());
+ }
+ methods.add(method);
+ }
+ return List.copyOf(methods);
+ }
+
+ private static Consumer<UnknownProperty> toUnknownPropertyCallback(
+ Object instance, Method method) {
+ return unknownProperty -> {
+ try {
+ method.invoke(instance, unknownProperty);
+ } catch (Exception e) {
+ Throwable cause = e.getCause() != null ? e.getCause() : e;
+ throw new IllegalArgumentException(
+ "Unable to invoke @OnUnknownDeserializationProperty callback "
+ + instance.getClass().getName()
+ + "."
+ + method.getName(),
+ cause);
+ }
+ };
+ }
}
diff --git
a/core/src/test/java/org/apache/hop/metadata/serializer/unknownproperty/PropertyWithLegacy.java
b/core/src/test/java/org/apache/hop/metadata/serializer/unknownproperty/PropertyWithLegacy.java
new file mode 100644
index 0000000000..367219f73c
--- /dev/null
+++
b/core/src/test/java/org/apache/hop/metadata/serializer/unknownproperty/PropertyWithLegacy.java
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.metadata.serializer.unknownproperty;
+
+import java.util.LinkedHashMap;
+import java.util.Map;
+import org.apache.hop.metadata.api.HopMetadata;
+import org.apache.hop.metadata.api.HopMetadataBase;
+import org.apache.hop.metadata.api.HopMetadataProperty;
+import org.apache.hop.metadata.api.IHopMetadata;
+import org.apache.hop.metadata.api.OnUnknownDeserializationProperty;
+import org.apache.hop.metadata.api.UnknownProperty;
+
+/** Test metadata type used to verify the {@link
OnUnknownDeserializationProperty} callback. */
+@HopMetadata(name = "PropertyWithLegacy", key = "property-with-legacy")
+public class PropertyWithLegacy extends HopMetadataBase implements
IHopMetadata {
+
+ /** The 'new' field, filled from annotated data. */
+ @HopMetadataProperty private String value;
+
+ /** Captures every unknown property seen by the callback, in insertion
order. */
+ private final Map<String, String> unknownProperties = new LinkedHashMap<>();
+
+ @OnUnknownDeserializationProperty
+ public void onUnknown(UnknownProperty property) {
+ unknownProperties.put(property.getName(), property.getValue());
+ }
+
+ /**
+ * @return value
+ */
+ public String getValue() {
+ return value;
+ }
+
+ /**
+ * @param value The value to set
+ */
+ public void setValue(String value) {
+ this.value = value;
+ }
+
+ public String getUnknown(String name) {
+ return unknownProperties.get(name);
+ }
+
+ public Map<String, String> getUnknownProperties() {
+ return unknownProperties;
+ }
+}
diff --git
a/core/src/test/java/org/apache/hop/metadata/serializer/unknownproperty/UnknownDeserializationPropertyTest.java
b/core/src/test/java/org/apache/hop/metadata/serializer/unknownproperty/UnknownDeserializationPropertyTest.java
new file mode 100644
index 0000000000..3a4a7e2391
--- /dev/null
+++
b/core/src/test/java/org/apache/hop/metadata/serializer/unknownproperty/UnknownDeserializationPropertyTest.java
@@ -0,0 +1,266 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.metadata.serializer.unknownproperty;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+import com.fasterxml.jackson.core.JsonFactory;
+import com.fasterxml.jackson.core.JsonParser;
+import java.io.IOException;
+import java.util.List;
+import java.util.function.Consumer;
+import java.util.stream.Stream;
+import lombok.Getter;
+import org.apache.hop.core.exception.HopException;
+import org.apache.hop.core.xml.XmlHandler;
+import org.apache.hop.metadata.api.HopMetadataBase;
+import org.apache.hop.metadata.api.OnUnknownDeserializationProperty;
+import org.apache.hop.metadata.api.UnknownProperty;
+import org.apache.hop.metadata.serializer.json.JsonMetadataParser;
+import org.apache.hop.metadata.serializer.memory.MemoryMetadataProvider;
+import org.apache.hop.metadata.serializer.xml.XmlMetadataUtil;
+import org.apache.hop.metadata.util.ReflectionUtil;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.MethodSource;
+import org.junit.jupiter.params.provider.ValueSource;
+import org.w3c.dom.Node;
+
+/**
+ * Verifies that the {@link
org.apache.hop.metadata.api.OnUnknownDeserializationProperty} callback
+ * is invoked for properties that do not map to any annotated field, in both
the XML and the JSON
+ * deserialization paths. This is how legacy/renamed keys are translated into
current fields.
+ */
+class UnknownDeserializationPropertyTest {
+
+ private final MemoryMetadataProvider provider = new MemoryMetadataProvider();
+
+ @Test
+ void xmlFiresCallbackForUnknownTag() throws Exception {
+ Node node =
+ XmlHandler.loadXmlString(
+ XmlHandler.aroundTag(
+ "element",
+
"<value>new</value><legacy_flag>Y</legacy_flag><legacy_other>42</legacy_other>"),
+ "element");
+ PropertyWithLegacy loaded =
+ XmlMetadataUtil.deSerializeFromXml(node, PropertyWithLegacy.class,
provider);
+
+ assertEquals("new", loaded.getValue());
+ // The two unknown tags were handed to the callback.
+ assertEquals("Y", loaded.getUnknown("legacy_flag"));
+ assertEquals("42", loaded.getUnknown("legacy_other"));
+ }
+
+ @Test
+ void xmlDoesNotFireForAnnotatedTags() throws Exception {
+ Node node =
+ XmlHandler.loadXmlString(
+ XmlHandler.aroundTag("element",
"<value>known</value><name>my-name</name>"), "element");
+ PropertyWithLegacy loaded =
+ XmlMetadataUtil.deSerializeFromXml(node, PropertyWithLegacy.class,
provider);
+
+ assertEquals("known", loaded.getValue());
+ assertNull(loaded.getUnknown("value"));
+ // name is an inherited annotated field (HopMetadataBase), not an unknown
property.
+ assertEquals("my-name", loaded.getName());
+ assertNull(loaded.getUnknown("name"));
+ }
+
+ @Test
+ void jsonFiresCallbackForUnknownKey() throws Exception {
+ PropertyWithLegacy loaded =
+ jsonRoundTrip("{\"name\":\"a\",\"value\":\"v\",\"legacy\":\"true\"}");
+
+ assertEquals("v", loaded.getValue());
+ assertEquals("true", loaded.getUnknown("legacy"));
+ assertEquals("a", loaded.getName());
+ }
+
+ @Test
+ void jsonDoesNotFireForAnnotatedKeys() throws Exception {
+ PropertyWithLegacy loaded =
jsonRoundTrip("{\"name\":\"b\",\"value\":\"w\"}");
+
+ assertEquals("w", loaded.getValue());
+ assertEquals("b", loaded.getName());
+ assertEquals(0, loaded.getUnknownProperties().size());
+ }
+
+ private PropertyWithLegacy jsonRoundTrip(String json) throws IOException,
HopException {
+ JsonFactory factory = new JsonFactory();
+ try (JsonParser parser = factory.createParser(json)) {
+ // Move to the START_OBJECT so the parser is positioned like the
serializer expects.
+ parser.nextToken();
+ JsonMetadataParser<PropertyWithLegacy> jsonParser =
+ new JsonMetadataParser<>(PropertyWithLegacy.class, provider);
+ return jsonParser.loadJsonObject(PropertyWithLegacy.class, parser);
+ }
+ }
+
+ @ParameterizedTest
+ @MethodSource("unknownStructuredValues")
+ void jsonSkipsUnknownStructuredValueAndKeepsLaterFields(String unknownJson)
throws Exception {
+ // An unknown object/array value must be fully consumed so a subsequent
known field ("name")
+ // still loads. Without skipping the subtree, the nested closing brace/box
would end the outer
+ // object and the trailing "name" would be dropped.
+ PropertyWithLegacy loaded =
+ jsonRoundTrip("{\"value\":\"v\",\"legacy\":" + unknownJson +
",\"name\":\"tail\"}");
+
+ assertEquals("v", loaded.getValue());
+ assertEquals("tail", loaded.getName());
+ // The unknown structured value cannot be represented as a scalar: the
callback saw null.
+ assertNull(loaded.getUnknown("legacy"));
+ }
+
+ static Stream<String> unknownStructuredValues() {
+ return Stream.of("{\"a\":1}", "[1, 2, 3]", "{}", "[]");
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"null", "true", "42", "\"null\"", "\"some text\"",
"\"\""})
+ void jsonScalarUnchangedExceptRealNull(String jsonValue) throws Exception {
+ // Scalars are passed through unchanged. Only a real JSON null (no quotes)
becomes Java null.
+ PropertyWithLegacy loaded =
+ jsonRoundTrip("{\"value\":\"v\",\"legacy\":" + jsonValue +
",\"name\":\"tail\"}");
+
+ if ("null".equals(jsonValue)) {
+ assertNull(loaded.getUnknown("legacy"));
+ } else if (jsonValue.startsWith("\"") && jsonValue.endsWith("\"")) {
+ // A JSON string literal ("null", "some text", ...) keeps its text,
quotes stripped.
+ assertEquals(jsonValue.substring(1, jsonValue.length() - 1),
loaded.getUnknown("legacy"));
+ } else {
+ // Numbers and booleans keep their text representation.
+ assertEquals(jsonValue, loaded.getUnknown("legacy"));
+ }
+ assertEquals("tail", loaded.getName());
+ }
+
+ @Test
+ void jsonStringLiteralNullStaysNonNull() throws Exception {
+ // A JSON string that reads "null" (quoted) is a real string, not a JSON
null, so it must not
+ // collapse to a Java null.
+ PropertyWithLegacy loaded =
+
jsonRoundTrip("{\"value\":\"v\",\"legacy\":\"null\",\"name\":\"tail\"}");
+ assertEquals("null", loaded.getUnknown("legacy"));
+ }
+
+ @Test
+ void multipleCallbacksAllFire() throws Exception {
+ Node node =
+ XmlHandler.loadXmlString(XmlHandler.aroundTag("element",
"<a>1</a><b>2</b>"), "element");
+ MultipleCallbacks loaded =
+ XmlMetadataUtil.deSerializeFromXml(node, MultipleCallbacks.class,
provider);
+ assertEquals("1", loaded.getA());
+ assertEquals("2", loaded.getB());
+ }
+
+ @Test
+ void publicOverrideMasksParentCallback() throws Exception {
+ Node node =
+ XmlHandler.loadXmlString(XmlHandler.aroundTag("element",
"<legacy>x</legacy>"), "element");
+ Child loaded = XmlMetadataUtil.deSerializeFromXml(node, Child.class,
provider);
+ // The child override runs; the parent callback must NOT run.
+ assertEquals("child:x", loaded.mapped);
+ assertNull(loaded.parentSaw);
+ }
+
+ @Test
+ void invalidSignatureThrows() {
+ assertThrows(
+ IllegalArgumentException.class,
+ () -> ReflectionUtil.onUnknownDeserializationPropertyCallbacks(new
BadSignature()));
+ }
+
+ @Test
+ void nonPublicAnnotatedMethodIsIgnored() {
+ List<Consumer<UnknownProperty>> callbacks =
+ ReflectionUtil.onUnknownDeserializationPropertyCallbacks(new
NonPublicCallback());
+ // Only the public callback is discovered; the private and protected ones
are ignored: exactly
+ // one callback.
+ assertEquals(1, callbacks.size());
+ }
+
+ //
---------------------------------------------------------------------------------------------
+ // Helper metadata types used by the tests above.
+
+ public static class MultipleCallbacks extends HopMetadataBase {
+ @OnUnknownDeserializationProperty
+ public void onA(UnknownProperty p) {
+ if ("a".equals(p.getName())) {
+ a = p.getValue();
+ }
+ }
+
+ @OnUnknownDeserializationProperty
+ public void onB(UnknownProperty p) {
+ if ("b".equals(p.getName())) {
+ b = p.getValue();
+ }
+ }
+
+ @Getter private String a;
+ @Getter private String b;
+ }
+
+ public static class ParentWithCallback extends HopMetadataBase {
+ String parentSaw;
+
+ @OnUnknownDeserializationProperty
+ public void onUnknown(UnknownProperty p) {
+ parentSaw = p.getValue();
+ }
+ }
+
+ public static class Child extends ParentWithCallback {
+ String mapped;
+
+ @Override
+ @OnUnknownDeserializationProperty
+ public void onUnknown(UnknownProperty p) {
+ mapped = "child:" + p.getValue();
+ }
+ }
+
+ public static class BadSignature extends HopMetadataBase {
+ @OnUnknownDeserializationProperty
+ public void bad(String wrong) {
+ // Wrong parameter type.
+ }
+ }
+
+ public static class NonPublicCallback extends HopMetadataBase {
+ private String seen;
+
+ @OnUnknownDeserializationProperty
+ private void onPrivate(UnknownProperty p) {
+ seen = p.getValue();
+ }
+
+ @OnUnknownDeserializationProperty
+ public void onPublic(UnknownProperty p) {
+ seen = p.getValue();
+ }
+
+ @SuppressWarnings("unused")
+ private String getSeen() {
+ return seen;
+ }
+ }
+}
diff --git a/docs/hop-dev-manual/modules/ROOT/pages/metadata-serialization.adoc
b/docs/hop-dev-manual/modules/ROOT/pages/metadata-serialization.adoc
index e04cbdd12a..c3bd92d526 100644
--- a/docs/hop-dev-manual/modules/ROOT/pages/metadata-serialization.adoc
+++ b/docs/hop-dev-manual/modules/ROOT/pages/metadata-serialization.adoc
@@ -272,6 +272,36 @@ This will serialize `Apache Hop` to:
----
====
+== Legacy and renamed keys
+
+When a property is renamed or a field changes type, existing serialized files
still carry the old key. To keep reading them you can mark a method with
`@OnUnknownDeserializationProperty`. The serializers invoke it once for every
property encountered during deserialization that does not map to any
`@HopMetadataProperty` field.
+
+This is the recommended way to translate a legacy or renamed key into a
current field. It works for both XML and JSON serialization.
+
+From `MetaInjectMeta.java`, where the legacy boolean
`allow_empty_stream_on_execution` was replaced by the `ConditionStreamBehavior`
enum:
+
+[source,java]
+----
+@OnUnknownDeserializationProperty
+public void onUnknownDeserializationProperty(UnknownProperty property) {
+ // Only translate the legacy boolean when the current
condition_stream_behavior enum was not
+ // part of the file. When both are present the current enum wins, so a file
that mixes the
+ // two can still represent ALWAYS.
+ if ("allow_empty_stream_on_execution".equals(property.getName())
+ && !conditionStreamBehaviorSet) {
+ conditionStreamBehavior =
ConditionStreamBehavior.fromLegacyBoolean(property.getValue());
+ }
+}
+----
+
+The serializers set annotated fields before firing this callback, and the
`conditionStreamBehaviorSet` flag (raised whenever the current enum value is
set, including during deserialization) therefore tells the callback whether a
file that contains both the old boolean and the new enum really carries the new
value. The legacy boolean is written only when the current tag was absent.
+
+The callback method must be `public` and take exactly one parameter of type
`UnknownProperty`, which exposes `getName()` (the key as found in the
serialized data) and `getValue()` (its raw textual value). You may declare
several such methods; they are all invoked for each unknown property.
+
+Discovery is based on `Class.getMethods()`, so private and protected methods
are ignored, and a public override in a subtype replaces the inherited method
(only the most-derived declaration is invoked, the overridden parent method is
not called). Any annotated method that does not take exactly one
`UnknownProperty` parameter raises an `IllegalArgumentException`.
+
+If you prefer, you can still read the raw serialized node directly in a
`convertLegacyXml(Node)` override for XML-only conversions.
+
== Lists
If you want to serialize List of objects, you can typically wrap them in a
group. An example from `CsvInputMeta.java`:
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/metainject.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/metainject.adoc
index 95026066d8..56d3f2bd15 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/metainject.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/metainject.adoc
@@ -114,6 +114,11 @@ The resulting pipeline will be your template pipeline with
the metadata already
|Streaming target transform|Select the target transform in your template
pipeline to receive data directly from the Streaming source transform.
|Run resulting pipeline|Select to inject metadata and run the template
pipeline.
If this option is not selected, metadata injection occurs, but the template
pipeline does not run.
+|When to run the resulting pipeline|Controls whether the injected template
pipeline runs when one or more of the input streams are empty.
+* *Run only when every input stream is not empty* - execution is skipped if
any input stream has no rows.
+* *Run when at least one input stream is not empty* - execution runs as soon
as one input stream has rows, even if other streams are empty.
+* *Run even when every input stream is empty* - execution runs even when all
input streams are empty (useful for templates that can run from constant
mappings alone).
+This replaces the legacy *Allow empty stream to run the resulting pipeline*
checkbox.
|===
diff --git a/integration-tests/mdi/0045-meta-inject-valiate.hpl
b/integration-tests/mdi/0045-meta-inject-valiate.hpl
index d198519044..939fe885ef 100644
--- a/integration-tests/mdi/0045-meta-inject-valiate.hpl
+++ b/integration-tests/mdi/0045-meta-inject-valiate.hpl
@@ -370,13 +370,13 @@ limitations under the License.
<format/>
<group/>
<length>-1</length>
- <name>allow_empty_stream_on_execution</name>
+ <name>condition_stream_behavior</name>
<precision>-1</precision>
<repeat>N</repeat>
<result_type>valueof</result_type>
<trim_type>none</trim_type>
<type>String</type>
- <xPath>allow_empty_stream_on_execution</xPath>
+ <xPath>condition_stream_behavior</xPath>
</field>
<field>
<currency/>
diff --git a/integration-tests/mdi/0045-meta-inject.hpl
b/integration-tests/mdi/0045-meta-inject.hpl
index bae83e9cc2..1b097e7cc0 100644
--- a/integration-tests/mdi/0045-meta-inject.hpl
+++ b/integration-tests/mdi/0045-meta-inject.hpl
@@ -140,9 +140,9 @@ limitations under the License.
<target_transform_name>ETL metadata injection</target_transform_name>
</mapping>
<mapping>
- <source_field>allow_empty_stream_on_execution</source_field>
+ <source_field>condition_stream_behavior</source_field>
<source_transform>files/meta-inject.xml</source_transform>
-
<target_attribute_key>ALLOW_EMPTY_STREAM_ON_EXECUTION</target_attribute_key>
+ <target_attribute_key>CONDITION_STREAM_BEHAVIOR</target_attribute_key>
<target_detail>N</target_detail>
<target_transform_name>ETL metadata injection</target_transform_name>
</mapping>
@@ -322,13 +322,13 @@ limitations under the License.
<format/>
<group/>
<length>-1</length>
- <name>allow_empty_stream_on_execution</name>
+ <name>condition_stream_behavior</name>
<precision>-1</precision>
<repeat>N</repeat>
<result_type>valueof</result_type>
<trim_type>none</trim_type>
<type>String</type>
- <xPath>allow_empty_stream_on_execution</xPath>
+ <xPath>condition_stream_behavior</xPath>
</field>
<field>
<currency/>
diff --git a/integration-tests/mdi/datasets/golden-meta-inject.csv
b/integration-tests/mdi/datasets/golden-meta-inject.csv
index a6eccc00dd..d34dd09792 100644
--- a/integration-tests/mdi/datasets/golden-meta-inject.csv
+++ b/integration-tests/mdi/datasets/golden-meta-inject.csv
@@ -1,2 +1,2 @@
-filename,run_configuration,source_transform,target_file,create_parent_folder,no_execution,allow_empty_stream_on_execution,stream_source_transform,stream_target_transform
-${PROJECT_HOME}/0038-json-input-template.hpl,local,sourceTransform,${java.io.tmpdir}/json-input.hpl,Y,Y,N,sourceTransform,targetTransform
+filename,run_configuration,source_transform,target_file,create_parent_folder,no_execution,condition_stream_behavior,stream_source_transform,stream_target_transform
+${PROJECT_HOME}/0038-json-input-template.hpl,local,sourceTransform,${java.io.tmpdir}/json-input.hpl,Y,Y,all-non-empty,sourceTransform,targetTransform
diff --git a/integration-tests/mdi/files/meta-inject.xml
b/integration-tests/mdi/files/meta-inject.xml
index 5e3bd0b99d..d8286f9c49 100644
--- a/integration-tests/mdi/files/meta-inject.xml
+++ b/integration-tests/mdi/files/meta-inject.xml
@@ -38,7 +38,7 @@
<target_file>${java.io.tmpdir}/json-input.hpl</target_file>
<create_parent_folder>Y</create_parent_folder>
<no_execution>Y</no_execution>
- <allow_empty_stream_on_execution>N</allow_empty_stream_on_execution>
+ <condition_stream_behavior>all-non-empty</condition_stream_behavior>
<stream_source_transform>sourceTransform</stream_source_transform>
<stream_target_transform>targetTransform</stream_target_transform>
<mappings>
diff --git a/integration-tests/mdi/metadata/dataset/golden-meta-inject.json
b/integration-tests/mdi/metadata/dataset/golden-meta-inject.json
index eda875c272..3e17cbf2cc 100644
--- a/integration-tests/mdi/metadata/dataset/golden-meta-inject.json
+++ b/integration-tests/mdi/metadata/dataset/golden-meta-inject.json
@@ -56,7 +56,7 @@
"field_length": -1,
"field_type": 2,
"field_precision": -1,
- "field_name": "allow_empty_stream_on_execution",
+ "field_name": "condition_stream_behavior",
"field_format": ""
},
{
diff --git a/integration-tests/mdi/metadata/unit-test/0045-meta-inject-valiate
UNIT.json b/integration-tests/mdi/metadata/unit-test/0045-meta-inject-valiate
UNIT.json
index 067eef21bd..a0f2cfffd9 100644
--- a/integration-tests/mdi/metadata/unit-test/0045-meta-inject-valiate
UNIT.json
+++ b/integration-tests/mdi/metadata/unit-test/0045-meta-inject-valiate
UNIT.json
@@ -71,8 +71,8 @@
{
"field_mappings": [
{
- "transform_field": "allow_empty_stream_on_execution",
- "data_set_field": "allow_empty_stream_on_execution"
+ "transform_field": "condition_stream_behavior",
+ "data_set_field": "condition_stream_behavior"
},
{
"transform_field": "create_parent_folder",
@@ -114,7 +114,7 @@
"target_file",
"create_parent_folder",
"no_execution",
- "allow_empty_stream_on_execution",
+ "condition_stream_behavior",
"stream_source_transform",
"stream_target_transform"
],
diff --git
a/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/ConditionStreamBehavior.java
b/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/ConditionStreamBehavior.java
new file mode 100644
index 0000000000..fa6d8817e5
--- /dev/null
+++
b/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/ConditionStreamBehavior.java
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.pipeline.transforms.metainject;
+
+import org.apache.hop.core.Const;
+import org.apache.hop.i18n.BaseMessages;
+import org.apache.hop.metadata.api.IEnumHasCode;
+import org.apache.hop.metadata.api.IEnumHasCodeAndDescription;
+
+/**
+ * Controls under which stream conditions the injected template pipeline is
executed.
+ *
+ * <p>This replaces the legacy boolean {@code allowEmptyStreamOnExecution}:
{@link #ALL_NON_EMPTY}
+ * and {@link #AT_LEAST_ONE_NON_EMPTY} preserve the old {@code false}/{@code
true} semantics, while
+ * {@link #ALWAYS} adds the ability to run even when every input stream is
empty (for templates that
+ * can run from constant mappings alone).
+ */
+public enum ConditionStreamBehavior implements IEnumHasCodeAndDescription {
+ /** Every input stream must produce at least one row. Equivalent to legacy
{@code false}. */
+ ALL_NON_EMPTY("all-non-empty",
"ConditionStreamBehavior.AllNonEmpty.Description"),
+
+ /** Run when at least one stream is non-empty even if others are empty.
Legacy {@code true}. */
+ AT_LEAST_ONE_NON_EMPTY(
+ "at-least-one-non-empty",
"ConditionStreamBehavior.AtLeastOneNonEmpty.Description"),
+
+ /** Run regardless of the stream contents, even when every input stream is
empty. */
+ ALWAYS("always", "ConditionStreamBehavior.Always.Description");
+
+ private static final Class<?> PKG = ConditionStreamBehavior.class;
+
+ private final String code;
+ private final String descriptionKey;
+
+ ConditionStreamBehavior(String code, String descriptionKey) {
+ this.code = code;
+ this.descriptionKey = descriptionKey;
+ }
+
+ @Override
+ public String getCode() {
+ return code;
+ }
+
+ @Override
+ public String getDescription() {
+ return BaseMessages.getString(PKG, descriptionKey);
+ }
+
+ /**
+ * @return {@code true} when every stream is required to be non-empty
({@link #ALL_NON_EMPTY}).
+ */
+ public boolean requiresAllNonEmpty() {
+ return this == ALL_NON_EMPTY;
+ }
+
+ /**
+ * @return {@code true} when execution is allowed even if every stream is
empty ({@link #ALWAYS}).
+ */
+ public boolean allowsAllEmpty() {
+ return this == ALWAYS;
+ }
+
+ /**
+ * Maps the legacy {@code allow_empty_stream_on_execution} boolean value
onto this enum. Before
+ * this enum existed the transform stored a plain boolean (as {@code
Y}/{@code N} in XML and
+ * {@code true}/{@code false} in JSON metadata / injection values).
+ *
+ * <p>A truthy value ({@code Y}/{@code yes}/{@code true}/{@code t}, per
{@link Const#toBoolean})
+ * meant "at least one non-empty stream was required" and maps to {@link
#AT_LEAST_ONE_NON_EMPTY};
+ * anything else maps to {@link #ALL_NON_EMPTY}.
+ *
+ * @param legacyValue the raw legacy boolean value, or {@code null}
+ * @return the equivalent enum constant, never {@code null}
+ */
+ public static ConditionStreamBehavior fromLegacyBoolean(String legacyValue) {
+ return Const.toBoolean(legacyValue)
+ ? ConditionStreamBehavior.AT_LEAST_ONE_NON_EMPTY
+ : ConditionStreamBehavior.ALL_NON_EMPTY;
+ }
+
+ /**
+ * Returns {@code true} when the given value is a legacy boolean token for
the {@code
+ * allow_empty_stream_on_execution} setting: {@code Y}/{@code N}/{@code
yes}/{@code no}/{@code
+ * true}/{@code false}/{@code t}/{@code f} (case-insensitive). This
deliberately excludes the
+ * current enum codes ({@code all-non-empty}, {@code
at-least-one-non-empty}, {@code always}) so
+ * they are never mistaken for a legacy value.
+ *
+ * @param value the value to test, or {@code null}
+ * @return {@code true} when the value is a legacy boolean token
+ */
+ public static boolean isLegacyBoolean(String value) {
+ if (value == null) {
+ return false;
+ }
+ // Case-insensitive comparison is locale-safe (matching Const.toBoolean),
so an uppercase
+ // "TRUE" still matches even when the JVM default locale (e.g. Turkish)
mangles lowercasing.
+ return "y".equalsIgnoreCase(value)
+ || "n".equalsIgnoreCase(value)
+ || "yes".equalsIgnoreCase(value)
+ || "no".equalsIgnoreCase(value)
+ || "true".equalsIgnoreCase(value)
+ || "false".equalsIgnoreCase(value)
+ || "t".equalsIgnoreCase(value)
+ || "f".equalsIgnoreCase(value);
+ }
+
+ public static String[] getDescriptions() {
+ return
IEnumHasCodeAndDescription.getDescriptions(ConditionStreamBehavior.class);
+ }
+
+ public static ConditionStreamBehavior lookupCode(String code) {
+ return IEnumHasCode.lookupCode(
+ ConditionStreamBehavior.class, code,
ConditionStreamBehavior.ALL_NON_EMPTY);
+ }
+
+ public static ConditionStreamBehavior lookupDescription(String description) {
+ return IEnumHasCodeAndDescription.lookupDescription(
+ ConditionStreamBehavior.class, description,
ConditionStreamBehavior.ALL_NON_EMPTY);
+ }
+}
diff --git
a/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInject.java
b/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInject.java
index 9a9d5c1e77..7833a0f909 100644
---
a/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInject.java
+++
b/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInject.java
@@ -73,6 +73,13 @@ import org.apache.hop.pipeline.transform.TransformMeta;
public class MetaInject extends BaseTransform<MetaInjectMeta, MetaInjectData> {
private static final Class<?> PKG = MetaInject.class;
+ /** Legacy injection target key, replaced by {@link
#CONDITION_STREAM_BEHAVIOR_INJECTION_KEY}. */
+ private static final String LEGACY_ALLOW_EMPTY_STREAM_INJECTION_KEY =
+ "ALLOW_EMPTY_STREAM_ON_EXECUTION";
+
+ /** Current injection target key for {@link ConditionStreamBehavior}. */
+ private static final String CONDITION_STREAM_BEHAVIOR_INJECTION_KEY =
"CONDITION_STREAM_BEHAVIOR";
+
public MetaInject(
TransformMeta transformMeta,
MetaInjectMeta meta,
@@ -83,6 +90,21 @@ public class MetaInject extends
BaseTransform<MetaInjectMeta, MetaInjectData> {
super(transformMeta, meta, data, copyNr, pipelineMeta, trans);
}
+ /**
+ * Decides whether execution should be skipped based on the stream state and
the configured {@link
+ * ConditionStreamBehavior}.
+ *
+ * @param receivedRows {@code true} when at least one input stream produced
a row.
+ * @param hasEmptyList {@code true} when at least one input stream produced
no rows.
+ * @param behavior The configured {@link ConditionStreamBehavior}.
+ * @return {@code true} when the template execution must be skipped.
+ */
+ static boolean shouldSkipExecution(
+ boolean receivedRows, boolean hasEmptyList, ConditionStreamBehavior
behavior) {
+ return (!receivedRows && !behavior.allowsAllEmpty())
+ || (hasEmptyList && behavior.requiresAllNonEmpty());
+ }
+
@Override
public boolean processRow() throws HopException {
// Read the data from all input transforms and keep it in memory...
@@ -102,9 +124,13 @@ public class MetaInject extends
BaseTransform<MetaInjectMeta, MetaInjectData> {
injectMetadataIntoTemplateTransforms();
- // Check if all previous transforms are returning data, unless
isAllowEmptyStreamOnExecution is
- // true then execute if at least one branch has data
- if (!data.receivedRows || (data.hasEmptyList &&
!meta.isAllowEmptyStreamOnExecution())) {
+ // Check if all previous transforms are returning data. The
ConditionStreamBehavior controls
+ // whether empty streams are tolerated: ALL_NON_EMPTY requires every
stream to have data,
+ // AT_LEAST_ONE_NON_EMPTY runs as soon as one stream has data, and ALWAYS
runs even when every
+ // stream is empty (e.g. templates relying on constant mappings alone).
+ //
+ if (shouldSkipExecution(
+ data.receivedRows, data.hasEmptyList,
meta.getConditionStreamBehavior())) {
setOutputDone();
return false;
}
@@ -405,9 +431,17 @@ public class MetaInject extends
BaseTransform<MetaInjectMeta, MetaInjectData> {
}
List<RowMetaAndData> sourceRows =
data.rowMap.get(mapping.getSourceTransformName());
+ // A mapping may target a legacy injection key (e.g.
ALLOW_EMPTY_STREAM_ON_EXECUTION) that
+ // before the property was renamed mapped to a @HopMetadataProperty field.
Resolve it up front
+ // so that lookupGroupKey and isTopLevelInjectionKey below, which test
membership against the
+ // current keys, see the normalized key. collectInjectionKeyValue
re-resolves internally, which
+ // is idempotent.
+ //
+ String targetAttributeKey = resolveInjectionKey(mapping);
+
// To which group does this belong?
//
- String groupKey = lookupGroupKey(injectionKeyGroupMap,
mapping.getTargetAttributeKey());
+ String groupKey = lookupGroupKey(injectionKeyGroupMap, targetAttributeKey);
if (groupKey != null) {
if (sourceRows != null && !sourceRows.isEmpty()) {
// We need to collect the data for the mapping
@@ -424,15 +458,14 @@ public class MetaInject extends
BaseTransform<MetaInjectMeta, MetaInjectData> {
// Detail flag is often used for grouped (list) keys. Scalar
HopMetadataProperty keys (e.g.
// ExecSqlRow UPDATE_STATS) are not part of any injection group; treat
them as simple keys.
//
- if (HopMetadataInjector.isTopLevelInjectionKey(
- targetMetaClass, mapping.getTargetAttributeKey())) {
+ if (HopMetadataInjector.isTopLevelInjectionKey(targetMetaClass,
targetAttributeKey)) {
collectInjectionKeyValue(this, mapping, sourceRows, injectionKeyData);
} else {
logBasic(
BaseMessages.getString(
PKG,
"MetaInject.TargetKeyNotFound.Warning",
- mapping.getTargetAttributeKey(),
+ targetAttributeKey,
targetTransformName));
}
} else {
@@ -604,7 +637,8 @@ public class MetaInject extends
BaseTransform<MetaInjectMeta, MetaInjectData> {
// Resolve variables in the constant, just like the legacy
newInjectionConstants() path.
//
injectionKeyData.put(
- mapping.getTargetAttributeKey(),
variables.resolve(mapping.getSourceField()));
+ resolveInjectionKey(mapping),
+ resolveInjectionValue(mapping,
variables.resolve(mapping.getSourceField())));
}
return;
}
@@ -622,7 +656,55 @@ public class MetaInject extends
BaseTransform<MetaInjectMeta, MetaInjectData> {
//
IValueMeta valueMeta = sourceRow.getRowMeta().getValueMeta(index);
Object valueData = valueMeta.getNativeDataType(sourceRow.getData()[index]);
- injectionKeyData.put(mapping.getTargetAttributeKey(), valueData);
+ injectionKeyData.put(resolveInjectionKey(mapping),
resolveInjectionValue(mapping, valueData));
+ }
+
+ /**
+ * Returns the injection key under which the mapped value must be stored.
The legacy {@code
+ * ALLOW_EMPTY_STREAM_ON_EXECUTION} target key, used by metadata-injection
mappings written before
+ * {@link ConditionStreamBehavior} existed, is translated to its modern
{@code
+ * CONDITION_STREAM_BEHAVIOR} counterpart. Any other key is returned
unchanged.
+ *
+ * <p>Resolving up front keeps every key comparison consistent, not just the
injection itself:
+ * {@code collectDataForOneMapping} (group and top-level lookups), {@code
+ * getUnavailableTargetKeys} (whether a saved key still maps to a property),
the dialog tree
+ * matching, and the enter-mapping target-row lookup all run against the
normalized key, so a
+ * legacy mapping is neither reported missing, offered for removal, nor
dropped on confirm.
+ *
+ * @param mapping the mapping being injected
+ * @return the normalized injection (target attribute) key
+ */
+ protected static String resolveInjectionKey(MetaInjectMapping mapping) {
+ if
(LEGACY_ALLOW_EMPTY_STREAM_INJECTION_KEY.equals(mapping.getTargetAttributeKey()))
{
+ return CONDITION_STREAM_BEHAVIOR_INJECTION_KEY;
+ }
+ return mapping.getTargetAttributeKey();
+ }
+
+ /**
+ * Returns the value to store under the {@link
#resolveInjectionKey(MetaInjectMapping) resolved}
+ * injection key. When the mapping ultimately targets {@link
+ * #CONDITION_STREAM_BEHAVIOR_INJECTION_KEY} and the injected value is a
legacy boolean token
+ * ({@code Y}/{@code N}/{@code true}/{@code false}), the boolean is mapped
to the equivalent
+ * {@link ConditionStreamBehavior} enum code so that the generic metadata
injector can resolve it
+ * ({@code Y}/{@code true} to {@code at-least-one-non-empty}, anything else
to {@code
+ * all-non-empty}). This covers both the mapping that still uses the old
{@code
+ * ALLOW_EMPTY_STREAM_ON_EXECUTION} key (remapped by {@link
+ * #resolveInjectionKey(MetaInjectMapping)}) and a mapping already renamed
to the new key but
+ * still fed legacy boolean values. The current enum codes and any other
value are returned
+ * unchanged.
+ *
+ * @param mapping the mapping being injected
+ * @param value the raw injected value
+ * @return the normalized value to store
+ */
+ private static Object resolveInjectionValue(MetaInjectMapping mapping,
Object value) {
+ if (value != null
+ &&
CONDITION_STREAM_BEHAVIOR_INJECTION_KEY.equals(resolveInjectionKey(mapping))
+ && ConditionStreamBehavior.isLegacyBoolean(value.toString())) {
+ return
ConditionStreamBehavior.fromLegacyBoolean(value.toString()).getCode();
+ }
+ return value;
}
HopTransformException putRows(IRowSet rowSet, RowProducer rowProducer) {
@@ -919,7 +1001,7 @@ public class MetaInject extends
BaseTransform<MetaInjectMeta, MetaInjectData> {
for (MetaInjectMapping mapping : mappings) {
if
(!unavailableTargetTransforms.contains(mapping.getTargetTransformName())) {
BeanInjectionInfo<?> info =
beanInfos.get(mapping.getTargetTransformName().toUpperCase());
- if (info != null &&
!info.getProperties().containsKey(mapping.getTargetAttributeKey())) {
+ if (info != null &&
!info.getProperties().containsKey(resolveInjectionKey(mapping))) {
missingKeys.add(mapping);
}
}
diff --git
a/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectDialog.java
b/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectDialog.java
index 6db0eda5f1..9c22b0c510 100644
---
a/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectDialog.java
+++
b/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectDialog.java
@@ -137,9 +137,10 @@ public class MetaInjectDialog extends BaseTransformDialog {
private Button wInheritAll;
- // allow to run execution if previous stream data are empty
+ // Controls under which stream conditions the template pipeline is executed.
//
- private Button wAllowEmptyStreamOnExecution;
+ private Label wlConditionStreamBehavior;
+ private CCombo wConditionStreamBehavior;
private CCombo wStreamingSourceTransform;
@@ -520,15 +521,24 @@ public class MetaInjectDialog extends BaseTransformDialog
{
fdNoExecution.top = new FormAttachment(wStreamingTargetTransform, 10);
wNoExecution.setLayoutData(fdNoExecution);
- wAllowEmptyStreamOnExecution = new Button(wOptionsComp, SWT.CHECK);
- wAllowEmptyStreamOnExecution.setText(
- BaseMessages.getString(PKG,
"MetaInjectDialog.AllowEmptyStreamOnExecution.Label"));
- PropsUi.setLook(wAllowEmptyStreamOnExecution);
- FormData fdAllowEmptyStreamOnExecution = new FormData();
- fdAllowEmptyStreamOnExecution.width = 350;
- fdAllowEmptyStreamOnExecution.left = new FormAttachment(0, 0);
- fdAllowEmptyStreamOnExecution.top = new FormAttachment(wNoExecution, 10);
- wAllowEmptyStreamOnExecution.setLayoutData(fdAllowEmptyStreamOnExecution);
+ wlConditionStreamBehavior = new Label(wOptionsComp, SWT.RIGHT);
+ wlConditionStreamBehavior.setText(
+ BaseMessages.getString(PKG,
"MetaInjectDialog.ConditionStreamBehavior.Label"));
+ PropsUi.setLook(wlConditionStreamBehavior);
+ FormData fdlConditionStreamBehavior = new FormData();
+ fdlConditionStreamBehavior.left = new FormAttachment(0, 0);
+ fdlConditionStreamBehavior.top = new FormAttachment(wNoExecution, 15);
+ wlConditionStreamBehavior.setLayoutData(fdlConditionStreamBehavior);
+
+ wConditionStreamBehavior =
+ new CCombo(wOptionsComp, SWT.SINGLE | SWT.LEFT | SWT.BORDER |
SWT.READ_ONLY);
+ PropsUi.setLook(wConditionStreamBehavior);
+ FormData fdConditionStreamBehavior = new FormData();
+ fdConditionStreamBehavior.right = new FormAttachment(100, 0);
+ fdConditionStreamBehavior.left = new FormAttachment(0, 0);
+ fdConditionStreamBehavior.top = new
FormAttachment(wlConditionStreamBehavior, 5);
+ wConditionStreamBehavior.setLayoutData(fdConditionStreamBehavior);
+
wConditionStreamBehavior.setItems(ConditionStreamBehavior.getDescriptions());
FormData fdOptionsComp = new FormData();
fdOptionsComp.left = new FormAttachment(0, 0);
@@ -996,7 +1006,10 @@ public class MetaInjectDialog extends BaseTransformDialog
{
wTargetFile.setText(Const.NVL(metaInjectMeta.getTargetFile(), ""));
wCreateParentFolder.setSelection(metaInjectMeta.isCreateParentFolder());
wNoExecution.setSelection(!metaInjectMeta.isNoExecution());
-
wAllowEmptyStreamOnExecution.setSelection(metaInjectMeta.isAllowEmptyStreamOnExecution());
+ wConditionStreamBehavior.setText(
+ ConditionStreamBehavior.lookupDescription(
+ metaInjectMeta.getConditionStreamBehavior().getDescription())
+ .getDescription());
wStreamingSourceTransform.setText(Const.NVL(metaInjectMeta.getStreamSourceTransformName(),
""));
wStreamingTargetTransform.setText(Const.NVL(metaInjectMeta.getStreamTargetTransformName(),
""));
@@ -1092,11 +1105,15 @@ public class MetaInjectDialog extends
BaseTransformDialog {
*/
private static MetaInjectMapping findMappingForTargetKey(
List<MetaInjectMapping> mappings, MetaInjectMapping template) {
+ // Both keys are normalized so a legacy injection key (e.g.
ALLOW_EMPTY_STREAM_ON_EXECUTION)
+ // that was renamed (to CONDITION_STREAM_BEHAVIOR) still matches the
current tree row; the
+ // mapping's source then stays visible and is not offered for removal.
+ //
for (MetaInjectMapping m : mappings) {
if (Const.NVL(m.getTargetTransformName(), "")
.equalsIgnoreCase(Const.NVL(template.getTargetTransformName(),
""))
- && Const.NVL(m.getTargetAttributeKey(), "")
- .equalsIgnoreCase(Const.NVL(template.getTargetAttributeKey(),
""))) {
+ && MetaInject.resolveInjectionKey(m)
+ .equalsIgnoreCase(MetaInject.resolveInjectionKey(template))) {
return m;
}
}
@@ -1239,7 +1256,8 @@ public class MetaInjectDialog extends BaseTransformDialog
{
meta.setTargetFile(wTargetFile.getText());
meta.setCreateParentFolder(wCreateParentFolder.getSelection());
meta.setNoExecution(!wNoExecution.getSelection());
-
meta.setAllowEmptyStreamOnExecution(wAllowEmptyStreamOnExecution.getSelection());
+ meta.setConditionStreamBehavior(
+
ConditionStreamBehavior.lookupDescription(wConditionStreamBehavior.getText()));
meta.setStreamSourceTransformName(wStreamingSourceTransform.getText());
meta.setStreamTargetTransformName(wStreamingTargetTransform.getText());
@@ -1271,6 +1289,21 @@ public class MetaInjectDialog extends
BaseTransformDialog {
}
}
+ /**
+ * Returns the normalized target attribute key under which {@code
enterMapping} must look up a
+ * saved mapping in the target rows. A mapping written before the {@code
+ * ALLOW_EMPTY_STREAM_ON_EXECUTION} injection key was renamed to {@code
CONDITION_STREAM_BEHAVIOR}
+ * still carries the legacy key; normalizing it (via {@link
MetaInject#resolveInjectionKey}) lets
+ * the target-row match succeed, so the mapping is pre-selected in the
mapping dialog and survives
+ * a confirm.
+ *
+ * @param mapping the saved mapping being matched
+ * @return the injection key to compare against the target rows
+ */
+ static String resolveTargetAttributeKey(MetaInjectMapping mapping) {
+ return MetaInject.resolveInjectionKey(mapping);
+ }
+
private record MappingTarget(TransformMeta transformMeta, String
attributeKey, boolean detail) {
@Override
public boolean equals(Object o) {
@@ -1389,7 +1422,7 @@ public class MetaInjectDialog extends BaseTransformDialog
{
if (targetTransformMeta != null) {
MappingTarget mapingTarget =
new MappingTarget(
- targetTransformMeta, mapping.getTargetAttributeKey(),
mapping.isTargetDetail());
+ targetTransformMeta, resolveTargetAttributeKey(mapping),
mapping.isTargetDetail());
targetIndex = mappingTargets.indexOf(mapingTarget);
}
if (sourceIndex >= 0 && targetIndex >= 0) {
diff --git
a/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectMeta.java
b/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectMeta.java
index ab1cfcac97..59f5f7dccb 100644
---
a/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectMeta.java
+++
b/plugins/transforms/metainject/src/main/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectMeta.java
@@ -38,6 +38,8 @@ import org.apache.hop.core.xml.XmlHandler;
import org.apache.hop.i18n.BaseMessages;
import org.apache.hop.metadata.api.HopMetadataProperty;
import org.apache.hop.metadata.api.IHopMetadataProvider;
+import org.apache.hop.metadata.api.OnUnknownDeserializationProperty;
+import org.apache.hop.metadata.api.UnknownProperty;
import org.apache.hop.pipeline.ISubPipelineAwareMeta;
import org.apache.hop.pipeline.PipelineHopMeta;
import org.apache.hop.pipeline.PipelineMeta;
@@ -105,10 +107,32 @@ public class MetaInjectMeta extends
BaseTransformMeta<MetaInject, MetaInjectData
private boolean noExecution;
@HopMetadataProperty(
- key = "allow_empty_stream_on_execution",
- injectionKey = "ALLOW_EMPTY_STREAM_ON_EXECUTION",
- injectionKeyDescription =
"MetaInject.Injection.ALLOW_EMPTY_STREAM_ON_EXECUTION")
- private boolean allowEmptyStreamOnExecution;
+ key = "condition_stream_behavior",
+ injectionKey = "CONDITION_STREAM_BEHAVIOR",
+ injectionKeyDescription =
"MetaInject.Injection.CONDITION_STREAM_BEHAVIOR",
+ storeWithCode = true)
+ private ConditionStreamBehavior conditionStreamBehavior =
ConditionStreamBehavior.ALL_NON_EMPTY;
+
+ /**
+ * True once the {@code condition_stream_behavior} value has been
deserialized (or set) on this
+ * instance. Used to decide whether a legacy {@code
allow_empty_stream_on_execution} value, which
+ * the serializer hands to the {@code @OnUnknownDeserializationProperty}
callback, may still be
+ * applied: the current enum wins whenever it is present, regardless of key
order. This flag is
+ * intentionally not a {@link HopMetadataProperty}, so it is never
serialized.
+ */
+ private transient boolean conditionStreamBehaviorSet;
+
+ /**
+ * Custom setter, overriding the Lombok-generated one, so that the {@code
+ * conditionStreamBehaviorSet} marker is raised whenever the value is set
(notably during
+ * deserialization).
+ *
+ * @param conditionStreamBehavior the behavior to set
+ */
+ public void setConditionStreamBehavior(ConditionStreamBehavior
conditionStreamBehavior) {
+ this.conditionStreamBehavior = conditionStreamBehavior;
+ this.conditionStreamBehaviorSet = true;
+ }
@HopMetadataProperty(
key = "stream_source_transform",
@@ -169,6 +193,26 @@ public class MetaInjectMeta extends
BaseTransformMeta<MetaInject, MetaInjectData
}
}
+ /**
+ * Reads the legacy {@code allow_empty_stream_on_execution} boolean property.
+ *
+ * <p>Before the {@link ConditionStreamBehavior} enum existed, this
transform stored a plain
+ * boolean under the tag {@code allow_empty_stream_on_execution} (as {@code
Y}/{@code N} in XML
+ * and {@code true}/{@code false} in JSON metadata). That tag no longer maps
to an annotated
+ * field, so this callback is fired by the serializer whenever it is still
found, translating the
+ * old value into the equivalent {@link ConditionStreamBehavior}.
+ */
+ @OnUnknownDeserializationProperty
+ public void onUnknownDeserializationProperty(UnknownProperty property) {
+ // Only translate the legacy boolean when the current
condition_stream_behavior enum was not
+ // part of the file. When both are present the current enum wins, so a
file that mixes the two
+ // can still represent ALWAYS.
+ if ("allow_empty_stream_on_execution".equals(property.getName())
+ && !conditionStreamBehaviorSet) {
+ conditionStreamBehavior =
ConditionStreamBehavior.fromLegacyBoolean(property.getValue());
+ }
+ }
+
@Override
public void getFields(
IRowMeta rowMeta,
diff --git
a/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_en_US.properties
b/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_en_US.properties
index 1ec16bd61b..5ab72d9fe2 100644
---
a/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_en_US.properties
+++
b/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_en_US.properties
@@ -40,6 +40,7 @@ MetaInject.Injection.STREAMING_SOURCE_TRANSFORM=The name of
the transform to use
MetaInject.Injection.STREAMING_TARGET_TRANSFORM=The name of the transform to
use for the target stream.
MetaInject.Injection.TARGET_FILE=The name of file where Hop will save the
pipeline template after injecting values.
MetaInject.Injection.ALLOW_EMPTY_STREAM_ON_EXECUTION=Allow empty stream on
execution?
+MetaInject.Injection.CONDITION_STREAM_BEHAVIOR=When to run the resulting
pipeline based on the input streams
MetaInject.Injection.CREATE_PARENT_FOLDER=Create parent folder?
MetaInject.Injection.RUN_CONFIGURATION=Run configuration
MetaInject.Log.ParentFolderCreated=Folder [{0}] was created.
@@ -77,7 +78,10 @@ MetaInjectDialog.InjectTab.TabTitle=Inject Metadata
MetaInjectDialog.InvalidMapping.Question=There are missing metadata injection
mappings. Do you want to remove them ?
MetaInjectDialog.InvalidMapping.Title=Invalid mapping
MetaInjectDialog.NoExecution.Label=Run resulting pipeline
-MetaInjectDialog.AllowEmptyStreamOnExecution.Label=Allow empty stream to run
the resulting pipeline
+MetaInjectDialog.ConditionStreamBehavior.Label=When to run the resulting
pipeline
+ConditionStreamBehavior.AllNonEmpty.Description=Run only when every input
stream is not empty
+ConditionStreamBehavior.AtLeastOneNonEmpty.Description=Run when at least one
input stream is not empty
+ConditionStreamBehavior.Always.Description=Run even when every input stream is
empty
MetaInjectDialog.OptionsTab.TabTitle=Options
MetaInjectDialog.Parameters.Title=Parameters
MetaInjectDialog.Parameters.Tooltip=Values handed to the template pipeline
diff --git
a/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_fr_FR.properties
b/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_fr_FR.properties
index abf076f407..489cc976e7 100644
---
a/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_fr_FR.properties
+++
b/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_fr_FR.properties
@@ -29,7 +29,10 @@ MetaInjectDialog.ErrorLoadingPipeline.DialogTitle=Erreur
lors du chargement du p
MetaInjectDialog.ErrorLoadingSpecifiedPipeline.Message=Erreur lors du
chargement et de la v\u00E9rification du pipeline s\u00E9lectionn\u00E9e.
MetaInjectDialog.ErrorLoadingSpecifiedPipeline.Title=Erreur
MetaInjectDialog.NoExecution.Label=Ex\u00E9cuter la pipeline r\u00E9sultante
-MetaInjectDialog.AllowEmptyStreamOnExecution.Label=Autoriser les streams vides
pour lancer l''ex\u00E9cution de la pipeline r\u00E9sultante
+MetaInjectDialog.ConditionStreamBehavior.Label=Quand ex\u00E9cuter la pipeline
r\u00E9sultante
+ConditionStreamBehavior.AllNonEmpty.Description=Ex\u00E9cuter seulement quand
chaque flux d''entr\u00E9e n''est pas vide
+ConditionStreamBehavior.AtLeastOneNonEmpty.Description=Ex\u00E9cuter quand au
moins un flux d''entr\u00E9e n''est pas vide
+ConditionStreamBehavior.Always.Description=Ex\u00E9cuter m\u00EAme quand
chaque flux d''entr\u00E9e est vide
MetaInjectDialog.Pipeline.Label=Pipeline
MetaInjectDialog.Shell.Title=Injection de m\u00E9tadonn\u00E9es
MetaInjectDialog.SourceFieldDialog.Title=Champ source
diff --git
a/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_pt_BR.properties
b/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_pt_BR.properties
index f6e7b23c6e..51e49592c3 100644
---
a/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_pt_BR.properties
+++
b/plugins/transforms/metainject/src/main/resources/org/apache/hop/pipeline/transforms/metainject/messages/messages_pt_BR.properties
@@ -81,7 +81,10 @@ MetaInjectDialog.InjectTab.FilterString.ToolTip=Procure e
devolva todos os itens
MetaInjectDialog.InvalidMapping.Question=Faltam mapas de injeção de metadados.
Você quer removê-los?
MetaInjectDialog.InvalidMapping.Title=Mapeamento inválido
MetaInjectDialog.NoExecution.Label=Correr o conduto resultante
-MetaInjectDialog.AllowEmptyStreamOnExecution.Label=Permita que o fluxo vazio
execute o conduto resultante.
+MetaInjectDialog.ConditionStreamBehavior.Label=Quando correr o conduto
resultante
+ConditionStreamBehavior.AllNonEmpty.Description=Correr apenas quando todos os
fluxos de entrada n\u00E3o est\u00E3o vazios
+ConditionStreamBehavior.AtLeastOneNonEmpty.Description=Correr quando pelo
menos um fluxo de entrada n\u00E3o est\u00E1 vazio
+ConditionStreamBehavior.Always.Description=Correr mesmo quando todos os fluxos
de entrada est\u00E3o vazios
MetaInjectDialog.RunConfiguration.Label=Executar configuração (opcional)
MetaInjectDialog.Shell.Title=Injeção de metadados de ETL
MetaInjectDialog.SourceTransform.Label=Transformar para ler de:
@@ -96,6 +99,7 @@ MetaInjectDialog.FilenameMissing.Message=O nome do modelo
está ausente.
MetaInjectDialog.SelfReference.Message=Este conduto não pode se injetar. Por
favor, selecione outro conduto modelo.
MetaInject.Injection.MAPPING_TARGET_DETAIL=O indicador de detalhe
MetaInject.Injection.ALLOW_EMPTY_STREAM_ON_EXECUTION=Permitir fluxo vazio na
execução ?
+MetaInject.Injection.CONDITION_STREAM_BEHAVIOR=Quando correr o conduto
resultante com base nos fluxos de entrada
MetaInject.Injection.CREATE_PARENT_FOLDER=Criar pasta superior ?
MetaInject.Injection.RUN_CONFIGURATION=Executar configuração
MetaInject.TargetKeyNotFound.Warning=ATENÇÃO: A chave do alvo de injeção '''''
não foi encontrada na transformação '''' e foi ignorada. Pode ter sido removida
em uma nova versão do Apache Hop.
diff --git
a/plugins/transforms/metainject/src/test/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectMetaTest.java
b/plugins/transforms/metainject/src/test/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectMetaTest.java
index 0e9f893e8e..0f6cc8655c 100644
---
a/plugins/transforms/metainject/src/test/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectMetaTest.java
+++
b/plugins/transforms/metainject/src/test/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectMetaTest.java
@@ -41,14 +41,21 @@ import org.apache.hop.core.row.value.ValueMetaPlugin;
import org.apache.hop.core.row.value.ValueMetaPluginType;
import org.apache.hop.core.row.value.ValueMetaString;
import org.apache.hop.core.variables.IVariables;
+import org.apache.hop.core.xml.XmlHandler;
import org.apache.hop.metadata.api.IHopMetadataProvider;
+import org.apache.hop.metadata.api.UnknownProperty;
import org.apache.hop.metadata.inject.HopMetadataInjector;
+import org.apache.hop.metadata.serializer.memory.MemoryMetadataProvider;
+import org.apache.hop.metadata.serializer.xml.XmlMetadataUtil;
import org.apache.hop.pipeline.PipelineMeta;
import org.apache.hop.pipeline.transform.TransformSerializationTestUtil;
import org.apache.hop.resource.IResourceNaming;
import org.apache.hop.resource.ResourceDefinition;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
+import org.w3c.dom.Node;
class MetaInjectMetaTest {
@@ -86,7 +93,7 @@ class MetaInjectMetaTest {
assertEquals("${java.io.tmpdir}/json-input.hpl", meta.getTargetFile());
assertTrue(meta.isCreateParentFolder());
assertTrue(meta.isNoExecution());
- assertFalse(meta.isAllowEmptyStreamOnExecution());
+ assertEquals(ConditionStreamBehavior.ALL_NON_EMPTY,
meta.getConditionStreamBehavior());
assertEquals("sourceTransform", meta.getSourceTransformName());
assertEquals("targetTransform", meta.getStreamTargetTransformName());
@@ -170,4 +177,156 @@ class MetaInjectMetaTest {
assertTrue(mappingKeys.contains("MAPPING_TARGET_FIELD"));
assertTrue(mappingKeys.contains("MAPPING_TARGET_DETAIL"));
}
+
+ private static MetaInjectMeta xmlRoundTrip(MetaInjectMeta meta) throws
Exception {
+ IHopMetadataProvider provider = new MemoryMetadataProvider();
+ Node node =
+ XmlHandler.loadXmlString(XmlHandler.aroundTag("transform",
meta.getXml()), "transform");
+ return XmlMetadataUtil.deSerializeFromXml(node, MetaInjectMeta.class,
provider);
+ }
+
+ @Test
+ void roundTripConditionStreamBehaviorNewCodes() throws Exception {
+ MetaInjectMeta meta = new MetaInjectMeta();
+ meta.setConditionStreamBehavior(ConditionStreamBehavior.ALWAYS);
+ assertEquals(ConditionStreamBehavior.ALWAYS,
xmlRoundTrip(meta).getConditionStreamBehavior());
+
+
meta.setConditionStreamBehavior(ConditionStreamBehavior.AT_LEAST_ONE_NON_EMPTY);
+ assertEquals(
+ ConditionStreamBehavior.AT_LEAST_ONE_NON_EMPTY,
+ xmlRoundTrip(meta).getConditionStreamBehavior());
+
+ meta.setConditionStreamBehavior(ConditionStreamBehavior.ALL_NON_EMPTY);
+ assertEquals(
+ ConditionStreamBehavior.ALL_NON_EMPTY,
xmlRoundTrip(meta).getConditionStreamBehavior());
+ }
+
+ @Test
+ void newSerializationWritesOnlyNewEnumField() throws Exception {
+ MetaInjectMeta meta = new MetaInjectMeta();
+ meta.setConditionStreamBehavior(ConditionStreamBehavior.ALWAYS);
+ String xml = meta.getXml();
+
+ // New files carry only the new enum tag, never the legacy boolean tag.
+
assertTrue(xml.contains("<condition_stream_behavior>always</condition_stream_behavior>"));
+ assertFalse(xml.contains("allow_empty_stream_on_execution"));
+ }
+
+ @Test
+ void legacyXmlYNMapping() throws Exception {
+ IHopMetadataProvider provider = new MemoryMetadataProvider();
+
+ // Legacy Y => enabled the checkbox => at least one non-empty stream.
+ Node nodeY =
+ XmlHandler.loadXmlString(
+ XmlHandler.aroundTag(
+ "transform",
XmlHandler.addTagValue("allow_empty_stream_on_execution", "Y")),
+ "transform");
+ MetaInjectMeta metaY =
+ XmlMetadataUtil.deSerializeFromXml(nodeY, MetaInjectMeta.class,
provider);
+ assertEquals(
+ ConditionStreamBehavior.AT_LEAST_ONE_NON_EMPTY,
metaY.getConditionStreamBehavior());
+
+ // Legacy N => checkbox disabled => every stream must be non-empty.
+ Node nodeN =
+ XmlHandler.loadXmlString(
+ XmlHandler.aroundTag(
+ "transform",
XmlHandler.addTagValue("allow_empty_stream_on_execution", "N")),
+ "transform");
+ MetaInjectMeta metaN =
+ XmlMetadataUtil.deSerializeFromXml(nodeN, MetaInjectMeta.class,
provider);
+ assertEquals(ConditionStreamBehavior.ALL_NON_EMPTY,
metaN.getConditionStreamBehavior());
+ }
+
+ @Test
+ void missingTagDefaultsToAllNonEmpty() throws Exception {
+ IHopMetadataProvider provider = new MemoryMetadataProvider();
+ // A pre-legacy transform with neither the old tag nor the new enum field.
+ Node node = XmlHandler.loadXmlString(XmlHandler.aroundTag("transform",
""), "transform");
+ MetaInjectMeta meta = XmlMetadataUtil.deSerializeFromXml(node,
MetaInjectMeta.class, provider);
+ assertEquals(ConditionStreamBehavior.ALL_NON_EMPTY,
meta.getConditionStreamBehavior());
+ }
+
+ /**
+ * When a file carries both the legacy {@code
allow_empty_stream_on_execution} tag and the current
+ * {@code condition_stream_behavior} tag - regardless of key order - the
current enum wins, so a
+ * mixed file can still represent {@link ConditionStreamBehavior#ALWAYS}.
The legacy value must
+ * never overwrite an explicit current value.
+ */
+ @ParameterizedTest
+ @CsvSource({
+ // newCode, legacyValue
+ "always, Y",
+ "always, N",
+ "at-least-one-non-empty, N",
+ "all-non-empty, Y"
+ })
+ void whenBothTagsPresentCurrentEnumWins(String newCode, String legacyValue)
throws Exception {
+ IHopMetadataProvider provider = new MemoryMetadataProvider();
+
+ // Legacy tag first, then the current enum tag.
+ String legacyFirst =
+ XmlHandler.aroundTag(
+ "transform",
+ XmlHandler.addTagValue("allow_empty_stream_on_execution",
legacyValue)
+ + XmlHandler.addTagValue("condition_stream_behavior",
newCode));
+ MetaInjectMeta loadedLegacyFirst =
+ XmlMetadataUtil.deSerializeFromXml(
+ XmlHandler.loadXmlString(legacyFirst, "transform"),
MetaInjectMeta.class, provider);
+
+ // Current enum tag first, then the legacy tag.
+ String enumFirst =
+ XmlHandler.aroundTag(
+ "transform",
+ XmlHandler.addTagValue("condition_stream_behavior", newCode)
+ + XmlHandler.addTagValue("allow_empty_stream_on_execution",
legacyValue));
+ MetaInjectMeta loadedEnumFirst =
+ XmlMetadataUtil.deSerializeFromXml(
+ XmlHandler.loadXmlString(enumFirst, "transform"),
MetaInjectMeta.class, provider);
+
+ // In both orders the current enum value is preserved.
+ assertEquals(
+ ConditionStreamBehavior.lookupCode(newCode),
+ loadedLegacyFirst.getConditionStreamBehavior());
+ assertEquals(
+ ConditionStreamBehavior.lookupCode(newCode),
loadedEnumFirst.getConditionStreamBehavior());
+ }
+
+ /**
+ * The legacy {@code allow_empty_stream_on_execution} key must translate to
the enum regardless of
+ * the deserialization flavor. The generic JSON/XML serializers fire
+ * {@code @OnUnknownDeserializationProperty} (verified for JSON in the core
{@code
+ * UnknownDeserializationPropertyTest}); here we exercise the
metainject-specific mapping logic,
+ * the {@code onUnknownDeserializationProperty} callback, directly.
+ */
+ @ParameterizedTest
+ @CsvSource({
+ // legacyValue, expectedEnum
+ "Y, AT_LEAST_ONE_NON_EMPTY",
+ "true, AT_LEAST_ONE_NON_EMPTY",
+ "N, ALL_NON_EMPTY",
+ "false, ALL_NON_EMPTY"
+ })
+ void legacyKeyMapsToEnum(String legacyValue, String expectedEnum) {
+ MetaInjectMeta meta = new MetaInjectMeta();
+ meta.onUnknownDeserializationProperty(
+ new UnknownProperty("allow_empty_stream_on_execution", legacyValue));
+
+ assertEquals(ConditionStreamBehavior.valueOf(expectedEnum),
meta.getConditionStreamBehavior());
+ }
+
+ /**
+ * A pre-legacy configuration carries neither the old {@code
allow_empty_stream_on_execution} key
+ * nor the current {@code condition_stream_behavior} key; the constructor
default {@link
+ * ConditionStreamBehavior#ALL_NON_EMPTY} must be preserved.
+ */
+ @Test
+ void withoutEitherKeyDefaultsToAllNonEmpty() {
+ MetaInjectMeta meta = new MetaInjectMeta();
+
+ assertEquals(ConditionStreamBehavior.ALL_NON_EMPTY,
meta.getConditionStreamBehavior());
+ // No legacy value was applied by the callback for any key.
+ meta.onUnknownDeserializationProperty(new
UnknownProperty("some_other_legacy", "anything"));
+ assertEquals(ConditionStreamBehavior.ALL_NON_EMPTY,
meta.getConditionStreamBehavior());
+ }
}
diff --git
a/plugins/transforms/metainject/src/test/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectTest.java
b/plugins/transforms/metainject/src/test/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectTest.java
index 7689f0926b..6e3e78e086 100644
---
a/plugins/transforms/metainject/src/test/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectTest.java
+++
b/plugins/transforms/metainject/src/test/java/org/apache/hop/pipeline/transforms/metainject/MetaInjectTest.java
@@ -19,6 +19,7 @@ package org.apache.hop.pipeline.transforms.metainject;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
@@ -45,6 +46,7 @@ import org.apache.hop.core.row.RowMeta;
import org.apache.hop.core.row.value.ValueMetaString;
import org.apache.hop.core.variables.Variables;
import org.apache.hop.metadata.api.IHopMetadataProvider;
+import org.apache.hop.metadata.inject.HopMetadataInjector;
import org.apache.hop.pipeline.Pipeline;
import org.apache.hop.pipeline.PipelineMeta;
import org.apache.hop.pipeline.engines.local.LocalPipelineEngine;
@@ -53,6 +55,8 @@ import org.apache.hop.pipeline.transform.ITransformMeta;
import org.apache.hop.pipeline.transform.TransformMeta;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
import org.mockito.Mockito;
class MetaInjectTest {
@@ -191,6 +195,272 @@ class MetaInjectTest {
assertEquals("resolved-value", injectionKeyData.get("SOME_KEY"));
}
+ /**
+ * A metadata-injection mapping written before {@link
ConditionStreamBehavior} existed targets the
+ * legacy {@code ALLOW_EMPTY_STREAM_ON_EXECUTION} injection key and feeds it
a boolean-ish value.
+ * Such a mapping must still work: the key is remapped to {@code
CONDITION_STREAM_BEHAVIOR} and
+ * the boolean is translated to the equivalent enum code (as seen by {@link
+ * ConditionStreamBehavior#fromLegacyBoolean}).
+ *
+ * <p>This verifies the constant-injection path (no source transform).
+ */
+ @ParameterizedTest
+ @CsvSource({
+ // legacy boolean value -> expected enum code under
CONDITION_STREAM_BEHAVIOR
+ "Y, at-least-one-non-empty",
+ "true, at-least-one-non-empty",
+ "TRUE, at-least-one-non-empty",
+ "yes, at-least-one-non-empty",
+ "N, all-non-empty",
+ "false, all-non-empty"
+ })
+ void legacyInjectionKeyConstantRemappedToEnumCode(String legacyValue, String
expectedCode)
+ throws Exception {
+ Variables variables = new Variables();
+
+ MetaInjectMapping mapping = new MetaInjectMapping();
+ mapping.setTargetAttributeKey("ALLOW_EMPTY_STREAM_ON_EXECUTION");
+ mapping.setSourceTransformName(null);
+ mapping.setSourceField(legacyValue);
+
+ Map<String, Object> injectionKeyData = new HashMap<>();
+ MetaInject.collectInjectionKeyValue(variables, mapping, null,
injectionKeyData);
+
+ // The legacy key is remapped to the current one, carrying the enum code.
+ assertNull(injectionKeyData.get("ALLOW_EMPTY_STREAM_ON_EXECUTION"));
+ assertEquals(expectedCode,
injectionKeyData.get("CONDITION_STREAM_BEHAVIOR"));
+ }
+
+ /**
+ * Same backward-compatibility remapping as {@link
+ * #legacyInjectionKeyConstantRemappedToEnumCode(String, String)} but for
the streamed path, where
+ * the value arrives as a typed boolean in a source row.
+ */
+ @ParameterizedTest
+ @CsvSource({"true, at-least-one-non-empty", "false, all-non-empty"})
+ void legacyInjectionKeyStreamedRemappedToEnumCode(String legacyValue, String
expectedCode)
+ throws Exception {
+ Variables variables = new Variables();
+
+ MetaInjectMapping mapping = new MetaInjectMapping();
+ mapping.setTargetAttributeKey("ALLOW_EMPTY_STREAM_ON_EXECUTION");
+ mapping.setSourceTransformName(TEST_SOURCE_TRANSFORM_NAME);
+ mapping.setSourceField(TEST_FIELD);
+
+ IRowMeta rowMeta = new RowMeta();
+ rowMeta.addValueMeta(new ValueMetaString(TEST_FIELD));
+ RowMetaAndData sourceRow = new RowMetaAndData(rowMeta, legacyValue);
+ List<RowMetaAndData> sourceRows = Collections.singletonList(sourceRow);
+
+ Map<String, Object> injectionKeyData = new HashMap<>();
+ MetaInject.collectInjectionKeyValue(variables, mapping, sourceRows,
injectionKeyData);
+
+ assertNull(injectionKeyData.get("ALLOW_EMPTY_STREAM_ON_EXECUTION"));
+ assertEquals(expectedCode,
injectionKeyData.get("CONDITION_STREAM_BEHAVIOR"));
+ }
+
+ /**
+ * End-to-end backward-compatibility check: a mapping that injects the
legacy {@code
+ * ALLOW_EMPTY_STREAM_ON_EXECUTION} key into a real {@link MetaInjectMeta}
target produces the
+ * expected {@link ConditionStreamBehavior}. This mirrors what {@code
+ * injectMetadataIntoTemplateTransform} does via {@link
HopMetadataInjector#inject}.
+ */
+ @ParameterizedTest
+ @CsvSource({
+ "Y, AT_LEAST_ONE_NON_EMPTY",
+ "true, AT_LEAST_ONE_NON_EMPTY",
+ "N, ALL_NON_EMPTY",
+ "false, ALL_NON_EMPTY"
+ })
+ void legacyInjectionKeyIntoMetaInjectMetaSetsEnum(String legacyValue, String
expectedEnum)
+ throws Exception {
+ MetaInjectMeta target = new MetaInjectMeta();
+
+ MetaInjectMapping mapping = new MetaInjectMapping();
+ mapping.setTargetAttributeKey("ALLOW_EMPTY_STREAM_ON_EXECUTION");
+ mapping.setSourceTransformName(null);
+ mapping.setSourceField(legacyValue);
+
+ Map<String, Object> injectionKeyData = new HashMap<>();
+ MetaInject.collectInjectionKeyValue(new Variables(), mapping, null,
injectionKeyData);
+
+ HopMetadataInjector.inject(null, target, injectionKeyData, new
HashMap<>());
+
+ assertEquals(
+ ConditionStreamBehavior.valueOf(expectedEnum),
target.getConditionStreamBehavior());
+ }
+
+ /**
+ * The reviewer's second legacy scenario: the mapping key was already
renamed to the new {@code
+ * CONDITION_STREAM_BEHAVIOR} but the upstream source still flows a legacy
boolean value ({@code
+ * Y}/{@code N}/{@code true}/{@code false}). Such a value used to be
rejected by the generic
+ * {@code convertCodeToEnumeration}; it must now be translated to the enum
code so the injection
+ * still lands on the right enum. This is the constant (no source transform)
variant.
+ */
+ @ParameterizedTest
+ @CsvSource({
+ "Y, AT_LEAST_ONE_NON_EMPTY",
+ "true, AT_LEAST_ONE_NON_EMPTY",
+ "N, ALL_NON_EMPTY",
+ "false, ALL_NON_EMPTY"
+ })
+ void newKeyWithLegacyBooleanValueStillInjectsEnum(String legacyValue, String
expectedEnum)
+ throws Exception {
+ MetaInjectMeta target = new MetaInjectMeta();
+
+ MetaInjectMapping mapping = new MetaInjectMapping();
+ mapping.setTargetAttributeKey("CONDITION_STREAM_BEHAVIOR");
+ mapping.setSourceTransformName(null);
+ mapping.setSourceField(legacyValue);
+
+ Map<String, Object> injectionKeyData = new HashMap<>();
+ MetaInject.collectInjectionKeyValue(new Variables(), mapping, null,
injectionKeyData);
+
+ HopMetadataInjector.inject(null, target, injectionKeyData, new
HashMap<>());
+
+ assertEquals(
+ ConditionStreamBehavior.valueOf(expectedEnum),
target.getConditionStreamBehavior());
+ }
+
+ /**
+ * Full runtime data-flow backward-compatibility check: a running pipeline
whose MetaInject source
+ * transforms produce rows, and whose mapping targets the legacy {@code
+ * ALLOW_EMPTY_STREAM_ON_EXECUTION} injection key, must still inject the
value into the template's
+ * MetaInjectMeta. The flowing source rows sit in {@code data.rowMap} and
are picked up by {@code
+ * collectDataForOneMapping} exactly like {@code
injectMetadataIntoTemplateTransform} does; the
+ * legacy key is remapped so {@code HopMetadataInjector.inject} lands on the
current enum.
+ */
+ @ParameterizedTest
+ @CsvSource({
+ // streamedLegacyValue, expectedEnum
+ "Y, AT_LEAST_ONE_NON_EMPTY",
+ "true, AT_LEAST_ONE_NON_EMPTY",
+ "N, ALL_NON_EMPTY",
+ "false, ALL_NON_EMPTY"
+ })
+ void legacyInjectionKeyFlowsThroughPipelineData(String streamedLegacyValue,
String expectedEnum)
+ throws Exception {
+ // A real source transform feeds one flowing row: the legacy boolean value
in "legacyFlag".
+ IRowMeta rowMeta = new RowMeta();
+ rowMeta.addValueMeta(new ValueMetaString("legacyFlag"));
+ RowMetaAndData sourceRow = new RowMetaAndData(rowMeta,
streamedLegacyValue);
+ metaInject.getData().rowMap = new HashMap<>();
+ metaInject
+ .getData()
+ .rowMap
+ .put(TEST_SOURCE_TRANSFORM_NAME, Collections.singletonList(sourceRow));
+
+ // The pipeline mapping still targets the legacy injection key that
existed before the enum.
+ MetaInjectMapping mapping = new MetaInjectMapping();
+ mapping.setTargetTransformName(TEST_TARGET_TRANSFORM_NAME);
+ mapping.setTargetAttributeKey("ALLOW_EMPTY_STREAM_ON_EXECUTION");
+ mapping.setSourceTransformName(TEST_SOURCE_TRANSFORM_NAME);
+ mapping.setSourceField("legacyFlag");
+
when(metaInject.getMeta().getMappings()).thenReturn(Collections.singletonList(mapping));
+
+ // This mirrors the head of injectMetadataIntoTemplateTransform: collect
then inject.
+ MetaInjectMeta target = new MetaInjectMeta();
+ Map<String, Set<String>> injectionKeyGroupMap =
+ HopMetadataInjector.findInjectionGroupKeys(MetaInjectMeta.class);
+ Map<String, Object> injectionKeyData = new HashMap<>();
+ metaInject.collectDataForOneMapping(
+ TEST_TARGET_TRANSFORM_NAME,
+ mapping,
+ injectionKeyGroupMap,
+ new HashMap<>(),
+ new HashMap<>(),
+ injectionKeyData,
+ MetaInjectMeta.class);
+
+ assertNull(injectionKeyData.get("ALLOW_EMPTY_STREAM_ON_EXECUTION"));
+ assertEquals("CONDITION_STREAM_BEHAVIOR",
injectionKeyData.keySet().iterator().next());
+
+ HopMetadataInjector.inject(null, target, injectionKeyData, new
HashMap<>());
+
+ assertEquals(
+ ConditionStreamBehavior.valueOf(expectedEnum),
target.getConditionStreamBehavior());
+ }
+
+ /**
+ * A saved mapping that still targets the legacy {@code
ALLOW_EMPTY_STREAM_ON_EXECUTION} injection
+ * key must NOT be reported as missing by {@link
MetaInject#getUnavailableTargetKeys}. That method
+ * consults the {@code BeanInjectionInfo} property map for {@code
CONDITION_STREAM_BEHAVIOR}; the
+ * legacy key is resolved to it, so the dialog neither hides the mapping nor
offers to delete it.
+ */
+ @Test
+ void getUnavailableTargetKeys_legacyKeyNotReportedMissing() {
+ // The injected pipeline holds a single MetaInject transform, so the
legacy injection key is
+ // looked up against its current CONDITION_STREAM_BEHAVIOR property.
+ MetaInjectMeta targetMeta = new MetaInjectMeta();
+ TransformMeta targetTransform = new
TransformMeta(TEST_TARGET_TRANSFORM_NAME, targetMeta);
+ PipelineMeta injectedPipeline = new PipelineMeta();
+ injectedPipeline.addTransform(targetTransform);
+
+ MetaInjectMapping legacy = new MetaInjectMapping();
+ legacy.setTargetTransformName(TEST_TARGET_TRANSFORM_NAME);
+ legacy.setTargetAttributeKey("ALLOW_EMPTY_STREAM_ON_EXECUTION"); //
pre-enum key
+ legacy.setSourceTransformName(null);
+ legacy.setSourceField("Y");
+
+ Set<MetaInjectMapping> missing =
+ MetaInject.getUnavailableTargetKeys(
+ Collections.singletonList(legacy), injectedPipeline,
Collections.emptySet());
+
+ // The legacy key resolves to the current property, so nothing is reported
missing.
+ assertTrue(missing.isEmpty());
+ }
+
+ /**
+ * Older pipelines may store a scalar key with {@code target_detail=Y}.
{@link
+ * MetaInject#collectDataForOneMapping} must resolve the legacy {@code
+ * ALLOW_EMPTY_STREAM_ON_EXECUTION} key before {@code
isTopLevelInjectionKey}, otherwise the
+ * target_detail branch logs {@code TargetKeyNotFound} and leaves the
template on {@code
+ * ALL_NON_EMPTY}. With the fix the value is collected under {@code
CONDITION_STREAM_BEHAVIOR}.
+ */
+ @ParameterizedTest
+ @CsvSource({
+ // streamedLegacyValue, expectedEnum
+ "Y, AT_LEAST_ONE_NON_EMPTY",
+ "N, ALL_NON_EMPTY"
+ })
+ void collectDataForOneMapping_legacyKeyTargetDetailY_collects(
+ String streamedLegacyValue, String expectedEnum) throws Exception {
+ IRowMeta rowMeta = new RowMeta();
+ rowMeta.addValueMeta(new ValueMetaString("legacyFlag"));
+ RowMetaAndData sourceRow = new RowMetaAndData(rowMeta,
streamedLegacyValue);
+ metaInject.getData().rowMap = new HashMap<>();
+ metaInject
+ .getData()
+ .rowMap
+ .put(TEST_SOURCE_TRANSFORM_NAME, Collections.singletonList(sourceRow));
+
+ MetaInjectMapping mapping = new MetaInjectMapping();
+ mapping.setTargetTransformName(TEST_TARGET_TRANSFORM_NAME);
+ mapping.setTargetAttributeKey("ALLOW_EMPTY_STREAM_ON_EXECUTION");
+ mapping.setTargetDetail(true); // older pipeline stores a scalar key with
target_detail=Y
+ mapping.setSourceTransformName(TEST_SOURCE_TRANSFORM_NAME);
+ mapping.setSourceField("legacyFlag");
+
+ Map<String, Set<String>> injectionKeyGroupMap =
+ HopMetadataInjector.findInjectionGroupKeys(MetaInjectMeta.class);
+ Map<String, Object> injectionKeyData = new HashMap<>();
+ metaInject.collectDataForOneMapping(
+ TEST_TARGET_TRANSFORM_NAME,
+ mapping,
+ injectionKeyGroupMap,
+ new HashMap<>(),
+ new HashMap<>(),
+ injectionKeyData,
+ MetaInjectMeta.class);
+
+ // The target_detail=Y scalar path must land on CONDITION_STREAM_BEHAVIOR,
not be warned away.
+ assertNull(injectionKeyData.get("ALLOW_EMPTY_STREAM_ON_EXECUTION"));
+ assertEquals("CONDITION_STREAM_BEHAVIOR",
injectionKeyData.keySet().iterator().next());
+ assertEquals(
+ ConditionStreamBehavior.valueOf(expectedEnum).getCode(),
+ injectionKeyData.get("CONDITION_STREAM_BEHAVIOR"));
+ }
+
/**
* Same regression as {@link
#collectInjectionKeyValue_resolvesVariablesInConstant()} but for a
* constant that belongs to an injection group (list) key.
@@ -251,6 +521,38 @@ class MetaInjectTest {
assertTrue(injectionGroupData.isEmpty());
}
+ /**
+ * A mapping saved before the {@code ALLOW_EMPTY_STREAM_ON_EXECUTION}
injection key was renamed
+ * still stores the legacy key. When {@code enterMapping} pre-selects it in
the target rows it
+ * must normalize that key (via {@link
MetaInjectDialog#resolveTargetAttributeKey}) to {@code
+ * CONDITION_STREAM_BEHAVIOR}, the key {@code BeanInjectionInfo} reports for
the current target
+ * row; otherwise {@code targetIndex == -1} and confirming the dialog drops
a mapping execution
+ * would have honored.
+ */
+ @Test
+ void resolveTargetAttributeKey_legacyKeyNormalizedToCurrentRowKey() {
+ MetaInjectMapping legacy = new MetaInjectMapping();
+ legacy.setTargetAttributeKey("ALLOW_EMPTY_STREAM_ON_EXECUTION");
+
+ // The normalized key is the one the target rows are built from, so the
lookup matches.
+ assertEquals("CONDITION_STREAM_BEHAVIOR",
MetaInjectDialog.resolveTargetAttributeKey(legacy));
+ }
+
+ /**
+ * Same normalization check but the other direction: a mapping that already
stores the current
+ * key, or any ordinary target key, is left untouched so non-legacy keys
still resolve exactly.
+ */
+ @Test
+ void resolveTargetAttributeKey_currentOrOtherKeysUnchanged() {
+ MetaInjectMapping current = new MetaInjectMapping();
+ current.setTargetAttributeKey("CONDITION_STREAM_BEHAVIOR");
+ assertEquals("CONDITION_STREAM_BEHAVIOR",
MetaInjectDialog.resolveTargetAttributeKey(current));
+
+ MetaInjectMapping ordinary = new MetaInjectMapping();
+ ordinary.setTargetAttributeKey("SOME_KEY");
+ assertEquals("SOME_KEY",
MetaInjectDialog.resolveTargetAttributeKey(ordinary));
+ }
+
/**
* Regression test for <a
href="https://github.com/apache/hop/issues/7246";>#7246</a>: when an
* injection group mixes a streamed key and a constant key, the constant
value must be merged into
@@ -364,6 +666,35 @@ class MetaInjectTest {
assertEquals("renamed2", groupBuffer.getBuffer().get(1)[1]);
}
+ @Test
+ void shouldSkipExecution_allNonEmpty() {
+ // ALL_NON_EMPTY: run only when every stream has data.
+ assertFalse(MetaInject.shouldSkipExecution(true, false,
ConditionStreamBehavior.ALL_NON_EMPTY));
+ assertTrue(MetaInject.shouldSkipExecution(false, true,
ConditionStreamBehavior.ALL_NON_EMPTY));
+ assertTrue(MetaInject.shouldSkipExecution(true, true,
ConditionStreamBehavior.ALL_NON_EMPTY));
+ }
+
+ @Test
+ void shouldSkipExecution_atLeastOneNonEmpty() {
+ // AT_LEAST_ONE_NON_EMPTY: run when at least one stream has data.
+ assertFalse(
+ MetaInject.shouldSkipExecution(
+ true, false, ConditionStreamBehavior.AT_LEAST_ONE_NON_EMPTY));
+ assertFalse(
+ MetaInject.shouldSkipExecution(true, true,
ConditionStreamBehavior.AT_LEAST_ONE_NON_EMPTY));
+ assertTrue(
+ MetaInject.shouldSkipExecution(
+ false, true, ConditionStreamBehavior.AT_LEAST_ONE_NON_EMPTY));
+ }
+
+ @Test
+ void shouldSkipExecution_always() {
+ // ALWAYS: run even when every stream is empty.
+ assertFalse(MetaInject.shouldSkipExecution(false, true,
ConditionStreamBehavior.ALWAYS));
+ assertFalse(MetaInject.shouldSkipExecution(true, true,
ConditionStreamBehavior.ALWAYS));
+ assertFalse(MetaInject.shouldSkipExecution(true, false,
ConditionStreamBehavior.ALWAYS));
+ }
+
@InjectionSupported(localizationPrefix = "", groups = "groups")
private static class InjectableTestTransformMeta
extends BaseTransformMeta<MetaInject, MetaInjectData> {
