This is an automated email from the ASF dual-hosted git repository.
mattcasters pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new 78596e0dfa Issue #8295 : Do not report a failed connection lookup as a
missing c… (#8301)
78596e0dfa is described below
commit 78596e0dfa9750d76ca979a07348351dee40dbe0
Author: Bart Maertens <[email protected]>
AuthorDate: Thu Sep 10 12:00:04 2026 +0200
Issue #8295 : Do not report a failed connection lookup as a missing c…
(#8301)
* Issue #8295 : Do not report a failed connection lookup as a missing
connection
The linter reported CONNECTION_DOES_NOT_EXIST for database connections that
were present in the project metadata all along.
ReferencedDatabaseConnectionChecker asked the metadata serializer whether a
connection existed and reported "does not exist" when that lookup *failed*
as
well as when it answered no. JsonMetadataSerializer.exists() throws whenever
VFS cannot reach the metadata folder, so one unreachable folder turned into
a
warning on every connection in the project.
There are now three distinct outcomes:
- the connection is verified absent : CONNECTION_DOES_NOT_EXIST, at warning
- the lookup failed, so nothing is known : CONNECTION_NOT_VERIFIED, at info,
naming the root cause on a single line
- whether the database can actually be reached is still never checked; this
is an existence check that never opens a JDBC connection
Staying silent on a failed lookup was not an option either: an unreadable
metadata folder would then produce a clean report that means nothing.
Because the save-time validator prompts on any remark, it now prompts only
at
warning level and above. Otherwise unreadable metadata would put its "save
anyway?" dialog in front of every save.
Two related hardenings:
- HopVfs falls back when the VFS namespace bound to a thread has been closed
underneath it. A closed DefaultFileSystemManager has dropped its
providers,
the local one included, so it reports an absolute path as "a relative
path,
and no base URI was provided". Executions take a namespace and release it,
and the binding is inherited by threads created while it is held, so the
linter's pool threads can outlive it. With one tenant the process wide
manager is used; with several the namespace builds its own connections
again rather than borrow another tenant's.
- ProjectsGuiPlugin switches the VFS namespace before firing
HopGuiProjectAfterEnabled, so a listener that starts background work is
not
handed a file system manager that is about to be closed.
Tested:
- ReferencedDatabaseConnectionCheckerTest: a lookup that throws is reported
as
CONNECTION_NOT_VERIFIED at info and its reason stays on one readable line,
while a genuinely missing connection still warns.
- ReferencedConnectionSaveValidatorTest: a connection that could not be
checked
does not interrupt a save, a missing one still does, and a real problem
still
gets through when mixed with unverifiable ones.
- HopVfsClosedNamespaceTest: an absolute local path still resolves after the
inherited namespace was closed, with and without variables, and a
namespace
is rebuilt rather than the process wide manager borrowed when tenants
share
the JVM.
- integration-tests/lint: a new IT project runs hop lint over two workflows
with a Check DB Connections action. Connections that exist produce no
CONNECTION_DOES_NOT_EXIST finding; a connection that is absent still
produces exactly one. Verified with run-tests-docker.sh PROJECT_NAME=lint.
- Hop GUI: the info finding renders correctly in the Problems tab, a save is
not interrupted by it, and a workflow whose connections all exist stays
clean.
* Issue #8295 : Address review feedback on startsWithScheme and test output
---------
Co-authored-by: mattcasters <[email protected]>
---
.gitignore | 1 +
.../main/java/org/apache/hop/core/vfs/HopVfs.java | 86 +++++-
.../hop/core/vfs/HopVfsClosedNamespaceTest.java | 136 ++++++++++
.../ReferencedDatabaseConnectionChecker.java | 61 ++++-
.../validation/messages/messages_en_US.properties | 1 +
.../ReferencedDatabaseConnectionCheckerTest.java | 77 ++++++
integration-tests/lint/dev-env-config.json | 9 +
integration-tests/lint/hop-config.json | 290 +++++++++++++++++++++
.../lint/main-0001-lint-check-db-connections.hwf | 221 ++++++++++++++++
.../metadata/pipeline-run-configuration/local.json | 17 ++
integration-tests/lint/metadata/rdbms/CRM.json | 26 ++
integration-tests/lint/metadata/rdbms/OPS.json | 26 ++
integration-tests/lint/metadata/rdbms/Vault.json | 26 ++
.../metadata/workflow-run-configuration/local.json | 9 +
integration-tests/lint/project-config.json | 13 +
.../lint/subject/existing-connections.hwf | 91 +++++++
.../lint/subject/missing-connection.hwf | 81 ++++++
.../apache/hop/projects/gui/ProjectsGuiPlugin.java | 15 +-
.../shared/ReferencedConnectionSaveValidator.java | 25 +-
.../ReferencedConnectionSaveValidatorTest.java | 95 +++++++
20 files changed, 1287 insertions(+), 19 deletions(-)
diff --git a/.gitignore b/.gitignore
index 64a232ef9b..752dd3ff69 100644
--- a/.gitignore
+++ b/.gitignore
@@ -72,6 +72,7 @@ integration-tests/http/output/
integration-tests/sftp/output/
# The PGP tests build a throwaway keyring and the files they sign under output/
integration-tests/pgp/output/
+integration-tests/lint/output/
integration-tests/spreadsheet/files/sample-file-append.xlsx
integration-tests/spreadsheet/files/sample-file-append-test.xlsx
integration-tests/spark-native/output
diff --git a/core/src/main/java/org/apache/hop/core/vfs/HopVfs.java
b/core/src/main/java/org/apache/hop/core/vfs/HopVfs.java
index 36d3a3c58f..c623c644eb 100644
--- a/core/src/main/java/org/apache/hop/core/vfs/HopVfs.java
+++ b/core/src/main/java/org/apache/hop/core/vfs/HopVfs.java
@@ -47,6 +47,7 @@ import org.apache.hop.core.Const;
import org.apache.hop.core.exception.HopException;
import org.apache.hop.core.exception.HopFileException;
import org.apache.hop.core.exception.HopRuntimeException;
+import org.apache.hop.core.logging.HopLogStore;
import org.apache.hop.core.logging.LogChannel;
import org.apache.hop.core.plugins.IPlugin;
import org.apache.hop.core.plugins.PluginRegistry;
@@ -171,13 +172,84 @@ public class HopVfs {
// namespace, with its own named VFS connections. Everything else uses the
process wide
// manager below, exactly as before. See issue #8106.
HopVfsNamespace namespace = HopVfsNamespaces.resolve(variables);
- if (namespace != null) {
- return namespace.getFileSystemManager();
+ DefaultFileSystemManager namespaceManager = managerOf(namespace);
+ if (namespaceManager != null) {
+ return namespaceManager;
}
bootstrapWith(variables);
return getFileSystemManager();
}
+ /**
+ * The file system manager of this namespace, or null when there is nothing
usable to resolve
+ * with.
+ *
+ * <p>A namespace is closed once the last user lets go of it, and a closed
{@link
+ * DefaultFileSystemManager} has dropped every provider it had - the local
one included. Handing
+ * it a path afterwards fails as {@code "because it is a relative path, and
no base URI was
+ * provided"}, even for an absolute local path, because there is no longer a
provider to claim it.
+ *
+ * <p>Whoever inherited the namespace is never told that it closed: the
binding is copied when a
+ * thread is created and never looked at again, and Hop GUI lets go of the
previous project's
+ * namespace while background work - the linter - is still running on it.
See issue #8295.
+ *
+ * <p>What to do about it depends on who else is in this JVM. With one
tenant, the process wide
+ * manager is the right answer and not merely a salvage: the work that
outlived the namespace
+ * belongs to the project that is open now, which is exactly what that
manager holds. With several
+ * tenants ({@link HopVfsNamespaces#isIsolated()}) it is the wrong answer -
the named connections
+ * on it are somebody else's - so the namespace builds its own connections
again instead. If even
+ * that fails, the closed manager is handed back and the caller gets the
error it would have got
+ * before: silently resolving one tenant's files through another's is worse
than failing.
+ *
+ * @param namespace the namespace to resolve with, may be null
+ * @return its manager, or null when the process wide manager should be used
instead
+ */
+ private static DefaultFileSystemManager managerOf(HopVfsNamespace namespace)
{
+ if (namespace == null) {
+ return null;
+ }
+ DefaultFileSystemManager manager = namespace.getFileSystemManager();
+ if (manager != null && manager.hasProvider("file")) {
+ return manager;
+ }
+
+ if (HopVfsNamespaces.isIsolated()) {
+ try {
+ namespace.rebuild();
+ } catch (Exception e) {
+ // Only the rebuild belongs in this try. Logging is what runs before
the log store exists,
+ // and a failure to say something must not be read as a failure to
rebuild.
+ if (HopLogStore.isInitialized()) {
+ LogChannel.GENERAL.logError(
+ "The VFS namespace of "
+ + namespace.getDescription()
+ + " was closed while still in use and could not be built
again. Files resolved"
+ + " through it keep failing: the process wide manager holds
the named connections"
+ + " of another tenant and is not used in its place.",
+ e);
+ }
+ return manager;
+ }
+ if (HopLogStore.isInitialized()) {
+ LogChannel.GENERAL.logBasic(
+ "The VFS namespace of "
+ + namespace.getDescription()
+ + " was closed while still in use. Its named connections were
read again.");
+ }
+ return namespace.getFileSystemManager();
+ }
+
+ // Resolving a file is one of the first things Hop does, long before there
is anywhere to log
+ // to, so never let saying this out loud be the thing that fails.
+ if (HopLogStore.isInitialized()) {
+ LogChannel.GENERAL.logDebug(
+ "The VFS namespace of "
+ + namespace.getDescription()
+ + " is closed. Resolving with the process wide file system
manager instead.");
+ }
+ return null;
+ }
+
/**
* Remember the variables to bootstrap the metadata driven providers with,
for as long as nothing
* bootstrapped them yet. These are the first variables we get to see, so
the named connections
@@ -361,11 +433,9 @@ public class HopVfs {
public static synchronized FileObject getFileObject(String vfsFilename)
throws HopFileException {
// Nothing to go on but the thread: the namespace of the execution running
on it, if any.
- HopVfsNamespace namespace = HopVfsNamespaces.getCurrent();
+ DefaultFileSystemManager namespaceManager =
managerOf(HopVfsNamespaces.getCurrent());
return resolveWith(
- vfsFilename,
- namespace == null ? getFileSystemManager() :
namespace.getFileSystemManager(),
- null);
+ vfsFilename, namespaceManager == null ? getFileSystemManager() :
namespaceManager, null);
}
/**
@@ -836,9 +906,9 @@ public class HopVfs {
*/
public static boolean startsWithScheme(String vfsFileName) {
// Nothing to go on but the thread: the namespace of the execution running
on it, if any.
- HopVfsNamespace namespace = HopVfsNamespaces.getCurrent();
+ DefaultFileSystemManager namespaceManager =
managerOf(HopVfsNamespaces.getCurrent());
return startsWithScheme(
- vfsFileName, namespace == null ? getFileSystemManager() :
namespace.getFileSystemManager());
+ vfsFileName, namespaceManager == null ? getFileSystemManager() :
namespaceManager);
}
private static boolean startsWithScheme(String vfsFileName,
DefaultFileSystemManager fsManager) {
diff --git
a/core/src/test/java/org/apache/hop/core/vfs/HopVfsClosedNamespaceTest.java
b/core/src/test/java/org/apache/hop/core/vfs/HopVfsClosedNamespaceTest.java
new file mode 100644
index 0000000000..fb55ae80c9
--- /dev/null
+++ b/core/src/test/java/org/apache/hop/core/vfs/HopVfsClosedNamespaceTest.java
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hop.core.vfs;
+
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNotSame;
+import static org.junit.jupiter.api.Assertions.assertSame;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.io.File;
+import org.apache.commons.vfs2.FileObject;
+import org.apache.commons.vfs2.impl.DefaultFileSystemManager;
+import org.apache.hop.core.exception.HopException;
+import org.apache.hop.core.scope.IHopScope;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.DisplayName;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+/**
+ * Issue #8295. A namespace is closed when the last user lets go of it -
switching project in Hop
+ * GUI does exactly that. Work already running on another thread keeps the
closed namespace, because
+ * the binding is inherited when the thread is created and is never refreshed.
+ *
+ * <p>A closed {@code DefaultFileSystemManager} has dropped its providers, the
local one included,
+ * so every path it is handed afterwards looks like a path with no scheme and
nothing to resolve it
+ * against. That is where the reported failure comes from:
+ *
+ * <pre>
+ * Could not find file with URI "/home/matt/git/.../run-retail-update.hwf"
because it is a
+ * relative path, and no base URI was provided.
+ * </pre>
+ *
+ * <p>An absolute local path is not a relative path. Resolving one must not
depend on whether some
+ * other thread has since let go of the namespace this one inherited.
+ */
+class HopVfsClosedNamespaceTest {
+
+ @AfterEach
+ void tearDown() {
+ HopVfsNamespaces.setScope(null);
+ HopVfsNamespaces.reset();
+ HopVfs.reset();
+ }
+
+ @Test
+ @DisplayName("An absolute local path still resolves after the inherited
namespace was closed")
+ void absolutePathResolvesAfterTheNamespaceWasClosed(@TempDir File tempDir)
throws Exception {
+ File file = new File(tempDir, "run-retail-update.hwf");
+ assertNotNull(file);
+
+ HopVfsNamespace namespace = new HopVfsNamespace("project being closed");
+ HopVfsNamespaces.bindThread(namespace);
+
+ // What HopGui does on the UI thread when the next project is opened: the
previous project's
+ // namespace has no users left, so it is closed. The background linter is
still running with it.
+ namespace.close();
+
+ FileObject resolved =
+ assertDoesNotThrow(
+ () -> HopVfs.getFileObject(file.getAbsolutePath()),
+ "An absolute local path must not be reported as a relative path
with no base URI");
+ assertNotNull(resolved);
+ }
+
+ @Test
+ @DisplayName("Resolving with variables also survives a closed namespace")
+ void absolutePathResolvesWithVariablesAfterTheNamespaceWasClosed(@TempDir
File tempDir)
+ throws HopException {
+ File file = new File(tempDir, "run-retail-update.hwf");
+
+ HopVfsNamespace namespace = new HopVfsNamespace("project being closed");
+ HopVfsNamespaces.bindThread(namespace);
+ namespace.close();
+
+ FileObject resolved =
+ assertDoesNotThrow(() -> HopVfs.getFileObject(file.getAbsolutePath(),
null));
+ assertNotNull(resolved);
+ }
+
+ /**
+ * Hop Web serves several tenants from one JVM, and there the process wide
manager holds somebody
+ * else's named connections. Borrowing it would resolve one tenant's files
through another's, so
+ * the namespace has to build its own connections again instead.
+ */
+ @Test
+ @DisplayName("With several tenants the namespace is rebuilt rather than the
process one borrowed")
+ void aClosedNamespaceIsRebuiltWhenTenantsShareTheJvm(@TempDir File tempDir)
throws Exception {
+ File file = new File(tempDir, "run-retail-update.hwf");
+
+ HopVfsNamespaces.setScope(IHopScope.process());
+ HopVfsNamespace namespace = new HopVfsNamespace("one session of several");
+ HopVfsNamespaces.bindThread(namespace);
+ DefaultFileSystemManager closed = namespace.getFileSystemManager();
+ namespace.close();
+
+ FileObject resolved = assertDoesNotThrow(() ->
HopVfs.getFileObject(file.getAbsolutePath()));
+ assertNotNull(resolved);
+ assertNotSame(
+ closed,
+ namespace.getFileSystemManager(),
+ "The namespace kept its closed file system manager instead of building
a new one");
+ assertSame(
+ namespace.getFileSystemManager(),
+ resolved.getFileSystem().getFileSystemManager(),
+ "The file was resolved outside this tenant's own namespace");
+ }
+
+ @Test
+ @DisplayName("startsWithScheme also survives a closed namespace")
+ void startsWithSchemeSurvivesAClosedNamespace() throws Exception {
+ HopVfsNamespace namespace = new HopVfsNamespace("project being closed");
+ HopVfsNamespaces.bindThread(namespace);
+ namespace.close();
+
+ assertTrue(
+ HopVfs.startsWithScheme("file:///tmp/test.hwf"),
+ "The scheme must still be recognized using the fallback file system
manager");
+ }
+}
diff --git
a/engine/src/main/java/org/apache/hop/metadata/validation/ReferencedDatabaseConnectionChecker.java
b/engine/src/main/java/org/apache/hop/metadata/validation/ReferencedDatabaseConnectionChecker.java
index 2d451f3e01..5058bd3f4f 100644
---
a/engine/src/main/java/org/apache/hop/metadata/validation/ReferencedDatabaseConnectionChecker.java
+++
b/engine/src/main/java/org/apache/hop/metadata/validation/ReferencedDatabaseConnectionChecker.java
@@ -22,6 +22,8 @@ import org.apache.hop.core.CheckResult;
import org.apache.hop.core.ICheckResult;
import org.apache.hop.core.ICheckResultSource;
import org.apache.hop.core.database.DatabaseMeta;
+import org.apache.hop.core.logging.HopLogStore;
+import org.apache.hop.core.logging.LogChannel;
import org.apache.hop.core.util.StringUtil;
import org.apache.hop.core.util.Utils;
import org.apache.hop.core.variables.IVariables;
@@ -49,8 +51,17 @@ public final class ReferencedDatabaseConnectionChecker {
public static final String ERROR_NOT_ASSIGNED = "CONNECTION_NOT_ASSIGNED";
public static final String ERROR_DOES_NOT_EXIST =
"CONNECTION_DOES_NOT_EXIST";
+ /**
+ * The connection could not be looked up at all, so nothing is known about
it. Reported at INFO:
+ * it says something about the metadata being unreadable, not about the file
being linted.
+ */
+ public static final String INFO_NOT_VERIFIED = "CONNECTION_NOT_VERIFIED";
+
private static final Class<?> PKG =
ReferencedDatabaseConnectionChecker.class;
+ /** How much of a failure reason fits in the problems list before it stops
being readable. */
+ private static final int MAX_REASON_LENGTH = 200;
+
private ReferencedDatabaseConnectionChecker() {}
public static List<ICheckResult> checkPipeline(
@@ -148,6 +159,31 @@ public final class ReferencedDatabaseConnectionChecker {
return remarks;
}
+ /**
+ * Why the lookup failed, in one line fit for a table cell.
+ *
+ * <p>A Hop exception carries the message of everything it wrapped, over
several lines and with
+ * the root cause repeated once per level. That reads badly in the problems
list, so this takes
+ * the root cause alone, on a single line, and caps it.
+ *
+ * @param e the failure
+ * @return a short single line reason, never null
+ */
+ private static String reason(Throwable e) {
+ Throwable root = e;
+ while (root.getCause() != null && root.getCause() != root) {
+ root = root.getCause();
+ }
+ String message = root.getMessage();
+ if (message == null || message.isBlank()) {
+ message = root.getClass().getSimpleName();
+ }
+ String oneLine = message.replaceAll("\\s+", " ").trim();
+ return oneLine.length() > MAX_REASON_LENGTH
+ ? oneLine.substring(0, MAX_REASON_LENGTH - 3) + "..."
+ : oneLine;
+ }
+
private static ICheckResult checkConnectionName(
String rawName,
String ownerKind,
@@ -182,15 +218,32 @@ public final class ReferencedDatabaseConnectionChecker {
return null;
}
} catch (Exception e) {
+ // Reaching the metadata failed, which says nothing about whether this
connection is in it.
+ // Reporting it as missing turns one unreachable metadata folder into a
warning on every
+ // connection in the project - see issue #8295. Saying nothing at all
would be worse still:
+ // an unreadable metadata folder would produce a clean report that means
nothing. So say
+ // what is actually known - that the connection could not be checked -
at INFO.
+ if (HopLogStore.isInitialized()) {
+ LogChannel.GENERAL.logDebug(
+ "Unable to look up database connection '"
+ + resolved
+ + "' referenced by "
+ + ownerKind
+ + " '"
+ + ownerName
+ + "' : "
+ + e.getMessage());
+ }
return new CheckResult(
- ICheckResult.TYPE_RESULT_WARNING,
- ERROR_DOES_NOT_EXIST,
+ ICheckResult.TYPE_RESULT_COMMENT,
+ INFO_NOT_VERIFIED,
BaseMessages.getString(
PKG,
- "ReferencedDatabaseConnectionChecker.DoesNotExist",
+ "ReferencedDatabaseConnectionChecker.NotVerified",
resolved,
ownerKind,
- ownerName),
+ ownerName,
+ reason(e)),
source);
}
diff --git
a/engine/src/main/resources/org/apache/hop/metadata/validation/messages/messages_en_US.properties
b/engine/src/main/resources/org/apache/hop/metadata/validation/messages/messages_en_US.properties
index 73b2d5a586..42879bb199 100644
---
a/engine/src/main/resources/org/apache/hop/metadata/validation/messages/messages_en_US.properties
+++
b/engine/src/main/resources/org/apache/hop/metadata/validation/messages/messages_en_US.properties
@@ -18,3 +18,4 @@ ReferencedDatabaseConnectionChecker.Kind.Transform=Transform
ReferencedDatabaseConnectionChecker.Kind.Action=Action
ReferencedDatabaseConnectionChecker.NotAssigned=No database connection is
assigned on {0} ''{1}''
ReferencedDatabaseConnectionChecker.DoesNotExist=Database connection ''{0}''
assigned on {1} ''{2}'' does not exist
+ReferencedDatabaseConnectionChecker.NotVerified=Database connection ''{0}''
assigned on {1} ''{2}'' could not be checked: the project metadata could not be
read ({3})
diff --git
a/engine/src/test/java/org/apache/hop/metadata/validation/ReferencedDatabaseConnectionCheckerTest.java
b/engine/src/test/java/org/apache/hop/metadata/validation/ReferencedDatabaseConnectionCheckerTest.java
index 590c6d2d4b..af06745537 100644
---
a/engine/src/test/java/org/apache/hop/metadata/validation/ReferencedDatabaseConnectionCheckerTest.java
+++
b/engine/src/test/java/org/apache/hop/metadata/validation/ReferencedDatabaseConnectionCheckerTest.java
@@ -17,6 +17,7 @@
package org.apache.hop.metadata.validation;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.mock;
@@ -25,6 +26,7 @@ import static org.mockito.Mockito.when;
import java.util.List;
import org.apache.hop.core.ICheckResult;
import org.apache.hop.core.database.DatabaseMeta;
+import org.apache.hop.core.exception.HopException;
import org.apache.hop.core.variables.Variables;
import org.apache.hop.metadata.api.HopMetadataProperty;
import org.apache.hop.metadata.api.HopMetadataPropertyType;
@@ -227,6 +229,81 @@ class ReferencedDatabaseConnectionCheckerTest {
ReferencedDatabaseConnectionChecker.ERROR_DOES_NOT_EXIST,
remarks.get(0).getErrorCode());
}
+ /**
+ * Issue #8295. The GUI linter reported CONNECTION_DOES_NOT_EXIST for
connections that were in the
+ * project metadata all along. It ran on a background thread whose VFS file
system manager had
+ * been closed underneath it, so {@code JsonMetadataSerializer.exists()}
threw instead of
+ * answering - and a failed lookup was reported as a missing connection.
+ *
+ * <p>An error reaching the metadata says nothing about whether the object
is there. Reporting it
+ * as missing turns every transient metadata problem into a warning on every
connection in the
+ * project. Saying nothing at all is no better: an unreadable metadata
folder would then produce a
+ * clean report. So it is reported for what it is, at INFO.
+ */
+ @Test
+ @SuppressWarnings("unchecked")
+ void aFailedLookupIsReportedAsUnverifiedRatherThanMissing() throws Exception
{
+ IHopMetadataProvider throwingProvider = mock(IHopMetadataProvider.class);
+ IHopMetadataSerializer<DatabaseMeta> throwingSerializer =
mock(IHopMetadataSerializer.class);
+
when(throwingProvider.getSerializer(DatabaseMeta.class)).thenReturn(throwingSerializer);
+ when(throwingSerializer.exists(anyString()))
+ .thenThrow(
+ new HopException(
+ "Unable to get VFS File object for filename "
+ + "'/project/metadata/rdbms/OPS.json' : Could not find
file with URI ... "
+ + "because it is a relative path, and no base URI was
provided."));
+
+ List<ICheckResult> remarks =
+ ReferencedDatabaseConnectionChecker.checkObject(
+ new ConnMeta("OPS"),
+ "Action",
+ "Check DB connections",
+ null,
+ variables,
+ throwingProvider);
+
+ assertEquals(1, remarks.size());
+ ICheckResult remark = remarks.get(0);
+ assertEquals(
+ ReferencedDatabaseConnectionChecker.INFO_NOT_VERIFIED,
+ remark.getErrorCode(),
+ "A lookup that fails is not evidence the connection is missing");
+ assertEquals(
+ ICheckResult.TYPE_RESULT_COMMENT,
+ remark.getType(),
+ "Being unable to read the metadata is not a defect in the file being
checked");
+ assertTrue(remark.getText().contains("OPS"));
+ assertTrue(remark.getText().contains("could not be checked"));
+ }
+
+ /**
+ * A Hop exception repeats the message of everything it wrapped, over
several lines. The problems
+ * list is a table, so the reason has to survive as one readable line.
+ */
+ @Test
+ @SuppressWarnings("unchecked")
+ void theReasonForAFailedLookupIsOneReadableLine() throws Exception {
+ IHopMetadataProvider throwingProvider = mock(IHopMetadataProvider.class);
+ IHopMetadataSerializer<DatabaseMeta> throwingSerializer =
mock(IHopMetadataSerializer.class);
+
when(throwingProvider.getSerializer(DatabaseMeta.class)).thenReturn(throwingSerializer);
+ when(throwingSerializer.exists(anyString()))
+ .thenThrow(
+ new HopException(
+ "\n\nUnable to get VFS File object for filename 'x' :"
+ + " Invalid URI escape sequence.\nInvalid URI escape
sequence.\n",
+ new IllegalStateException("Invalid URI escape sequence
\"%te\".")));
+
+ List<ICheckResult> remarks =
+ ReferencedDatabaseConnectionChecker.checkObject(
+ new ConnMeta("OPS"), "Action", "Check DB", null, variables,
throwingProvider);
+
+ String text = remarks.get(0).getText();
+ assertFalse(text.contains("\n"), "The reason must not wrap over several
lines: " + text);
+ assertTrue(
+ text.contains("Invalid URI escape sequence \"%te\"."),
+ "The root cause is what says why it failed: " + text);
+ }
+
/** A Dummy transform that also carries an annotated connection name, for
pipeline-level tests. */
static class ConnTransformMeta extends DummyMeta {
@HopMetadataProperty(hopMetadataPropertyType =
HopMetadataPropertyType.RDBMS_CONNECTION)
diff --git a/integration-tests/lint/dev-env-config.json
b/integration-tests/lint/dev-env-config.json
new file mode 100644
index 0000000000..a1ca51bc29
--- /dev/null
+++ b/integration-tests/lint/dev-env-config.json
@@ -0,0 +1,9 @@
+{
+ "variables": [
+ {
+ "name": "SAMPLE",
+ "value": "sampleValue",
+ "description": "A sample variable value"
+ }
+ ]
+}
\ No newline at end of file
diff --git a/integration-tests/lint/hop-config.json
b/integration-tests/lint/hop-config.json
new file mode 100644
index 0000000000..d9e1e6562e
--- /dev/null
+++ b/integration-tests/lint/hop-config.json
@@ -0,0 +1,290 @@
+{
+ "variables": [
+ {
+ "name": "HOP_LENIENT_STRING_TO_NUMBER_CONVERSION",
+ "value": "N",
+ "description": "System wide flag to allow lenient string to number
conversion for backward compatibility. If this setting is set to \"Y\", an
string starting with digits will be converted successfully into a number.
(example: 192.168.1.1 will be converted into 192 or 192.168 or 192168 depending
on the decimal and grouping symbol). The default (N) will be to throw an error
if non-numeric symbols are found in the string."
+ },
+ {
+ "name": "HOP_COMPATIBILITY_DB_IGNORE_TIMEZONE",
+ "value": "N",
+ "description": "System wide flag to ignore timezone while writing
date/timestamp value to the database."
+ },
+ {
+ "name": "HOP_LOG_SIZE_LIMIT",
+ "value": "0",
+ "description": "The log size limit for all pipelines and workflows that
don't have the \"log size limit\" property set in their respective properties."
+ },
+ {
+ "name": "HOP_EMPTY_STRING_DIFFERS_FROM_NULL",
+ "value": "N",
+ "description": "NULL vs Empty String. If this setting is set to Y, an
empty string and null are different. Otherwise they are not."
+ },
+ {
+ "name": "HOP_MAX_LOG_SIZE_IN_LINES",
+ "value": "0",
+ "description": "The maximum number of log lines that are kept internally
by Hop. Set to 0 to keep all rows (default)"
+ },
+ {
+ "name": "HOP_MAX_LOG_TIMEOUT_IN_MINUTES",
+ "value": "1440",
+ "description": "The maximum age (in minutes) of a log line while being
kept internally by Hop. Set to 0 to keep all rows indefinitely (default)"
+ },
+ {
+ "name": "HOP_MAX_WORKFLOW_TRACKER_SIZE",
+ "value": "5000",
+ "description": "The maximum number of workflow trackers kept in memory"
+ },
+ {
+ "name": "HOP_MAX_ACTIONS_LOGGED",
+ "value": "5000",
+ "description": "The maximum number of action results kept in memory for
logging purposes."
+ },
+ {
+ "name": "HOP_MAX_LOGGING_REGISTRY_SIZE",
+ "value": "10000",
+ "description": "The maximum number of logging registry entries kept in
memory for logging purposes."
+ },
+ {
+ "name": "HOP_LOG_TAB_REFRESH_DELAY",
+ "value": "1000",
+ "description": "The hop log tab refresh delay."
+ },
+ {
+ "name": "HOP_LOG_TAB_REFRESH_PERIOD",
+ "value": "1000",
+ "description": "The hop log tab refresh period."
+ },
+ {
+ "name": "HOP_PLUGIN_CLASSES",
+ "value": null,
+ "description": "A comma delimited list of classes to scan for plugin
annotations"
+ },
+ {
+ "name": "HOP_PLUGIN_PACKAGES",
+ "value": null,
+ "description": "A comma delimited list of packages to scan for plugin
annotations (warning: slow!!)"
+ },
+ {
+ "name": "HOP_TRANSFORM_PERFORMANCE_SNAPSHOT_LIMIT",
+ "value": "0",
+ "description": "The maximum number of transform performance snapshots to
keep in memory. Set to 0 to keep all snapshots indefinitely (default)"
+ },
+ {
+ "name": "HOP_ROWSET_GET_TIMEOUT",
+ "value": "50",
+ "description": "The name of the variable that optionally contains an
alternative rowset get timeout (in ms). This only makes a difference for
extremely short lived pipelines."
+ },
+ {
+ "name": "HOP_ROWSET_PUT_TIMEOUT",
+ "value": "50",
+ "description": "The name of the variable that optionally contains an
alternative rowset put timeout (in ms). This only makes a difference for
extremely short lived pipelines."
+ },
+ {
+ "name": "HOP_CORE_TRANSFORMS_FILE",
+ "value": null,
+ "description": "The name of the project variable that will contain the
alternative location of the hop-transforms.xml file. You can use this to
customize the list of available internal transforms outside of the codebase."
+ },
+ {
+ "name": "HOP_CORE_WORKFLOW_ACTIONS_FILE",
+ "value": null,
+ "description": "The name of the project variable that will contain the
alternative location of the hop-workflow-actions.xml file."
+ },
+ {
+ "name": "HOP_SERVER_OBJECT_TIMEOUT_MINUTES",
+ "value": "1440",
+ "description": "This project variable will set a time-out after which
waiting, completed or stopped pipelines and workflows will be automatically
cleaned up. The default value is 1440 (one day)."
+ },
+ {
+ "name": "HOP_PIPELINE_PAN_JVM_EXIT_CODE",
+ "value": null,
+ "description": "Set this variable to an integer that will be returned as
the Pan JVM exit code."
+ },
+ {
+ "name": "HOP_DISABLE_CONSOLE_LOGGING",
+ "value": "N",
+ "description": "Set this variable to Y to disable standard Hop logging
to the console. (stdout)"
+ },
+ {
+ "name": "HOP_REDIRECT_STDERR",
+ "value": "N",
+ "description": "Set this variable to Y to redirect stderr to Hop
logging."
+ },
+ {
+ "name": "HOP_REDIRECT_STDOUT",
+ "value": "N",
+ "description": "Set this variable to Y to redirect stdout to Hop
logging."
+ },
+ {
+ "name": "HOP_DEFAULT_NUMBER_FORMAT",
+ "value": null,
+ "description": "The name of the variable containing an alternative
default number format"
+ },
+ {
+ "name": "HOP_DEFAULT_BIGNUMBER_FORMAT",
+ "value": null,
+ "description": "The name of the variable containing an alternative
default bignumber format"
+ },
+ {
+ "name": "HOP_DEFAULT_INTEGER_FORMAT",
+ "value": null,
+ "description": "The name of the variable containing an alternative
default integer format"
+ },
+ {
+ "name": "HOP_DEFAULT_DATE_FORMAT",
+ "value": null,
+ "description": "The name of the variable containing an alternative
default date format"
+ },
+ {
+ "name": "HOP_DEFAULT_TIMESTAMP_FORMAT",
+ "value": null,
+ "description": "The name of the variable containing an alternative
default timestamp format"
+ },
+ {
+ "name": "HOP_DEFAULT_SERVLET_ENCODING",
+ "value": null,
+ "description": "Defines the default encoding for servlets, leave it
empty to use Java default encoding"
+ },
+ {
+ "name": "HOP_FAIL_ON_LOGGING_ERROR",
+ "value": "N",
+ "description": "Set this variable to Y when you want the
workflow/pipeline fail with an error when the related logging process (e.g. to
a database) fails."
+ },
+ {
+ "name": "HOP_AGGREGATION_MIN_NULL_IS_VALUED",
+ "value": "N",
+ "description": "Set this variable to Y to set the minimum to NULL if
NULL is within an aggregate. Otherwise by default NULL is ignored by the MIN
aggregate and MIN is set to the minimum value that is not NULL. See also the
variable HOP_AGGREGATION_ALL_NULLS_ARE_ZERO."
+ },
+ {
+ "name": "HOP_AGGREGATION_ALL_NULLS_ARE_ZERO",
+ "value": "N",
+ "description": "Set this variable to Y to return 0 when all values
within an aggregate are NULL. Otherwise by default a NULL is returned when all
values are NULL."
+ },
+ {
+ "name": "HOP_COMPATIBILITY_TEXT_FILE_OUTPUT_APPEND_NO_HEADER",
+ "value": "N",
+ "description": "Set this variable to Y for backward compatibility for
the Text File Output transform. Setting this to Ywill add no header row at all
when the append option is enabled, regardless if the file is existing or not."
+ },
+ {
+ "name": "HOP_PASSWORD_ENCODER_PLUGIN",
+ "value": "Hop",
+ "description": "Specifies the password encoder plugin to use by ID (Hop
is the default)."
+ },
+ {
+ "name": "HOP_SYSTEM_HOSTNAME",
+ "value": null,
+ "description": "You can use this variable to speed up hostname lookup.
Hostname lookup is performed by Hop so that it is capable of logging the server
on which a workflow or pipeline is executed."
+ },
+ {
+ "name": "HOP_SERVER_JETTY_ACCEPTORS",
+ "value": null,
+ "description": "A variable to configure jetty option: acceptors for
Carte"
+ },
+ {
+ "name": "HOP_SERVER_JETTY_ACCEPT_QUEUE_SIZE",
+ "value": null,
+ "description": "A variable to configure jetty option: acceptQueueSize
for Carte"
+ },
+ {
+ "name": "HOP_SERVER_JETTY_RES_MAX_IDLE_TIME",
+ "value": null,
+ "description": "A variable to configure jetty option:
lowResourcesMaxIdleTime for Carte"
+ },
+ {
+ "name":
"HOP_COMPATIBILITY_MERGE_ROWS_USE_REFERENCE_STREAM_WHEN_IDENTICAL",
+ "value": "N",
+ "description": "Set this variable to Y for backward compatibility for
the Merge Rows (diff) transform. Setting this to Y will use the data from the
reference stream (instead of the comparison stream) in case the compared rows
are identical."
+ },
+ {
+ "name": "HOP_SPLIT_FIELDS_REMOVE_ENCLOSURE",
+ "value": "false",
+ "description": "Set this variable to false to preserve enclosure symbol
after splitting the string in the Split fields transform. Changing it to true
will remove first and last enclosure symbol from the resulting string chunks."
+ },
+ {
+ "name": "HOP_ALLOW_EMPTY_FIELD_NAMES_AND_TYPES",
+ "value": "false",
+ "description": "Set this variable to TRUE to allow your pipeline to pass
'null' fields and/or empty types."
+ },
+ {
+ "name": "HOP_GLOBAL_LOG_VARIABLES_CLEAR_ON_EXPORT",
+ "value": "false",
+ "description": "Set this variable to false to preserve global log
variables defined in pipeline / workflow Properties -> Log panel. Changing it
to true will clear it when export pipeline / workflow."
+ },
+ {
+ "name": "HOP_FILE_OUTPUT_MAX_STREAM_COUNT",
+ "value": "1024",
+ "description": "This project variable is used by the Text File Output
transform. It defines the max number of simultaneously open files within the
transform. The transform will close/reopen files as necessary to insure the max
is not exceeded"
+ },
+ {
+ "name": "HOP_FILE_OUTPUT_MAX_STREAM_LIFE",
+ "value": "0",
+ "description": "This project variable is used by the Text File Output
transform. It defines the max number of milliseconds between flushes of files
opened by the transform."
+ },
+ {
+ "name": "HOP_USE_NATIVE_FILE_DIALOG",
+ "value": "N",
+ "description": "Set this value to Y if you want to use the system file
open/save dialog when browsing files"
+ },
+ {
+ "name": "HOP_AUTO_CREATE_CONFIG",
+ "value": "Y",
+ "description": "Set this value to N if you don't want to automatically
create a hop configuration file (hop-config.json) when it's missing"
+ }
+ ],
+ "LocaleDefault": "en_BE",
+ "guiProperties": {
+ "FontFixedSize": "13",
+ "MaxUndo": "100",
+ "DarkMode": "Y",
+ "FontNoteSize": "13",
+ "ShowOSLook": "Y",
+ "FontFixedStyle": "0",
+ "FontNoteName": ".AppleSystemUIFont",
+ "FontFixedName": "Monospaced",
+ "FontGraphStyle": "0",
+ "FontDefaultSize": "13",
+ "GraphColorR": "255",
+ "FontGraphSize": "13",
+ "IconSize": "32",
+ "BackgroundColorB": "255",
+ "FontNoteStyle": "0",
+ "FontGraphName": ".AppleSystemUIFont",
+ "FontDefaultName": ".AppleSystemUIFont",
+ "GraphColorG": "255",
+ "UseGlobalFileBookmarks": "Y",
+ "FontDefaultStyle": "0",
+ "GraphColorB": "255",
+ "BackgroundColorR": "255",
+ "BackgroundColorG": "255",
+ "WorkflowDialogStyle": "RESIZE,MAX,MIN",
+ "LineWidth": "1",
+ "ContextDialogShowCategories": "Y"
+ },
+ "projectsConfig": {
+ "enabled": true,
+ "projectMandatory": true,
+ "environmentMandatory": false,
+ "defaultProject": "default",
+ "defaultEnvironment": null,
+ "standardParentProject": "default",
+ "standardProjectsFolder": null,
+ "projectConfigurations": [
+ {
+ "projectName": "default",
+ "projectHome": "${HOP_CONFIG_FOLDER}",
+ "configFilename": "project-config.json"
+ }
+ ],
+ "lifecycleEnvironments": [
+ {
+ "name": "dev",
+ "purpose": "Testing",
+ "projectName": "default",
+ "configurationFiles": [
+ "${PROJECT_HOME}/dev-env-config.json"
+ ]
+ }
+ ],
+ "projectLifecycles": []
+ }
+}
\ No newline at end of file
diff --git a/integration-tests/lint/main-0001-lint-check-db-connections.hwf
b/integration-tests/lint/main-0001-lint-check-db-connections.hwf
new file mode 100644
index 0000000000..9dd425f8d2
--- /dev/null
+++ b/integration-tests/lint/main-0001-lint-check-db-connections.hwf
@@ -0,0 +1,221 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+-->
+<workflow>
+ <name>main-0001-lint-check-db-connections</name>
+ <name_sync_with_filename>Y</name_sync_with_filename>
+ <description>Issue #8295: the linter must not report
CONNECTION_DOES_NOT_EXIST for connections that are in the project
metadata.</description>
+ <extended_description/>
+ <workflow_version/>
+ <created_user>-</created_user>
+ <created_date>2026/09/09 08:00:00.000</created_date>
+ <modified_user>-</modified_user>
+ <modified_date>2026/09/09 08:00:00.000</modified_date>
+ <parameters>
+ </parameters>
+ <actions>
+ <action>
+ <name>Start</name>
+ <description/>
+ <type>SPECIAL</type>
+ <attributes/>
+ <DayOfMonth>1</DayOfMonth>
+ <hour>12</hour>
+ <intervalMinutes>60</intervalMinutes>
+ <intervalSeconds>0</intervalSeconds>
+ <minutes>0</minutes>
+ <repeat>N</repeat>
+ <schedulerType>0</schedulerType>
+ <weekDay>1</weekDay>
+ <parallel>N</parallel>
+ <xloc>96</xloc>
+ <yloc>96</yloc>
+ <attributes_hac/>
+ </action>
+ <action>
+ <name>Lint the Check DB connections workflows</name>
+ <description>Runs hop lint over two subject workflows and checks the
CONNECTION_DOES_NOT_EXIST findings.</description>
+ <type>SHELL</type>
+ <attributes/>
+ <filename/>
+ <work_directory>${PROJECT_HOME}</work_directory>
+ <arg_from_previous>N</arg_from_previous>
+ <exec_per_row>N</exec_per_row>
+ <set_logfile>N</set_logfile>
+ <logfile/>
+ <set_append_logfile>N</set_append_logfile>
+ <logext/>
+ <add_date>N</add_date>
+ <add_time>N</add_time>
+ <insertScript>Y</insertScript>
+ <script>#!/bin/bash
+# Issue #8295: "Invalid linter warnings in action Check DB Connections".
+#
+# The linter reported CONNECTION_DOES_NOT_EXIST for database connections that
are present in the
+# project metadata. The checker behind that rule
(ReferencedDatabaseConnectionChecker) asks the
+# metadata serializer whether the connection exists, and used to report "does
not exist" when that
+# lookup *failed* as well as when it answered no. A failed lookup is not
evidence of absence.
+#
+# Note on ${} below: Hop resolves its own variables in this script before
running it, so only Hop
+# variables are written with braces. Shell variables are written without them.
+
+set -u
+
+PROJECT_DIR="${PROJECT_HOME}"
+
+# Finding the hop CLI: run-tests.sh sets HOP_LOCATION but does not export it,
so it does not reach
+# this shell - printenv rather than $HOP_LOCATION, which would abort under
"set -u". The test image
+# puts the client on the PATH, which is what normally answers here.
+HOP_CLI="$(printenv HOP_LOCATION 2>/dev/null || true)"
+if [ -n "$HOP_CLI" ] && [ -x "$HOP_CLI/hop" ]; then
+ HOP_CLI="$HOP_CLI/hop"
+elif command -v hop > /dev/null 2>/dev/null; then
+ HOP_CLI="$(command -v hop)"
+elif [ -x /opt/hop/hop ]; then
+ HOP_CLI=/opt/hop/hop
+else
+ echo "FAIL: no hop CLI found (HOP_LOCATION is not set, hop is not on the
PATH, and there is"
+ echo " nothing at /opt/hop/hop). The linter cannot be tested without
it."
+ exit 1
+fi
+echo "Using hop CLI: $HOP_CLI"
+
+# hop lint reads the project metadata (metadata/rdbms) through the project
that HOP_CONFIG_FOLDER
+# points at, exactly like the rest of this test suite.
+export HOP_CONFIG_FOLDER="$PROJECT_DIR"
+
+RULE=CONNECTION_DOES_NOT_EXIST
+REPORT_DIR="$PROJECT_DIR/output"
+mkdir -p "$REPORT_DIR"
+
+failures=0
+
+# Run hop lint over one file and leave the JSON report in $2. hop lint exits 1
when a finding
+# reaches --fail-on (ERROR by default), so only an exit code above 1 means the
linter itself
+# failed to run.
+run_lint() {
+ subject="$1"
+ report="$2"
+ rm -f "$report"
+ "$HOP_CLI" lint "$subject" --format json --output "$report"
+ rc=$?
+ if [ $rc -gt 1 ]; then
+ echo "FAIL: hop lint could not lint $subject (exit code $rc)"
+ return 1
+ fi
+ if [ ! -s "$report" ]; then
+ echo "FAIL: hop lint wrote no report for $subject"
+ return 1
+ fi
+ return 0
+}
+
+count_rule() {
+ grep -c "$RULE" "$1" 2>/dev/null || true
+}
+
+echo "=== Subject 1: every referenced connection exists (OPS, Vault, CRM) ==="
+EXISTING_REPORT="$REPORT_DIR/lint-existing-connections.json"
+if run_lint "$PROJECT_DIR/subject/existing-connections.hwf"
"$EXISTING_REPORT"; then
+ found=$(count_rule "$EXISTING_REPORT")
+ if [ "$found" -ne 0 ]; then
+ echo "FAIL: expected no $RULE findings, got $found."
+ echo " OPS, Vault and CRM are all present in
$PROJECT_DIR/metadata/rdbms."
+ echo "----- report -----"
+ cat "$EXISTING_REPORT"
+ echo "------------------"
+ failures=$((failures + 1))
+ else
+ echo "OK: no $RULE findings for connections that exist."
+ fi
+else
+ failures=$((failures + 1))
+fi
+
+echo "=== Subject 2: the referenced connection really is absent
(NoSuchConnection) ==="
+MISSING_REPORT="$REPORT_DIR/lint-missing-connection.json"
+if run_lint "$PROJECT_DIR/subject/missing-connection.hwf" "$MISSING_REPORT";
then
+ found=$(count_rule "$MISSING_REPORT")
+ if [ "$found" -ne 1 ]; then
+ echo "FAIL: expected exactly one $RULE finding, got $found."
+ echo "----- report -----"
+ cat "$MISSING_REPORT"
+ echo "------------------"
+ failures=$((failures + 1))
+ elif ! grep -q "NoSuchConnection" "$MISSING_REPORT"; then
+ echo "FAIL: the $RULE finding does not name NoSuchConnection."
+ echo "----- report -----"
+ cat "$MISSING_REPORT"
+ echo "------------------"
+ failures=$((failures + 1))
+ else
+ echo "OK: the missing connection is still reported."
+ fi
+else
+ failures=$((failures + 1))
+fi
+
+if [ $failures -ne 0 ]; then
+ echo "$failures lint assertion(s) failed."
+ exit 1
+fi
+
+echo "All lint assertions passed."
+exit 0
+</script>
+ <loglevel>Basic</loglevel>
+ <parallel>N</parallel>
+ <nr>0</nr>
+ <xloc>320</xloc>
+ <yloc>96</yloc>
+ <attributes_hac/>
+ </action>
+ <action>
+ <name>Abort workflow</name>
+ <description/>
+ <type>ABORT</type>
+ <attributes/>
+ <always_log_rows>N</always_log_rows>
+ <message>The Hop linter reported incorrect CONNECTION_DOES_NOT_EXIST
findings. See issue #8295.</message>
+ <parallel>N</parallel>
+ <xloc>560</xloc>
+ <yloc>208</yloc>
+ <attributes_hac/>
+ </action>
+ </actions>
+ <hops>
+ <hop>
+ <from>Start</from>
+ <to>Lint the Check DB connections workflows</to>
+ <enabled>Y</enabled>
+ <evaluation>Y</evaluation>
+ <unconditional>Y</unconditional>
+ </hop>
+ <hop>
+ <from>Lint the Check DB connections workflows</from>
+ <to>Abort workflow</to>
+ <enabled>Y</enabled>
+ <evaluation>N</evaluation>
+ <unconditional>N</unconditional>
+ </hop>
+ </hops>
+ <notepads>
+ </notepads>
+ <attributes/>
+</workflow>
diff --git
a/integration-tests/lint/metadata/pipeline-run-configuration/local.json
b/integration-tests/lint/metadata/pipeline-run-configuration/local.json
new file mode 100644
index 0000000000..63794efcaf
--- /dev/null
+++ b/integration-tests/lint/metadata/pipeline-run-configuration/local.json
@@ -0,0 +1,17 @@
+{
+ "engineRunConfiguration": {
+ "Local": {
+ "feedback_size": "50000",
+ "sample_size": "100",
+ "sample_type_in_gui": "Last",
+ "rowset_size": "10000",
+ "safe_mode": false,
+ "show_feedback": false,
+ "topo_sort": false,
+ "gather_metrics": false
+ }
+ },
+ "configurationVariables": [],
+ "name": "local",
+ "description": "Runs your pipelines locally with the standard local Hop
pipeline engine"
+}
\ No newline at end of file
diff --git a/integration-tests/lint/metadata/rdbms/CRM.json
b/integration-tests/lint/metadata/rdbms/CRM.json
new file mode 100644
index 0000000000..36663e9bd4
--- /dev/null
+++ b/integration-tests/lint/metadata/rdbms/CRM.json
@@ -0,0 +1,26 @@
+{
+ "rdbms": {
+ "H2": {
+ "databaseName": "/tmp/hop-lint-it-CRM",
+ "pluginId": "H2",
+ "accessType": 0,
+ "hostname": "",
+ "password": "Encrypted ",
+ "pluginName": "H2",
+ "port": "",
+ "attributes": {
+ "SUPPORTS_TIMESTAMP_DATA_TYPE": "N",
+ "QUOTE_ALL_FIELDS": "N",
+ "SUPPORTS_BOOLEAN_DATA_TYPE": "N",
+ "FORCE_IDENTIFIERS_TO_LOWERCASE": "N",
+ "PRESERVE_RESERVED_WORD_CASE": "Y",
+ "SQL_CONNECT": "",
+ "FORCE_IDENTIFIERS_TO_UPPERCASE": "N",
+ "PREFERRED_SCHEMA_NAME": ""
+ },
+ "manualUrl": "",
+ "username": ""
+ }
+ },
+ "name": "CRM"
+}
diff --git a/integration-tests/lint/metadata/rdbms/OPS.json
b/integration-tests/lint/metadata/rdbms/OPS.json
new file mode 100644
index 0000000000..67d75b6abb
--- /dev/null
+++ b/integration-tests/lint/metadata/rdbms/OPS.json
@@ -0,0 +1,26 @@
+{
+ "rdbms": {
+ "H2": {
+ "databaseName": "/tmp/hop-lint-it-OPS",
+ "pluginId": "H2",
+ "accessType": 0,
+ "hostname": "",
+ "password": "Encrypted ",
+ "pluginName": "H2",
+ "port": "",
+ "attributes": {
+ "SUPPORTS_TIMESTAMP_DATA_TYPE": "N",
+ "QUOTE_ALL_FIELDS": "N",
+ "SUPPORTS_BOOLEAN_DATA_TYPE": "N",
+ "FORCE_IDENTIFIERS_TO_LOWERCASE": "N",
+ "PRESERVE_RESERVED_WORD_CASE": "Y",
+ "SQL_CONNECT": "",
+ "FORCE_IDENTIFIERS_TO_UPPERCASE": "N",
+ "PREFERRED_SCHEMA_NAME": ""
+ },
+ "manualUrl": "",
+ "username": ""
+ }
+ },
+ "name": "OPS"
+}
diff --git a/integration-tests/lint/metadata/rdbms/Vault.json
b/integration-tests/lint/metadata/rdbms/Vault.json
new file mode 100644
index 0000000000..b237a59cbf
--- /dev/null
+++ b/integration-tests/lint/metadata/rdbms/Vault.json
@@ -0,0 +1,26 @@
+{
+ "rdbms": {
+ "H2": {
+ "databaseName": "/tmp/hop-lint-it-Vault",
+ "pluginId": "H2",
+ "accessType": 0,
+ "hostname": "",
+ "password": "Encrypted ",
+ "pluginName": "H2",
+ "port": "",
+ "attributes": {
+ "SUPPORTS_TIMESTAMP_DATA_TYPE": "N",
+ "QUOTE_ALL_FIELDS": "N",
+ "SUPPORTS_BOOLEAN_DATA_TYPE": "N",
+ "FORCE_IDENTIFIERS_TO_LOWERCASE": "N",
+ "PRESERVE_RESERVED_WORD_CASE": "Y",
+ "SQL_CONNECT": "",
+ "FORCE_IDENTIFIERS_TO_UPPERCASE": "N",
+ "PREFERRED_SCHEMA_NAME": ""
+ },
+ "manualUrl": "",
+ "username": ""
+ }
+ },
+ "name": "Vault"
+}
diff --git
a/integration-tests/lint/metadata/workflow-run-configuration/local.json
b/integration-tests/lint/metadata/workflow-run-configuration/local.json
new file mode 100644
index 0000000000..e37a93039a
--- /dev/null
+++ b/integration-tests/lint/metadata/workflow-run-configuration/local.json
@@ -0,0 +1,9 @@
+{
+ "engineRunConfiguration": {
+ "Local": {
+ "safe_mode": false
+ }
+ },
+ "name": "local",
+ "description": "Runs your workflows locally with the standard local Hop
workflow engine"
+}
\ No newline at end of file
diff --git a/integration-tests/lint/project-config.json
b/integration-tests/lint/project-config.json
new file mode 100644
index 0000000000..6a91171e1c
--- /dev/null
+++ b/integration-tests/lint/project-config.json
@@ -0,0 +1,13 @@
+{
+ "metadataBaseFolder" : "${PROJECT_HOME}/metadata",
+ "unitTestsBasePath" : "${PROJECT_HOME}",
+ "dataSetsCsvFolder" : "${PROJECT_HOME}/datasets",
+ "enforcingExecutionInHome" : true,
+ "config" : {
+ "variables" : [ {
+ "name" : "HOP_LICENSE_HEADER_FILE",
+ "value" : "${PROJECT_HOME}/../asf-header.txt",
+ "description" : "This will automatically serialize the ASF license
header into pipelines and workflows in the integration test projects"
+ } ]
+ }
+}
\ No newline at end of file
diff --git a/integration-tests/lint/subject/existing-connections.hwf
b/integration-tests/lint/subject/existing-connections.hwf
new file mode 100644
index 0000000000..ac366adcd1
--- /dev/null
+++ b/integration-tests/lint/subject/existing-connections.hwf
@@ -0,0 +1,91 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+-->
+<workflow>
+ <name>existing-connections</name>
+ <name_sync_with_filename>Y</name_sync_with_filename>
+ <description>Linter subject: every connection referenced here exists in
metadata/rdbms.</description>
+ <extended_description/>
+ <workflow_version/>
+ <created_user>-</created_user>
+ <created_date>2026/09/09 08:00:00.000</created_date>
+ <modified_user>-</modified_user>
+ <modified_date>2026/09/09 08:00:00.000</modified_date>
+ <parameters>
+ </parameters>
+ <actions>
+ <action>
+ <name>Start</name>
+ <description/>
+ <type>SPECIAL</type>
+ <attributes/>
+ <DayOfMonth>1</DayOfMonth>
+ <hour>12</hour>
+ <intervalMinutes>60</intervalMinutes>
+ <intervalSeconds>0</intervalSeconds>
+ <minutes>0</minutes>
+ <repeat>N</repeat>
+ <schedulerType>0</schedulerType>
+ <weekDay>1</weekDay>
+ <parallel>N</parallel>
+ <xloc>144</xloc>
+ <yloc>80</yloc>
+ <attributes_hac/>
+ </action>
+ <action>
+ <name>Check DB connections</name>
+ <description/>
+ <type>CHECK_DB_CONNECTIONS</type>
+ <attributes/>
+ <connections>
+ <connection>
+ <name>OPS</name>
+ <waitfor>200</waitfor>
+ <waittime>millisecond</waittime>
+ </connection>
+ <connection>
+ <name>Vault</name>
+ <waitfor>200</waitfor>
+ <waittime>millisecond</waittime>
+ </connection>
+ <connection>
+ <name>CRM</name>
+ <waitfor>200</waitfor>
+ <waittime>millisecond</waittime>
+ </connection>
+ </connections>
+ <parallel>N</parallel>
+ <xloc>384</xloc>
+ <yloc>80</yloc>
+ <attributes_hac/>
+ </action>
+ </actions>
+ <hops>
+ <hop>
+ <from>Start</from>
+ <to>Check DB connections</to>
+ <enabled>Y</enabled>
+ <evaluation>Y</evaluation>
+ <unconditional>Y</unconditional>
+ </hop>
+ </hops>
+ <notepads>
+ </notepads>
+ <attributes/>
+</workflow>
diff --git a/integration-tests/lint/subject/missing-connection.hwf
b/integration-tests/lint/subject/missing-connection.hwf
new file mode 100644
index 0000000000..29ad8f9932
--- /dev/null
+++ b/integration-tests/lint/subject/missing-connection.hwf
@@ -0,0 +1,81 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+-->
+<workflow>
+ <name>missing-connection</name>
+ <name_sync_with_filename>Y</name_sync_with_filename>
+ <description>Linter subject: the referenced connection is deliberately
absent from metadata/rdbms.</description>
+ <extended_description/>
+ <workflow_version/>
+ <created_user>-</created_user>
+ <created_date>2026/09/09 08:00:00.000</created_date>
+ <modified_user>-</modified_user>
+ <modified_date>2026/09/09 08:00:00.000</modified_date>
+ <parameters>
+ </parameters>
+ <actions>
+ <action>
+ <name>Start</name>
+ <description/>
+ <type>SPECIAL</type>
+ <attributes/>
+ <DayOfMonth>1</DayOfMonth>
+ <hour>12</hour>
+ <intervalMinutes>60</intervalMinutes>
+ <intervalSeconds>0</intervalSeconds>
+ <minutes>0</minutes>
+ <repeat>N</repeat>
+ <schedulerType>0</schedulerType>
+ <weekDay>1</weekDay>
+ <parallel>N</parallel>
+ <xloc>144</xloc>
+ <yloc>80</yloc>
+ <attributes_hac/>
+ </action>
+ <action>
+ <name>Check DB connections</name>
+ <description/>
+ <type>CHECK_DB_CONNECTIONS</type>
+ <attributes/>
+ <connections>
+ <connection>
+ <name>NoSuchConnection</name>
+ <waitfor>200</waitfor>
+ <waittime>millisecond</waittime>
+ </connection>
+ </connections>
+ <parallel>N</parallel>
+ <xloc>384</xloc>
+ <yloc>80</yloc>
+ <attributes_hac/>
+ </action>
+ </actions>
+ <hops>
+ <hop>
+ <from>Start</from>
+ <to>Check DB connections</to>
+ <enabled>Y</enabled>
+ <evaluation>Y</evaluation>
+ <unconditional>Y</unconditional>
+ </hop>
+ </hops>
+ <notepads>
+ </notepads>
+ <attributes/>
+</workflow>
diff --git
a/plugins/misc/projects/src/main/java/org/apache/hop/projects/gui/ProjectsGuiPlugin.java
b/plugins/misc/projects/src/main/java/org/apache/hop/projects/gui/ProjectsGuiPlugin.java
index da331aa884..310090d840 100644
---
a/plugins/misc/projects/src/main/java/org/apache/hop/projects/gui/ProjectsGuiPlugin.java
+++
b/plugins/misc/projects/src/main/java/org/apache/hop/projects/gui/ProjectsGuiPlugin.java
@@ -321,6 +321,16 @@ public class ProjectsGuiPlugin {
hopGui.getEventsHandler().fire(projectName,
HopGuiEvents.ProjectActivated.name());
hopGui.getEventsHandler().fire(projectName,
HopGuiEvents.MetadataChanged.name());
+ // The project that just opened has its own VFS connections: take its
namespace, and let go
+ // of the one of the project we came from. Not a full reset - in Hop Web
that would empty the
+ // file system manager of every other session as well.
+ //
+ // Before the extension point below, not after: letting go of the
previous namespace closes
+ // it, and a listener that starts background work - the linter walks the
whole project - would
+ // otherwise be handed a file system manager that is about to be closed
underneath it. See
+ // issue #8295.
+ hopGui.useVfsNamespaceOfOpenProject();
+
// Inform the outside world that we're enabled another project
//
ExtensionPointHandler.callExtensionPoint(
@@ -329,11 +339,6 @@ public class ProjectsGuiPlugin {
HopExtensionPoint.HopGuiProjectAfterEnabled.name(),
project);
- // The project that just opened has its own VFS connections: take its
namespace, and let go
- // of the one of the project we came from. Not a full reset - in Hop Web
that would empty the
- // file system manager of every other session as well.
- hopGui.useVfsNamespaceOfOpenProject();
-
// Finally, warn about metadata elements in this project which we can't
load.
// They're ignored so the project itself opens just fine.
//
diff --git
a/ui/src/main/java/org/apache/hop/ui/hopgui/file/shared/ReferencedConnectionSaveValidator.java
b/ui/src/main/java/org/apache/hop/ui/hopgui/file/shared/ReferencedConnectionSaveValidator.java
index 1fdd52b5df..bc0aa45a11 100644
---
a/ui/src/main/java/org/apache/hop/ui/hopgui/file/shared/ReferencedConnectionSaveValidator.java
+++
b/ui/src/main/java/org/apache/hop/ui/hopgui/file/shared/ReferencedConnectionSaveValidator.java
@@ -69,8 +69,29 @@ public final class ReferencedConnectionSaveValidator {
workflowMeta, variables, metadataProvider));
}
- static boolean confirmRemarks(Shell shell, List<ICheckResult> remarks) {
- if (remarks == null || remarks.isEmpty()) {
+ /**
+ * The remarks worth stopping a save for: a connection that is really
missing, or none assigned.
+ *
+ * <p>A connection the checker could not look up at all is reported too (see
{@link
+ * ReferencedDatabaseConnectionChecker#INFO_NOT_VERIFIED}), but that says
the metadata could not
+ * be read rather than that anything is wrong with the file being saved.
Prompting on it would put
+ * this dialog in front of every single save for as long as the metadata is
unreachable.
+ *
+ * @param remarks every remark the checker produced, may be null
+ * @return the ones at warning level or above, never null
+ */
+ static List<ICheckResult> blockingRemarks(List<ICheckResult> remarks) {
+ if (remarks == null) {
+ return List.of();
+ }
+ return remarks.stream()
+ .filter(remark -> remark.getType() >= ICheckResult.TYPE_RESULT_WARNING)
+ .toList();
+ }
+
+ static boolean confirmRemarks(Shell shell, List<ICheckResult> allRemarks) {
+ List<ICheckResult> remarks = blockingRemarks(allRemarks);
+ if (remarks.isEmpty()) {
return true;
}
if (shell == null || shell.isDisposed()) {
diff --git
a/ui/src/test/java/org/apache/hop/ui/hopgui/file/shared/ReferencedConnectionSaveValidatorTest.java
b/ui/src/test/java/org/apache/hop/ui/hopgui/file/shared/ReferencedConnectionSaveValidatorTest.java
new file mode 100644
index 0000000000..060c01b1bf
--- /dev/null
+++
b/ui/src/test/java/org/apache/hop/ui/hopgui/file/shared/ReferencedConnectionSaveValidatorTest.java
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hop.ui.hopgui.file.shared;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.List;
+import org.apache.hop.core.CheckResult;
+import org.apache.hop.core.ICheckResult;
+import org.apache.hop.metadata.validation.ReferencedDatabaseConnectionChecker;
+import org.junit.jupiter.api.DisplayName;
+import org.junit.jupiter.api.Test;
+
+/**
+ * Which remarks are worth interrupting a save for. See issue #8295: a
connection the checker could
+ * not look up is reported, but it describes unreachable metadata rather than
a problem with the
+ * file being saved, so it must not put a dialog in front of every save.
+ */
+class ReferencedConnectionSaveValidatorTest {
+
+ private static ICheckResult remark(int type, String code) {
+ return new CheckResult(type, code, code, null);
+ }
+
+ @Test
+ @DisplayName("A connection that does not exist still stops the save")
+ void missingConnectionBlocks() {
+ List<ICheckResult> blocking =
+ ReferencedConnectionSaveValidator.blockingRemarks(
+ List.of(
+ remark(
+ ICheckResult.TYPE_RESULT_WARNING,
+
ReferencedDatabaseConnectionChecker.ERROR_DOES_NOT_EXIST)));
+
+ assertEquals(1, blocking.size());
+ }
+
+ @Test
+ @DisplayName("A connection that could not be checked does not stop the save")
+ void unverifiedConnectionDoesNotBlock() {
+ List<ICheckResult> blocking =
+ ReferencedConnectionSaveValidator.blockingRemarks(
+ List.of(
+ remark(
+ ICheckResult.TYPE_RESULT_COMMENT,
+ ReferencedDatabaseConnectionChecker.INFO_NOT_VERIFIED)));
+
+ assertTrue(
+ blocking.isEmpty(),
+ "Unreadable metadata would otherwise put this dialog in front of every
save");
+ }
+
+ @Test
+ @DisplayName("A real problem still gets through when mixed with ones that
could not be checked")
+ void mixedRemarksKeepTheRealProblem() {
+ List<ICheckResult> blocking =
+ ReferencedConnectionSaveValidator.blockingRemarks(
+ List.of(
+ remark(
+ ICheckResult.TYPE_RESULT_COMMENT,
+ ReferencedDatabaseConnectionChecker.INFO_NOT_VERIFIED),
+ remark(
+ ICheckResult.TYPE_RESULT_WARNING,
+ ReferencedDatabaseConnectionChecker.ERROR_NOT_ASSIGNED),
+ remark(
+ ICheckResult.TYPE_RESULT_COMMENT,
+ ReferencedDatabaseConnectionChecker.INFO_NOT_VERIFIED)));
+
+ assertEquals(1, blocking.size());
+ assertEquals(
+ ReferencedDatabaseConnectionChecker.ERROR_NOT_ASSIGNED,
blocking.get(0).getErrorCode());
+ }
+
+ @Test
+ @DisplayName("Nothing to report is not a reason to prompt")
+ void emptyAndNullAreNotBlocking() {
+
assertTrue(ReferencedConnectionSaveValidator.blockingRemarks(null).isEmpty());
+
assertTrue(ReferencedConnectionSaveValidator.blockingRemarks(List.of()).isEmpty());
+ }
+}
