This is an automated email from the ASF dual-hosted git repository.
jongyoul pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 206b6a319d [ZEPPELIN-6208] Enable DuckDB support in JDBC Interpreter
by excluding incompatible default properties
206b6a319d is described below
commit 206b6a319dd4e30bf8c5df7331847f440724fff0
Author: HyeonUk Kang <[email protected]>
AuthorDate: Sun Aug 9 22:53:41 2026 +0900
[ZEPPELIN-6208] Enable DuckDB support in JDBC Interpreter by excluding
incompatible default properties
What is this PR for?
The JDBC interpreter forwards all of an interpreter's prefixed properties
(default.*) straight to the JDBC driver when opening a connection. Many of
these keys are consumed by Zeppelin itself or by the DBCP connection pool —
e.g. driver, url, precode, statementPrecode,
completer.ttlInSeconds, validationQuery, maxIdle — and are not valid JDBC
connection properties.
Lenient drivers (PostgreSQL, MySQL) silently ignore unknown properties,
so this went unnoticed. But strict drivers reject the connection outright. For
example, DuckDB fails with:
SQLException: Invalid Input Error: The following options were not
recognized:
completer.ttlInSeconds, url, driver, maxIdle
This currently makes it impossible to use DuckDB (and other strict
drivers such as MS SQL Server) with the JDBC interpreter.
Until now this was worked around with a driver-specific whitelist that
only applied when the driver class was Presto/Trino (PRESTO_PROPERTIES). That
approach is hard to maintain: every new strict driver needs its own if branch
and allow-list, and it also strips any legitimate driver
property the user added but that wasn't on the list.
This PR replaces that with a generic, driver-agnostic deny-list:
- Introduces NON_DRIVER_PROPERTIES, a single source of truth listing only
Zeppelin-internal and DBCP pool keys (user/password are deliberately kept,
since they are standard JDBC properties).
- Adds toDriverProperties(), which returns a filtered copy of the
properties handed to the driver, leaving the original untouched (the previous
Presto code mutated the shared per-user Properties in place — a latent bug).
- Removes the Presto/Trino special-case branch and the PRESTO_PROPERTIES
whitelist; the generic filter subsumes it.
- Adds an optional escape hatch, zeppelin.jdbc.driver.excludeProperties
(comma-separated), so operators can exclude additional keys for future drivers
without a code change.
Result: DuckDB, MS SQL Server, Trino/Presto, and any future strict driver
work through one consistent rule, while genuine driver properties (e.g. SSL,
useSSL, sslmode) still pass through unchanged.
What type of PR is it?
Bug Fix
Todos
- [x] Replace Presto/Trino whitelist with a generic internal-property
deny-list
- [x] Keep user/password and arbitrary driver properties (e.g. SSL)
flowing to the driver
- [x] Add unit tests for the filtering logic
- [x] Add end-to-end tests against real strict drivers (DuckDB embedded,
Trino)
| Property setting (1) | Property setting (2) |
|:---:|:---:|
| <img width="2507" height="963" alt="스크린샷 2026-06-25 오후 9 20 23"
src="https://github.com/user-attachments/assets/9c32bab8-d6d5-4096-b90a-5fa59e434509";
/> | <img width="2083" height="139" alt="스크린샷 2026-06-25 오후 9 20 29"
src="https://github.com/user-attachments/assets/2e0d2320-59eb-4efb-b28d-0f9bf5e7bc5f";
/> |
**2. Result — Before / After**
| Before — DuckDB connection error | After — connected successfully |
|:---:|:---:|
| <img width="2462" height="206" alt="before-duckdb-error"
src="https://github.com/user-attachments/assets/9b204002-9373-44a1-8d9b-1e41da30e4f6";
/> | <img width="2475" height="517" alt="after-success"
src="https://github.com/user-attachments/assets/417141ca-f780-4e58-8f55-0c32f9d6102c";
/>
What is the Jira issue?
- [ZEPPELIN-6208](https://issues.apache.org/jira/browse/ZEPPELIN-6208)
How should this be tested?
Automated tests added in JDBCInterpreterTest:
- testToDriverProperties — internal/pool keys are removed; user,
password, and arbitrary driver props (SSL) are kept; the original Properties is
not mutated.
- testToDriverPropertiesWithUserDefinedExcludes —
zeppelin.jdbc.driver.excludeProperties strips additional user-specified keys.
- testDuckDbConnectionWithInternalProperties — end-to-end: configures the
interpreter with internal keys present and runs CREATE/INSERT/SELECT against an
embedded DuckDB (no server needed). Fails on the old code, passes here.
- testTrinoConnectionWithInternalProperties — end-to-end against a real
Trino coordinator; auto-skipped via assumeTrue when none is reachable on
localhost:8080.
mvn -pl jdbc -am test -Dtest=JDBCInterpreterTest
Manual: add a JDBC interpreter with
default.driver=org.duckdb.DuckDBDriver, default.url=jdbc:duckdb:, add the
org.duckdb:duckdb_jdbc dependency in the interpreter settings, leave an
internal key such as default.completer.ttlInSeconds=120, and run a query — it
now connects successfully.
Questions:
- Does the license files need to update? - X
- Is there breaking changes for older versions? - X
- Does this need documentation? - Optional
Closes #5276 from hyunw9/ZEPPELIN-6208.
Signed-off-by: Jongyoul Lee <[email protected]>
---
jdbc/pom.xml | 16 +++
.../org/apache/zeppelin/jdbc/JDBCInterpreter.java | 73 +++++++----
.../apache/zeppelin/jdbc/JDBCInterpreterTest.java | 142 +++++++++++++++++++++
3 files changed, 209 insertions(+), 22 deletions(-)
diff --git a/jdbc/pom.xml b/jdbc/pom.xml
index 7225529055..67cba4d95d 100644
--- a/jdbc/pom.xml
+++ b/jdbc/pom.xml
@@ -43,6 +43,8 @@
<!--test library versions-->
<mockrunner.jdbc.version>1.0.8</mockrunner.jdbc.version>
+ <duckdb.jdbc.version>1.3.1.0</duckdb.jdbc.version>
+ <trino.jdbc.version>481</trino.jdbc.version>
</properties>
<dependencies>
@@ -67,6 +69,20 @@
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.duckdb</groupId>
+ <artifactId>duckdb_jdbc</artifactId>
+ <version>${duckdb.jdbc.version}</version>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>io.trino</groupId>
+ <artifactId>trino-jdbc</artifactId>
+ <version>${trino.jdbc.version}</version>
+ <scope>test</scope>
+ </dependency>
+
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
diff --git a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
index 90c34614b5..aa486fec43 100644
--- a/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
+++ b/jdbc/src/main/java/org/apache/zeppelin/jdbc/JDBCInterpreter.java
@@ -147,14 +147,29 @@ public class JDBCInterpreter extends KerberosInterpreter {
private static final String DBCP_STRING = "jdbc:apache:commons:dbcp:";
private static final String MAX_ROWS_KEY = "zeppelin.jdbc.maxRows";
- private static final Set<String> PRESTO_PROPERTIES = new
HashSet<>(Arrays.asList(
- "user", "password",
- "socksProxy", "httpProxy", "clientTags", "applicationNamePrefix",
"accessToken",
- "SSL", "SSLKeyStorePath", "SSLKeyStorePassword", "SSLTrustStorePath",
- "SSLTrustStorePassword", "KerberosRemoteServiceName",
"KerberosPrincipal",
- "KerberosUseCanonicalHostname", "KerberosServicePrincipalPattern",
- "KerberosConfigPath", "KerberosKeytabPath",
"KerberosCredentialCachePath",
- "extraCredentials", "roles", "sessionProperties"));
+ /**
+ * Properties that Zeppelin consumes internally (or hands to the DBCP
connection
+ * pool) and therefore must NOT be forwarded to the JDBC driver as connection
+ * properties.
+ *
+ * Note: "user" and "password" are deliberately excluded from this set
because
+ * they are standard JDBC connection properties and must reach the driver.
+ */
+ private static final Set<String> NON_DRIVER_PROPERTIES = new
HashSet<>(Arrays.asList(
+ // connection metadata consumed by the interpreter itself
+ DRIVER_KEY, URL_KEY,
+ // SQL hooks executed by the interpreter, not the driver
+ PRECODE_KEY, STATEMENT_PRECODE_KEY,
+ // auto-completion settings
+ COMPLETER_TTL_KEY, COMPLETER_SCHEMA_FILTERS_KEY,
+ // proxy / credential settings handled by the interpreter
+ "proxy.user.property", JDBC_JCEKS_FILE, JDBC_JCEKS_CREDENTIAL_KEY,
+ // DBCP connection-pool settings applied in configConnectionPool()
+ "validationQuery", "testOnBorrow", "testOnCreate", "testOnReturn",
+ "testWhileIdle", "timeBetweenEvictionRunsMillis", "maxWaitMillis",
+ "maxIdle", "minIdle", "maxTotal"));
+
+ static final String DRIVER_EXCLUDE_PROPERTIES_KEY =
"zeppelin.jdbc.driver.excludeProperties";
private static final String ALLOW_LOAD_LOCAL = "allowLoadLocal";
@@ -486,28 +501,42 @@ public class JDBCInterpreter extends KerberosInterpreter {
connectionPool.setMaxWaitMillis(maxWaitMillis);
}
+ /**
+ * Builds the property set handed to the JDBC driver: a copy of {@code
properties}
+ * with all Zeppelin-internal and connection-pool keys ({@link
#NON_DRIVER_PROPERTIES})
+ * removed. The original is left untouched so it can still be used for pool
+ * configuration and lookups. Additional keys can be excluded via
+ * {@value #DRIVER_EXCLUDE_PROPERTIES_KEY} (comma-separated).
+ */
+ // package private for testing purposes
+ Properties toDriverProperties(Properties properties) {
+ Set<String> excludes = new HashSet<>(NON_DRIVER_PROPERTIES);
+ String userExcludes = getProperty(DRIVER_EXCLUDE_PROPERTIES_KEY);
+ if (StringUtils.isNotBlank(userExcludes)) {
+ for (String key : userExcludes.split(",")) {
+ excludes.add(key.trim());
+ }
+ }
+
+ Properties driverProperties = new Properties();
+ for (String key : properties.stringPropertyNames()) {
+ if (!excludes.contains(key)) {
+ driverProperties.setProperty(key, properties.getProperty(key));
+ }
+ }
+ return driverProperties;
+ }
+
private void createConnectionPool(String url, String user,
Properties properties) throws SQLException, ClassNotFoundException {
LOGGER.info("Creating connection pool for url: {}, user: {}", url, user);
- /* Remove properties that is not valid properties for presto/trino by
checking driver key.
- * - Presto: com.facebook.presto.jdbc.PrestoDriver
- * - Trino(ex. PrestoSQL): io.trino.jdbc.TrinoDriver /
io.prestosql.jdbc.PrestoDriver
- */
String driverClass = properties.getProperty(DRIVER_KEY);
- if (driverClass != null &&
(driverClass.equals("com.facebook.presto.jdbc.PrestoDriver")
- || driverClass.equals("io.prestosql.jdbc.PrestoDriver")
- || driverClass.equals("io.trino.jdbc.TrinoDriver"))) {
- for (String key : properties.stringPropertyNames()) {
- if (!PRESTO_PROPERTIES.contains(key)) {
- properties.remove(key);
- }
- }
- }
+ Properties driverProperties = toDriverProperties(properties);
ConnectionFactory connectionFactory =
- new DriverManagerConnectionFactory(url, properties);
+ new DriverManagerConnectionFactory(url, driverProperties);
PoolableConnectionFactory poolableConnectionFactory = new
PoolableConnectionFactory(
connectionFactory, null);
diff --git
a/jdbc/src/test/java/org/apache/zeppelin/jdbc/JDBCInterpreterTest.java
b/jdbc/src/test/java/org/apache/zeppelin/jdbc/JDBCInterpreterTest.java
index cc5002b22a..f688e81310 100644
--- a/jdbc/src/test/java/org/apache/zeppelin/jdbc/JDBCInterpreterTest.java
+++ b/jdbc/src/test/java/org/apache/zeppelin/jdbc/JDBCInterpreterTest.java
@@ -59,12 +59,14 @@ import static
org.apache.zeppelin.jdbc.JDBCInterpreter.DEFAULT_PRECODE;
import static
org.apache.zeppelin.jdbc.JDBCInterpreter.DEFAULT_STATEMENT_PRECODE;
import static org.apache.zeppelin.jdbc.JDBCInterpreter.DEFAULT_URL;
import static org.apache.zeppelin.jdbc.JDBCInterpreter.DEFAULT_USER;
+import static
org.apache.zeppelin.jdbc.JDBCInterpreter.DRIVER_EXCLUDE_PROPERTIES_KEY;
import static org.apache.zeppelin.jdbc.JDBCInterpreter.PRECODE_KEY_TEMPLATE;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
+import static org.junit.jupiter.api.Assumptions.assumeTrue;
/**
@@ -189,6 +191,146 @@ public class JDBCInterpreterTest extends
BasicJDBCTestCaseAdapter {
assertEquals("1000", jdbcInterpreter.getProperty(COMMON_MAX_LINE));
}
+ @Test
+ void testToDriverProperties() {
+ JDBCInterpreter jdbcInterpreter = new JDBCInterpreter(new Properties());
+
+ Properties properties = new Properties();
+ // genuine JDBC driver properties, must be kept
+ properties.setProperty("user", "trino_user");
+ properties.setProperty("password", "secret");
+ properties.setProperty("SSL", "true");
+ // Zeppelin-internal / pool properties, must be removed (ZEPPELIN-6208)
+ properties.setProperty("driver", "io.trino.jdbc.TrinoDriver");
+ properties.setProperty("url", "jdbc:trino://localhost:8080");
+ properties.setProperty("precode", "set time zone 'UTC'");
+ properties.setProperty("statementPrecode", "set time zone 'UTC'");
+ properties.setProperty("completer.ttlInSeconds", "120");
+ properties.setProperty("completer.schemaFilters", "public");
+ properties.setProperty("validationQuery", "show databases");
+ properties.setProperty("maxIdle", "8");
+
+ Properties driverProperties =
jdbcInterpreter.toDriverProperties(properties);
+
+ assertEquals(3, driverProperties.size());
+ assertEquals("trino_user", driverProperties.getProperty("user"));
+ assertEquals("secret", driverProperties.getProperty("password"));
+ assertEquals("true", driverProperties.getProperty("SSL"));
+ assertFalse(driverProperties.containsKey("driver"));
+ assertFalse(driverProperties.containsKey("url"));
+ assertFalse(driverProperties.containsKey("precode"));
+ assertFalse(driverProperties.containsKey("statementPrecode"));
+ assertFalse(driverProperties.containsKey("completer.ttlInSeconds"));
+ assertFalse(driverProperties.containsKey("completer.schemaFilters"));
+ assertFalse(driverProperties.containsKey("validationQuery"));
+ assertFalse(driverProperties.containsKey("maxIdle"));
+
+ // the original properties must not be mutated
+ assertTrue(properties.containsKey("driver"));
+ assertTrue(properties.containsKey("url"));
+ }
+
+ @Test
+ void testToDriverPropertiesWithUserDefinedExcludes() {
+ Properties config = new Properties();
+ config.setProperty(DRIVER_EXCLUDE_PROPERTIES_KEY, "SSL, customKey");
+ JDBCInterpreter jdbcInterpreter = new JDBCInterpreter(config);
+
+ Properties properties = new Properties();
+ properties.setProperty("user", "trino_user");
+ properties.setProperty("SSL", "true");
+ properties.setProperty("customKey", "customValue");
+
+ Properties driverProperties =
jdbcInterpreter.toDriverProperties(properties);
+
+ assertEquals(1, driverProperties.size());
+ assertEquals("trino_user", driverProperties.getProperty("user"));
+ assertFalse(driverProperties.containsKey("SSL"));
+ assertFalse(driverProperties.containsKey("customKey"));
+ }
+
+ /**
+ * End-to-end check that a strict JDBC driver (DuckDB) can connect even
though
+ * Zeppelin-internal properties are present in the interpreter configuration.
+ * Before ZEPPELIN-6208 those keys were forwarded to the driver and DuckDB
+ * rejected the connection. DuckDB runs in-process, so no external server is
+ * needed.
+ */
+ @Test
+ void testDuckDbConnectionWithInternalProperties()
+ throws IOException, InterpreterException {
+ Properties properties = new Properties();
+ properties.setProperty("default.driver", "org.duckdb.DuckDBDriver");
+ properties.setProperty("default.url", "jdbc:duckdb:");
+ properties.setProperty("default.user", "");
+ properties.setProperty("default.password", "");
+ // Internal keys that DuckDB's strict driver would reject if forwarded
+ properties.setProperty("default.completer.ttlInSeconds", "120");
+ properties.setProperty("default.completer.schemaFilters", "");
+ properties.setProperty("common.max_count", "1000");
+ JDBCInterpreter t = new JDBCInterpreter(properties);
+ t.open();
+
+ String sqlQuery = "CREATE TABLE pokes (id INTEGER, name VARCHAR); " +
+ "INSERT INTO pokes VALUES (1, 'a'), (2, 'b'); " +
+ "SELECT * FROM pokes ORDER BY id;";
+ InterpreterResult interpreterResult = t.interpret(sqlQuery, context);
+
+ assertEquals(InterpreterResult.Code.SUCCESS, interpreterResult.code());
+ List<InterpreterResultMessage> resultMessages =
context.out.toInterpreterResultMessage();
+ InterpreterResultMessage tableMessage = resultMessages.stream()
+ .filter(m -> m.getType() == InterpreterResult.Type.TABLE)
+ .reduce((first, second) -> second)
+ .orElseThrow(() -> new AssertionError("No TABLE result produced"));
+ assertEquals("id\tname\n1\ta\n2\tb\n", tableMessage.getData());
+ }
+
+ private static boolean isTrinoAvailable(String host, int port) {
+ try (java.net.Socket socket = new java.net.Socket()) {
+ socket.connect(new java.net.InetSocketAddress(host, port), 1000);
+ return true;
+ } catch (IOException e) {
+ return false;
+ }
+ }
+
+ /**
+ * End-to-end check against a real Trino coordinator. Like DuckDB, Trino has
a
+ * strict driver that rejects unknown connection properties; before
ZEPPELIN-6208
+ * these were stripped by a driver-specific whitelist, now by the generic
filter.
+ * Skipped automatically when no Trino server is reachable on localhost:8080.
+ */
+ @Test
+ void testTrinoConnectionWithInternalProperties()
+ throws IOException, InterpreterException {
+ assumeTrue(isTrinoAvailable("localhost", 8080),
+ "Trino coordinator not reachable on localhost:8080, skipping");
+
+ Properties properties = new Properties();
+ properties.setProperty("default.driver", "io.trino.jdbc.TrinoDriver");
+ properties.setProperty("default.url", "jdbc:trino://localhost:8080");
+ properties.setProperty("default.user", "test");
+ properties.setProperty("default.password", "");
+ // Internal keys that Trino's strict driver would reject if forwarded
+ properties.setProperty("default.completer.ttlInSeconds", "120");
+ properties.setProperty("default.completer.schemaFilters", "");
+ properties.setProperty("common.max_count", "1000");
+ JDBCInterpreter t = new JDBCInterpreter(properties);
+ t.open();
+
+ String sqlQuery =
+ "SELECT nationkey AS id, name FROM tpch.tiny.nation ORDER BY nationkey
LIMIT 2;";
+ InterpreterResult interpreterResult = t.interpret(sqlQuery, context);
+
+ assertEquals(InterpreterResult.Code.SUCCESS, interpreterResult.code());
+ List<InterpreterResultMessage> resultMessages =
context.out.toInterpreterResultMessage();
+ InterpreterResultMessage tableMessage = resultMessages.stream()
+ .filter(m -> m.getType() == InterpreterResult.Type.TABLE)
+ .reduce((first, second) -> second)
+ .orElseThrow(() -> new AssertionError("No TABLE result produced"));
+ assertEquals("id\tname\n0\tALGERIA\n1\tARGENTINA\n",
tableMessage.getData());
+ }
+
@Test
void testSelectQuery() throws IOException, InterpreterException {
Properties properties = new Properties();
