This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new eb1b5996c9 [KYUUBI #6815] JDBC Engine supports Oracle
eb1b5996c9 is described below
commit eb1b5996c9b89de829e81d88a858c6f7b1fe9def
Author: naive-zhang <[email protected]>
AuthorDate: Mon Dec 2 23:41:57 2024 +0800
[KYUUBI #6815] JDBC Engine supports Oracle
# Description
Currently, Kyuubi supports JDBC engines with limited dialects, and I extend
the dialects to support Oracle.
* Introduce Oracle support in JDBC Engine
* Adding dialects and tests for Oracle
## Types of changes :bookmark:
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan ๐งช
Add tests of `OperationWithOracleEngineSuite`, `OracleOperationSuite`,
`OracleSessionSuite` and `OracleStatementSuite`.
---
# Checklist ๐
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes #6815 from naive-zhang/jdbc-oracle.
Closes #6815
0ffad5b6b [native-zhang] add some brief comments on the caller side for the
implementation of Oracle JDBC engine
6f469a135 [naive-zhang] Merge branch 'apache:master' into jdbc-oracle
ae70710e6 [Cheng Pan] Update
externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/dialect/OracleSQLDialect.scala
171d06b9e [native-zhang] use another implementation of transform decimal
into int, in engine instead of KyuubiBaseResultSet
7cb74d28e [naive-zhang] Merge branch 'apache:master' into jdbc-oracle
ccd7cae8b [naive-zhang] remove redundant override methods in
OracleSQLDialect.scala
a7da4a646 [naive-zhang] remove redundant impl of getTableTypesOperation in
OracleSQLDialect.scala
70b49fcba [naive-zhang] Use the single line string if SQL fits in one line,
otherwise write it in a pretty style
e58348460 [naive-zhang] Update
externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/dialect/OracleSQLDialect.scala
b33e97a08 [naive-zhang] remove redundant testcontainers-scala-oracle-xe
dependency in pom.xml
4c967b98e [naive-zhang] use gvenzl/oracle-free:23.5-slim with
docker-compose for test case
0215e6d49 [naive-zhang] Merge branch 'apache:master' into jdbc-oracle
d688b4706 [naive-zhang] change oracle image into
gvenzl/oracle-free:23.5-slim
abf983727 [naive-zhang] fix code style checking error in KyuubiConf.scala
d1e82edb1 [naive-zhang] fix code style checking error in settings.md
aa2e2e9ba [naive-zhang] adjust wired space in OracleSQLDialect
b43cea421 [naive-zhang] add oracle configuration for
kyuubi.engine.jdbc.connection.provider
397c1cfec [naive-zhang] Merge branch 'apache:master' into jdbc-oracle
2f1b5ed0b [naive-zhang] add jdbc support for Oracle
Lead-authored-by: naive-zhang <[email protected]>
Co-authored-by: native-zhang <[email protected]>
Co-authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
docs/configuration/settings.md | 2 +-
externals/kyuubi-jdbc-engine/pom.xml | 6 ++
...i.engine.jdbc.connection.JdbcConnectionProvider | 1 +
...g.apache.kyuubi.engine.jdbc.dialect.JdbcDialect | 1 +
.../engine/jdbc/dialect/OracleSQLDialect.scala | 114 +++++++++++++++++++++
.../jdbc/oracle/OracleConnectionProvider.scala | 26 +++++
.../engine/jdbc/oracle/OracleSchemaHelper.scala | 36 +++++++
.../jdbc/oracle/OracleTRowSetGenerator.scala | 46 +++++++++
.../kyuubi/engine/jdbc/schema/SchemaHelper.scala | 2 +-
.../src/test/resources/oracle-compose.yml | 42 ++++++++
.../oracle/OperationWithOracleEngineSuite.scala | 61 +++++++++++
.../engine/jdbc/oracle/OracleOperationSuite.scala | 104 +++++++++++++++++++
.../engine/jdbc/oracle/OracleSessionSuite.scala | 40 ++++++++
.../engine/jdbc/oracle/OracleStatementSuite.scala | 81 +++++++++++++++
.../engine/jdbc/oracle/WithOracleContainer.scala | 55 ++++++++++
.../engine/jdbc/oracle/WithOracleEngine.scala | 38 +++++++
.../org/apache/kyuubi/config/KyuubiConf.scala | 5 +-
pom.xml | 7 ++
18 files changed, 664 insertions(+), 3 deletions(-)
diff --git a/docs/configuration/settings.md b/docs/configuration/settings.md
index a1b714d0d7..fede369a84 100644
--- a/docs/configuration/settings.md
+++ b/docs/configuration/settings.md
@@ -163,7 +163,7 @@ You can configure the Kyuubi properties in
`$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.engine.jdbc.connection.password | <undefined>
| The password is used for connecting to server
[...]
| kyuubi.engine.jdbc.connection.propagateCredential | false
| Whether to use the session's user and password to connect to database
[...]
| kyuubi.engine.jdbc.connection.properties
|| The additional properties are used for connecting to server
[...]
-| kyuubi.engine.jdbc.connection.provider | <undefined>
| A JDBC connection provider plugin for the Kyuubi Server to establish
a connection to the JDBC URL. The configuration value should be a subclass of
`org.apache.kyuubi.engine.jdbc.connection.JdbcConnectionProvider`. Kyuubi
provides the following built-in implementations: <li>doris: For establishing
Doris connections.</li> <li>mysql: For establishing MySQL connections.</li>
<li>phoenix: For establishing [...]
+| kyuubi.engine.jdbc.connection.provider | <undefined>
| A JDBC connection provider plugin for the Kyuubi Server to establish
a connection to the JDBC URL. The configuration value should be a subclass of
`org.apache.kyuubi.engine.jdbc.connection.JdbcConnectionProvider`. Kyuubi
provides the following built-in implementations: <li>doris: For establishing
Doris connections.</li> <li>mysql: For establishing MySQL connections.</li>
<li>phoenix: For establishing [...]
| kyuubi.engine.jdbc.connection.url | <undefined>
| The server url that engine will connect to
[...]
| kyuubi.engine.jdbc.connection.user | <undefined>
| The user is used for connecting to server
[...]
| kyuubi.engine.jdbc.deploy.mode | LOCAL
| Configures the jdbc engine deploy mode, The value can be 'local',
'yarn'. In local mode, the engine operates on the same node as the
KyuubiServer. In YARN mode, the engine runs within the Application Master (AM)
container of YARN.
[...]
diff --git a/externals/kyuubi-jdbc-engine/pom.xml
b/externals/kyuubi-jdbc-engine/pom.xml
index a69b55c406..4a3ee70934 100644
--- a/externals/kyuubi-jdbc-engine/pom.xml
+++ b/externals/kyuubi-jdbc-engine/pom.xml
@@ -115,6 +115,12 @@
<classifier>http</classifier>
<scope>test</scope>
</dependency>
+
+ <dependency>
+ <groupId>com.oracle.database.jdbc</groupId>
+ <artifactId>ojdbc8</artifactId>
+ <scope>test</scope>
+ </dependency>
</dependencies>
<build>
diff --git
a/externals/kyuubi-jdbc-engine/src/main/resources/META-INF/services/org.apache.kyuubi.engine.jdbc.connection.JdbcConnectionProvider
b/externals/kyuubi-jdbc-engine/src/main/resources/META-INF/services/org.apache.kyuubi.engine.jdbc.connection.JdbcConnectionProvider
index 637de2e1e7..d7184dba4b 100644
---
a/externals/kyuubi-jdbc-engine/src/main/resources/META-INF/services/org.apache.kyuubi.engine.jdbc.connection.JdbcConnectionProvider
+++
b/externals/kyuubi-jdbc-engine/src/main/resources/META-INF/services/org.apache.kyuubi.engine.jdbc.connection.JdbcConnectionProvider
@@ -19,6 +19,7 @@
org.apache.kyuubi.engine.jdbc.clickhouse.ClickHouseConnectionProvider
org.apache.kyuubi.engine.jdbc.doris.DorisConnectionProvider
org.apache.kyuubi.engine.jdbc.impala.ImpalaConnectionProvider
org.apache.kyuubi.engine.jdbc.mysql.MySQLConnectionProvider
+org.apache.kyuubi.engine.jdbc.oracle.OracleConnectionProvider
org.apache.kyuubi.engine.jdbc.phoenix.PhoenixConnectionProvider
org.apache.kyuubi.engine.jdbc.postgresql.PostgreSQLConnectionProvider
org.apache.kyuubi.engine.jdbc.starrocks.StarRocksConnectionProvider
diff --git
a/externals/kyuubi-jdbc-engine/src/main/resources/META-INF/services/org.apache.kyuubi.engine.jdbc.dialect.JdbcDialect
b/externals/kyuubi-jdbc-engine/src/main/resources/META-INF/services/org.apache.kyuubi.engine.jdbc.dialect.JdbcDialect
index 6c0838c0d0..ddd0383d3b 100644
---
a/externals/kyuubi-jdbc-engine/src/main/resources/META-INF/services/org.apache.kyuubi.engine.jdbc.dialect.JdbcDialect
+++
b/externals/kyuubi-jdbc-engine/src/main/resources/META-INF/services/org.apache.kyuubi.engine.jdbc.dialect.JdbcDialect
@@ -19,6 +19,7 @@ org.apache.kyuubi.engine.jdbc.dialect.ClickHouseDialect
org.apache.kyuubi.engine.jdbc.dialect.DorisDialect
org.apache.kyuubi.engine.jdbc.dialect.ImpalaDialect
org.apache.kyuubi.engine.jdbc.dialect.MySQLDialect
+org.apache.kyuubi.engine.jdbc.dialect.OracleSQLDialect
org.apache.kyuubi.engine.jdbc.dialect.PhoenixDialect
org.apache.kyuubi.engine.jdbc.dialect.PostgreSQLDialect
org.apache.kyuubi.engine.jdbc.dialect.StarRocksDialect
diff --git
a/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/dialect/OracleSQLDialect.scala
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/dialect/OracleSQLDialect.scala
new file mode 100644
index 0000000000..2f3c3c3c9a
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/dialect/OracleSQLDialect.scala
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kyuubi.engine.jdbc.dialect
+
+import java.sql.{Connection, ResultSet, Statement}
+import java.util
+
+import scala.collection.JavaConverters._
+import scala.collection.mutable.ArrayBuffer
+
+import org.apache.commons.lang3.StringUtils
+
+import org.apache.kyuubi.engine.jdbc.oracle.{OracleSchemaHelper,
OracleTRowSetGenerator}
+import org.apache.kyuubi.engine.jdbc.schema.{JdbcTRowSetGenerator,
SchemaHelper}
+import org.apache.kyuubi.operation.meta.ResultSetSchemaConstant._
+import org.apache.kyuubi.session.Session
+
+class OracleSQLDialect extends JdbcDialect {
+
+ override def createStatement(connection: Connection, fetchSize: Int):
Statement = {
+ val statement =
+ connection.createStatement(ResultSet.TYPE_FORWARD_ONLY,
ResultSet.CONCUR_READ_ONLY)
+ if (connection.getAutoCommit) {
+ statement.setFetchSize(fetchSize)
+ }
+ statement
+ }
+
+ override def getTablesQuery(
+ catalog: String,
+ schema: String,
+ tableName: String,
+ tableTypes: util.List[String]): String = {
+ val tTypes =
+ if (tableTypes == null || tableTypes.isEmpty) {
+ Set()
+ } else {
+ tableTypes.asScala.toSet
+ }
+ val query = new StringBuilder(
+ "SELECT OWNER AS TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE AS TABLE_TYPE FROM
ALL_CATALOG")
+
+ val filters = ArrayBuffer[String]()
+ if (StringUtils.isNotBlank(schema)) {
+ filters += s"OWNER LIKE '$schema'"
+ }
+
+ if (StringUtils.isNotBlank(tableName)) {
+ filters += s"$TABLE_NAME LIKE '$tableName'"
+ }
+
+ if (tTypes.nonEmpty) {
+ filters += s"(${
+ tTypes.map { tableType => s"$TABLE_TYPE = '$tableType'" }
+ .mkString(" OR ")
+ })"
+ }
+
+ if (filters.nonEmpty) {
+ query.append(" WHERE ")
+ query.append(filters.mkString(" AND "))
+ }
+
+ query.toString()
+ }
+
+ override def getColumnsQuery(
+ session: Session,
+ catalogName: String,
+ schemaName: String,
+ tableName: String,
+ columnName: String): String = {
+ val query = new StringBuilder(
+ "SELECT OWNER AS TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME FROM
ALL_TAB_COLUMNS")
+
+ val filters = ArrayBuffer[String]()
+ if (StringUtils.isNotEmpty(schemaName)) {
+ filters += s"OWNER LIKE '$schemaName'"
+ }
+ if (StringUtils.isNotEmpty(tableName)) {
+ filters += s"$TABLE_NAME LIKE '$tableName'"
+ }
+ if (StringUtils.isNotEmpty(columnName)) {
+ filters += s"$COLUMN_NAME LIKE '$columnName'"
+ }
+
+ if (filters.nonEmpty) {
+ query.append(" WHERE ")
+ query.append(filters.mkString(" AND "))
+ }
+
+ query.toString()
+ }
+
+ override def getTRowSetGenerator(): JdbcTRowSetGenerator = new
OracleTRowSetGenerator
+
+ override def getSchemaHelper(): SchemaHelper = new OracleSchemaHelper
+
+ override def name(): String = "oracle"
+}
diff --git
a/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleConnectionProvider.scala
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleConnectionProvider.scala
new file mode 100644
index 0000000000..86c2685dc8
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleConnectionProvider.scala
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.engine.jdbc.oracle
+
+import org.apache.kyuubi.engine.jdbc.connection.JdbcConnectionProvider
+
+class OracleConnectionProvider extends JdbcConnectionProvider {
+ override val name: String = classOf[OracleConnectionProvider].getName
+ // use oracle jdbc class for connection
+ override val driverClass: String = "oracle.jdbc.OracleDriver"
+}
diff --git
a/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleSchemaHelper.scala
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleSchemaHelper.scala
new file mode 100644
index 0000000000..29d708d93c
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleSchemaHelper.scala
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kyuubi.engine.jdbc.oracle
+
+import java.sql.Types
+
+import org.apache.kyuubi.engine.jdbc.schema.SchemaHelper
+import org.apache.kyuubi.shaded.hive.service.rpc.thrift.TTypeDesc
+
+class OracleSchemaHelper extends SchemaHelper {
+ override protected def toTTypeDesc(sqlType: Int, precision: Int, scale:
Int): TTypeDesc = {
+ sqlType match {
+ // case for int, returns NUMERIC type in Oracle JDBC
+ case Types.NUMERIC if scale == 0 =>
+ super.toTTypeDesc(Types.INTEGER, precision, scale)
+ // except for int
+ case Types.NUMERIC =>
+ super.toTTypeDesc(Types.DECIMAL, precision, scale)
+ case _ => super.toTTypeDesc(sqlType, precision, scale)
+ }
+ }
+}
diff --git
a/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleTRowSetGenerator.scala
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleTRowSetGenerator.scala
new file mode 100644
index 0000000000..e3f3802457
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleTRowSetGenerator.scala
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kyuubi.engine.jdbc.oracle
+
+import java.sql.Types
+
+import org.apache.kyuubi.engine.jdbc.schema.{Column,
DefaultJdbcTRowSetGenerator}
+import org.apache.kyuubi.shaded.hive.service.rpc.thrift.{TColumn, TColumnValue}
+
+class OracleTRowSetGenerator extends DefaultJdbcTRowSetGenerator {
+
+ override def toIntegerTColumn(rows: Seq[Seq[_]], ordinal: Int): TColumn = {
+ // define convertFunc in asIntegerTColumn for int type
+ asIntegerTColumn(rows, ordinal, (rows, ordinal) =>
Integer.parseInt(rows(ordinal).toString))
+ }
+
+ override def toIntegerTColumnValue(row: Seq[_], ordinal: Int): TColumnValue
= {
+ asIntegerTColumnValue(row, ordinal, x => Integer.parseInt(x.toString))
+ super.toIntegerTColumnValue(row, ordinal)
+ }
+
+ override def getColumnType(schema: Seq[Column], ordinal: Int): Int = {
+ schema(ordinal).sqlType match {
+ // case for int, returns NUMERIC type in Oracle JDBC
+ case Types.NUMERIC if schema(ordinal).scale == 0 =>
+ Types.INTEGER
+ case Types.NUMERIC =>
+ Types.DECIMAL
+ case _ => super.getColumnType(schema, ordinal)
+ }
+ }
+}
diff --git
a/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/schema/SchemaHelper.scala
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/schema/SchemaHelper.scala
index 16d46fc36f..b6ea403523 100644
---
a/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/schema/SchemaHelper.scala
+++
b/externals/kyuubi-jdbc-engine/src/main/scala/org/apache/kyuubi/engine/jdbc/schema/SchemaHelper.scala
@@ -41,7 +41,7 @@ abstract class SchemaHelper {
tColumnDesc
}
- private def toTTypeDesc(sqlType: Int, precision: Int, scale: Int): TTypeDesc
= {
+ protected def toTTypeDesc(sqlType: Int, precision: Int, scale: Int):
TTypeDesc = {
val typeEntry = new TPrimitiveTypeEntry(toTTypeId(sqlType))
typeEntry.setTypeQualifiers(toTTypeQualifiers(sqlType, precision, scale))
val tTypeDesc = new TTypeDesc()
diff --git a/externals/kyuubi-jdbc-engine/src/test/resources/oracle-compose.yml
b/externals/kyuubi-jdbc-engine/src/test/resources/oracle-compose.yml
new file mode 100644
index 0000000000..17fd3c3c82
--- /dev/null
+++ b/externals/kyuubi-jdbc-engine/src/test/resources/oracle-compose.yml
@@ -0,0 +1,42 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+version: "3"
+services:
+
+ # Oracle service (label used to access the service container)
+ oracle:
+
+ # Docker Hub image (feel free to change the tag "latest" to any other
available one)
+ image: gvenzl/oracle-free:23.5-slim
+
+ # Provide passwords and other environment variables to container
+ environment:
+ ORACLE_RANDOM_PASSWORD: true
+ APP_USER: kyuubi
+ APP_USER_PASSWORD: oracle
+
+ # Forward Oracle port
+ ports:
+ - "1521"
+
+ # Provide healthcheck script options for startup
+ healthcheck:
+ test: healthcheck.sh
+ interval: 10s
+ timeout: 5s
+ retries: 10
\ No newline at end of file
diff --git
a/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OperationWithOracleEngineSuite.scala
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OperationWithOracleEngineSuite.scala
new file mode 100644
index 0000000000..fef3755809
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OperationWithOracleEngineSuite.scala
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kyuubi.engine.jdbc.oracle
+
+import org.apache.kyuubi.config.KyuubiConf
+import org.apache.kyuubi.engine.jdbc.connection.ConnectionProvider
+import org.apache.kyuubi.operation.HiveJDBCTestHelper
+import org.apache.kyuubi.shaded.hive.service.rpc.thrift.{TGetInfoReq,
TGetInfoType}
+
+class OperationWithOracleEngineSuite extends OracleOperationSuite with
HiveJDBCTestHelper {
+
+ override protected def jdbcUrl: String = jdbcConnectionUrl
+
+ test("oracle - test for Jdbc engine getInfo") {
+ val metaData = ConnectionProvider.create(kyuubiConf).getMetaData
+
+ withSessionConf(Map(KyuubiConf.SERVER_INFO_PROVIDER.key -> "ENGINE"))()() {
+ withSessionHandle { (client, handle) =>
+ val req = new TGetInfoReq()
+ req.setSessionHandle(handle)
+ req.setInfoType(TGetInfoType.CLI_DBMS_NAME)
+ assert(client.GetInfo(req).getInfoValue.getStringValue ==
metaData.getDatabaseProductName)
+
+ val req2 = new TGetInfoReq()
+ req2.setSessionHandle(handle)
+ req2.setInfoType(TGetInfoType.CLI_DBMS_VER)
+ assert(
+ client.GetInfo(req2).getInfoValue.getStringValue ==
metaData.getDatabaseProductVersion)
+
+ val req3 = new TGetInfoReq()
+ req3.setSessionHandle(handle)
+ req3.setInfoType(TGetInfoType.CLI_MAX_COLUMN_NAME_LEN)
+ assert(client.GetInfo(req3).getInfoValue.getLenValue ==
metaData.getMaxColumnNameLength)
+
+ val req4 = new TGetInfoReq()
+ req4.setSessionHandle(handle)
+ req4.setInfoType(TGetInfoType.CLI_MAX_SCHEMA_NAME_LEN)
+ assert(client.GetInfo(req4).getInfoValue.getLenValue ==
metaData.getMaxSchemaNameLength)
+
+ val req5 = new TGetInfoReq()
+ req5.setSessionHandle(handle)
+ req5.setInfoType(TGetInfoType.CLI_MAX_TABLE_NAME_LEN)
+ assert(client.GetInfo(req5).getInfoValue.getLenValue ==
metaData.getMaxTableNameLength)
+ }
+ }
+ }
+}
diff --git
a/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleOperationSuite.scala
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleOperationSuite.scala
new file mode 100644
index 0000000000..1cec7e2cd1
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleOperationSuite.scala
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kyuubi.engine.jdbc.oracle
+
+import scala.collection.mutable.ArrayBuffer
+
+import org.apache.kyuubi.operation.HiveJDBCTestHelper
+import org.apache.kyuubi.operation.meta.ResultSetSchemaConstant.{COLUMN_NAME,
TABLE_NAME, TABLE_TYPE}
+
+abstract class OracleOperationSuite extends WithOracleEngine with
HiveJDBCTestHelper {
+ test("oracle - get tables") {
+ case class Table(catalog: String, schema: String, tableName: String,
tableType: String)
+
+ withJdbcStatement() { statement =>
+ val meta = statement.getConnection.getMetaData
+ val resultBuffer = ArrayBuffer[Table]()
+
+ var tables = meta.getTables(null, null, null, null)
+ while (tables.next()) {
+ resultBuffer +=
+ Table(
+ null,
+ null,
+ tables.getString(TABLE_NAME),
+ tables.getString(TABLE_TYPE))
+ }
+ assert(resultBuffer.contains(Table(null, null, "DUAL", "TABLE")))
+ assert(resultBuffer.contains(Table(null, null, "DUAL", "SYNONYM")))
+ assert(resultBuffer.contains(Table(null, null, "NLS_SESSION_PARAMETERS",
"VIEW")))
+ resultBuffer.clear()
+
+ statement.execute("create table T_PEOPLE (ID INTEGER not null " +
+ "constraint \"T_PEOPLE_pk\" primary key, NAME VARCHAR2(64))")
+
+ tables = meta.getTables(null, null, "T_PEOPLE", Array("TABLE"))
+ while (tables.next()) {
+ val table = Table(
+ null,
+ null,
+ tables.getString(TABLE_NAME),
+ tables.getString(TABLE_TYPE))
+ assert(table == Table(null, null, "T_PEOPLE", "TABLE"))
+ }
+
+ tables = meta.getTables(null, null, "%PEOPLE", Array("TABLE"))
+ while (tables.next()) {
+ val table = Table(
+ null,
+ null,
+ tables.getString(TABLE_NAME),
+ tables.getString(TABLE_TYPE))
+ assert(table == Table(null, null, "T_PEOPLE", "TABLE"))
+ }
+
+ statement.execute("DROP TABLE T_PEOPLE")
+ }
+ }
+
+ test("oracle - get columns") {
+ case class Column(tableName: String, columnName: String)
+
+ withJdbcStatement() { statement =>
+ val meta = statement.getConnection.getMetaData
+ val resultBuffer = ArrayBuffer[Column]()
+
+ var columns = meta.getColumns(null, null, null, null)
+ while (columns.next()) {
+ resultBuffer +=
+ Column(
+ columns.getString(TABLE_NAME),
+ columns.getString(COLUMN_NAME))
+ }
+ assert(resultBuffer.contains(Column("DUAL", "DUMMY")))
+ resultBuffer.clear()
+
+ statement.execute("create table T_PEOPLE (ID INTEGER not null " +
+ "constraint \"T_PEOPLE_pk\" primary key)")
+
+ columns = meta.getColumns(null, null, "%PEOPLE", null)
+ while (columns.next()) {
+ val column = Column(
+ columns.getString(TABLE_NAME),
+ columns.getString(COLUMN_NAME))
+ assert(column == Column("T_PEOPLE", "ID"))
+ }
+
+ statement.execute("DROP TABLE T_PEOPLE")
+ }
+ }
+}
diff --git
a/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleSessionSuite.scala
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleSessionSuite.scala
new file mode 100644
index 0000000000..9a9843617b
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleSessionSuite.scala
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kyuubi.engine.jdbc.oracle
+
+import org.apache.kyuubi.operation.HiveJDBCTestHelper
+
+class OracleSessionSuite extends WithOracleEngine with HiveJDBCTestHelper {
+ test("oracle session suite") {
+ withJdbcStatement() { statement =>
+ {
+ val resultSet = statement.executeQuery(
+ "SELECT '1' AS ID FROM DUAL")
+ val metadata = resultSet.getMetaData
+ for (i <- 1 to metadata.getColumnCount) {
+ assert(metadata.getColumnName(i) == "ID")
+ }
+ while (resultSet.next()) {
+ val id = resultSet.getObject(1)
+ assert(id == "1")
+ }
+ }
+ }
+ }
+
+ override protected def jdbcUrl: String = jdbcConnectionUrl
+}
diff --git
a/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleStatementSuite.scala
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleStatementSuite.scala
new file mode 100644
index 0000000000..21d95173db
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/OracleStatementSuite.scala
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kyuubi.engine.jdbc.oracle
+
+import java.sql.Timestamp
+
+import org.apache.kyuubi.operation.HiveJDBCTestHelper
+
+class OracleStatementSuite extends WithOracleEngine with HiveJDBCTestHelper {
+
+ test("oracle - test select") {
+ withJdbcStatement() { statement =>
+ statement.execute("create table T_TEST (ID INTEGER not null " +
+ "constraint \"T_TEST_PK\" primary key, NAME VARCHAR2(64), USER_ID
LONG, SCORE NUMBER(8,2))")
+ statement.execute("INSERT INTO T_TEST(ID, NAME, USER_ID, SCORE) " +
+ "VALUES (1, 'Bob', 43254353,89.92)")
+ val resultSet = statement.executeQuery("SELECT * FROM T_TEST")
+ while (resultSet.next()) {
+ val id = resultSet.getObject(1)
+ assert(id == 1)
+ val name = resultSet.getObject(2)
+ assert(name == "Bob")
+ val user_id = resultSet.getObject(3)
+ assert(user_id == "43254353")
+ val score = resultSet.getObject(4)
+ assert(score == new java.math.BigDecimal("89.92"))
+ }
+ statement.execute("DROP TABLE T_TEST")
+ }
+ }
+
+ test("oracle - test types") {
+ withJdbcStatement() { statement =>
+ statement.execute("CREATE TABLE TYPE_TEST(INT_COL INTEGER, NUM_8_COL
NUMBER(8, 0), " +
+ " NUM_16_4_COL NUMBER(16, 4), DECIMAL_COL DECIMAL(8, 2), DATE_COL
DATE, " +
+ "TIMESTAMP_COL TIMESTAMP, CHAR_COL CHAR(10), VARCHAR2_COL
VARCHAR2(255), " +
+ "NCHAR_COL NCHAR(10), NCHAR2_COL NVARCHAR2(255), LONG_COL LONG,
FLOAT_COL FLOAT, " +
+ "REAL_COL REAL)")
+ statement.execute("INSERT INTO TYPE_TEST( INT_COL, NUM_8_COL,
NUM_16_4_COL," +
+ " DECIMAL_COL, DATE_COL, TIMESTAMP_COL, CHAR_COL, VARCHAR2_COL,
NCHAR_COL, " +
+ "NCHAR2_COL, LONG_COL, FLOAT_COL, REAL_COL) " +
+ "VALUES ( 1, 2, 3.1415, 0.61, TO_DATE('2024-11-07', 'YYYY-MM-DD'), " +
+ "TO_TIMESTAMP('2024-11-07 22:03:01.324', 'YYYY-MM-DD HH24:MI:SS.FF3'),
" +
+ "'pi', 'alice', 'bob', 'siri', 'alex', 1.432, 3.432)")
+
+ val resultSet1 = statement.executeQuery("SELECT * FROM TYPE_TEST")
+ while (resultSet1.next()) {
+ assert(resultSet1.getObject(1) == 1)
+ assert(resultSet1.getObject(2) == 2)
+ assert(resultSet1.getObject(3) == new java.math.BigDecimal("3.1415"))
+ assert(resultSet1.getObject(4) == new java.math.BigDecimal("0.61"))
+ assert(resultSet1.getObject(5) == Timestamp.valueOf("2024-11-07
00:00:00"))
+ assert(resultSet1.getObject(6) == Timestamp.valueOf("2024-11-07
22:03:01.324"))
+ assert(resultSet1.getObject(7) == "pi ")
+ assert(resultSet1.getObject(8) == "alice")
+ assert(resultSet1.getObject(9) == "bob ")
+ assert(resultSet1.getObject(10) == "siri")
+ assert(resultSet1.getObject(11) == "alex")
+ assert(resultSet1.getObject(12) == new java.math.BigDecimal("1.432"))
+ assert(resultSet1.getObject(13) == new java.math.BigDecimal("3.432"))
+ }
+ statement.execute("DROP TABLE TYPE_TEST")
+ }
+ }
+
+ override protected def jdbcUrl: String = jdbcConnectionUrl
+}
diff --git
a/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/WithOracleContainer.scala
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/WithOracleContainer.scala
new file mode 100644
index 0000000000..148b0ea90f
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/WithOracleContainer.scala
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.engine.jdbc.oracle
+
+import java.io.File
+import java.time.Duration
+
+import com.dimafeng.testcontainers.{DockerComposeContainer, ExposedService}
+import
org.testcontainers.containers.wait.strategy.DockerHealthcheckWaitStrategy
+
+import org.apache.kyuubi.Utils
+import org.apache.kyuubi.engine.jdbc.WithJdbcServerContainer
+
+trait WithOracleContainer extends WithJdbcServerContainer {
+ private val ORACLE_PORT = 1521
+ private val ORACLE_SERVICE_NAME = "oracle"
+ protected val ORACLE_USER_NAME = "kyuubi"
+ protected val ORACLE_PASSWORD = "oracle"
+
+ override val containerDef: DockerComposeContainer.Def = {
+ // Use docker compose for healthcheck. Without healthcheck the test case
may fail,
+ // because the container is pulled up but the oracle service is not ready
yet.
+ DockerComposeContainer
+ .Def(
+ composeFiles = new File(Utils.getContextOrKyuubiClassLoader
+ .getResource("oracle-compose.yml").toURI),
+ exposedServices = Seq[ExposedService](
+ ExposedService(
+ ORACLE_SERVICE_NAME,
+ ORACLE_PORT,
+ waitStrategy =
+ new
DockerHealthcheckWaitStrategy().withStartupTimeout(Duration.ofMinutes(2)))))
+ }
+
+ protected def oracleJdbcUrl: String = withContainers { container =>
+ val feHost: String = container.getServiceHost(ORACLE_SERVICE_NAME,
ORACLE_PORT)
+ val fePort: Int = container.getServicePort(ORACLE_SERVICE_NAME,
ORACLE_PORT)
+ s"jdbc:oracle:thin:@//$feHost:$fePort/FREEPDB1"
+ }
+}
diff --git
a/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/WithOracleEngine.scala
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/WithOracleEngine.scala
new file mode 100644
index 0000000000..f30e6d1951
--- /dev/null
+++
b/externals/kyuubi-jdbc-engine/src/test/scala/org/apache/kyuubi/engine/jdbc/oracle/WithOracleEngine.scala
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kyuubi.engine.jdbc.oracle
+
+import org.apache.kyuubi.config.KyuubiConf.{ENGINE_JDBC_CONNECTION_PASSWORD,
ENGINE_JDBC_CONNECTION_URL, ENGINE_JDBC_CONNECTION_USER,
ENGINE_JDBC_DRIVER_CLASS, ENGINE_JDBC_SHORT_NAME, ENGINE_SHARE_LEVEL,
ENGINE_TYPE}
+import org.apache.kyuubi.config.KyuubiReservedKeys.KYUUBI_SESSION_USER_KEY
+import org.apache.kyuubi.engine.jdbc.WithJdbcEngine
+
+trait WithOracleEngine extends WithJdbcEngine with WithOracleContainer {
+
+ override def withKyuubiConf: Map[String, String] = withContainers {
container =>
+ Map(
+ ENGINE_SHARE_LEVEL.key -> "SERVER",
+ ENGINE_JDBC_CONNECTION_URL.key -> oracleJdbcUrl,
+ ENGINE_JDBC_CONNECTION_USER.key -> ORACLE_USER_NAME,
+ ENGINE_JDBC_CONNECTION_PASSWORD.key -> ORACLE_PASSWORD,
+ ENGINE_TYPE.key -> "jdbc",
+ ENGINE_JDBC_SHORT_NAME.key -> "oracle",
+ KYUUBI_SESSION_USER_KEY -> "kyuubi",
+ ENGINE_JDBC_DRIVER_CLASS.key -> "oracle.jdbc.OracleDriver")
+ }
+
+}
diff --git
a/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
b/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
index 436ac5fd16..e26033bf0f 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
@@ -3245,7 +3245,8 @@ object KyuubiConf {
"<li>postgresql: For establishing PostgreSQL connections.</li>" +
"<li>starrocks: For establishing StarRocks connections.</li>" +
"<li>impala: For establishing Impala connections.</li>" +
- "<li>clickhouse: For establishing clickhouse connections.</li>")
+ "<li>clickhouse: For establishing clickhouse connections.</li>" +
+ "<li>oracle: For establishing oracle connections.</li>")
.version("1.6.0")
.stringConf
.transform {
@@ -3263,6 +3264,8 @@ object KyuubiConf {
"org.apache.kyuubi.engine.jdbc.impala.ImpalaConnectionProvider"
case "ClickHouse" | "clickhouse" | "ClickHouseConnectionProvider" =>
"org.apache.kyuubi.engine.jdbc.clickhouse.ClickHouseConnectionProvider"
+ case "Oracle" | "oracle" | "OracleConnectionProvider" =>
+ "org.apache.kyuubi.engine.jdbc.oracle.OracleConnectionProvider"
case other => other
}
.createOptional
diff --git a/pom.xml b/pom.xml
index e53326796d..cf72c7195a 100644
--- a/pom.xml
+++ b/pom.xml
@@ -187,6 +187,7 @@
<paimon.artifact>paimon-spark-${spark.binary.version}</paimon.artifact>
<phoenix.version>6.0.0</phoenix.version>
<postgresql.version>42.7.2</postgresql.version>
+ <ojdbc.version>23.2.0.0</ojdbc.version>
<prometheus.version>0.16.0</prometheus.version>
<protobuf.version>3.25.5</protobuf.version>
<scalatest.version>3.2.16</scalatest.version>
@@ -1091,6 +1092,12 @@
<version>${postgresql.version}</version>
</dependency>
+ <dependency>
+ <groupId>com.oracle.database.jdbc</groupId>
+ <artifactId>ojdbc8</artifactId>
+ <version>${ojdbc.version}</version>
+ </dependency>
+
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
