This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new e3fd64b96765 [SPARK-46525][DOCKER][TESTS][FOLLOWUP] Fix docker-integration-tests on Apple Silicon for db2 and oracle with third-party docker environments e3fd64b96765 is described below commit e3fd64b96765dc04994798e79161026bd1361230 Author: Kent Yao <y...@apache.org> AuthorDate: Wed Jan 17 07:19:06 2024 -0800 [SPARK-46525][DOCKER][TESTS][FOLLOWUP] Fix docker-integration-tests on Apple Silicon for db2 and oracle with third-party docker environments ### What changes were proposed in this pull request? SPARK-46525 makes docker-integration-tests pass most tests on Apple Silicon except DB2 and Oracle. This PR modifies DockerJDBCIntegrationSuite to make it compatible with some of the third-party docker environments, such as the Colima docker environment. Developers can quickly bring up these tests for local testing after a simple pre-setup process. ### Why are the changes needed? Make it possible to test and debug locally for developers on Apple Silicon platforms. ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? - Passed OracleIntegrationSuite on my Apple Silicon Mac locally ``` [info] OracleIntegrationSuite: [info] - SPARK-33034: ALTER TABLE ... add new columns (21 seconds, 839 milliseconds) [info] .... [info] Run completed in 3 minutes, 16 seconds. [info] Total number of tests run: 26 [info] Suites: completed 1, aborted 0 [info] Tests: succeeded 26, failed 0, canceled 0, ignored 11, pending 0 [info] All tests passed. [success] Total time: 326 s (05:26), completed Jan 5, 2024, 7:10:36 PM ``` - Containers ran normally on my Apple Silicon Mac locally during and after tests above ``` 5ea7cc54165b ibmcom/db2:11.5.6.0a "/var/db2_setup/lib/…" 36 minutes ago Up 36 minutes 22/tcp, 55000/tcp, 60006-60007/tcp, 0.0.0.0:57898->50000/tcp, :::57898->50000/tcp strange_ritchie d31122b8a504 gvenzl/oracle-free:23.3 "container-entrypoin…" About an hour ago Up About an hour 0.0.0.0:64193->1521/tcp, :::64193->1521/tcp priceless_wright 75f9943fd4b6 mariadb:10.5.12 "/docker-entrypoint/…" 2 hours ago Up 2 hours 0.0.0.0:55052->3306/tcp, :::55052->3306/tcp angry_ganguly ``` ### Was this patch authored or co-authored using generative AI tooling? no Closes #44612 from yaooqinn/SPARK-46525-F. Authored-by: Kent Yao <y...@apache.org> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- .../scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala | 3 ++- .../test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala index fde228939dd4..4b5820c2302f 100644 --- a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala +++ b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DockerJDBCIntegrationSuite.scala @@ -156,7 +156,7 @@ abstract class DockerJDBCIntegrationSuite .newHostConfig() .withNetworkMode("bridge") .withPrivileged(db.privileged) - .withPortBindings(PortBinding.parse(s"$dockerIp:$externalPort:${db.jdbcPort}")) + .withPortBindings(PortBinding.parse(s"$externalPort:${db.jdbcPort}")) if (db.usesIpc) { hostConfig.withIpcMode("host") @@ -197,6 +197,7 @@ abstract class DockerJDBCIntegrationSuite } } catch { case NonFatal(e) => + logError(s"Failed to initialize Docker container for ${this.getClass.getName}", e) try { afterAll() } finally { diff --git a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala index 98d57d6a7475..fa563ae54a96 100644 --- a/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala +++ b/connector/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleDatabaseOnDocker.scala @@ -57,7 +57,7 @@ class OracleDatabaseOnDocker extends DatabaseOnDocker with Logging { val newBind = new Bind( dir.getAbsolutePath, new Volume("/docker-entrypoint-initdb.d"), - AccessMode.ro) + AccessMode.DEFAULT) hostConfigBuilder.withBinds(hostConfigBuilder.getBinds :+ newBind: _*) } catch { case e: Exception => --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org