phanhuyn opened a new pull request, #44459: URL: https://github.com/apache/spark/pull/44459
Hi, thanks for checking the PR. This is a small bug fix to make Scala Spark works with Clickhouse's array type. Let me know if this would cause problem on other DBs. Also let me know if I should add more details to the PR. ### Why are the changes needed? The PR is to fix issue describe at: https://github.com/ClickHouse/clickhouse-java/issues/1505 When using spark to write an array of string to Clickhouse, the Clickhouse JDBC driver throws `java.lang.IllegalArgumentException: Unknown data type: string` exception. The exception was due to Spark JDBC utils passing an invalid type value `string` (should be `String`). The original type values retrieved from Clickhouse JDBC is correct, but Spark JDBC utils attempts to convert to type string to lower case: https://github.com/apache/spark/blob/6b931530d75cb4f00236f9c6283de8ef450963ad/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala#L639 ### What changes were proposed in this pull request? - Remove the lowercase cast. The string value retrieved from JDBC driver implementation should be passed as-is, Spark shouldn't try to modify the value. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Follow the reproduce step at: https://github.com/ClickHouse/clickhouse-java/issues/1505 - Create ClickHouse table with array string type - Write scala Spark job to write to clickhouse - Verify the change fixes the issue ### Was this patch authored or co-authored using generative AI tooling? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
