Seth Fitzsimmons created SPARK-28100:
----------------------------------------
Summary: Unable to override JDBC types for ByteType when providing
a custom JdbcDialect for Postgres
Key: SPARK-28100
URL: https://issues.apache.org/jira/browse/SPARK-28100
Project: Spark
Issue Type: Bug
Components: Input/Output
Affects Versions: 2.4.3
Reporter: Seth Fitzsimmons
{{AggregatedDialect}} defines [{{getJDBCType}}
|https://github.com/apache/spark/blob/1217996f1574f758d8cccc1c4e3846452d24b35b/sql/core/src/main/scala/org/apache/spark/sql/jdbc/AggregatedDialect.scala#L41-L43]
as:
{{dialects.flatMap(_.getJDBCType(dt)).headOption}}
However, when attempting to write a {{ByteType}}, {{PostgreDialect}} currently
throws:
https://github.com/apache/spark/blob/1217996f1574f758d8cccc1c4e3846452d24b35b/sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala#L83
This prevents any other registered dialects from providing a JDBC type for
{{ByteType}} as the last one (Spark's default {{PostgresDialect}}) will raise
an uncaught exception.
https://github.com/apache/spark/pull/24845 addresses this by providing a
mapping, but the general problem holds if any {{JdbcDialect}} implementations
ever throw in {{getJDBCType}} for any reason.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
