[ https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984705#comment-16984705 ]
Hyukjin Kwon commented on SPARK-28152: -------------------------------------- it was reverted in branch-2.4 at https://github.com/apache/spark/commit/00b61e36958118e98c6dbfa0515c11c8672a62ac > Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlServerDialect > ------------------------------------------------------------------------- > > Key: SPARK-28152 > URL: https://issues.apache.org/jira/browse/SPARK-28152 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.3, 3.0.0 > Reporter: Shiv Prashant Sood > Assignee: Shiv Prashant Sood > Priority: Minor > Fix For: 3.0.0 > > > ShortType and FloatTypes are not correctly mapped to right JDBC types when > using JDBC connector. This results in tables and spark data frame being > created with unintended types. The issue was observed when validating against > SQLServer. > Some example issue > * Write from df with column type results in a SQL table of with column type > as INTEGER as opposed to SMALLINT. Thus a larger table that expected. > * read results in a dataframe with type INTEGER as opposed to ShortType > FloatTypes have a issue with read path. In the write path Spark data type > 'FloatType' is correctly mapped to JDBC equivalent data type 'Real'. But in > the read path when JDBC data types need to be converted to Catalyst data > types ( getCatalystType) 'Real' gets incorrectly gets mapped to 'DoubleType' > rather than 'FloatType'. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org