[ https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan updated SPARK-28152: -------------------------------- Labels: (was: release-notes) > Mapped ShortType to SMALLINT and FloatType to REAL for MsSqlServerDialect > ------------------------------------------------------------------------- > > Key: SPARK-28152 > URL: https://issues.apache.org/jira/browse/SPARK-28152 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.3, 3.0.0 > Reporter: Shiv Prashant Sood > Assignee: Shiv Prashant Sood > Priority: Minor > Fix For: 3.0.0 > > > ShortType and FloatTypes are not correctly mapped to right JDBC types when > using JDBC connector. This results in tables and spark data frame being > created with unintended types. The issue was observed when validating against > SQLServer. > Some example issue > * Write from df with column type results in a SQL table of with column type > as INTEGER as opposed to SMALLINT. Thus a larger table that expected. > * read results in a dataframe with type INTEGER as opposed to ShortType > FloatTypes have a issue with read path. In the write path Spark data type > 'FloatType' is correctly mapped to JDBC equivalent data type 'Real'. But in > the read path when JDBC data types need to be converted to Catalyst data > types ( getCatalystType) 'Real' gets incorrectly gets mapped to 'DoubleType' > rather than 'FloatType'. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org