[ https://issues.apache.org/jira/browse/SPARK-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903100#comment-14903100 ]
Suresh Thalamati commented on SPARK-10756: ------------------------------------------ This issue is similar to https://issues.apache.org/jira/browse/SPARK-9078. Fix might address saving to Teradata also. Default table exists check query is changed to : "SELECT * FROM $table WHERE 1=0". > DataFrame write to teradata using jdbc not working, tries to create table > each time irrespective of table existence > ------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-10756 > URL: https://issues.apache.org/jira/browse/SPARK-10756 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.5.0 > Reporter: Amit > > DataFrame write to teradata using jdbc not working, tries to create table > each time irrespective of table existence. > Whenever it goes to persist dataframe it checks for the table existence by > using query "SELECT 1 FROM $table LIMIT 1" and the keyword limit is not > supported in teradata. So the exception thrown by teradata for keyword is > understood as exception for table not exist in Spark and then Spark runs the > create table command irrespective of table was present. > So Create table command execution fails with the exception of table already > exist hence saving of data frame fails > Below is the method of JDBCUtils class > /** > * Returns true if the table already exists in the JDBC database. > */ > def tableExists(conn: Connection, table: String): Boolean = { > // Somewhat hacky, but there isn't a good way to identify whether a table > exists for all > // SQL database systems, considering "table" could also include the > database name. > Try(conn.prepareStatement(s"SELECT 1 FROM $table LIMIT > 1").executeQuery().next()).isSuccess > } > In case of teradata, It returns false for every save/write operation > irrespective of the fact that table was present. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org