org.netezza.error.NzSQLException: ERROR: Invalid datatype - TEXT

Hi All, I am using Spark jdbc df to store data into Netezza , I think spark is trying to create table using data type TEXT for string column , netezza doesn't support data type text. how to overwrite spark method to use VARCHAR instead of data type text ? val

Re: org.netezza.error.NzSQLException: ERROR: Invalid datatype - TEXT

Please take a look at getJDBCType() in: sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala You can register dialect for Netezza as shown in sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala Cheers On Tue, Jan 26, 2016 at 7:26 AM, kali.tumm...@gmail.com <

Re: org.netezza.error.NzSQLException: ERROR: Invalid datatype - TEXT

Fixed by creating a new netezza Dialect and registered in jdbcDialects using JdbcDialects.registerDialect(NetezzaDialect) method (spark/sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala) package com.citi.ocean.spark.elt /** * Created by st84879 on 26/01/2016. */ import

Re: org.netezza.error.NzSQLException: ERROR: Invalid datatype - TEXT

Thanks Ted trick worked, need to commit this feature in next spark release. Thanks Sri Sent from my iPhone > On 26 Jan 2016, at 15:49, Ted Yu wrote: > > Please take a look at getJDBCType() in: > sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala > >