If that is the only way, then we will do that. I just hope Sqoop or Netessa is able to do it. After all, it is a pretty standard ETL requirement.
From: Abraham Elmahrek [mailto:[email protected]] Sent: Monday, February 09, 2015 2:31 PM To: [email protected] Subject: Re: any way to truncate data based on column length? Hey man, Why not use Hive (https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTableAsSelect(CTAS)) CTAS to sanitize your data first? You should be able to use CTAS in conjunction with SUBSTR (https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF). -Abe On Mon, Feb 9, 2015 at 12:00 PM, Frank Luo <[email protected]<mailto:[email protected]>> wrote: We are using HDP 2.2 with Sqoop 1.4.5.2.2.0.0-2041. When loading data to Netezza tables, we found some input data length is larger than what defined in Netezza, and when that happens, the loading fails. I am wondering if there is way to tell Sqoop/Netezza driver to truncate the data when too large to fit. Here is message from NTZ log: 1: 484(308) [6, VARCHAR(255)] text field too long for column, "xxx” Thx Frank
