On Feb 25, 2012, at 1:11 AM, Jarek Jarcec Cecho wrote:
> would you mind running sqoop again with parameter --verbose and sending
> entire output?
Sure. With --verbose on the create-hive-table command, I get the following:
[hduser@cmsvm01 ~]$ sqoop create-hive-table --connect
jdbc:postgresql://cmsgp/kdi_big --table kdi_eligibility --username xxxxxx -P
--verbose --fields-terminated-by ','
12/02/27 10:06:07 DEBUG tool.BaseSqoopTool: Enabled debug logging.
Enter password:
12/02/27 10:06:18 DEBUG sqoop.ConnFactory: Loaded manager factory:
com.cloudera.sqoop.manager.DefaultManagerFactory
12/02/27 10:06:18 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
com.cloudera.sqoop.manager.DefaultManagerFactory
12/02/27 10:06:18 DEBUG manager.DefaultManagerFactory: Trying with scheme:
jdbc:postgresql:
12/02/27 10:06:18 INFO manager.SqlManager: Using default fetchSize of 1000
12/02/27 10:06:18 DEBUG sqoop.ConnFactory: Instantiated ConnManager
com.cloudera.sqoop.manager.PostgresqlManager@39e87719
12/02/27 10:06:19 INFO hive.HiveImport: Loading uploaded data into Hive
12/02/27 10:06:19 DEBUG hive.HiveImport: Hive.inputTable: kdi_eligibility
12/02/27 10:06:19 DEBUG hive.HiveImport: Hive.outputTable: kdi_eligibility
12/02/27 10:06:19 DEBUG manager.SqlManager: No connection paramenters
specified. Using regular API for making connection.
12/02/27 10:06:19 DEBUG manager.SqlManager: Using fetchSize for next query: 1000
12/02/27 10:06:19 INFO manager.SqlManager: Executing SQL statement: SELECT t.*
FROM "kdi_eligibility" AS t LIMIT 1
12/02/27 10:06:19 WARN hive.TableDefWriter: Column link_key had to be cast to a
less precise type in Hive
12/02/27 10:06:19 WARN hive.TableDefWriter: Column birth_dt had to be cast to a
less precise type in Hive
12/02/27 10:06:19 WARN hive.TableDefWriter: Column death_dt had to be cast to a
less precise type in Hive
12/02/27 10:06:19 ERROR sqoop.Sqoop: Got exception running Sqoop:
java.lang.NullPointerException
java.lang.NullPointerException
at
com.cloudera.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:151)
at com.cloudera.sqoop.hive.HiveImport.importTable(HiveImport.java:193)
at
com.cloudera.sqoop.tool.CreateHiveTableTool.run(CreateHiveTableTool.java:60)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
And with --verbose on the import command, this:
[hduser@cmsvm01 ~]$ sqoop import --connect jdbc:postgresql://cmsgp/kdi_big
--username xxxxxx -P --table kdi_eligibility --hive-import -m 1 --direct
--verbose
12/02/27 10:04:41 DEBUG tool.BaseSqoopTool: Enabled debug logging.
Enter password:
12/02/27 10:04:57 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for
output. You can override
12/02/27 10:04:57 INFO tool.BaseSqoopTool: delimiters with
--fields-terminated-by, etc.
12/02/27 10:04:57 DEBUG sqoop.ConnFactory: Loaded manager factory:
com.cloudera.sqoop.manager.DefaultManagerFactory
12/02/27 10:04:57 DEBUG sqoop.ConnFactory: Trying ManagerFactory:
com.cloudera.sqoop.manager.DefaultManagerFactory
12/02/27 10:04:57 DEBUG manager.DefaultManagerFactory: Trying with scheme:
jdbc:postgresql:
12/02/27 10:04:57 INFO manager.SqlManager: Using default fetchSize of 1000
12/02/27 10:04:57 DEBUG sqoop.ConnFactory: Instantiated ConnManager
com.cloudera.sqoop.manager.DirectPostgresqlManager@1bbb60c3
12/02/27 10:04:57 INFO tool.CodeGenTool: Beginning code generation
12/02/27 10:04:57 DEBUG manager.SqlManager: No connection paramenters
specified. Using regular API for making connection.
12/02/27 10:04:57 DEBUG manager.SqlManager: Using fetchSize for next query: 1000
12/02/27 10:04:57 INFO manager.SqlManager: Executing SQL statement: SELECT t.*
FROM "kdi_eligibility" AS t LIMIT 1
12/02/27 10:04:58 ERROR tool.ImportTool: Imported Failed: Column name
'........pg.dropped.16........' not in table
And, for reference, the table is defined as follows:
CREATE TABLE kdi_eligibility (
state varchar(2) NOT NULL,
fyq varchar(5) NULL,
link_key numeric(8, 0) NOT NULL,
rectype varchar(1) NULL,
ident varchar(20) NULL,
xxx varchar(1) NULL,
county varchar(3) NULL,
zip varchar(5) NULL,
hhh varchar(12) NULL,
casenum varchar(12) NULL,
birth_dt datetime NULL,
death_dt datetime NULL,
idnum varchar(9) NULL,
s_indicator varchar(1) NULL,
e_indicator varchar(1) NULL,
xrefnum varchar(12) NULL
)
Thanks!
