[
https://issues.apache.org/jira/browse/SQOOP-473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242878#comment-13242878
]
Cheolsoo Park commented on SQOOP-473:
-------------------------------------
Bilung suggested the following fix for this problem, and I like his suggestion.
For all-tables import, the table name is not specified, so in the
getColumnTypeNames() method of ConnManager class, the 'else' path is taken in
the following code:
{code}
if (null != tableName) {
// We're generating a class based on a table import.
columnTypeNames = getColumnTypeNamesForTable(tableName);
} else {
// This is based on an arbitrary query.
String query = sqlQuery;
if (query.indexOf(SqlManager.SUBSTITUTE_TOKEN) == -1) {
throw new RuntimeException("Query [" + query + "] must contain '"
+ SqlManager.SUBSTITUTE_TOKEN + "' in WHERE clause.");
}
columnTypeNames = getColumnTypeNamesForQuery(query);
}
{code}
However, this is wrong since all-tables import is not free form query import.
To fix the problem, I am going to modify the getColumnTypeNames() method so
that tableName can be given as an argument in addition to columnName and
sqlType. This change requires to modify the toJavaType() method as well, but at
all the places where toJavaType() is called, tableName is available so only
minor changes are needed.
In addition, I am going to modify the toHiveType() and toAvroType() methods so
that the signature of these methods can stay consistent with that of
toJavaType() (although this is not required).
> Sqoop import-all-tables fails with MySQL connector
> --------------------------------------------------
>
> Key: SQOOP-473
> URL: https://issues.apache.org/jira/browse/SQOOP-473
> Project: Sqoop
> Issue Type: Bug
> Components: connectors
> Affects Versions: 1.4.1-incubating
> Reporter: Prashant Gokhale
> Assignee: Cheolsoo Park
>
> Sqoop command
> sqoop import-all-tables --connect jdbc:mysql://xxxx/mysqltestdb2
> --username=xxxx --warehouse-dir output-dir/alltables
> Output:
> {noformat}
> 12/03/29 10:42:20 INFO manager.MySQLManager: Preparing to use a MySQL
> streaming resultset.
> 12/03/29 10:42:21 INFO tool.CodeGenTool: Beginning code generation
> 12/03/29 10:42:21 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM `testtable` AS t LIMIT 1
> 12/03/29 10:42:21 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.sqoop.manager.ConnManager.getColumnTypeNames(ConnManager.java:306)
> at
> org.apache.sqoop.manager.MySQLManager.toJavaType(MySQLManager.java:237)
> at org.apache.sqoop.orm.ClassWriter.toJavaType(ClassWriter.java:253)
> at org.apache.sqoop.orm.ClassWriter.generateFields(ClassWriter.java:480)
> at
> org.apache.sqoop.orm.ClassWriter.generateClassForColumns(ClassWriter.java:1306)
> at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1138)
> at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
> at
> org.apache.sqoop.tool.ImportAllTablesTool.run(ImportAllTablesTool.java:64)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> {noformat}
> I have not tested with other connectors.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
