[ https://issues.apache.org/jira/browse/SQOOP-473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242878#comment-13242878 ]
Cheolsoo Park commented on SQOOP-473: ------------------------------------- Bilung suggested the following fix for this problem, and I like his suggestion. For all-tables import, the table name is not specified, so in the getColumnTypeNames() method of ConnManager class, the 'else' path is taken in the following code: {code} if (null != tableName) { // We're generating a class based on a table import. columnTypeNames = getColumnTypeNamesForTable(tableName); } else { // This is based on an arbitrary query. String query = sqlQuery; if (query.indexOf(SqlManager.SUBSTITUTE_TOKEN) == -1) { throw new RuntimeException("Query [" + query + "] must contain '" + SqlManager.SUBSTITUTE_TOKEN + "' in WHERE clause."); } columnTypeNames = getColumnTypeNamesForQuery(query); } {code} However, this is wrong since all-tables import is not free form query import. To fix the problem, I am going to modify the getColumnTypeNames() method so that tableName can be given as an argument in addition to columnName and sqlType. This change requires to modify the toJavaType() method as well, but at all the places where toJavaType() is called, tableName is available so only minor changes are needed. In addition, I am going to modify the toHiveType() and toAvroType() methods so that the signature of these methods can stay consistent with that of toJavaType() (although this is not required). > Sqoop import-all-tables fails with MySQL connector > -------------------------------------------------- > > Key: SQOOP-473 > URL: https://issues.apache.org/jira/browse/SQOOP-473 > Project: Sqoop > Issue Type: Bug > Components: connectors > Affects Versions: 1.4.1-incubating > Reporter: Prashant Gokhale > Assignee: Cheolsoo Park > > Sqoop command > sqoop import-all-tables --connect jdbc:mysql://xxxx/mysqltestdb2 > --username=xxxx --warehouse-dir output-dir/alltables > Output: > {noformat} > 12/03/29 10:42:20 INFO manager.MySQLManager: Preparing to use a MySQL > streaming resultset. > 12/03/29 10:42:21 INFO tool.CodeGenTool: Beginning code generation > 12/03/29 10:42:21 INFO manager.SqlManager: Executing SQL statement: SELECT > t.* FROM `testtable` AS t LIMIT 1 > 12/03/29 10:42:21 ERROR sqoop.Sqoop: Got exception running Sqoop: > java.lang.NullPointerException > java.lang.NullPointerException > at > org.apache.sqoop.manager.ConnManager.getColumnTypeNames(ConnManager.java:306) > at > org.apache.sqoop.manager.MySQLManager.toJavaType(MySQLManager.java:237) > at org.apache.sqoop.orm.ClassWriter.toJavaType(ClassWriter.java:253) > at org.apache.sqoop.orm.ClassWriter.generateFields(ClassWriter.java:480) > at > org.apache.sqoop.orm.ClassWriter.generateClassForColumns(ClassWriter.java:1306) > at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1138) > at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82) > at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390) > at > org.apache.sqoop.tool.ImportAllTablesTool.run(ImportAllTablesTool.java:64) > at org.apache.sqoop.Sqoop.run(Sqoop.java:145) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69) > at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) > at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) > at org.apache.sqoop.Sqoop.main(Sqoop.java:238) > at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57) > {noformat} > I have not tested with other connectors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira