wanghao created KYLIN-3934: ------------------------------ Summary: sqoop import param '--null-string' result in null value become blank string in hive table Key: KYLIN-3934 URL: https://issues.apache.org/jira/browse/KYLIN-3934 Project: Kylin Issue Type: Bug Components: Others Affects Versions: v2.6.0 Reporter: wanghao
when column value from jdbc is null, sqoop will write it into hive table with blank string. eg jdbc: A | B 1 | 1 2 | 2 a | null hive table: A | B 1 | 1 2 | 2 a | because of this, when I use count(distinct B), it return 3, not 2, and it can lead to other problems {code:java} String cmd = String.format(Locale.ROOT, "%s/bin/sqoop import" + generateSqoopConfigArgString() + "--connect \"%s\" --driver %s --username %s --password %s --query \"%s AND \\$CONDITIONS\" " + "--target-dir %s/%s --split-by %s --boundary-query \"%s\" --null-string '' " + "--fields-terminated-by '%s' --num-mappers %d", sqoopHome, connectionUrl, driverClass, jdbcUser, jdbcPass, selectSql, jobWorkingDir, hiveTable, splitColumn, bquery, filedDelimiter, mapperNum); {code} the param '–null=string' should be '\\N' instead of blank string ''. I resolved this problem by replace the param. But it needs be configured in kylin.properties -- This message was sent by Atlassian JIRA (v7.6.3#76005)