Eric Lin created SQOOP-3473:
-------------------------------
Summary: --autoreset-to-one-mapper does not work well with --query
Key: SQOOP-3473
URL: https://issues.apache.org/jira/browse/SQOOP-3473
Project: Sqoop
Issue Type: Bug
Affects Versions: 1.4.7
Reporter: Eric Lin
In sqoop doc, it mentions that *--autoreset-to-one-mapper* Cannot be used with
*--split-by <col>* option. However, when running Sqoop command with
*--autoreset-to-one-mapper* and *--query*, it fails and says *--split-by* is
missing. Example below:
{noformat}
sqoop import --connect jdbc:mysql://mysql-host.com/test --username username
--password password --query "SELECT * FROM test_table WHERE 1=1 AND
\$CONDITIONS" --delete-target-dir --target-di/tmp/test_table_data
--autoreset-to-one-mapper
Warning:
/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p3573.3750/bin/../lib/sqoop/../accumulo
does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
20/04/23 17:07:18 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.3
20/04/23 17:07:18 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
When importing query results in parallel, you must specify --split-by.
Try --help for usage instructions.
{noformat}
It fails here, and I think we should add extra checking to skip it if
*--autoreset-to-one-mapper* is already passed. I tested in older version, but I
can see code logic is the same in the latest version
https://github.com/apache/sqoop/blob/release-1.4.7-rc0/src/java/org/apache/sqoop/tool/ImportTool.java#L1068-L1072
To workaround, simply add "-m 1" at the end to force with one mapper.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
