[
https://issues.apache.org/jira/browse/SQOOP-3473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Lin updated SQOOP-3473:
----------------------------
Description:
In sqoop doc, it mentions that *autoreset-to-one-mapper* Cannot be used with
*split-by <col>* option. However, when running Sqoop command with
*autoreset-to-one-mapper* and *--query*, it fails and says *split-by* is
missing. Example below:
{noformat}
sqoop import --connect jdbc:mysql://mysql-host.com/test --username username
--password password --query "SELECT * FROM test_table WHERE 1=1 AND
\$CONDITIONS" --delete-target-dir --target-di/tmp/test_table_data
--autoreset-to-one-mapper
Warning:
/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p3573.3750/bin/../lib/sqoop/../accumulo
does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
20/04/23 17:07:18 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.3
20/04/23 17:07:18 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
When importing query results in parallel, you must specify --split-by.
Try --help for usage instructions.
{noformat}
It fails here, and I think we should add extra checking to skip it if
*autoreset-to-one-mapper* is already passed. I tested in older version, but I
can see code logic is the same in the latest version
https://github.com/apache/sqoop/blob/release-1.4.7-rc0/src/java/org/apache/sqoop/tool/ImportTool.java#L1068-L1072
To workaround, simply add "-m 1" at the end to force with one mapper.
was:
In sqoop doc, it mentions that *--autoreset-to-one-mapper* Cannot be used with
*--split-by <col>* option. However, when running Sqoop command with
*--autoreset-to-one-mapper* and *--query*, it fails and says *--split-by* is
missing. Example below:
{noformat}
sqoop import --connect jdbc:mysql://mysql-host.com/test --username username
--password password --query "SELECT * FROM test_table WHERE 1=1 AND
\$CONDITIONS" --delete-target-dir --target-di/tmp/test_table_data
--autoreset-to-one-mapper
Warning:
/opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p3573.3750/bin/../lib/sqoop/../accumulo
does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
20/04/23 17:07:18 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.3
20/04/23 17:07:18 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
When importing query results in parallel, you must specify --split-by.
Try --help for usage instructions.
{noformat}
It fails here, and I think we should add extra checking to skip it if
*--autoreset-to-one-mapper* is already passed. I tested in older version, but I
can see code logic is the same in the latest version
https://github.com/apache/sqoop/blob/release-1.4.7-rc0/src/java/org/apache/sqoop/tool/ImportTool.java#L1068-L1072
To workaround, simply add "-m 1" at the end to force with one mapper.
> --autoreset-to-one-mapper does not work well with --query
> ---------------------------------------------------------
>
> Key: SQOOP-3473
> URL: https://issues.apache.org/jira/browse/SQOOP-3473
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.7
> Reporter: Eric Lin
> Priority: Major
>
> In sqoop doc, it mentions that *autoreset-to-one-mapper* Cannot be used with
> *split-by <col>* option. However, when running Sqoop command with
> *autoreset-to-one-mapper* and *--query*, it fails and says *split-by* is
> missing. Example below:
> {noformat}
> sqoop import --connect jdbc:mysql://mysql-host.com/test --username username
> --password password --query "SELECT * FROM test_table WHERE 1=1 AND
> \$CONDITIONS" --delete-target-dir --target-di/tmp/test_table_data
> --autoreset-to-one-mapper
> Warning:
> /opt/cloudera/parcels/CDH-5.13.3-1.cdh5.13.3.p3573.3750/bin/../lib/sqoop/../accumulo
> does not exist! Accumulo imports will fail.
> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> 20/04/23 17:07:18 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.13.3
> 20/04/23 17:07:18 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> When importing query results in parallel, you must specify --split-by.
> Try --help for usage instructions.
> {noformat}
> It fails here, and I think we should add extra checking to skip it if
> *autoreset-to-one-mapper* is already passed. I tested in older version, but I
> can see code logic is the same in the latest version
> https://github.com/apache/sqoop/blob/release-1.4.7-rc0/src/java/org/apache/sqoop/tool/ImportTool.java#L1068-L1072
> To workaround, simply add "-m 1" at the end to force with one mapper.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
