[
https://issues.apache.org/jira/browse/SQOOP-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szabolcs Vasas updated SQOOP-3021:
----------------------------------
Description:
The following Sqoop command fails with a javac error:
{code}
sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --query
"select C1_INT,C4_VARCHAR20, REGEXP_REPLACE(TRIM(C4_VARCHAR20),'\:','!') from
T1_IMPORT WHERE \$CONDITIONS" --target-dir regex_imp --delete-target-dir -m 1
{code}
The reason is that the REGEXP_REPLACE expression contains a backslash character
which does not get escaped in ClassWriter and an invalid string gets generated
into the Java code.
SQOOP-2864 solved this problem for the double quote character we need to
generalize that solution.
was:
I've seen a user who created table with column names containing double quotes
and while code generation, we quite spectacularly failed:
{code}
16/03/02 12:14:13 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
/usr/lib/hadoop-mapreduce
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error:
')' expected
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error:
not a statement
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error:
';' expected
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error:
not a statement
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:325: error:
';' expected
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error:
')' expected
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error:
not a statement
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error:
';' expected
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error:
not a statement
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:332: error:
';' expected
__sqoop$field_map.put(""tellmewhy"", this._tellmewhy_);
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:340: error:
')' expected
else if (""tellmewhy"".equals(__fieldName)) {
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:340: error:
';' expected
else if (""tellmewhy"".equals(__fieldName)) {
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:340: error:
';' expected
else if (""tellmewhy"".equals(__fieldName)) {
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:343: error:
'else' without 'if'
else if ("'single'".equals(__fieldName)) {
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:355: error:
')' expected
else if (""tellmewhy"".equals(__fieldName)) {
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:355: error:
';' expected
else if (""tellmewhy"".equals(__fieldName)) {
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:355: error:
';' expected
else if (""tellmewhy"".equals(__fieldName)) {
^
/tmp/sqoop-root/compile/60f084d5441147b848b007d2a18b504d/bofa.java:359: error:
'else' without 'if'
else if ("'single'".equals(__fieldName)) {
^
18 errors
16/03/02 12:14:14 ERROR tool.ImportTool: Encountered IOException running import
job: java.io.IOException: Error returned by javac
at
org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:217)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:108)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
{code}
I've looked into it and the problem is that we've started preserving raw column
names inside the generated class, but we did not escape properly the case when
the column name contain double quotes.
> ClassWriter fails if a column name contains a backslash character
> -----------------------------------------------------------------
>
> Key: SQOOP-3021
> URL: https://issues.apache.org/jira/browse/SQOOP-3021
> Project: Sqoop
> Issue Type: Bug
> Affects Versions: 1.4.6
> Reporter: Szabolcs Vasas
> Assignee: Szabolcs Vasas
> Fix For: 1.4.7
>
>
> The following Sqoop command fails with a javac error:
> {code}
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --query
> "select C1_INT,C4_VARCHAR20, REGEXP_REPLACE(TRIM(C4_VARCHAR20),'\:','!') from
> T1_IMPORT WHERE \$CONDITIONS" --target-dir regex_imp --delete-target-dir -m 1
> {code}
> The reason is that the REGEXP_REPLACE expression contains a backslash
> character which does not get escaped in ClassWriter and an invalid string
> gets generated into the Java code.
> SQOOP-2864 solved this problem for the double quote character we need to
> generalize that solution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)