[
https://issues.apache.org/jira/browse/MAPREDUCE-1327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leonid Furman updated MAPREDUCE-1327:
-------------------------------------
Affects Version/s: 0.22.0
Release Note:
I generated the patch based on the build from the revision "830358". Therefore,
the changes in this patch may not be compatible with the latest build (which
contains compilation errors - see revision "896302".
Once the build has no compiling errors, I can resubmit this patch again.
Otherwise, feel free to manually apply the code changes I made to the main
trunk after it is reviewed and approved.
Hadoop Flags: [Incompatible change]
Status: Patch Available (was: Open)
The motivation for this fix is the following. Oracle releases 10.2+ introduced
new data types "TIMESTAMP WITH TIME ZONE" and "TIMESTAMP WITH LOCAL TIME ZONE",
and Sqoop did not handle these types properly. To be specific, it did not find
the proper Java data types mapping for those SQL columns, which caused the
exception described earlier. This error occurred in two situations:
1. Sqoop was attempting to map the table metadata to Java data types while
running table import.
2. Sqoop was attempting to map the table metadata to Java data types while
loading data to hive.
Hence, I added two methods "toJavaType" and "toHiveType" method to SqlManager
interface, and implemented these methods in OracleManager class. From now on,
any new data types that are very specific to database vendor and not yet
supported can be handled properly.
Please feel free to provide feedback.
Thanks,
Leonid.
> Oracle database import via sqoop fails when a table contains the column types
> such as TIMESTAMP(6) WITH LOCAL TIME ZONE and TIMESTAMP(6) WITH TIME ZONE
> -------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1327
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1327
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/sqoop
> Affects Versions: 0.22.0
> Reporter: Leonid Furman
> Fix For: 0.22.0
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> When Oracle table contains the columns "TIMESTAMP(6) WITH LOCAL TIME ZONE"
> and "TIMESTAMP(6) WITH TIME ZONE", Sqoop fails to map values for those
> columns to valid Java data types, resulting in the following exception:
> ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.sqoop.orm.ClassWriter.generateFields(ClassWriter.java:253)
> at
> org.apache.hadoop.sqoop.orm.ClassWriter.generateClassForColumns(ClassWriter.java:701)
> at
> org.apache.hadoop.sqoop.orm.ClassWriter.generate(ClassWriter.java:597)
> at org.apache.hadoop.sqoop.Sqoop.generateORM(Sqoop.java:75)
> at org.apache.hadoop.sqoop.Sqoop.importTable(Sqoop.java:87)
> at org.apache.hadoop.sqoop.Sqoop.run(Sqoop.java:175)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.sqoop.Sqoop.main(Sqoop.java:201)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> I have modified the code for Hadoop and Sqoop so this bug is fixed on my
> machine. Please let me know if you would like me to generate the patch and
> upload it to this ticket.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
