http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_large_objects
use --inline-lob-limit 0. -Abe On Wed, Jul 15, 2015 at 9:22 PM, Suraj Nayak <[email protected]> wrote: > Hi All, > > I tried --map-column-java com.cloudera.sqoop.lib.BlobRef by using --query > "select CAST(email as BLOB) from table_name where \$CONDITIONS", and > --map-column-java org.apache.hadoop.io.BytesWritable. Still the same > issue. > > > Sqoop Output: 20 16 3a 34 fa 40 f6 fc 24 3d 5f b9 c6 63 b5 5f 54 77 > Required output : [image: Inline image 1] > > > > > > On Wed, Jul 15, 2015 at 9:04 PM, Suraj Nayak <[email protected]> wrote: > >> One correction in Sqoop version. Its 1.4.4 not 1.4.5. >> >> On Wed, Jul 15, 2015 at 8:56 PM, Suraj Nayak <[email protected]> wrote: >> >>> Is it by setting --map-column-java email=com.cloudera.sqoop.lib.BlobRef? >>> >>> On Wed, Jul 15, 2015 at 8:11 PM, Suraj Nayak <[email protected]> wrote: >>> >>>> Hi Abe, >>>> >>>> I think the 1st option suits my use case best. But, what is the >>>> method/way to tell Sqoop to use Large Object Loading? >>>> >>>> Thanks! >>>> >>>> On Wed, Jul 15, 2015 at 7:50 PM, Abraham Elmahrek <[email protected]> >>>> wrote: >>>> >>>>> Hey man, >>>>> >>>>> I think Sqoop does this intentionally: >>>>> https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/mapreduce/RawKeyTextOutputFormat.java#L65 >>>>> . >>>>> >>>>> I see two things that may get you beyond this point: >>>>> >>>>> 1. You can try type casting to a BLOB and tell Sqoop to use Large >>>>> Object Loading. >>>>> 2. Try the --as-avrodatafile option and write as Avro. >>>>> >>>>> -Abe >>>>> >>>>> On Wed, Jul 15, 2015 at 7:23 PM, Suraj Nayak <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Abe, >>>>>> >>>>>> Thanks for highlighting missing required info quickly. Below are the >>>>>> details: >>>>>> >>>>>> - *Version:* Sqoop 1.4.5 >>>>>> - *Sqoop Command: *sqoop import --connect >>>>>> jdbc:teradata://aa.bb.cc.internal/DATABASE=someDB --username sqoop_usr >>>>>> --password sqoop_usr --table ENCRYPTED_TBL --fields-terminated-by >>>>>> \\001 -m >>>>>> 1 --target-dir /tmp/ENC_TBL --connection-manager >>>>>> "org.apache.sqoop.manager.GenericJdbcManager" --driver >>>>>> com.teradata.jdbc.TeraDriver >>>>>> - *Table structure:* id:varchar, count:int, first_name:binary, >>>>>> email:binary, column5:varchar. >>>>>> >>>>>> Binary is used as the data is encrypted. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jul 15, 2015 at 6:44 PM, Abraham Elmahrek <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hey man, >>>>>>> >>>>>>> Need some details to help: >>>>>>> >>>>>>> - What version of Sqoop? >>>>>>> - Sqoop command? >>>>>>> - Database table structure (preferably a describe on the database) >>>>>>> >>>>>>> -Abe >>>>>>> >>>>>>> On Wed, Jul 15, 2015 at 6:42 PM, Suraj Nayak <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> > Hi Sqoop Users and Developers, >>>>>>> > >>>>>>> > How can i import a Binary data column in a table into HDFS without >>>>>>> > converting it into String. >>>>>>> > >>>>>>> > I have encrypted data in RDBMS, I need to import this column as is >>>>>>> without >>>>>>> > converting it into string. As of now, Sqoop is typecasting the >>>>>>> data into >>>>>>> > String/text and decryption is failing in Hadoop. >>>>>>> > >>>>>>> > Can someone provide pointers to solve this? Any workaround? >>>>>>> > >>>>>>> > -- >>>>>>> > Thanks >>>>>>> > Suraj Nayak M >>>>>>> > >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thanks >>>>>> Suraj Nayak M >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Thanks >>>> Suraj Nayak M >>>> >>> >>> >>> >>> -- >>> Thanks >>> Suraj Nayak M >>> >> >> >> >> -- >> Thanks >> Suraj Nayak M >> > > > > -- > Thanks > Suraj Nayak M >
