Hi Abe, Thanks for your input. With --inline-lob-limit 0, am able to get the data as BLOB in external store *_lob/large_obj_task_1436909150341_0086_m_0000000.lob*. The record contain *externalLob(lf,_lob/large_obj_task_1436909150341_0086_m_0000000.lob,284,18) *as data in the column. How can I read this?
Also, is there a way to get the actual data instead of this kind of external reference? The size of the encrypted column is very small. I think should not be more than 128 bytes. Thanks Suraj On Wed, Jul 15, 2015 at 10:02 PM, Abraham Elmahrek <[email protected]> wrote: > http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_large_objects > > use --inline-lob-limit 0. > > -Abe > > On Wed, Jul 15, 2015 at 9:22 PM, Suraj Nayak <[email protected]> wrote: > >> Hi All, >> >> I tried --map-column-java com.cloudera.sqoop.lib.BlobRef by using >> --query "select CAST(email as BLOB) from table_name where \$CONDITIONS", >> and --map-column-java org.apache.hadoop.io.BytesWritable. Still the same >> issue. >> >> >> Sqoop Output: 20 16 3a 34 fa 40 f6 fc 24 3d 5f b9 c6 63 b5 5f 54 77 >> Required output : [image: Inline image 1] >> >> >> >> >> >> On Wed, Jul 15, 2015 at 9:04 PM, Suraj Nayak <[email protected]> wrote: >> >>> One correction in Sqoop version. Its 1.4.4 not 1.4.5. >>> >>> On Wed, Jul 15, 2015 at 8:56 PM, Suraj Nayak <[email protected]> wrote: >>> >>>> Is it by setting --map-column-java email=com.cloudera.sqoop.lib.BlobRef? >>>> >>>> On Wed, Jul 15, 2015 at 8:11 PM, Suraj Nayak <[email protected]> wrote: >>>> >>>>> Hi Abe, >>>>> >>>>> I think the 1st option suits my use case best. But, what is the >>>>> method/way to tell Sqoop to use Large Object Loading? >>>>> >>>>> Thanks! >>>>> >>>>> On Wed, Jul 15, 2015 at 7:50 PM, Abraham Elmahrek <[email protected]> >>>>> wrote: >>>>> >>>>>> Hey man, >>>>>> >>>>>> I think Sqoop does this intentionally: >>>>>> https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/mapreduce/RawKeyTextOutputFormat.java#L65 >>>>>> . >>>>>> >>>>>> I see two things that may get you beyond this point: >>>>>> >>>>>> 1. You can try type casting to a BLOB and tell Sqoop to use Large >>>>>> Object Loading. >>>>>> 2. Try the --as-avrodatafile option and write as Avro. >>>>>> >>>>>> -Abe >>>>>> >>>>>> On Wed, Jul 15, 2015 at 7:23 PM, Suraj Nayak <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Abe, >>>>>>> >>>>>>> Thanks for highlighting missing required info quickly. Below are the >>>>>>> details: >>>>>>> >>>>>>> - *Version:* Sqoop 1.4.5 >>>>>>> - *Sqoop Command: *sqoop import --connect >>>>>>> jdbc:teradata://aa.bb.cc.internal/DATABASE=someDB --username >>>>>>> sqoop_usr >>>>>>> --password sqoop_usr --table ENCRYPTED_TBL --fields-terminated-by >>>>>>> \\001 -m >>>>>>> 1 --target-dir /tmp/ENC_TBL --connection-manager >>>>>>> "org.apache.sqoop.manager.GenericJdbcManager" --driver >>>>>>> com.teradata.jdbc.TeraDriver >>>>>>> - *Table structure:* id:varchar, count:int, first_name:binary, >>>>>>> email:binary, column5:varchar. >>>>>>> >>>>>>> Binary is used as the data is encrypted. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Jul 15, 2015 at 6:44 PM, Abraham Elmahrek <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey man, >>>>>>>> >>>>>>>> Need some details to help: >>>>>>>> >>>>>>>> - What version of Sqoop? >>>>>>>> - Sqoop command? >>>>>>>> - Database table structure (preferably a describe on the >>>>>>>> database) >>>>>>>> >>>>>>>> -Abe >>>>>>>> >>>>>>>> On Wed, Jul 15, 2015 at 6:42 PM, Suraj Nayak <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>> > Hi Sqoop Users and Developers, >>>>>>>> > >>>>>>>> > How can i import a Binary data column in a table into HDFS without >>>>>>>> > converting it into String. >>>>>>>> > >>>>>>>> > I have encrypted data in RDBMS, I need to import this column as >>>>>>>> is without >>>>>>>> > converting it into string. As of now, Sqoop is typecasting the >>>>>>>> data into >>>>>>>> > String/text and decryption is failing in Hadoop. >>>>>>>> > >>>>>>>> > Can someone provide pointers to solve this? Any workaround? >>>>>>>> > >>>>>>>> > -- >>>>>>>> > Thanks >>>>>>>> > Suraj Nayak M >>>>>>>> > >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks >>>>>>> Suraj Nayak M >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks >>>>> Suraj Nayak M >>>>> >>>> >>>> >>>> >>>> -- >>>> Thanks >>>> Suraj Nayak M >>>> >>> >>> >>> >>> -- >>> Thanks >>> Suraj Nayak M >>> >> >> >> >> -- >> Thanks >> Suraj Nayak M >> > > -- Thanks Suraj Nayak M
