It looks like Sqoop2 may be a good fit for you. It allows you to write your own connector - so you can basically copy the HDFS connector, add your custom code to transform the data and use your new connector instead.
On Thu, Apr 9, 2015 at 10:49 AM, KayVajj <[email protected]> wrote: > Hi, > > I have a question if we can override some methods in the QueryResult.java in > the sqoop generated Java file and use it during the import to customize a > few things. What I'm exactly trying to do is some clean up and > transformations before I write it to HDFS. I feel if this is possible it > will efficient rather doing it after the fact as we have to do expensive > reads from HDFS. > > Also I wanted to see if we can plugin some code in the process of writing it > to HDFS. Lets say If I want to write the record to some other store the only > way I see write another MR job which reads the files written by SQOOP and > process it the way I need. > > I really appreciate your inputs. > > Thanks > Kay
