I am trying to do incremental import of a table from DB into hive using sqoop import Then since the sqoop incrremental import is not able to replace the old records with new ones or write to the same directory as the previous import , i had to do the incremental import to another directory and then merge them.
This merge works fine if i imported only to HDFS. But if i imported directly to hive in the form of internal tables then merge doesn't work. I mean if the inputs to sqoop merge are normal HDFS directories or hive internal table directories it always writes to HDFS only but doesnt write the merge output to hive internal table once again. I am asking why cant it write? On Sat, Dec 7, 2013 at 10:35 PM, Jarek Jarcec Cecho <[email protected]>wrote: > Hi Lovely, > Would you mind iterating a bit about your use case? What you are trying to > accomplish? > > Jarcec > > On Fri, Dec 06, 2013 at 06:03:55AM -0800, lovely kasi wrote: > > Sqoop import uses --hive-table option to import the data to hive and the > > final result appears like an hive internal table .But why doesn't the > sqoop > > merge do the same thing.The sqoop merge can merge two HDFS directories > and > > also data from hive internal tables but it doesn't write the output in > the > > same way to an hive internal table. > > > > > > Thanks, > > Lovely >
