Now I have redefine "MultiHfileOutputFormat" and "HfileOutputFormat" code. And it works fine for my solution. It generate data for multiple families, for multiple table and create multiple HFiles accordingly to related Family and save it at different paths.
Thanks Andrew for indicating right direction and Thanks all community for help On Tue, Sep 27, 2011 at 11:45 AM, Arsalan Bilal <[email protected]>wrote: > Now this time, i am trying to achieve my objective by playing with > HfileOutputFormat and MultiHfileOutputFormat. But did not achieve my target. > So i need your help > I am using two > context.write(key, kv1); > context.write(key, kv2); > then it should write/generate data/Hfile on two different paths for two > different tables. I am trying to achieve this. Plz help. > > > > On Mon, Sep 26, 2011 at 8:26 PM, Stack <[email protected]> wrote: > >> Can you play around w/ the class to make it work for your case? >> St.Ack >> >> On Mon, Sep 26, 2011 at 9:24 AM, Arsalan Bilal <[email protected]> >> wrote: >> > @Andrew >> > >> > Can u help me? U suggest that Try this: https://gist.github.com/1237770, i >> > saw it, How we can we use it actually? >> > i try to use it as >> job.setOutputFormatClass(MultiHFileOutputFormat.class); >> > but it generate different column Families(not correct) inside one >> > folder(recursively). I need to generate different column families in >> > different folders. >> > And there is also lack of configureIncrementalLoad Function in your >> pointed >> > file. >> > >> > Plz help me Andrew, Stack, i need your help. >> > >> > >> > On Mon, Sep 26, 2011 at 6:48 PM, Shuja Rehman <[email protected]> >> wrote: >> > >> >> @arslan, >> >> I did not get chance to look at it. >> >> >> >> >> >> On Mon, Sep 26, 2011 at 12:11 PM, Arsalan Bilal < >> [email protected]>wrote: >> >> >> >>> I am also facing same kind of problem, I am using one Map only and >> need to >> >>> generate multiple Hfiles for multiple table for insertion in Hbase? >> >>> Have you found any solution ? Can u help me to find solution of this >> one? >> >>> >> >>> >> >>> On Fri, Sep 23, 2011 at 8:11 PM, Andrew Purtell <[email protected] >> >wrote: >> >>> >> >>>> Try this: https://gist.github.com/1237770 >> >>>> >> >>>> See line 135. >> >>>> >> >>>> Best regards, >> >>>> >> >>>> >> >>>> - Andy >> >>>> >> >>>> >> >>>> Problems worthy of attack prove their worth by hitting back. - Piet >> Hein >> >>>> (via Tom White) >> >>>> >> >>>> >> >>>> >________________________________ >> >>>> >From: Shuja Rehman <[email protected]> >> >>>> >To: [email protected]; Andrew Purtell <[email protected]> >> >>>> >Sent: Friday, September 23, 2011 8:18 AM >> >>>> >Subject: Re: Creation of Hfiles for multiple tables using Single >> Bulk >> >>>> Load Job? >> >>>> > >> >>>> > >> >>>> >andrew, >> >>>> > >> >>>> >can you guide me where to change and what to change? it will help. >> >>>> >thnx >> >>>> > >> >>>> > >> >>>> >On Thu, Sep 22, 2011 at 9:35 PM, Shuja Rehman < >> [email protected]> >> >>>> wrote: >> >>>> > >> >>>> >Can you update it so I can use it? :) >> >>>> >> >> >>>> >>Thanks >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> >>On Thu, Sep 22, 2011 at 8:40 PM, Andrew Purtell < >> [email protected]> >> >>>> wrote: >> >>>> >> >> >>>> >>> From: Shuja Rehman <[email protected]> >> >>>> >>> >> >>>> >>> >> >>>> >>>> I am using bulk load to insert data into hbase. Its runs fine if >> I >> >>>> need to >> >>>> >>>> insert in one table. But Now, I have the requirement in which I >> need >> >>>> to >> >>>> >>>> insert data into more than one table. >> >>>> >>> >> >>>> >>> >> >>>> >>>We started some support for this here: >> >>>> https://issues.apache.org/jira/browse/HBASE-3727 >> >>>> >>> >> >>>> >>>It needs to be updated for HFile v2, I think, a couple of single >> line >> >>>> changes. >> >>>> >>> >> >>>> >>>Best regards, >> >>>> >>> >> >>>> >>> >> >>>> >>> - Andy >> >>>> >>> >> >>>> >>>Problems worthy of attack prove their worth by hitting back. - >> Piet >> >>>> Hein (via Tom White) >> >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >>>----- Original Message ----- >> >>>> >>>> From: Shuja Rehman <[email protected]> >> >>>> >>>> To: [email protected] >> >>>> >>>> Cc: >> >>>> >>>> Sent: Thursday, September 22, 2011 8:01 AM >> >>>> >>>> Subject: Creation of Hfiles for multiple tables using Single >> Bulk >> >>>> Load Job? >> >>>> >>>> >> >>>> >>>> Hi >> >>>> >>>> >> >>>> >>>> I am using bulk load to insert data into hbase. Its runs fine if >> I >> >>>> need to >> >>>> >>>> insert in one table. But Now, I have the requirement in which I >> need >> >>>> to >> >>>> >>>> insert data into more than one table. But it seems that using >> bulk >> >>>> load, we >> >>>> >>>> can generate hfiles for 1 table only. So the question is that >> is >> >>>> there any >> >>>> >>>> solution which enable us to generate hfiles in multiple output >> paths >> >>>> and >> >>>> >>>> then we can load these different files in different hbase >> tables. >> >>>> >>>> >> >>>> >>>> Thanks in advance >> >>>> >>>> >> >>>> >>>> >> >>>> >>>> -- >> >>>> >>>> Regards >> >>>> >>>> Shuja-ur-Rehman Baig >> >>>> >>> >> >>>> >>>> <http://pk.linkedin.com/in/shujamughal> >> >>>> >>>> >> >>>> >>> >> >>>> >> >> >>>> >> >> >>>> >>-- >> >>>> >>Regards >> >>>> >>Shuja-ur-Rehman Baig >> >>>> >> >> >>>> >> >> >>>> >> >> >>>> > >> >>>> > >> >>>> >-- >> >>>> >Regards >> >>>> >Shuja-ur-Rehman Baig >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Best Regards, >> >>> Arsalan Bilal >> >>> >> >>> >> >>> >> >> >> >> >> >> -- >> >> Regards >> >> Shuja-ur-Rehman Baig >> >> <http://pk.linkedin.com/in/shujamughal> >> >> >> >> >> > >> > >> > -- >> > Best Regards, >> > Arsalan Bilal >> > >> > > > > -- > Best Regards, > Arsalan Bilal > > > -- Best Regards, Arsalan Bilal
