Hi,
i am facing a similar problem. I Need to read a large file to put into
different hbase tables. Until now i have done it with
MultiTableOutputFormat directly from the Mapper. Thats works ok, but i
believe it will become quite slow when i try larger files. But I thought
it is a good chance to try bulk load as well and compare them because i
think it can be much faster in my scenario.
It works fast and like a charm for one table. But i want to avoid
reading the file a few times.
Did you finally manage to make it work? Also with the lack of
"configureIncrementalLoad" in the MultiHFileOutputFormat class? I am
having only one family.
Regards,
Christopher
Am -10.01.-28163 20:59, schrieb Arsalan Bilal:
Now I have redefine "MultiHfileOutputFormat" and "HfileOutputFormat" code.
And it works fine for my solution. It generate data for multiple families,
for multiple table and create multiple HFiles accordingly to related Family
and save it at different paths.
Thanks Andrew for indicating right direction and Thanks all community for
help
On Tue, Sep 27, 2011 at 11:45 AM, Arsalan Bilal<charsalanbi...@gmail.com>wrote:
Now this time, i am trying to achieve my objective by playing with
HfileOutputFormat and MultiHfileOutputFormat. But did not achieve my target.
So i need your help
I am using two
context.write(key, kv1);
context.write(key, kv2);
then it should write/generate data/Hfile on two different paths for two
different tables. I am trying to achieve this. Plz help.
On Mon, Sep 26, 2011 at 8:26 PM, Stack<st...@duboce.net> wrote:
Can you play around w/ the class to make it work for your case?
St.Ack
On Mon, Sep 26, 2011 at 9:24 AM, Arsalan Bilal<charsalanbi...@gmail.com>
wrote:
@Andrew
Can u help me? U suggest that Try this: https://gist.github.com/1237770, i
saw it, How we can we use it actually?
i try to use it as
job.setOutputFormatClass(MultiHFileOutputFormat.class);
but it generate different column Families(not correct) inside one
folder(recursively). I need to generate different column families in
different folders.
And there is also lack of configureIncrementalLoad Function in your
pointed
file.
Plz help me Andrew, Stack, i need your help.
On Mon, Sep 26, 2011 at 6:48 PM, Shuja Rehman<shujamug...@gmail.com>
wrote:
@arslan,
I did not get chance to look at it.
On Mon, Sep 26, 2011 at 12:11 PM, Arsalan Bilal<
charsalanbi...@gmail.com>wrote:
I am also facing same kind of problem, I am using one Map only and
need to
generate multiple Hfiles for multiple table for insertion in Hbase?
Have you found any solution ? Can u help me to find solution of this
one?
On Fri, Sep 23, 2011 at 8:11 PM, Andrew Purtell<apurt...@apache.org
wrote:
Try this: https://gist.github.com/1237770
See line 135.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet
Hein
(via Tom White)
________________________________
From: Shuja Rehman<shujamug...@gmail.com>
To: user@hbase.apache.org; Andrew Purtell<apurt...@apache.org>
Sent: Friday, September 23, 2011 8:18 AM
Subject: Re: Creation of Hfiles for multiple tables using Single
Bulk
Load Job?
andrew,
can you guide me where to change and what to change? it will help.
thnx
On Thu, Sep 22, 2011 at 9:35 PM, Shuja Rehman<
shujamug...@gmail.com>
wrote:
Can you update it so I can use it? :)
Thanks
On Thu, Sep 22, 2011 at 8:40 PM, Andrew Purtell<
apurt...@apache.org>
wrote:
From: Shuja Rehman<shujamug...@gmail.com>
I am using bulk load to insert data into hbase. Its runs fine if
I
need to
insert in one table. But Now, I have the requirement in which I
need
to
insert data into more than one table.
We started some support for this here:
https://issues.apache.org/jira/browse/HBASE-3727
It needs to be updated for HFile v2, I think, a couple of single
line
changes.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. -
Piet
Hein (via Tom White)
----- Original Message -----
From: Shuja Rehman<shujamug...@gmail.com>
To: user@hbase.apache.org
Cc:
Sent: Thursday, September 22, 2011 8:01 AM
Subject: Creation of Hfiles for multiple tables using Single
Bulk
Load Job?
Hi
I am using bulk load to insert data into hbase. Its runs fine if
I
need to
insert in one table. But Now, I have the requirement in which I
need
to
insert data into more than one table. But it seems that using
bulk
load, we
can generate hfiles for 1 table only. So the question is that
is
there any
solution which enable us to generate hfiles in multiple output
paths
and
then we can load these different files in different hbase
tables.
Thanks in advance
--
Regards
Shuja-ur-Rehman Baig
<http://pk.linkedin.com/in/shujamughal>
--
Regards
Shuja-ur-Rehman Baig
--
Regards
Shuja-ur-Rehman Baig
--
Best Regards,
Arsalan Bilal
--
Regards
Shuja-ur-Rehman Baig
<http://pk.linkedin.com/in/shujamughal>
--
Best Regards,
Arsalan Bilal
--
Best Regards,
Arsalan Bilal
