On Mon, Apr 19, 2010 at 10:55 AM, Renaud Delbru wrote:
Also, do someone have some advices/examples on how to write and run unit
> tests involving hbase ?
>
Specifically about this - I found HBaseTestingUtility very useful, available
in trunk (backported to 0.20.5+ as well, I guess). For PE and
Thank you, guys. Your comments are quite helpful for me.
Andrey
> Jean-Daniel Cryans writes:
>
>
> That can be done in a couple of seconds using the normal HBase client
> in a multithreaded process, fed by a message queue if you feel like
> it. What were you trying to achieve using MR?
>
> J-D
Yes, that's a point as well, to use or not to use the MapReduce.
My advice is to use a scanner with explicit start/end key (best) or filters
(still good), not temporary tables (not so good).
HBase 0.21 will have MultiGet, so that would be another option then.
- Andy
> From: Geoff Hendrey
[...]
> As I understand it, you have a table, and you need to do
>
el_se...@hotmail.com]
Sent: Tuesday, April 20, 2010 9:37 AM
To: hbase-user@hadoop.apache.org
Subject: RE: Get operation in HBase Map-Reduce methods
Going back to the OP's question... using get() within a M/R, the answer
is yes.
However you have a problem in that you need to have to somehow deter
ng a mapper that takes a list in memory as its input source.
>
> Now here's the point where someone from Cloudera, Yahoo! or somewhere else
> says that even that piece is trivial and here's how to do it. :-)
>
> -Mike
>
>
>> > Date: Tue, 20 Apr 201
source.
Now here's the point where someone from Cloudera, Yahoo! or somewhere else says
that even that piece is trivial and here's how to do it. :-)
-Mike
> Date: Tue, 20 Apr 2010 10:15:52 +0200
> Subject: Re: Get operation in HBase Map-Reduce methods
> From: jdcry...@apache.or
Hi Andrey,
thanks a lot, your classes are quite useful.
However, my problem was not related to the HBaseClusterTestCase, but it
was a problem in my code.
Thanks
--
Renaud Delbru
On 19/04/10 19:39, Andrey S wrote:
2010/4/19 Renaud Delbru
Hi,
I am trying to create a unit test using the
That can be done in a couple of seconds using the normal HBase client
in a multithreaded process, fed by a message queue if you feel like
it. What were you trying to achieve using MR?
J-D
On Tue, Apr 20, 2010 at 12:54 PM, Andrey wrote:
> Yes, about 1k rows currently. In the future it may happen
Yes, about 1k rows currently. In the future it may happen to be more: some tens
of thousands.
Andrey
What are the numbers like? Is it 1k rows you need to process? 1M? 10B?
Your question is more about scaling (or the need to).
J-D
On Tue, Apr 20, 2010 at 8:39 AM, Andrey wrote:
> Dear All,
>
> Assumed, I've got a list of rowIDs of a HBase table. I want to get each row by
> its rowID, do some oper
Dear All,
Assumed, I've got a list of rowIDs of a HBase table. I want to get each row by
its rowID, do some operations with its values, and store the results somewhere
subsequently. Is there a good way to do this in a Map-Reduce manner?
As far as I understand, a mapper usually takes a Scan to for
2010/4/19 Renaud Delbru
> Hi,
>
> I am trying to create a unit test using the HBaseClusterTestCase and the
> RowCounter example.
> I am able to spin up a hbase table, load data inside, access the data
> (lookup and scan), but whenever I am trying to launch a map/reduce job
> (TableMapper), the ma
Hi,
I am trying to create a unit test using the HBaseClusterTestCase and the
RowCounter example.
I am able to spin up a hbase table, load data inside, access the data
(lookup and scan), but whenever I am trying to launch a map/reduce job
(TableMapper), the map/reduce functions are never execut
Hi All,
I have a tutorial on Hbase MapReduce here :
http://sujee.net/tech/articles/hbase-map-reduce-freq-counter/
It is rated PG-13 (i.e. for beginners). Uses v0.20+ mapreduce APIs.
I'd appreciate any comments & feedback from this group.
thanks
Sujee
http://sujee.net
gt; >>> what I have read the comparison portion would occur during the map
>>> step,
>>> >>> and
>>> >>> the reduce step would then aggregate the totals to come up with a
>>> >>> percent
>>> >>> matched for eac
t;>>
>> >>> What i'm trying to find are some simple map/reduce hbase examples for
>> >>> 0.20
>> >>> API. Can anyone point me in the direction of some?
>> >>
>> >>
>> >> See content of org.apache.ha
t
> >>> matched for each field.
> >>>
> >>> What i'm trying to find are some simple map/reduce hbase examples for
> >>> 0.20
> >>> API. Can anyone point me in the direction of some?
> >>
> >>
> >> See conte
ield.
>>>
>>> What i'm trying to find are some simple map/reduce hbase examples for
>>> 0.20
>>> API. Can anyone point me in the direction of some?
>>
>>
>> See content of org.apache.hadoop.hbase.mapreduce package. Checkout in
>> part
Or better,
http://kdpeterson.net/blog/2009/09/minimal-hbase-mapreduce-example.html
St.Ack
On Fri, Sep 25, 2009 at 2:32 PM, stack wrote:
> On Fri, Sep 25, 2009 at 2:09 PM, terryg wrote:
>
>>
>> I'm working on a map/reduce java program for performing a comparison of
>> columns in a single record a
On Fri, Sep 25, 2009 at 2:09 PM, terryg wrote:
>
> I'm working on a map/reduce java program for performing a comparison of
> columns in a single record and then reporting the percent matched. From
> what I have read the comparison portion would occur during the map step,
> and
> the reduce step
rcent
matched for each field.
What i'm trying to find are some simple map/reduce hbase examples for 0.20
API. Can anyone point me in the direction of some?
--
View this message in context:
http://www.nabble.com/Looking-for-simple-hbase-map-reduce-example-tp25619336p25619336.html
Sen
I thank you are looking for this jira for reduce assignment
https://issues.apache.org/jira/browse/HBASE-1199
Regions are assigned in the map to be ran on the server hosting the region
but if there is an idle server it will pull the next map task.
Billy
"bharath vissapragada"
wrote in
mess
Hi all ,
Can any one tell me where i can access some docs which gives a good
explanation about how a map-reduce scheduler in Hbase
i.e., How Map regions are created (to minimize data flow through network)
and how the reduce phases are performed so that we can minimize
the flow of keys and values a
w.nabble.com/file/p23823017/HBaseDBUploader.java
HBaseDBUploader.java
http://www.nabble.com/file/p23823017/DBInputFormat.java DBInputFormat.java
--
View this message in context:
http://www.nabble.com/HBase-Map-Reduce-tp23823017p23823017.html
Sent from the HBase User mailing list archive at Nabble.com.
Rafael,
Like it says in the documentation, the RowCounter is a very good example.
Else have a look at: http://wiki.apache.org/hadoop/Hbase/MapReduce
J-D
On Sat, Aug 23, 2008 at 10:35 AM, Rafael Turk <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I've been trying to implement a
Hello,
I've been trying to implement an HBase map & reduce job.
Going through the API documentation at
http://hadoop.apache.org/hbase/docs/r0.2.0/api/index.html I've found an nice
example, but its outdated as its refers to HBase 0.1 branch
Can anyone provide an updated ver
To: hbase-user@hadoop.apache.org
Subject: Re: Adding version number and timestamps to Hbase map reduce
(Sent this morning from the wrong email address, sorry if this gets in
the moderators way ...)
I need to be able to retrieve timestamps through the thrift interface
- after sending the query
(Sent this morning from the wrong email address, sorry if this gets in
the moderators way ...)
I need to be able to retrieve timestamps through the thrift interface
- after sending the query below this morning, I got my hands dirty &
looked into it properly. I've now implemented two thrift
From the mailing list last month:
In HBase-0.1.x, you cannot get the timestamps. In trunk (0.2.0) you
will.
--- Jim Kellerman, Senior Engineer; Powerset
-Original Message- From: Yair Even-Zohar [mailto:[EMAIL PROTECTED]
] Sent: Tuesday, July 15, 2008 8:05 AM
I got several questions:
Subject: Adding version number and timestamps to Hbase map reduce
>
> I got several questions:
>
>
>
> 1) Can we retrieve the timestamp for a given version for a data
> point in a column? I see in the documentation that
> HTable.get() may return array of versions but I
I got several questions:
1) Can we retrieve the timestamp for a given version for a data
point in a column? I see in the documentation that HTable.get() may
return array of versions but I don't see how to get the extract the
timestamp for each version.
2) Did anyone code a mapReduc
32 matches
Mail list logo
