Thanks for the explanation. Makes perfect sense now that you've explained it. That would incur a
huge write overhead so I see whey we don't keep the counts.
~Jeff
On 3/16/2011 2:59 PM, Matt Corgan wrote:
Jeff,
The problem is that when hbase receives a put or delete, it doesn't know if
the p
Back to the issue of keeping a count, I've often wondered if this
would be easy to do without much cost at compaction time? It of course
wouldn't be a true real-time total but something like a
compactedRowCount. It could be a useful metric to expose via JMX to
get a feel for growth over time.
On
Works. Thanks.
Viv
On Wed, Mar 16, 2011 at 6:21 PM, Ted Yu wrote:
> The connection loss was due to inability of finding zookeeper quorum
>
> Use the commandline in my previous email.
>
>
> On Wed, Mar 16, 2011 at 3:18 PM, Vivek Krishna wrote:
>
>> Oops. sorry about the environment.
>>
>> I am
The connection loss was due to inability of finding zookeeper quorum
Use the commandline in my previous email.
On Wed, Mar 16, 2011 at 3:18 PM, Vivek Krishna wrote:
> Oops. sorry about the environment.
>
> I am using hadoop-0.20.2-CDH3B4, and hbase-0.90.1-CDH3B4
> and zookeeper-3.3.2-CDH3B4.
>
>
Oops. sorry about the environment.
I am using hadoop-0.20.2-CDH3B4, and hbase-0.90.1-CDH3B4
and zookeeper-3.3.2-CDH3B4.
I was able to configure jars and run the command,
hadoop jar /usr/lib/hbase/hbase-0.90.1-CDH3B4.jar rowcounter test,
but I get
java.io.IOException: Cannot create a record rea
In the future, describe your environment a bit.
The way I approach this is:
find the correct commandline from
src/main/java/org/apache/hadoop/hbase/mapreduce/package-info.java
Then I issue:
[hadoop@us01-ciqps1-name01 hbase]$ HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase
classpath` ${HADOOP_HOME}/bin/
I guess it is using the mapred class
11/03/16 20:58:27 INFO mapred.JobClient: Task Id :
attempt_201103161245_0005_m_04_0, Status : FAILED
java.io.IOException: Cannot create a record reader because of a previous
error. Please look at the previous logs lines from the task's full log for
more det
Jeff,
The problem is that when hbase receives a put or delete, it doesn't know if
the put is overwriting an existing row or inserting a new one, and it
doesn't know if whether the requested row was there to delete. This isn't
known until read or compaction time.
So to keep the counter up to date
Since we have lived so long without this information, I guess we can hold
for longer :-)
Another issue I am working on is to reduce memory footprint. See the
following discussion thread:
One of the regionserver aborted, then the master shut down itself
We have to bear in mind that there would be a
Cc: Stack
Subject: Re: Row Counters
Just a random thought. What about keeping a per region row count? Then if you
needed to get a row
count for a table you'd just have to query each region once and sum. Seems
like it wouldn't be too
expensive because you'd just have a row c
Just a random thought. What about keeping a per region row count? Then if you needed to get a row
count for a table you'd just have to query each region once and sum. Seems like it wouldn't be too
expensive because you'd just have a row counter variable. It maybe more complicated than I'm mak
$ ./bin/hadoop jar hbase*.jar rowcounter
Search for related discusson on search-hadoop
On Wed, Mar 16, 2011 at 1:35 PM, Vivek Krishna wrote:
> 1. How do I count rows fast in hbase?
>
> First I tired count 'test' , takes ages.
>
> Saw that I could use RowCounter, but looks like it is deprecated
On Wed, Mar 16, 2011 at 1:35 PM, Vivek Krishna wrote:
> 1. How do I count rows fast in hbase?
>
> First I tired count 'test' , takes ages.
>
> Saw that I could use RowCounter, but looks like it is deprecated.
It is not. Make sure you are using the one from mapreduce package as
opposed to mapre
1. How do I count rows fast in hbase?
First I tired count 'test' , takes ages.
Saw that I could use RowCounter, but looks like it is deprecated. When I
try to use it, I get
java.io.IOException: Cannot create a record reader because of a previous
error. Please look at the previous logs lines f
14 matches
Mail list logo
