Bear in mind each region will return its top n, then you will have to run
another top n in your client code. This introduce a numerical error : top
on top.
On Thursday, April 10, 2014, Bogala, Chandra Reddy chandra.bog...@gs.com
wrote:
Hi,
I am planning to write endpoint coprocessor to
There is a property you can tune to lower default num of retries from 10 to
any number like 2.
On Wednesday, April 9, 2014, kanwal kanwal...@gmail.com wrote:
I'm currently running into an issue on my local setup where my application
is
unable to connect to the hbase table but I'm successfully
Well, I guessed that, what it doesn't make too much sense because it's
so slow. I only have right now 100 rows with 1000 versions each row.
I have checked the size of the dataset and each row is about 700Kbytes
(around 7Gb, 100rowsx1000versions). So, it should only check 100 rows x
700Kbytes =
Last test I have done it's to reduce the number of versions to 100.
So, right now, I have 100 rows with 100 versions each one.
Times are: (I got the same times for blocksize of 64Ks and 1Mb)
100row-1000versions + blockcache- 80s.
100row-1000versions + No blockcache- 70s.
100row-*100*versions +
Yes, I've already stopped the balancer and manually moved the regions to
other servers. Now I'm decommissioning the dfs data node on the server.
After that I will reboot the server.
δΊ 2014/4/9 22:28, Jean-Marc Spaggiari ει:
Hum.
Disable load balancer, and move all the regions manually to
What is the max version setting u have done for ur table cf? When u set
some a value, HBase has to keep all those versions. During a scan it will
read all those versions. In 94 version the default value for the max
versions is 3. I guess you have set some bigger value. If u have not,
mind
Yes, I have tried with two different values for that value of versions,
1000 and maximum value for integers.
But, I want to keep those versions. I don't want to keep just 3
versions. Imagine that I want to record a new version each minute and
store a day, those are 1440 versions.
Why is
Today I was able to catch an error during a mapreduce job that actually
mimes the rowCount more or less.
The error I saw is:
ould not sync. Requesting close of hlog
java.io.IOException: Reflection
at
In the storage layer (HFiles in HDFS) all versions of a particular cell
will be staying together. (Yes it has to be lexicographically ordered
KVs). So during a scan we will have to read all the version data. At this
storage layer it doesn't know the versions stuff etc.
-Anoop-
On Fri, Apr 11,
Sorry, I didn't get it why it should read all the timestamps and not
just the newest it they're sorted and you didn't specific any timestamp
in your filter.
On 11/04/14 12:13, Anoop John wrote:
In the storage layer (HFiles in HDFS) all versions of a particular cell
will be staying together.
Hi Lars,
So just to continue on that, when we are do MR jobs with HBase, this should
be disable too since we will read the entire table, right? Is this done by
default or it's something the client should setup manually? On my own code
I setup this manually. I looked into
I read something interesting about it in HBase TDG.
Page 344:
The StoreScanner class combines the store files and memstore that the
Store instance
contains. It is also where the exclusion happens, based on the Bloom
filter, or the timestamp. If you are asking for versions that are not more
than
In your previous example:
scan 'table1', {FILTER = ValueFilter(=, 'binary:5')}
there was no expression w.r.t. timestamp. See the following javadoc from
Scan.java:
* To only retrieve columns within a specific range of version timestamps,
* execute {@link #setTimeRange(long, long)
Hi James
Sorry for being late.
I have tested the same scenario. This works fine with Phoenix. :-)
Phoenix uses its own Filter not SCVF. In Phoenix Filter hasFilterRow() is
not implemented and by default it returns false. So the old 94 way of
filtering happens even in 98.1 code also and so
Okay, thank you, I'll check it this Monday. I didn't know that Scan checks
all the versions.
So, I was checking each column and each version although it just showed me
the newest version because I didn't indicate anything about the VERSIONS
attribute. It makes sense that it takes so long.
Thank you. I am aware of this challenge. How to call below coprocessor from
client. Can I call this coprocessor from hbase shell?. I am new to Hbase. So
may be asking very dumb questions.
Thanks,
Chandra
-Original Message-
From: Asaf Mesika [mailto:asaf.mes...@gmail.com]
Sent:
Please take a look at :
hbase-shell/src/main/ruby/hbase/security.rb
for example on how coprocessor is activated from shell.
Cheers
On Fri, Apr 11, 2014 at 11:06 AM, Bogala, Chandra Reddy
chandra.bog...@gs.com wrote:
Thank you. I am aware of this challenge. How to call below coprocessor
On Fri, Apr 11, 2014 at 6:54 AM, Jean-Marc Spaggiari
jean-m...@spaggiari.org wrote:
Hi Lars,
So just to continue on that, when we are do MR jobs with HBase, this should
be disable too since we will read the entire table, right? Is this done by
default or it's something the client should
Yep. For all of our M/R jobs we do indeed disable the caching of blocks.
In fact TableInputFormat sets cache blocks to false currently anyway.
-- Lars
From: Jean-Marc Spaggiari jean-m...@spaggiari.org
To: user user@hbase.apache.org; lars hofhansl
Hey folks,
Stack published a writeup I did on the HBase blog on the effects of rowkey
size, column-name size, CF compression, data block encoding and KV storage
approach on HFile size. For example, had large row keys vs. small row keys,
used Snappy vs. LZO vs. etc., used prefix vs. fast-diff,
Ok. I see it in TableInputFormat:
// false by default, full table scans generate too much BC churn
scan.setCacheBlocks((conf.getBoolean(SCAN_CACHEBLOCKS, false)));
So ne need to to it too in initTableMapperJob I guess...
Thanks,
JM
2014-04-11 16:53 GMT-04:00 lars hofhansl
Nice writeup, Doug.
Do you have plan to profile Prefix Tree data block encoding ?
Cheers
On Fri, Apr 11, 2014 at 3:14 PM, Doug Meil doug.m...@explorysmedical.comwrote:
Hey folks,
Stack published a writeup I did on the HBase blog on the effects of rowkey
size, column-name size, CF
Thanks Yu. My understanding is, this coprocessor is available as part of Hbase
server components. So I should be able to attach this coprocessor to any of my
tables by using alter table command.
alter 'demo-table','COPROCESSOR' ='.jar|class|priority|args'
Then from hbase shell, I
23 matches
Mail list logo
