Hi Ted
There is no problem in execution of Custom filters the issue is why are
they more resource exhaustive.
So i have two questions
1. Are custom filter work in similar fashion as CoPro works? i.e working
parallel on all RS.
If yes why are they more resource exhaustive
2. What exactly we mean
Maybe you can refer the Maven Repository and Artifacts for MapR
http://doc.mapr.com/display/MapR/Maven+Repository+and+Artifacts+for+MapR to
set pom
2014-06-18 13:33 GMT+08:00 Chen Wang chen.apache.s...@gmail.com:
Is this error indicating that I basically need a hbase mapr client?
currently my
I actually tried that already, but it didn't work..I added
dependency
groupIdorg.apache.hbase/groupId
artifactIdhbase/artifactId
version0.94.9-mapr-1308/version
/dependency
and removed the original hbase dependency..
On Wed, Jun 18, 2014 at 12:05 AM, Rabbit's Foot
Have you asked this question on MapR mailing list ?
Cheers
On Jun 18, 2014, at 12:14 AM, Chen Wang chen.apache.s...@gmail.com wrote:
I actually tried that already, but it didn't work..I added
dependency
groupIdorg.apache.hbase/groupId
artifactIdhbase/artifactId
hi,mailist:
i use hbase export tool export hbase data to HDFS,but it very
slow,the first 4map finished in 1min ,but the fifth map task run 35min
before it finished ,why?
A normal scan is not executed parallely on regions. It is serial way. And
a Filter is a way to filter out some data during the scan. So Filter
works along with scan. There is nothing like Filter execute parellel or so.
What was working paralle is the Endpoints. So if u impl one and call a
hi all,
the hbase client send too much requests and the some region server down.
1. region server down because of gc pause
I can see it from log:
[JvmPauseMonitor] util.JvmPauseMonitor: Detected pause in JVM
or host machine (eg GC): pause of approximately 3056ms
I
In case anyone interested, I switched to TableOutputFormat to unblock
myself..
job.setOutputFormatClass(TableOutputFormat.class);
job.getConfiguration().set(TableOutputFormat.OUTPUT_TABLE, myTable);
job.setOutputKeyClass(ImmutableBytesWritable.class);
What do you mean by down? Does it crash?
The server does not block on 0.96, it immediately sends back an exception.
(See HBASE-9467)
The client is implicitly slowed down by the retries, w/o blocking on the
server. It's managed by the hbase client itself, and it's transparent for
the client
I mean client slow itself down. e.g.
my client code(one of many threads)
while(true){
// process data and generate data need to insert to hbase
ListPut puts=...;
table.put(puts);
}
the client (I mean threads created by hbase api, not by codes) is
slowed down by
and also there so many Puts maintained by background hbase threads
that consuming too much resources
On Wed, Jun 18, 2014 at 6:54 PM, Li Li fancye...@gmail.com wrote:
I mean client slow itself down. e.g.
my client code(one of many threads)
while(true){
// process data and
Hi Anoop here is the code for Custom Filter implementation
public class MyCustomFilter extends FilterBase {
/**
* @param args
* @author vikram singh chandel
*/
private byte[] value = null;
private boolean filterRow = true;
public MyCustomFilter() {
super();
}
public MyCustomFilter(byte[]
puts will block at a point if the servers cannot follow.
See the settings starting with hbase.client in the hbase book.
As a first try, I would recommend to set hbase.client.max.perserver.
tasks to 1 in the client. You may also want to change the buffer size (
hbase.client.write.buffer)...
On
thanks, I will try it.
btw, by put threads will buffer 50,000 Gets and 50,000 Puts. it check
whether the data exist by get, then if not exist, insert by put.
while(true){
//process data
cache.add(data);
if(cache.size==50_000){
ListGet gets=...; //
Not sure why u want a custom Filter for this.. This is like SCVF. Infact
simpler than SCVF.. Seems u have only one cf:column . Is it?
wrt performance
if (Bytes.compareTo(value, kv.getValue()) == 0) {
Try replacing this with
if (Bytes.compareTo(value, 0, value.length, kv.getValueArray(),
Hi,
I have noticed that from time to time, one hbase node (out of 6) is more
active.
More active = its replication queue is higher, disk IO is higher and so on.
Is there a reason for that? Will it be the same one all the time, or it will
randomly move between nodes?
--
View this message in
Sure:
org.apache.hadoop.hbase.ipc.CallerDisconnectedException: Aborting on region
HDS-accounts,\xAE\xF6ds6296301|acco|62ef251681194,1394645253040.1082dfb8ae0fca3fbd39e362a9d15471.,
call
get(HDS-accounts,\xAE\xF6ds6296301|acco|62ef251681194,1394645253040.1082dfb8ae0fca3fbd39e362a9d15471.,
ubuntu@namenode:~$ hbase/bin/start-hbase.sh
Error: Could not find or load main class org.apache.hadoop.hbase.util.
HBaseConfTool
Error: Could not find or load main class
org.apache.hadoop.hbase.zookeeper.ZKServerTool
starting master, logging to
What you described sounds like a hot spot.
I have never heard of 'leader node' in hbase.
One cause for hotspot could be due to (suboptimal) schema design.
Cheers
On Wed, Jun 18, 2014 at 12:56 AM, yanivG yaniv.yancov...@gmail.com wrote:
Hi,
I have noticed that from time to time, one hbase
Do you have a large region that the fifth map task might be working on ?
Cheers
On Wed, Jun 18, 2014 at 2:05 AM, ch huang justlo...@gmail.com wrote:
hi,mailist:
i use hbase export tool export hbase data to HDFS,but it very
slow,the first 4map finished in 1min ,but the fifth
Hi All
I basically have a thread pool which has the task of pushing data into
HBASE. However, it is seen that wheneever the data rate is very high, which
means that i have many records to be pushed into hbase at one time. I get
the following exception which is thrown by the put api of hbase
Hello all,
I'm trying to view the master status of a 6 node (0.94.10; hadoop 1.1.2)
cluster but I keep getting a timeout exception.
The rest of the cluster is operating quite normally. From the exception, it
seems like the list tables function (required to display the web UI) is
timing out for
Have you checked region server log on 10.100.101.221
http://hdpmgr001.pse.movenetworks.com/10.100.101.221:6 ?
Cheers
On Wed, Jun 18, 2014 at 10:19 AM, Tom Brown tombrow...@gmail.com wrote:
Hello all,
I'm trying to view the master status of a 6 node (0.94.10; hadoop 1.1.2)
cluster but I
That server is the master and is not a regionserver.
--Tom
On Wed, Jun 18, 2014 at 11:29 AM, Ted Yu yuzhih...@gmail.com wrote:
Have you checked region server log on 10.100.101.221
http://hdpmgr001.pse.movenetworks.com/10.100.101.221:6 ?
Cheers
On Wed, Jun 18, 2014 at 10:19 AM, Tom
Could this happen if the master is running too many RPC tasks and can't
keep up? What about if there's too many connections to the server?
--Tom
On Wed, Jun 18, 2014 at 11:33 AM, Tom Brown tombrow...@gmail.com wrote:
That server is the master and is not a regionserver.
--Tom
On Wed, Jun
There was a recent thread related to using HTablePool :
http://search-hadoop.com/m/DHED4zrOq61/HBase+with+multiple+threadssubj=+Discuss+HBase+with+multiple+threads
Please take a look.
On Wed, Jun 18, 2014 at 10:20 AM, arunas sivaram.ar...@gmail.com wrote:
Hi All
I basically have a thread
I don't have much experience with commons pool framework.
Using HConnection effectively should be the way to go.
Cheers
On Wed, Jun 18, 2014 at 5:18 PM, arunas sivaram.ar...@gmail.com wrote:
Thanks Ted, The link was indeed helpful.
The issue is that whenever we hit this exception, since
hi,maillist:
i have A ,B,C,D,E,F six boxes,
A,B,C installed zookeeper and i installed hdfs on all six boxes,A (NN)
,B,C,D,E,F(DN)
i installed old hbase cluster on A,B,C
A hbase-master
B hbase-regionserver
C hbase-regionserver
for some reason ,i stoped the hbase cluster ,and
hi,maillist:
i have two hbase cluster , use same zookeeper cluster
,but the program is always connect to the old cluster ,not new ,even i
changed hbase-site.xml file
ch,
Not sure if this is dated, but
http://mail-archives.apache.org/mod_mbox/hbase-user/201010.mbox/%3CAANLkTimzn8hWAVhAqga2f=uz4ta22fxwt7qoe3y+m...@mail.gmail.com%3E
might be a good place to start.
-Dima
On Wed, Jun 18, 2014 at 8:10 PM, ch huang justlo...@gmail.com wrote:
hi,maillist:
What values for 'zookeeper.znode.parent' do you use in the hbase-site.xml
files ?
Cheers
On Wed, Jun 18, 2014 at 8:22 PM, ch huang justlo...@gmail.com wrote:
hi,maillist:
i have two hbase cluster , use same zookeeper cluster
,but the program is always connect to the old
On Wed, Jun 18, 2014 at 10:23 AM, Chen Wang chen.apache.s...@gmail.com wrote:
java.io.IOException: java.io.IOException: No FileSystem for scheme: maprfs
Make sure CLASSPATH and LIBRARY_PATH settings are correct
CLASSPATH should contain the $MAPR_HOME/lib/maprfs-1.0.3-mapr-3.0.0.jar
LIB_PATH
LinkedIn
Jean-Marc,
I'd like to add you to my professional network on LinkedIn.
- bhalchandra
bhalchandra bhutkar
Student at College of Engineering Pune
Pune Area, India
Confirm that you know bhalchandra bhutkar:
LinkedIn
Andrew,
I'd like to add you to my professional network on LinkedIn.
- bhalchandra
bhalchandra bhutkar
Student at College of Engineering Pune
Pune Area, India
Confirm that you know bhalchandra bhutkar:
LinkedIn
Jean-Marc,
I'd like to add you to my professional network on LinkedIn.
- bhalchandra
bhalchandra bhutkar
Student at College of Engineering Pune
Pune Area, India
Confirm that you know bhalchandra bhutkar:
35 matches
Mail list logo
