Re: Java Commited Virtual Memory significally larged then Heap Memory

My bad. All things work. Thanks for Todd Lipcon :) 2011/1/11 Andrey Stepachev > I tried to set MALLOC_ARENA_MAX=2. But still the same issue like in LZO > problem thread. All those 65M blocks here. And JVM continues to eat memory > on heavy write load. And yes, I use "improved" kernel > Linux 2.

Re: incrementColumnValue

"increment" and "decrement" are not idempotent. Map/Reduce requires you do things in an idempotent fashion (the same task may get executed multiple times, even simultaneously). On Tue, Jan 11, 2011 at 4:07 PM, Billy Pearson wrote: > Is there a way to make a mapreduce job and use incrementColumnVa

Re: problem with LZO compressor on write only loads

Yes, I tried. 2011/1/12 Sandy Pratt > I'm curious if you've tried -XX:MaxDirectMemorySize=256m (or whatever > value). > > > -Original Message- > > From: Andrey Stepachev [mailto:oct...@gmail.com] > > Sent: Tuesday, January 11, 2011 12:58 > > To: user@hbase.apache.org > > Subject: Re: pro

Re: Verbose logging with compression

Sounds like all upside to me... was a little tricky to notice since it still compresses without them Matt On Tue, Jan 11, 2011 at 10:14 PM, Stack wrote: > Oh. Yeah. Makes sense. We used to bundle the native libs but we > seem to have dropped them. We should add them back? > St.Ack > > On T

Re: KeeperErrorCode = NoNode for /hbase/master when running from local m/r job

Here is the M/R job log: http://pastebin.com/pbwgKup4 And Zookeper messages during the job execution: http://pastebin.com/rxNZDD7x On 1/11/11, Jean-Daniel Cryans wrote: > hbase.master isn't used since 0.20.0 > > Can you post a log we can look at? Ideally the full log from the job, > on pasteb

Re: Verbose logging with compression

Oh. Yeah. Makes sense. We used to bundle the native libs but we seem to have dropped them. We should add them back? St.Ack On Tue, Jan 11, 2011 at 3:24 PM, Matt Corgan wrote: > Turns out this is what happens if you don't have the native libraries set up > correctly.  The data still gets compr

Re: KeeperErrorCode = NoNode for /hbase/master when running from local m/r job

hbase.master isn't used since 0.20.0 Can you post a log we can look at? Ideally the full log from the job, on pastebin.com or similar. J-D On Tue, Jan 11, 2011 at 5:24 PM, jason wrote: > Hi all, > > I am trying to connect to a remote hbase from my local M/R job (to > populate an hbase table wit

Re: incrementColumnValue

Hey, It is not possible, nor alas would it be a good idea. Speculative execution can cause jobs to run twice with the "results" discarded from one. The hbase output format doesnt really have a good way to 'discard' results, since we are outputting to a table not to a file that can be tossed. Fu

KeeperErrorCode = NoNode for /hbase/master when running from local m/r job

Hi all, I am trying to connect to a remote hbase from my local M/R job (to populate an hbase table with results) Since my job is local I am not using any configuration files but pass all the conf properties with -D arguments. I am passing "hbase.zookeeper.quorum" and "hbase.master" paths in the C

incrementColumnValue

Is there a way to make a mapreduce job and use incrementColumnValue in place of Put? I am trying to move a job over from thrift and have to be able to use incrementColumnValue as a output but I can not seams to work it out with out calling HTable every map. small example would be nice if any

Re: performance numbers for single query(not reads/sec but single query)

You should check like below http://blog.rajeevsharma.in/2009/06/secondary-indexes-in-hbase.html source samples https://issues.apache.org/jira/browse/HBASE-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel Th

Re: Cluster Wide Pauses

java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode) On Tue, Jan 11, 2011 at 2:57 PM, Wayne wrote: > What is shared across all nodes that could stop everything? Originally I > suspected the node with the .MET

performance numbers for single query(not reads/sec but single query)

I read something somewhere that had performance using a self-created index(with either lily or the tableindex or something) and it has some numbers in there. I was trying to redig that up since we are looking at in-memory datagrid technology and I figured we could just use hbase instead. (we do a

Re: Verbose logging with compression

Turns out this is what happens if you don't have the native libraries set up correctly. The data still gets compressed using the pure java codec, but it doesn't cache the codec and gives you a warning each time it creates it for each block. On Mon, Jan 10, 2011 at 2:41 PM, Stack wrote: > Thats

Re: Cluster Wide Pauses

What is shared across all nodes that could stop everything? Originally I suspected the node with the .META. table and GC pauses but could never find the smoking gun. What JVM are you running? We use u23 and I am starting to consider rolling back to u16 per the previous post. On Tue, Jan 11, 2011 a

Re: Cluster Wide Pauses

I am running YCSB to load a modest amount of data (2 M rows) and see something similar. For example: 10 sec: 135327 operations; 13505.69 current ops/sec; [INSERT AverageLatency(ms)=0.84] 20 sec: 204440 operations; 6883.08 current ops/sec; [INSERT AverageLatency(ms)=3.51] 30 sec: 250166 operatio

Cluster Wide Pauses

We have very frequent cluster wide pauses that stop all reads and writes for seconds. We are constantly loading data to this cluster of 10 nodes. These pauses can happen as frequently as every minute but sometimes are not seen for 15+ minutes. Basically watching the Region server list with request

Re: version recommendation needed

Thanks! for suggestions. -Dani http://www.cc.gatech.edu/~iar3/ On Mon, Jan 10, 2011 at 1:58 PM, Jean-Daniel Cryans wrote: > Most people run the latest of either CentOS or Ubuntu I think. Java > u16 is know to be very stable, u18 is to be avoided like the plague. > Regarding HBase, 0.20.6 is the

Re: CPU Wait Problems

That was our first thought as we had motherboard controller based raid 0 in place. We have since rebuilt all nodes with jbod using the recommended etx4 partition creation and mount parameters. So far so good. On Tue, Jan 11, 2011 at 4:24 PM, Ted Dunning wrote: > I have seen this also with evil d

Re: CPU Wait Problems

I have seen this also with evil disk controllers on the edge of dying. On Tue, Jan 11, 2011 at 12:10 PM, Wayne wrote: > Thanks a lot for the heads up on this. We have only seen this once, but if > we start seeing it more we will definitely try to go back to a previous > version. We are using 1.6

RE: problem with LZO compressor on write only loads

I'm curious if you've tried -XX:MaxDirectMemorySize=256m (or whatever value). > -Original Message- > From: Andrey Stepachev [mailto:oct...@gmail.com] > Sent: Tuesday, January 11, 2011 12:58 > To: user@hbase.apache.org > Subject: Re: problem with LZO compressor on write only loads > > Not

Re: problem with LZO compressor on write only loads

Not only with LZO, but with regular gzip I got the same issue (on sun and jrocket jvm). Looks like some bug for me. Don't know how to beat this bug. 2011/1/3 Friso van Vollenhoven > Hi all, > > I seem to run into a problem that occurs when using LZO compression on a > heavy write only load. I am

Re: Java Commited Virtual Memory significally larged then Heap Memory

I tried to set MALLOC_ARENA_MAX=2. But still the same issue like in LZO problem thread. All those 65M blocks here. And JVM continues to eat memory on heavy write load. And yes, I use "improved" kernel Linux 2.6.34.7-0.5. 2011/1/11 Xavier Stevens > Are you using a newer linux kernel with the new

Re: CPU Wait Problems

Thanks a lot for the heads up on this. We have only seen this once, but if we start seeing it more we will definitely try to go back to a previous version. We are using 1.6u23. Are you using the Sun JVM? We were previously working with cassandra and found the openJDK 1.6u17 to be a lot better for o

Re: Thrift WAL

Lars Francke already did a lot of work towards having the Thrift API match the java one in this jira: https://issues.apache.org/jira/browse/HBASE-1744 J-D On Tue, Jan 11, 2011 at 1:46 AM, Jan Lukavský wrote: > Hi J-D, >  should an issue by filled on this? We might try to provide a patch, after >

Re: Hadoop replication in 0.20

Thats right. Its only available in 0.90/0.89.X. St.Ack On Tue, Jan 11, 2011 at 8:44 AM, Joel Halbert wrote: > I use a single Master (non-distributed) HBase instance, but I would like > to replicate to one other machine. > > I found a description of replication functionality: > http://hbase.apach

Re: java.net.SocketException: Too many open files

Whats Alex says. You can see what the user running hbase sees for ulimit by looking in the log. Its the first thing printed. Grep ulimit. St.Ack On Tue, Jan 11, 2011 at 12:59 AM, 陈加俊 wrote: > I set the env as fallows: > > $ ulimit -n > 65535 > >  $ ulimit -a > core file size          (blocks,

Re: CPU Wait Problems

Which jdk are you using? We've had similar problems with jdk1.6u22 on Ubuntu 10.04 in Amazon EC2. Nodes would lock up for 20-40+ minutes. We haven't done any conclusive tests yet, but we haven't seen the same problems after down rev'ing to jdk1.6u16. -brent On Mon, Jan 10, 2011 at 12:59 PM, W

Re: HBase replication in 0.20

Apologies the title of this post should have beeen "HBase replication in 0.2" - changed. On Tue, 2011-01-11 at 16:44 +, Joel Halbert wrote: > I use a single Master (non-distributed) HBase instance, but I would like > to replicate to one other machine. > > I found a description of replication

Re: java.net.SocketException: Too many open files

Make sure you've set the limit for correct user. Also check out this info (check out "File descriptor limits" section): http://www.cloudera.com/blog/2009/03/configuration-parameters-what-can-you-just-ignore Alex Baranau Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBas

Hadoop replication in 0.20

I use a single Master (non-distributed) HBase instance, but I would like to replicate to one other machine. I found a description of replication functionality: http://hbase.apache.org/docs/r0.89.20100726/replication.html but I note that the documentation is for r0.89, and I am using 0.20. Is the

Re: Java Commited Virtual Memory significally larged then Heap Memory

Are you using a newer linux kernel with the new and "improved" memory allocator? If so try setting this in hadoop-env.sh: export MALLOC_ARENA_MAX= Maybe start by setting it to 4. You can thank Todd Lipcon if this works for you. Cheers, -Xavier On 1/11/11 7:24 AM, Andrey Stepachev wrote: > N

Re: Strange regionserver behavior with GZ compression

I have not tested GZ compression on a 256mb region size yet. When I start a new round of testing I will, thanks for the idea, -chris On Mon, Jan 10, 2011 at 12:54 PM, Sandy Pratt wrote: > Chris, > > I'm curious if this happens when hbase.hregion.max.filesize is set to the > default 256m. Have

Re: Strange regionserver behavior with GZ compression

Hi Stack, Thanks for taking a look. I think I caught a regionserver compacting: http://pastebin.com/y9BQaVeJ http://pastebin.com/ZMxwEX5j thanks again, -chris On Mon, Jan 10, 2011 at 1:52 PM, Stack wrote: > Odd. Mind thread dumping the regionserver a few times and > pastebining it during a

Re: HTable.put(List puts) perform batch insert?

Re HBaseHUT J-D was correct: you will gain speed with it in case you need Get & Put operation to perform your updates. Don't forget to play with writeToWAL, writeBuffer (with autoFlush=false) attributes! Alex Baranau Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase

Re: Java Commited Virtual Memory significally larged then Heap Memory

No. I don't use LZO. I tried even remove any native support (i.e. all .so from class path) and use java gzip. But nothing. 2011/1/11 Friso van Vollenhoven > Are you using LZO by any chance? If so, which version? > > Friso > > > On 11 jan 2011, at 15:57, Andrey Stepachev wrote: > > > After start

Re: Java Commited Virtual Memory significally larged then Heap Memory

Are you using LZO by any chance? If so, which version? Friso On 11 jan 2011, at 15:57, Andrey Stepachev wrote: > After starting the hbase in jroсkit found the same memory leakage. > > After the launch > > Every 2,0 s: date & & ps - sort =- rss-eopid, rss, vsz, pcpu | head > Tue Jan 11 16:49:3

Re: Java Commited Virtual Memory significally larged then Heap Memory

After starting the hbase in jroсkit found the same memory leakage. After the launch Every 2,0 s: date & & ps - sort =- rss-eopid, rss, vsz, pcpu | head Tue Jan 11 16:49:31 2011 11 16:49:31 MSK 2011 PID RSS VSZ% CPU 7863 2547760 5576744 78.7 JR dumps: Total mapped 5576740KB (reserve

Re: Thrift WAL

Hi J-D, should an issue by filled on this? We might try to provide a patch, after agreement on where to put this option. Should it be member of Mutation, or a new call should be created (eg. mutateRowsNoWal())? Thanks, Jan On 7.1.2011 19:21, Jean-Daniel Cryans wrote: Not by design, it's re

Re: Region loadbalancing

Hi, sure, we are experiencing the following: - regions are getting unavailable for much less time, so clients are no longer failing (in fact some of them usually still fail on RetriesExhausedException caused by "failed setting up proxy", but there are only few of them) - on the other hand

java.net.SocketException: Too many open files

I set the env as fallows: $ ulimit -n 65535 $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 63943 max locked memory (

Re: HBase tuning - minimise table read latency

No, the second table is too large to fit in memory. On Mon, 2011-01-10 at 11:26 -0800, Stack wrote: > Mark the second-table in-memory in the schema. And for the first, > have it not use cache at all. This way, cache should only have > content from the table that is read. Does the second table