Hudson build is back to normal : HBase-TRUNK #1737

See

Build failed in Hudson: HBase-TRUNK #1736

See Changes: [apurtell] HBASE-3257 Coprocessors: Extend server side API to include HLog operations [todd] HBASE-3509. Add metric for flush queue length [todd] HBASE-3510. Add thread name for IPC reader threads [todd] HBASE-3511.

Re: Loaded Coprocessor

> is there any way to check which Coprocessors Classes are > currently loaded? Preferebly using the shell... Not yet. Good idea, filed: https://issues.apache.org/jira/browse/HBASE-3512 > And related: Can one dynamically load Coprocessors for a > running hbase system? Currently coprocessors are l

Re: requests count per HRegion and rebalance

This approach would introduce dependency on the underlying JVM - vendor, version, etc because the log format can change across JVM versions. On Mon, Feb 7, 2011 at 12:03 PM, tsuna wrote: > On Mon, Feb 7, 2011 at 10:59 AM, Todd Lipcon wrote: > > Unfortunately you can't get at any of the really j

RE: CHANGES.txt in trunk vs branch?

I like the approach Todd outlines, but have been doing it as Stack describes. +1 on moving to Todd/Hadoop approach. In addition, for JIRA, we should probably put the fix version as the lowest common denominator rather than everywhere it is applied. (if fixed in 0.90 branch, it's not really a b

Re: CHANGES.txt in trunk vs branch?

Thanks Todd. Sounds good. I 'think' that in the past we would do different to Hadoop in that we'd just bundle the fix issue under the explicit fix version on branch but under the current trunk version when we added the fix into trunk; i.e. using your version example below, we'd put fix under 0.90

CHANGES.txt in trunk vs branch?

Hey all, I wanted to clarify/ask what our policy is for updating CHANGES.txt on trunk when something is committed to both trunk and branch. In the Hadoop project we add the patch to the branch section on both trunk and branch. That is, if we're working on 0.92 and put a patch in 0.90.1, it will g

Re: requests count per HRegion and rebalance

On Mon, Feb 7, 2011 at 10:59 AM, Todd Lipcon wrote: > Unfortunately you can't get at any of the really juicy details like CMS FLS > statistics :( Do you have to resort to reading back your own GC log to figure this stuff out? :( -- Benoit "tsuna" Sigoure Software Engineer @ www.StumbleUpon.com

Re: Loaded Coprocessor

Hi JS. > is there any way to check which Coprocessors Classes are currently loaded? Preferebly using the shell... No. Not yet. But that's a reasonable requirement. You can open a jira for it. Right now you have to check HTableDescriptor (for table coprocessors) or configurations (for system-

Re: requests count per HRegion and rebalance

http://download.oracle.com/javase/1.5.0/docs/api/java/lang/management/ManagementFactory.html#getGarbageCollectorMXBeans() exposes some of the info Unfortunately you can't get

Re: requests count per HRegion and rebalance

This means we need horrible stuff to write good stuff :-) On Mon, Feb 7, 2011 at 10:49 AM, tsuna wrote: > On Mon, Feb 7, 2011 at 10:26 AM, Ted Yu wrote: > > About 'number of GC cycles per GC type', is there API to get this > > information ? > > As far as I know this is only exposed through JMX.

Re: requests count per HRegion and rebalance

On Mon, Feb 7, 2011 at 10:26 AM, Ted Yu wrote: > About 'number of GC cycles per GC type', is there API to get this > information ? As far as I know this is only exposed through JMX. I'm sure there is a way to use some horrible APIs in the JDK to "efficiently" access JMX stuff from the local JVM.

Re: requests count per HRegion and rebalance

About 'number of GC cycles per GC type', is there API to get this information ? Regards On Mon, Feb 7, 2011 at 10:02 AM, tsuna wrote: > On Mon, Feb 7, 2011 at 9:12 AM, Ted Yu wrote: > > This is a good suggestion. > > We also need to consider the situation where source RS is under heavy > load,

Re: requests count per HRegion and rebalance

On Mon, Feb 7, 2011 at 9:12 AM, Ted Yu wrote: > This is a good suggestion. > We also need to consider the situation where source RS is under heavy load, > facing potential GC pause. In this case we shouldn't expect source RS to > carry on too much computation. Whenever things fail unexpectedly, w

Re: requests count per HRegion and rebalance

Sebastian: I was going to say please make an issue with this good stuff but I notice you just started in. Good on you Sebastian. Thanks, St.Ack On Mon, Feb 7, 2011 at 8:30 AM, Sebastian Bauer wrote: > path-1 with AtomicLong: > http://paste.org/pastebin/view/28532 > > On 07.02.2011 17:16, Seb

Re: requests count per HRegion and rebalance

JIRA created: https://issues.apache.org/jira/browse/HBASE-3507 On 07.02.2011 18:07, Stack wrote: Benoît: Stick the below up in an issue. Its good stuff. St.Ack On Mon, Feb 7, 2011 at 8:44 AM, tsuna wrote: Hey this is pretty cool, I've been wanting something like that for a while. It's a l

Re: requests count per HRegion and rebalance

its trunk version: Path: . URL: https://svn.apache.org/repos/asf/hbase/trunk Repository Root: https://svn.apache.org/repos/asf Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68 Revision: 1067865 Node Kind: directory Schedule: normal Last Changed Author: stack Last Changed Rev: 1067598 Last Ch

Re: requests count per HRegion and rebalance

What version of HBase do you use ? Looking at the line numbers, you are using version older than 0.90 Also, incrementing counter in startRegionOperation() doesn't consider the following call (in 0.90 RC3): public Integer obtainRowLock(final byte [] row) throws IOException { startRegionOperat

Re: requests count per HRegion and rebalance

Yeah i know move take a while but we have small cluster(3 machines) and when few hot regions land in one machine then we had about 4000/1000/1000 requests on machines after a rebalancing we have now about 1900/2200/1900 so its worked :) maybe someone can write tool to take few regions from ove

Re: requests count per HRegion and rebalance

This is a good suggestion. We also need to consider the situation where source RS is under heavy load, facing potential GC pause. In this case we shouldn't expect source RS to carry on too much computation. On Mon, Feb 7, 2011 at 8:44 AM, tsuna wrote: > Hey this is pretty cool, I've been wanting

Re: requests count per HRegion and rebalance

Benoît: Stick the below up in an issue. Its good stuff. St.Ack On Mon, Feb 7, 2011 at 8:44 AM, tsuna wrote: > Hey this is pretty cool, I've been wanting something like that for a > while.  It's a little bit rough but the idea is there.  Ultimately, I > hope HBase will be able to provide fine-g

Re: requests count per HRegion and rebalance

Hey this is pretty cool, I've been wanting something like that for a while. It's a little bit rough but the idea is there. Ultimately, I hope HBase will be able to provide fine-grained metrics on a per-region basis and use that to do load balancing. The problem right now is that load balancing i

Re: requests count per HRegion and rebalance

path-1 with AtomicLong: http://paste.org/pastebin/view/28532 On 07.02.2011 17:16, Sebastian Bauer wrote: then why hbase have this? in HRegionServer.java:206 // Request counter private volatile AtomicInteger requestCount = new AtomicInteger(); On 07.02.2011 17:00, Ted Yu wrote: In HRegion.java

Re: requests count per HRegion and rebalance

then why hbase have this? in HRegionServer.java:206 // Request counter private volatile AtomicInteger requestCount = new AtomicInteger(); On 07.02.2011 17:00, Ted Yu wrote: In HRegion.java: final AtomicInteger requestsCount = new AtomicInteger(0); AtomicLong should be used. On Mon, Feb 7,

Re: requests count per HRegion and rebalance

In HRegion.java: final AtomicInteger requestsCount = new AtomicInteger(0); AtomicLong should be used. On Mon, Feb 7, 2011 at 8:43 AM, Sebastian Bauer wrote: > Hi, > > Path-1 add another mertic for HRegion to count request made to region. > > Path-2 add another command to hbase shell to grab a

Re: requests count per HRegion and rebalance

On 07.02.2011 17:43, Sebastian Bauer wrote: Hi, Path-1 add another mertic for HRegion to count request made to region. Path-2 add another command to hbase shell to grab all regions, sort by requests from Path-1 and move in round-robin algorithm to servers Path-1: http://paste.org/pastebin/v

requests count per HRegion and rebalance

Hi, Path-1 add another mertic for HRegion to count request made to region. Path-2 add another command to hbase shell to grab all regions, sort by requests from Path-1 and move in round-robin algorithm to servers Path-1: http://paste.org/pastebin/view/28526 Path-2: http://paste.org/pastebin/v

Loaded Coprocessor

Hi, is there any way to check which Coprocessors Classes are currently loaded? Preferebly using the shell... And related: Can one dynamically load Coprocessors for a running hbase system?   Thanks for your support! JS