Re: Monitoring

2011-07-29 Thread Andrew Purtell
Have you tried Ganglia2?     http://sourceforge.net/apps/trac/ganglia/wiki/ganglia-web-2   Best regards,     - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) > >From: Otis Gospodnetic >To: "user@hbase.apache.org"

Re: HBase backup and outage scenarios in practice?

2011-07-29 Thread Otis Gospodnetic
Hola, If there is indeed something out out of date there, please do let me know - I'd like to keep that post up to date, esp. since it's linked from the HBase Book.  Thanks, Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Hadoop - HBase Hadoop ecosystem search :: http://search-hado

Re: data structure

2011-07-29 Thread Otis Gospodnetic
Hi Andre,   In the process of developing some of our HBase-based products we've built a generic aggregation framework that is nice, flexible and extensible, and it sounds like it could build those reports you are after.  We run MR aggregate job that reads raw data (e.g. your impression data) fro

Re: Monitoring

2011-07-29 Thread Otis Gospodnetic
Hello, I was just looking at Ganglia that one of our customers uses for their 200+ node HBase cluster.  I do not find Ganglia very nice.  For example, I don't think I can make it render some metric for N nodes on a single chart for example, or that I can pick a very specific time period, etc.  

Re: Flume & HBase integration status

2011-07-29 Thread Eric Sammer
This, by the way, is an extremely well written article. Thanks Alex! On Fri, Jul 29, 2011 at 5:57 AM, Alex Baranau wrote: > Just published a post about current state of Flume & HBase integration > (HBase sinks for Flume) at > http://blog.sematext.com/2011/07/28/flume-and-hbase-integration. > > Mi

Re: Filter Rows on last 4 bytes

2011-07-29 Thread Jean-Daniel Cryans
You need to write your own comparator (sorry if I didn't say it plainly like that the first time). J-D On Fri, Jul 29, 2011 at 12:15 AM, Shuja Rehman wrote: > Yes, I have look that  But I did not find any comparator which takes > starting index and length to match. I was thinking to have such ty

Re: HBase / YCSB

2011-07-29 Thread Gary Helmling
Is it possible that you have mismatched versions of either the hbase jar or hadoop jar on the ycsb client versus the servers? In almost all cases where I've run into mysterious rpc hangs right off the bat it's been attributable to forgetting to update a jar file or an older version still being pres

Re: HBase / YCSB

2011-07-29 Thread Jeff Whiting
Check the region server logs. If they are blocking on something it should show up there. For cdh3 the logs are in /var/log/hbase/. Also you may want to turn on debug level for your logging (either in log4j or in the web interface). Finally all of your requests are going to just one region se

HBase / YCSB

2011-07-29 Thread Eric Hauser
Hi, I've been doing different experiments with a 5-node cluster with YCSB.  We have been testing a number of different configurations, so I have been constantly been wiping our cluster up and setting it up again since we configure everything via Chef.   At one point, I was able to get the following

Re: Back_up Master operation?

2011-07-29 Thread Stack
On Fri, Jul 29, 2011 at 6:20 AM, shanmuganathan.r wrote: >      If I will kill the active master ,than How much time it will take to > allocate the regionservers to the new master? > No regions are allocated when new master assumes master role. >      Where I see the Backup-master runs properl

Re: Problem with hbase.client.ipc.pool.type=threadlocal in trunk

2011-07-29 Thread Ted Yu
I am waiting for Karthick to shed some light on this. On Fri, Jul 29, 2011 at 9:47 AM, Stack wrote: > Sounds good. Issue? > St.Ack > > On Thu, Jul 28, 2011 at 10:46 PM, Ted Yu wrote: > > For HBaseClient, at least the javadoc doesn't match: > > > > * @param config configuration > > * @retur

Re: Design/Schema questions

2011-07-29 Thread Stack
You are a good man Alex. On Fri, Jul 29, 2011 at 5:48 AM, Alex Baranau wrote: > Just published a post about Flume & HBase integration which might be > helpful. It describes the possible issues & workarounds for them. > > Alex Baranau > > Sematext :: http://sematext.com/ :: Solr - Lucene - Nu

Re: GZ better than LZO?

2011-07-29 Thread Chris Tarnas
Your region distribution across the nodes is not great, for both cases most of your data is going to one server, spreading the regions out across multiple servers would be best. How many different vehicle_ids are being used, and are they all sequential integers in your tests? Hbase performs bet

Re: Problem with hbase.client.ipc.pool.type=threadlocal in trunk

2011-07-29 Thread Stack
Sounds good. Issue? St.Ack On Thu, Jul 28, 2011 at 10:46 PM, Ted Yu wrote: > For HBaseClient, at least the javadoc doesn't match: > >   * @param config configuration >   * @return either a {@link PoolType#Reusable} or {@link > PoolType#ThreadLocal} >   */ >  private static PoolType getPoolType(C

Re: MR on HBase - java.io.IOException: Pass a Delete or a Put

2011-07-29 Thread Stack
Study the mapreduce examples in unit tests or under our mapreduce package. Below looks fine to me. Maybe its how the job is configured. St.Ack P.S. you don't have to find our src in random locations; e.g. our paul smiths' apache home dir. Our src is here: hbase.org St.Ack On Fri, Jul 29, 2011

Re: Column Indexing - Top N Columns

2011-07-29 Thread Barış Can Daylık
Number of rows will be 6 million. So in the worst case the table will be square, but on average 100K columns won't be exceeded. If I'm not mistaken columns are sorted by column names and not the values, does result.raw return columns sorted by their values? If it does so, does it sort them whe

Re: MR on HBase - java.io.IOException: Pass a Delete or a Put

2011-07-29 Thread Narayanan K
Hi Stack/Suraj, I tried my MR code on HBase to 0.90.x version. But I am getting the same exception after the Map stage is complete: *java.io.IOException: Pass a Delete or a Put*. I did a search on web and found the *TableOutputFormat* source code where the *write* method is throwing the "*Pass a

Re: GZ better than LZO?

2011-07-29 Thread lars hofhansl
For what's it worth I had similar observations. I simulated heavy write load and I found that NO compression was the fastest, followed by GZ, followed by LZO. After the tests I did a major_compact of the tables, and I included that time in the total. Also these tests where done with a single reg

Re: Region is not online: -ROOT-,,0

2011-07-29 Thread Xian Woo
hmm, I am not sure about that , let me check again. Thank you about that. ^_^ 2011/7/29 Gan, Xiyun > Refer to https://issues.apache.org/jira/browse/HBASE-3669 > Probably it's a bug. > > Is this issue reproducible? > > On Fri, Jul 29, 2011 at 1:22 PM, Xian Woo wrote: > > Actually it happened w

Back_up Master operation?

2011-07-29 Thread shanmuganathan.r
Hi All, I am running the HBase in fully distributed mode. I want to improve my the HBase configuration. So I move to the Backup_Master. I include the conf/backup-masters file and i given the IP where i want to run the Backup-master. I start the hbase conventionally. The Master is running

Flume & HBase integration status

2011-07-29 Thread Alex Baranau
Just published a post about current state of Flume & HBase integration (HBase sinks for Flume) at http://blog.sematext.com/2011/07/28/flume-and-hbase-integration. Might be useful for those who are looking at this topic. Alex Baranau Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch -

Re: Design/Schema questions

2011-07-29 Thread Alex Baranau
Just published a post about Flume & HBase integration which might be helpful. It describes the possible issues & workarounds for them. Alex Baranau Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase On Wed, Jul 27, 2011 at 9:39 PM, Mark wrote: > Unfortunately that wa

RE: GZ better than LZO?

2011-07-29 Thread Steinmaurer Thomas
Hi Chris! Your questions are somehow hard to answer for me, because I'm not really in charge for the test cluster from an administration/setup POV. Basically, when running: http://xxx:60010/master.jsp I see 7 region servers. Each with a "maxHeap" value of 995. When clicking on the different tab

Re: Region is not online: -ROOT-,,0

2011-07-29 Thread Gan, Xiyun
Refer to https://issues.apache.org/jira/browse/HBASE-3669 Probably it's a bug. Is this issue reproducible? On Fri, Jul 29, 2011 at 1:22 PM, Xian Woo wrote: > Actually it happened when I did a test to manually reboot one of the  region > servers.  Note that the region server I rebooted was the on

Re: Filter Rows on last 4 bytes

2011-07-29 Thread Shuja Rehman
Yes, I have look that But I did not find any comparator which takes starting index and length to match. I was thinking to have such type of comparator. Filter rf = new RowFilter(CompareOp.EQUAL, new BinaryComparator(Bytes.toBytes(ID), StartingIndex, Length); And, about restriction, yes first fou