Re: Hbase cluster configuration

2009-02-04 Thread Michael Dagaev
Andrew, thanks. Looks like I should think of scalability of a single region server. I will probably think of it again and ask questions on the list later. M. On Wed, Feb 4, 2009 at 1:03 AM, Andrew Purtell wrote: > Hi Michael, > > I have found that trial and error is necessary now. There are no >

Backup Again

2009-02-04 Thread Michael Dagaev
Hi, all I read HBASE-974 and HBASE-643 mentioned on the list but what do you think about copying tables from the production to a backup Hbase cluster ? I guess we do need a big iron for such a backup cluster. I understand that the copy can be implemented with MR but for now we can implement i

Re: Backup Again

2009-02-04 Thread Chris K Wensel
Hey Michael You could probably use Cascading to migrate data between HBase clusters. http://wiki.apache.org/hadoop/Hbase/Cascading But the code currently doesn't support multiple HBase cluster clients in a single JVM, but I'm sure it can be coded in quickly. (the code is hosted at github, so

Re: Backup Again

2009-02-04 Thread Chris K Wensel
Currently no. but we would love you to patch that in. If you clone the repo and get it working, I'll merge it back from your repo. I'm thinking it is as simple as handing the URL to HBaseTap. thoughts? ckw On Feb 4, 2009, at 8:38 AM, Michael Dagaev wrote: Thanks, Chris BTW, is it possible

Re: Backup Again

2009-02-04 Thread Michael Dagaev
> Currently no. but we would love you to patch that in. If you clone the repo > and get it working, I'll merge it back from your repo. Currently, I would like just to work around that problem :) What about loading the Hbase client classes with different class loaders? M.

Re: Backup Again

2009-02-04 Thread Michael Dagaev
Thanks, Chris BTW, is it possible to run a few HBase clients in a single JVM? On Wed, Feb 4, 2009 at 6:28 PM, Chris K Wensel wrote: > Hey Michael > > You could probably use Cascading to migrate data between HBase clusters. > http://wiki.apache.org/hadoop/Hbase/Cascading > > But the code currentl

Re: Backup Again

2009-02-04 Thread Chris K Wensel
I think that would be slightly more troublesome than just passing an url around. most of the heavy lifting is done inside TableInput/ OutputFormat (which is part of HBase). You need to pass reasonable properties all the way down. I think the Streamy guys might be touching on some of this, th

column family names

2009-02-04 Thread Toby White
From the HBase documentation (http://wiki.apache.org/hadoop/Hbase/HbaseArchitecture ) 'A column name has the form ":" where and can be arbitrary byte arrays' yet in HColumnDescriptor.java: for (int i = 0; i < (b.length - 1); i++) { if (Character.isLetterOrDigit(b[i]) || b[i] ==

Re: Backup Again

2009-02-04 Thread Michael Dagaev
Looks like we can work around the problem just by changing the "hbase.rootdir" value used as a key in the map. M. On Wed, Feb 4, 2009 at 7:00 PM, Chris K Wensel wrote: > I think that would be slightly more troublesome than just passing an url > around. most of the heavy lifting is done inside Ta

Re: column family names

2009-02-04 Thread stack
Yeah, the architecture document is starting to showing its age. Needs a nip and tuck. No big reasons for family name being 'text'/'printable'. Is it an onerous restriction? St.Ack On Wed, Feb 4, 2009 at 8:09 AM, Toby White wrote: > From the HBase documentation ( > http://wiki.apache.org/ha

Re: hbase.client.scanner.caching

2009-02-04 Thread stack
Want to file an issue Billy? It'd be one thing if we were over-reporting our requests/second. In that case we might be a little slow to the fix but under-reporting, thats a different matter altogether. St.Ack On Tue, Feb 3, 2009 at 11:36 PM, Billy Pearson wrote: > hbase.client.scanner.caching m

Re: column family names

2009-02-04 Thread Toby White
I quite wanted to use a hyphen in the name (so that it could be a direct mapping to an HTTP header field name), but hbase wouldn't let me. Not massively onerous - it's easy to translate; it's just a slight awkwardness. Toby On 4 Feb 2009, at 17:49, stack wrote: Yeah, the architecture docume

Re: column family names

2009-02-04 Thread stack
File an issue and a patch and we'll commit it. Seems like a reasonable expectation for a column family name. Thanks Toby, St.Ack On Wed, Feb 4, 2009 at 9:53 AM, Toby White wrote: > I quite wanted to use a hyphen in the name (so that it could be a direct > mapping to an HTTP header field name),

Re: column family names

2009-02-04 Thread Toby White
Thanks, see HBASE-1184. Toby On 4 Feb 2009, at 17:59, stack wrote: File an issue and a patch and we'll commit it. Seems like a reasonable expectation for a column family name. Thanks Toby, St.Ack On Wed, Feb 4, 2009 at 9:53 AM, Toby White wrote: I quite wanted to use a hyphen in the name

Row Filters in TableInputFormatBase

2009-02-04 Thread Dave Latham
In order to speed up a map reduce job operating on HBase input data, we recently added a RowFilter to the input format. However, when trying to execute it, map tasks (one per region) that used to take 1-2 minutes began timing out after 10 minutes. So I dug in to TableInputFormatBase to see how it

HBase Hackathon Wrap-up

2009-02-04 Thread Jonathan Gray
For those that were not able to attend the hackathon last weekend in LA, I wrote a blog post about what we worked on. http://streamydev.wordpress.com/2009/02/04/hbase-hackathon-wrap-up/ Thanks again to everyone who came JG