Re: Using HBase on other file systems

2010-05-12 Thread Jeff Hammerbacher
Some projects sacrifice stability and manageability for performance (see, e.g., http://gluster.org/pipermail/gluster-users/2009-October/003193.html). On Wed, May 12, 2010 at 11:15 AM, Edward Capriolo wrote: > On Wed, May 12, 2010 at 1:30 PM, Andrew Purtell > wrote: > > > Before recommending Glus

Re: Enabling Indexing in HBase

2010-05-12 Thread Seraph Imalia
Hi, I'm working with Michelan... We are actually using the HBaseConfiguration object - which is why we were confused when the client was trying to connect to zookeeper on localhost. Even stranger was that all other functions work fine - getting a table, putting and getting data. It is onl

Re: HBase client hangs after upgrade to 0.20.4 when used from reducer

2010-05-12 Thread Todd Lipcon
Hi Friso, Also, if you can capture a jstack of the regionservers at thie time that would be great. -Todd On Wed, May 12, 2010 at 9:26 AM, Jean-Daniel Cryans wrote: > Friso, > > Unfortunately it's hard to determine the cause with the provided > information, the client call you pasted is pretty m

data redundancy in hbase tables for read performance

2010-05-12 Thread N Kapshoo
For the model I am designing, read speed is the highest priority. That being said, I have a Customers table with information about Claims. Here is the design today: Table: Customers RowId: CustomerId Family: Claims Column: ClaimId Value: JSON(ClaimId, Status, Description, From) I am storing the

Re: Using HBase on other file systems

2010-05-12 Thread Edward Capriolo
On Wed, May 12, 2010 at 1:30 PM, Andrew Purtell wrote: > Before recommending Gluster I suggest you set up a test cluster and then > randomly kill bricks. > > Also as pointed out in another mail, you'll want to colocate TaskTrackers > on Gluster bricks to get I/O locality, yet there is no way for

Re: Problem with performance with many columns in column familie

2010-05-12 Thread Sebastian Bauer
path has stupid bug with double lock... Index: core/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java === --- core/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java (wersja 942215) +++ core/src/main/java/org

Stargate WAR target

2010-05-12 Thread Andrew Purtell
Anybody use it? - Andy

Re: Using HBase on other file systems

2010-05-12 Thread Andrew Purtell
Before recommending Gluster I suggest you set up a test cluster and then randomly kill bricks. Also as pointed out in another mail, you'll want to colocate TaskTrackers on Gluster bricks to get I/O locality, yet there is no way for Gluster to export stripe locations back to Hadoop. It seems

Re: Enabling Indexing in HBase

2010-05-12 Thread Jean-Daniel Cryans
Yes, you can also create a HBaseConfiguration object and configure it with those exact configs (that you then provide to HTable). J-D On Wed, May 12, 2010 at 1:22 AM, Michelan Arendse wrote: > Thank you. I have added the configuration folder to my client class path and > it worked. > > Now I am

Re: HBase client hangs after upgrade to 0.20.4 when used from reducer

2010-05-12 Thread Jean-Daniel Cryans
Friso, Unfortunately it's hard to determine the cause with the provided information, the client call you pasted is pretty much normal i.e. the client is waiting to receive a result from a region server. The fact that you can't shut down the master when this happens is very concerning. Do you stil

Re: Enabling IHbase

2010-05-12 Thread Stack
You saw this package doc over in the ihbase's new home on github? http://github.com/ykulbak/ihbase/blob/master/src/main/java/org/apache/hadoop/hbase/client/idx/package.html It'll read better if you build the javadoc. There is also this: http://github.com/ykulbak/ihbase/blob/master/README St.Ack

HBase client hangs after upgrade to 0.20.4 when used from reducer

2010-05-12 Thread Friso van Vollenhoven
Hi all, I am using Hadoop (0.20.2) and HBase to periodically import data (every 15 minutes). There are a number of import processes, but generally they all create a sequence file on HDFS, which is then run through a MapReduce job. The MapReduce uses the identity mapper (the input file is a Hado

Re: Enabling IHbase

2010-05-12 Thread Renato Marroquín Mogrovejo
Hi Alex, Thanks for your help, but I meant something more like a how-to set it up thing, or like a tutorial of it (= I also read these ones if anyone else is interested. http://blog.sematext.com/2010/03/31/hbase-digest-march-2010/ http://search-hadoop.com/m/5MBst1uL87b1 Renato M. 2010/5/12 al

Re: Problem with performance with many columns in column familie

2010-05-12 Thread Sebastian Bauer
I figured out what is taking so long, test data was 1 row with 10 columns and 1 with 100 when i try to increament column this huge row data didnt land in MemStore and times was(test in python after warmup): before path: #get one column from big row 1 0:00:00.919464 #get one column from sm

Re: Enabling IHbase

2010-05-12 Thread alex kamil
regarding usage this may be helpful https://issues.apache.org/jira/browse/HBASE-2167 On Wed, May 12, 2010 at 10:48 AM, alex kamil wrote: > Renato, > > just noticed you are looking for *Indexed *Hbase > > i found this > http://blog.reactive.org/2010/03/indexed-hbase-it-might-not-be-what-you.html

Re: Enabling IHbase

2010-05-12 Thread alex kamil
Renato, just noticed you are looking for *Indexed *Hbase i found this http://blog.reactive.org/2010/03/indexed-hbase-it-might-not-be-what-you.html Alex On Wed, May 12, 2010 at 10:42 AM, alex kamil wrote: > > http://www.google.com/search?hl=en&source=hp&q=hbase+tutorial&aq=f&aqi=g-p1g-sx3g1g-s

Re: Enabling IHbase

2010-05-12 Thread alex kamil
http://www.google.com/search?hl=en&source=hp&q=hbase+tutorial&aq=f&aqi=g-p1g-sx3g1g-sx4g-msx1&aql=&oq=&gs_rfai= On Wed, May 12, 2010 at 10:25 AM, Renato Marroquín Mogrovejo < renatoj.marroq...@gmail.com> wrote: > Hi eveyone, > > I just read about IHbase and seems like something I could give it a

Enabling IHbase

2010-05-12 Thread Renato Marroquín Mogrovejo
Hi eveyone, I just read about IHbase and seems like something I could give it a try, but I haven't been able to find information (besides descriptions and advantages) regarding to how to install it or use it. Thanks in advance. Renato M.

Re: Using HBase on other file systems

2010-05-12 Thread Edward Capriolo
On Tuesday, May 11, 2010, Jeff Hammerbacher wrote: > Hey Edward, > > I do think that if you compare GoogleFS to HDFS, GFS looks more full >> featured. >> > > What features are you missing? Multi-writer append was explicitly called out > by Sean Quinlan as a bad idea, and rolled back. From internal

Re: Regarding IntSet implementation

2010-05-12 Thread Ram Kulbak
Hi Lekhnath, The IntSets are package protected so that their callers will always use the IntSet interface, thus preventing manipulation of the IntSet after it was built and hiding implementation details. It seems to me that having an index which can spill to disk may be a handy feature, perhaps yo

RE: Enabling Indexing in HBase

2010-05-12 Thread Michelan Arendse
Thank you. I have added the configuration folder to my client class path and it worked. Now I am faced with another issue, since this application will be used in ColdFusion is there a way of making this work without having the configuration as part of the class path? -Original Message-