Re: DBInputFormat

2010-02-12 Thread Gaurav Vashishth
Yes, I have looked them. I was looking for examples where I can have my own split function which overrides the behavious of DBInputSplit. By default, it feches the rows from the table and splits them and each split having different number of rows. This is what I want to control, I need splits to

Re: DBInputFormat

2010-02-12 Thread Amandeep Khurana
You can find examples on how to use DBInputFormat on the internet. And if you want a sample input format, just read any of the existing ones... Amandeep Khurana Computer Science Graduate Student University of California, Santa Cruz On Fri, Feb 12, 2010 at 10:39 PM, Gaurav Vashishth wrote: > >

Re: DBInputFormat

2010-02-12 Thread Gaurav Vashishth
Ok thanks for the reply. Do you have any sample code which can demonstrate how to do this? -Gaurav Amandeep Khurana wrote: > > DBInputFormat splits the count() from the RDBMS table into the number of > mappers. If you want to split using your own scheme, you'll have to write > your own input f

RE: stargate logging

2010-02-12 Thread Patterson, Josh
Andy, Yeah, I've copied the jars down, that's how I'm doing it too. Ok, I didn't realize it would pick up the base log4j.properties file, but does make sense. Thanks! Josh Patterson TVA -Original Message- From: Andrew Purtell [mailto:apurt...@apache.org] Sent: Friday, February 12, 2010

Re: stargate logging

2010-02-12 Thread Andrew Purtell
I have always just merged the Stargate contrib with core for deployment: Copy the Stargate jar to HBase ./ Copy the jars from ./contrib/stargate/lib to ./lib Then use ./bin/hbase-daemon.sh or ./bin/hbase to start Stargate. It will pick up log4j.properties in ./conf. Doing otherwise seems like

Re: searching values in hbase tables

2010-02-12 Thread Dan Washusen
Comments also inline... On 13 February 2010 03:15, Jean-Daniel Cryans wrote: > Inline. > > J-D > > On Fri, Feb 12, 2010 at 5:38 AM, Muhammad Mudassar >wrote: > > > Hi > > I have hbase table with 3 column families and some number of rows stored > in > > it here I want to ask that how I can searc

Re: DBInputFormat

2010-02-12 Thread Amandeep Khurana
DBInputFormat splits the count() from the RDBMS table into the number of mappers. If you want to split using your own scheme, you'll have to write your own input format or tweak the existing one. Amandeep Khurana Computer Science Graduate Student University of California, Santa Cruz On Fri, Feb

Re: filtering on a prefix of the row key

2010-02-12 Thread Stack
On Fri, Feb 12, 2010 at 2:17 AM, Bruno Dumon wrote: > On Thu, Feb 11, 2010 at 7:41 PM, Stack wrote: >> On Wed, Feb 10, 2010 at 7:16 AM, Bruno Dumon wrote: >>> Hi, >>> >>> I would like a filter that accepts rows as long as the first X bytes >>> of the row key are less than or equal to a certain b

Re: DBInputFormat

2010-02-12 Thread Stack
On Fri, Feb 12, 2010 at 4:32 AM, Gaurav Vashishth wrote: > > I have the Map Reduce function whose job is to process the database , MySql, > and give us some output. For this purpose, I have created the map reduce > fucntion and have used the DBInputFormat, but Im confused in how the > JobTracker w

Re: HBase Insert Performance

2010-02-12 Thread Patrick Hunt
In general when determining the number of ZooKeeper serving nodes to deploy (the size of an ensemble) you need to think in terms of reliability, and not performance. Reliability: A single ZooKeeper server (standalone) is essentially a coordinator with no reliability (a single serving node fai

Re: HBase Insert Performance

2010-02-12 Thread Jean-Daniel Cryans
If you have 1 cluster and it's very small, as you point out HBase isn't intense on ZK (yet) so using only 1 ZK is ok. Another setup like we have here at stumbleupon is multiple clusters using the same quorum. In this case it makes sense to get 3 or 5 nodes and in our case the hardware is beefy eno

Re: searching values in hbase tables

2010-02-12 Thread Jean-Daniel Cryans
Inline. J-D On Fri, Feb 12, 2010 at 5:38 AM, Muhammad Mudassar wrote: > Hi > I have hbase table with 3 column families and some number of rows stored in > it here I want to ask that how I can search values from the table (like > select name from employee where age='35': query of sql ) using api

stargate logging

2010-02-12 Thread Patterson, Josh
I cant seem to find the stargate REST server log4j.properties file --- how would I find or set that up? Josh Patterson TVA

searching values in hbase tables

2010-02-12 Thread Muhammad Mudassar
Hi I have hbase table with 3 column families and some number of rows stored in it here I want to ask that how I can search values from the table (like select name from employee where age='35': query of sql ) using api in java some code would be helpful. i am having problem to define row keys curent

Re: HBase Insert Performance

2010-02-12 Thread Michał Podsiadłowski
Hey all, I was asking about minimum number of zookeepers and usually everybody was saying odd number >=3. Are there any reasons for this. Have you encounter any problems from single zookeeper? As far as know already hbase is doing very very little operations using zookeeper so load on it is insign

DBInputFormat

2010-02-12 Thread Gaurav Vashishth
I have the Map Reduce function whose job is to process the database , MySql, and give us some output. For this purpose, I have created the map reduce fucntion and have used the DBInputFormat, but Im confused in how the JobTracker will produce the splits here. I want that first 'n' records from th

Re: HBase Insert Performance

2010-02-12 Thread Gaurav Vashishth
Ryan, I have setup the custer as suggested by you. Now I have Master,namemode and zookeeper on same machine and have 8 region servers running as data nodes and with this configuration I was able to get the insertion speed of around 18K records/sec. Though Im still using 4GB ram, will upgrade it

Re: filtering on a prefix of the row key

2010-02-12 Thread Bruno Dumon
On Thu, Feb 11, 2010 at 7:41 PM, Stack wrote: > On Wed, Feb 10, 2010 at 7:16 AM, Bruno Dumon wrote: >> Hi, >> >> I would like a filter that accepts rows as long as the first X bytes >> of the row key are less than or equal to a certain byte array. >> > > Would an inclusivestoprow filter work for