Re: thread safety of incrementColumnValue

2011-07-20 Thread Doug Meil
Hi there- I think there are two subjects here: 1) the fact that HTable isn't thread-safe 2) how counters work Even if you are incrementing counters, you shouldn't be sharing HTable instances across threads. Counters get updated atomically on the RS, not on the client. Counter behavior isn

thread safety of incrementColumnValue

2011-07-20 Thread large data
I have an HTable instance instantiated as part of a singleton service. This singleton service is called from different threads from different parts of the app. Reading through the HTable docs suggests not to use single HTable instance for updates, if it's true how can incrementColumnValue provide t

Re: Partial replication

2011-07-20 Thread Jean-Daniel Cryans
Actually it's on the family level. J-D On Wed, Jul 6, 2011 at 1:53 PM, Stack wrote: > Replication is table-scoped. > St.Ack > > On Tue, Jul 5, 2011 at 1:46 AM, James Ram wrote: >> Hi, >> >> Is it possible to replicate a particular dataset to another cluster >> instead of replicating the whole d

Re: HBase & MapReduce & Zookeeper

2011-07-20 Thread Stack
My guess is that it needs to ask the master for the regions so it can make the splits used by mapper tasks (to find master, needs to ask zk, etc.). Check it out yourself under the mapreduce package? St.Ack On Wed, Jul 20, 2011 at 3:06 PM, Andre Reiter wrote: > Hi Ted, > thanks for the reply, >

August 2011 San Francisco Hadoop User Group Meetup

2011-07-20 Thread Aaron Kimball
Hadoop Fans, It's my pleasure to announce the next monthly San Francisco Hadoop user group meetup. *When*: Wednesday, August 10, 2011 6:00 PM to 8:00 PM *Where*: USF Harney Science Center Golden Gate Ave and Chabot Terrace San Francisco, CA 94117 This meetup will be hosted at the University of Sa

Re: HBase & MapReduce & Zookeeper

2011-07-20 Thread Andre Reiter
Hi Ted, thanks for the reply, at the moment i'm hust wondering, why the client creates a zookeeper connection at all all the client has to do, is to schedule a MR job, which is done by connecting to the jobtracker and to provide all the needed stuff, config, some extra resources in the distrib

Re: HBase & MapReduce & Zookeeper

2011-07-20 Thread Ted Yu
This seems to be cdh related. >> either the map HConnectionManager.HBASE_INSTANCES does not contain the connection for the current config You need to pass the same conf object. In trunk, I added the following: public static void deleteStaleConnection(HConnection connection) { See http://zhihon

Re: HBase & MapReduce & Zookeeper

2011-07-20 Thread Andre Reiter
unfortunatelly there was no such LOG entry... :-( our versions: hadoop-0.20.2-CDH3B4 hbase-0.90.1-CDH3B4 zookeeper-3.3.2-CDH3B4 either the map HConnectionManager.HBASE_INSTANCES does not contain the connection for the current config, or HConnectionImplementation.zooKeeper is null but the zooke

Re: HBase & MapReduce & Zookeeper

2011-07-20 Thread Ted Yu
Andre: So you didn't see the following in client log (HConnectionManager line 1067) ? LOG.info("Closed zookeeper sessionid=0x" + Long.toHexString(this.zooKeeper.getZooKeeper().getSessionId())); HConnectionManager.deleteConnection(conf, true) is supposed to close zk connection in 0

Re: HBase & MapReduce & Zookeeper

2011-07-20 Thread Andre Reiter
Hi St.Ack, actually calling HConnectionManager.deleteConnection(conf, true); does not close the connection to the zookeeper i still can see the connection established... andre Stack wrote: Then similarly, can you do the deleteConnection above in your client or reuse the Configuration client

Re: HBase rest RowSpec bug on startRow and endRow

2011-07-20 Thread Hailun Yan
Thanks, Andy. On Mon, Jul 18, 2011 at 2:49 PM, allan yan wrote: > Hi, > I don't know how to report issue to HBase team. So I just post it here. > There might be a bug for REST web service to get rows with given startRow > and endRow. > For example, to get a list of rows with startRow=testrow1, en

Re: HBase & MapReduce & Zookeeper

2011-07-20 Thread Stack
Then similarly, can you do the deleteConnection above in your client or reuse the Configuration client-side that you use setting up the job? St.Ack On Wed, Jul 20, 2011 at 12:13 AM, Andre Reiter wrote: > Hi Stack, > > just to make clear, actually the connections to the zookeeper being kept are >

Re: HBase standalone start issue

2011-07-20 Thread Stack
On Wed, Jul 20, 2011 at 1:19 AM, Jeff wrote: > Caused by: java.net.ConnectException: Connection refused This is root of the prob. Its odd though because you are standalone. Any exceptions before those you post here? Anything about this box that preventing process connect to itself on different

Re: HMaster jvm crashes and imbalance cluster

2011-07-20 Thread Stack
On Wed, Jul 20, 2011 at 8:22 AM, Xu-Feng Mao wrote: > Hi, > We're running a 25-node regionserver hbase cluster, using cdh3u0. > 1. We run into several jvm crashes on master today. It seems like jvm > issues, as I attached the hs_error_pid files > with this message. Just want to confirm that if thi

Re: Hbase row key size - can we use big row keys

2011-07-20 Thread Stack
On Wed, Jul 20, 2011 at 7:00 AM, Srikanth P. Shreenivas wrote: > We are trying to design a HBase table, and we seem to be tending towards > packing 3 fields into our row key, as we need to be able to do random access > using these 3 fields. > The row key is turning out to be around 93 characters

HMaster jvm crashes and imbalance cluster

2011-07-20 Thread Xu-Feng Mao
Hi, We're running a 25-node regionserver hbase cluster, using cdh3u0. 1. We run into several jvm crashes on master today. It seems like jvm issues, as I attached the hs_error_pid files with this message. Just want to confirm that if this is really a jvm issue, or maybe some master issue trigger t

Re: How can I use shell to view multiple versions?

2011-07-20 Thread Stack
2011/7/20 勇胡 : > Maybe I have to update my hbase. > Yes. This would be a good idea. St.Ack

Re: How can I use shell to view multiple versions?

2011-07-20 Thread 勇胡
Thanks for your response. I just found the information from mailing archive. It is as follows: "In the shell you can ask for more versions by saying: get 'table', 'row', {COLUMN => 'family', VERSIONS => 4}". By the way, my hbase version is 0.90.2, I used help 'get' to see the examples and I didn't

Re: How can I use shell to view multiple versions?

2011-07-20 Thread Ted Yu
If you look at help of get command, you would see: hbase> get 't1', 'r1', {COLUMN => 'c1', TIMERANGE => [ts1, ts2], VERSIONS => 4} On Wed, Jul 20, 2011 at 7:09 AM, 勇胡 wrote: > Hello, > > I created a table named 'score', and I issue put command twice: > > put 'score','tom','course:math','1.2' >

Re: Hbase - limit connection per IP?

2011-07-20 Thread Barney Frank
I believe you are hitting the default zookeeper limit. If hbase is not managing zookeeper, edit zoo.cfg on the zookeeper server and add maxClientCnxns=0 and restart the server. Otherwise there is a similar setting on hbase, but can't recall it. On Tue, Jul 19, 2011 at 11:36 PM, King JKing wrote:

How can I use shell to view multiple versions?

2011-07-20 Thread 勇胡
Hello, I created a table named 'score', and I issue put command twice: put 'score','tom','course:math','1.2' put 'score','tom','course:math','1.3' Now the cell has two data versions, I want to read these two versions, so I issue a command, get 'score', 'tom', {COLUMN=>'course:math',VERSION=>2}

Hbase row key size - can we use big row keys

2011-07-20 Thread Srikanth P. Shreenivas
Hi, We are trying to design a HBase table, and we seem to be tending towards packing 3 fields into our row key, as we need to be able to do random access using these 3 fields. The row key is turning out to be around 93 characters in size. Is it okay to use a 100 character long row keys? It wil

HBase standalone start issue

2011-07-20 Thread Jeff
Hi, I am trying to start HBase 0.90.3 in standalone mode and I get the following errors. I follow the instructions in the QuckStart at apache site. I set the JAVA_HOME and the conf/hbase-site.xml accordingly. I start HBase with bin/start-hbase.sh script and the start the shell using bin/start-hb

Re: HBase & MapReduce & Zookeeper

2011-07-20 Thread Andre Reiter
Hi Stack, just to make clear, actually the connections to the zookeeper being kept are not on our mappers (tasktrackers) but on the client, which schedules the MR job i think, the mappers are just fine, as they are andre Stack wrote: Can you reuse Configuration instances though the "configu

Re: HBase & MapReduce & Zookeeper

2011-07-20 Thread Stack
Can you reuse Configuration instances though the "configuration" changes? Else in your Mapper#cleanup, call HTable.close() then try HConnectionManager.deleteConnection(table.getConfiguration()) after close (could be issue with executors used by multi* operations not completing before delete of con