Re: Rowkey design and presplit table

2013-03-06 Thread Asaf Mesika
I would convert each id to long and then use Bytes.toBytes to convert this long to a byte array. If it is an int then even better. Now, write all 3 longs one after another to one array which will be your rowkey. This gives you: * fixed size * small row key - 3*8 bytes if you use long and 3*4 for in

Re: Odd WARN in hbase 0.94.2

2013-03-06 Thread Bryan Beaudreault
I'll add that this did fix my issue. The dn.50010 proc file is now in place and I have not see any other exceptions. Thanks guys! On Wed, Mar 6, 2013 at 10:45 PM, Anoop Sam John wrote: > Hi Kevin > Thanks for the information. In HBase book, we have added a > note on how to use

RE: Odd WARN in hbase 0.94.2

2013-03-06 Thread Anoop Sam John
Hi Kevin Thanks for the information. In HBase book, we have added a note on how to use short circuit reads.. Can we update it accordingly? Which version of HDFS need this attribute to be present in DN side also? It would be great if you can file a JIRA and give a change in the de

Re: how do counters work, client api?

2013-03-06 Thread Jeff Kolesky
One interesting post to read is from Lars H: http://hadoop-hbase.blogspot.com/2012/11/hbase-mvcc-and-built-in-atomic.html It describes how the atomic counter operations work. Jeff On Wed, Mar 6, 2013 at 12:25 PM, S Ahmed wrote: > Where can I read up on how counters work with hbase? > > Does it

Re: Odd WARN in hbase 0.94.2

2013-03-06 Thread Bryan Beaudreault
Great, thanks for the help Kevin. He's correct that we only have dfs.client.read.shortcircuit set in our hbase-site.xml, and we don't have dfs.domain.socket.path set anywhere. I'll get that added. Thanks again! On Wed, Mar 6, 2013 at 3:43 PM, Kevin O'dell wrote: > Hi Byran, > > One of enginee

Re: Odd WARN in hbase 0.94.2

2013-03-06 Thread Kevin O'dell
Hi Byran, One of engineers wanted me to pass this recommendation along: The issue here is mostly likely a missing configuration. We need to have this configuration on both the DataNode and RegionServer: dfs.domain.socket.path /var/run/hadoop-hdfs/dn._PORT dfs.client.read.shortcircuit

Re: Unable to fully decommission a RS

2013-03-06 Thread Nicolas Liochon
Yes, decommissioning the regionserver does not mean decommissioning the datanode. Here, if I understand well your first step, you migrated the regions to other regions servers. Physically, the data was still on the previous machine, with the hdfs datanode. It's not used anymore for writes if all ot

Unable to fully decommission a RS

2013-03-06 Thread Yves Langisch
Hi, I'm trying to decommission a RS, i.e. migrate all regions to different nodes in order to shutdown that specific node. First I've moved all regions to different nodes and made sure that there are no regions anymore on this node: ... ip-11-11-111-111.ec2.internal:60020 1362060123906