Thrift and coprocessors

2012-03-19 Thread Ben West
Hi all, We use thrift to access HBase, and I've been playing around with endpoint coprocessors. I'm wondering how I can use thrift to access these - it seems like they're mostly supported with Java clients. So far, I've just been adding each function to the thrift schema and then manually edit

Table that won't delete

2012-03-26 Thread Ben West
Hello all, I accidentally created a table using LZO compression, and I don't have everything set up for LZO. So I tried to alter the table, but the shell gave me an error that the table was disabled. Tried to enable, and it said the table was enabled. Couldn't drop the table because it thinks t

Re: Table that won't delete

2012-03-26 Thread Ben West
I fixed this of course right after sending to the mailing list. I had to delete the node from zookeeper and restart HBase. - Original Message - From: Ben West To: "user@hbase.apache.org" Cc: Sent: Monday, March 26, 2012 4:40 PM Subject: Table that won't delete

Re: Regionserver assignment

2012-04-03 Thread Ben West
Hey Balaji, If you dislike HBase's default region assignments, you can use the "move" command in shell to change where they're hosted.  -Ben - Original Message - From: Balaji k To: user@hbase.apache.org Cc: Sent: Monday, April 2, 2012 11:49 PM Subject: Regionserver assignment Hi,  

NotAllMetaRegionsOnlineException

2011-09-18 Thread Ben West
Hey All, I'm following the quickstart guide (http://hbase.apache.org/quickstart.html). I downloaded the tar, unzipped and started just fine. But when I try to move on to the next step, I get: ~/S/h/h/bin> ./hbase shell HBase Shell; enter 'help' for list of supported commands. Type "exit" to lea

REST questions

2011-09-23 Thread Ben West
Hey all, I'm trying to use the REST API and I have a couple questions: 1. If I save a value with a timestamp of t, it seems like I have to query for values of t+1 (or +2 etc.). So does the timestamp portion of the API mean "find the most recent value written before this timestamp"? (As opposed

RE: NotAllMetaRegionsOnlineException

2011-09-25 Thread Ben West
ure that your machine DNS resolve the host properly. -Stuti -Original Message----- From: Ben West [mailto:bwsithspaw...@yahoo.com] Sent: Monday, September 19, 2011 3:27 AM To: user@hbase.apache.org Subject: NotAllMetaRegionsOnlineException Hey All, I'm following the quickstart guide (http:

Spaces disappear in HBase?

2011-10-02 Thread Ben West
Hey all, I'm running the standalone HBase server (0.90.4) and REST client (version 0.0.2). When I POST data and then GET it back, the data is changed; particularly the spaces seem to be removed. Does anyone know what's going on? Here is a python script replicating my problem; I have a table nam

Re: Spaces disappear in HBase?

2011-10-09 Thread Ben West
having trouble with this.) -Ben - Original Message - From: Andrew Purtell To: "user@hbase.apache.org" ; Ben West Cc: Sent: Monday, October 3, 2011 6:50 PM Subject: Re: Spaces disappear in HBase? Keys and values need to be base64 encoded in all non-binary representations,

HMaster issues

2011-10-18 Thread Ben West
Hey All, I'm having an issue I can't debug. The shell gives me a MasterNotRunningException, but I can connect to the master via the web interface. I'm using the Cloudera demo VM, which has a standalone instance of HBase. I'm attaching the HBase-master log file. The only thing I can see is a wa

Custom timestamps

2011-10-19 Thread Ben West
Hi all, We're storing timestamped data in HBase; from lurking on the mailing list it seems like the recommendation is usually to make the timestamp part of the row key. I'm curious why this is - is scanning over rows more efficient than scanning over timestamps within a cell?  The book says: "

Re: Custom timestamps

2011-10-20 Thread Ben West
Thanks Stack. We are indeed using locks outside of HBase, but I hadn't heard about the problems with HBase's locks. Good to know. -Ben - Original Message - From: Stack To: user@hbase.apache.org; Ben West Cc: Sent: Wednesday, October 19, 2011 5:24 PM Subject: Re: Custom

Re: Custom timestamps

2011-10-20 Thread Ben West
Message - From: Ben West To: "user@hbase.apache.org" Cc: Sent: Thursday, October 20, 2011 9:13 AM Subject: Re: Custom timestamps Thanks Stack. We are indeed using locks outside of HBase, but I hadn't heard about the problems with HBase's locks. Good to know. -Ben --

Re: HMaster issues

2011-10-20 Thread Ben West
iel Cryans To: user@hbase.apache.org; Ben West Cc: Sent: Tuesday, October 18, 2011 1:23 PM Subject: Re: HMaster issues This line: java.lang.NoSuchMethodException: org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.getFileLength() Is because it's using the local filesystem and no

Importtsv error

2011-10-24 Thread Ben West
Hey all, I'm getting this error: $ hadoop jar /usr/lib/hbase/hbase-0.90.3-cdh3u1.jar importtsv -libjars /usr/lib/hbase/lib/guava-r06.jar Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/zookeeper/KeeperException ... I found a few threads [1,2] which seem to be the same th

Re: Importtsv error

2011-10-24 Thread Ben West
This worked, thanks! - Original Message - From: Ravikumar MAV To: user@hbase.apache.org; Ben West Cc: Sent: Monday, October 24, 2011 2:51 PM Subject: Re: Importtsv error You can try export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:`hbase classpath` in your shell On Mon, Oct 24, 2011 at

Meta region hotspotting

2011-12-08 Thread Ben West
Hey all, We have a cluster with four region servers and about 2,000 regions. We're using the REST server, and we've noticed that whatever region is hosting META gets 3-5x the number of requests that the other regions do. It's my understanding that the client should cache the row start/end loca

Re: Meta region hotspotting

2011-12-09 Thread Ben West
Thanks Stack. Is there any way I can log requests going to .META. vs. region servers? I've tried setting various portions of HBase to DEBUG but I haven't found the magic combo yet. - Original Message - From: Stack To: user@hbase.apache.org; Ben West Cc: Sent: Thursday,

Re: Meta region hotspotting

2011-12-15 Thread Ben West
is sound plausible? I can submit a JIRA asking for maxSize to be a config param if so. We have >> 10 simultaneous requests. - Original Message ----- From: Ben West To: Stack ; "user@hbase.apache.org" Cc: Sent: Friday, December 9, 2011 10:52 AM Subject: Re: Meta region hotspo

Re: Meta region hotspotting

2011-12-23 Thread Ben West
ect: Re: Meta region hotspotting Can u try w/o a limit or with upped limit and see if a diff?    Sounds plausible yes but looks like u could make a fact with some small experiments Thanks On Dec 15, 2011, at 12:25 PM, Ben West wrote: > Digging into this further, I see the following in HT

HBase 0.92rc3 rest performance

2012-01-17 Thread Ben West
Hi all We're trying out .92rc3 instead of .90.4, and for the most part everything seems fine. But we have a simple test of REST performance which is basically a large number of cURL jobs getting random rows, and this test is running *a lot* slower under .92. When we run just a single client do

Re: HBase 0.92rc3 rest performance

2012-01-18 Thread Ben West
server since thrift and native > clients stayed the same. > > Can you provide us your test so we can do testing on our side too? > > Maybe doing a few jstacks on the REST server could point out the > obvious bottlenecks. > > J-D > > On Tue, Jan 17, 2012 at 1

Re: Meta region hotspotting

2012-01-19 Thread Ben West
your help everyone, -Ben - Original Message - From: Jack Levin To: user@hbase.apache.org Cc: Ben West Sent: Monday, December 26, 2011 7:32 PM Subject: Re: Meta region hotspotting Some time ago, we had a situation where our REST server was slammed with queries that did not find any matches for

Re: Newbee Question : POC Idea for Hadoop Application

2012-01-31 Thread Ben West
There are lots of free, large data sets: * http://www.readwriteweb.com/archives/where_to_find_open_data_on_the.php * http://www.quora.com/Data/Where-can-I-get-large-datasets-open-to-the-public Just find one that interests you. There probably aren't many fields in which people didn't wish there w

Re: randomWrite tests gives random results

2012-02-06 Thread Ben West
You can try turning on verbose garbage collection logs and see if the slow times correspond to a GC pause. Cloudera has a series of blog posts regarding GC pauses in HBase and how to avoid them: http://www.cloudera.com/blog/2011/02/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffer