Re: mini-hbase configuration for tests

2011-09-01 Thread Eric Charles
Hi JD, I tried without success to hack our test architecture to evaluate the effect of those configs. I stopped to avoid diverging to a kind of 'hbase performance testing'. Definitely good to note that hbase.client.pause and hbase.master.event.waiting.time can be tuned for tests speed. Thx

Re: mini-hbase configuration for tests

2011-08-29 Thread Jean-Daniel Cryans
(cleaning up my unread emails hehe) > We are using 0.90.x. Which exact timing param can we set? (I'm not sure > which one looking http://hbase.apache.org/book/config.files.html#hbase.site) > > We set configuration via htu.getConfiguration().set... and not via > hbase-default.xml. Is it OK to do li

Re: mini-hbase configuration for tests

2011-08-21 Thread Eric Charles
On 16/08/11 05:33, Stack wrote: On Mon, Aug 15, 2011 at 5:12 PM, Garrett Wu wrote: I have a bunch of integration tests that spend a lot of time creating and deleting tables in a mini hbase cluster started using HBaseTestingUtility. Disabling and deleting a table seems to take a second or two

Re: mini-hbase configuration for tests

2011-08-17 Thread Garrett Wu
Thanks for the suggestions. I tweaked jobclient.completion.poll.interval and hbase.regionserver.msginterval, but that didn't seem to do much. I'll just not delete the tables, which is fine since they're all in a mini hbase anyway. On Mon, Aug 15, 2011 at 5:37 PM, Bill Graham wrote: > Hey Garre

Re: mini-hbase configuration for tests

2011-08-15 Thread Stack
On Mon, Aug 15, 2011 at 5:12 PM, Garrett Wu wrote: > I have a bunch of integration tests that spend a lot of time creating and > deleting tables in a mini hbase cluster started using HBaseTestingUtility. >  Disabling and deleting a table seems to take a second or two each time > -- I'm guessing it

Re: mini-hbase configuration for tests

2011-08-15 Thread Bill Graham
Hey Garrett, I'm not sure about a config setting but in Pig we changed TestHBaseStorage to delete all rows of tables instead of truncate them. This was faster since the tables are typically small in tests. See Dymitriy's note in the deleteAllRows method here: http://svn.apache.org/repos/asf/pig/t

mini-hbase configuration for tests

2011-08-15 Thread Garrett Wu
I have a bunch of integration tests that spend a lot of time creating and deleting tables in a mini hbase cluster started using HBaseTestingUtility. Disabling and deleting a table seems to take a second or two each time -- I'm guessing it's some sort of heartbeat interval it's waiting for. Is the