Hi Eric, You are right on both issues. There should be an option to turn of autoflushing when creating the HBaseStore. I think the gora properties file is a nice place for this option.
And indeed a flush should flush all threadlocal instances. I implemented the threadlocal code for the HBaseStore, because at the time there was no synchronization at all for the HBaseStore. The solution for the flushing problem would be to use a readwrite lock so that all threads can use their HTable instance freely (read-lock) except when flushing, because that will block actions until the flush is done (write-lock). That way all HTable instances will get flushed. A nicer solution would be to see if there are other (best-practices) ways to implement multithreading HBase client code. I will shortly file Jira issues and patch them. Ferdy. On Wed, Apr 4, 2012 at 2:45 AM, Eric Newton <eric.new...@gmail.com> wrote: > Hi Lewis, > > I changed gora-hbase in my local copy to always turn off auto flush, and > then I flush as-needed. It looks as though I might need to make sure that > I flush with the same thread I write with, since it is using thread-local > storage to pick up the correct client interface. I don't see that as a > major problem, it's just a surprising result of the way the connection was > implemented. I could be wrong, though. I only spent about 10 minutes > looking into it, in order to figure out why HBase was slower than I > expected. > > -Eric > > On Tue, Apr 3, 2012 at 4:23 PM, Lewis John Mcgibbney < > lewis.mcgibb...@gmail.com> wrote: > > > Hi Eric, > > > > On Tue, Apr 3, 2012 at 4:04 PM, Eric Newton <eric.new...@gmail.com> > wrote: > > > > > Is there any particular reason that gora-hbase uses auto flush on every > > > HTable connection? > > > > > > > I'm not using the HBase module at the moment so will wait for others to > > chime in, however maybe you could comment on an alternative > implementation > > (or simply to remove auto flush making the flush configurable instead?) I > > would be really interested to hear your comments. > > > > > > > > > > This makes Keith Turner's excellent goraci test run very slowly for > > hbase. > > > > > > > Well it's something we should defo look into then. Hopefully we can > > actually get this test suite integrated into our CI build soon, until > then > > thanks for pointing out the problem. > > > > > > > > > > Also, I was unable to subscribe using user-subscr...@gora.apache.org. > > > > > > > Yeah currently we don't have user list, all traffic has been coming > through > > dev@. I was actually going to wait until after our next release before > > getting user@ sorted as most of the work going on has been development > > since graduation. I'll progress with logging an issue with INFRA though, > > thanks for pointing this out. > > > > Lewis > > >
