Re: ConcurrentSkipListMap performance in the MemStore

Maybe we should open an issue, I may not have time soon. Also using AtomicIntegerArray's makes more sense as the memory footprint goes down, as does the garbage generated. CSLM implements hairy delete handling code, we don't need that. Skip lists are sorted Log(n) seek/insert-able linked lists.

JIRAs whose patches are ready to be committed

Modifying subject. On Tue, Jun 14, 2011 at 6:31 AM, Ted Yu wrote: > I am grateful for this opportunity to better serve community. > Currently I have patches for 3927 and 3904 which I think are close to being > ready. > Please kindly review them. > > Thanks > On Tue, Jun 14, 2011 at 4:49 AM, St

Re: ConcurrentSkipListMap performance in the MemStore

Woah. That looks promising Jason. Can you hack up TestMemStore.java and have it see if above will even pass the CSLM hbase tests? St.Ack On Tue, Jun 14, 2011 at 5:23 PM, Jason Rutherglen wrote: > Here's the implementation we can presumably drop in and benchmark: > > https://github.com/mspiegel/

Build failed in Jenkins: HBase-TRUNK #1971

See Changes: [stack] HBASE-3940 HBase daemons should log version info at startup and possibly periodically -- [...truncated 3479 lines...] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elap

Re: ConcurrentSkipListMap performance in the MemStore

Here's the implementation we can presumably drop in and benchmark: https://github.com/mspiegel/lockfreeskiptree On Tue, Jun 14, 2011 at 4:56 PM, Jason Rutherglen wrote: > Here's a paper on a ConcurrentSkipTree: > > http://www.cs.virginia.edu/~ms6ep/publications/michael-spiegel-dissertation.pdf >

Re: ConcurrentSkipListMap performance in the MemStore

Here's a paper on a ConcurrentSkipTree: http://www.cs.virginia.edu/~ms6ep/publications/michael-spiegel-dissertation.pdf On Tue, Jun 14, 2011 at 4:40 PM, Jason Rutherglen wrote: > Right, I think one naive approach would be to take CSLM, convert it to > use AtomicIntegerArray instead of objects (b

Re: ConcurrentSkipListMap performance in the MemStore

Right, I think one naive approach would be to take CSLM, convert it to use AtomicIntegerArray instead of objects (basically emulate objects in blocks of ints), and of course not implement delete functionality. The first challenge is making an AtomicIntegerArray based linked list where the value is

Re: ConcurrentSkipListMap performance in the MemStore

Or do as Todd suggested a while back and pull in CSLM and hack it to suit our needs; its public domain. St.Ack On Tue, Jun 14, 2011 at 3:23 PM, Jason Rutherglen wrote: >> See this J: >> http://www.cloudera.com/blog/2011/03/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-3/

Re: ConcurrentSkipListMap performance in the MemStore

> See this J: > http://www.cloudera.com/blog/2011/03/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-3/ Interesting. In Lucene for realtime search we have the same problem where the best/only option right now for the realtime terms dictionary is to use ConcurrentSkipListMa

Re: TestMultipleTimestamps hangs?

Ok thanks! On Tue, Jun 14, 2011 at 11:41 AM, Stack wrote: > It was broke for a while and then fixed last night w/ this commit: > > http://svn.apache.org/viewvc?view=revision&revision=1135384 > > Sorry about that Jason, > > St.Ack > > On Mon, Jun 13, 2011 at 12:29 PM, Jason Rutherglen > wrote: >>

Build failed in Jenkins: HBase-TRUNK #1970

See Changes: [stack] doc fix from leif wickland up on mailing list [todd] HBASE-3982. Improvements to TestHFileSeek. -- [...truncated 3328 lines...] [INFO] Copying 4 resources [INFO] Copying 3 resou

Re: TestMultipleTimestamps hangs?

It was broke for a while and then fixed last night w/ this commit: http://svn.apache.org/viewvc?view=revision&revision=1135384 Sorry about that Jason, St.Ack On Mon, Jun 13, 2011 at 12:29 PM, Jason Rutherglen wrote: > Looks like it's endlessly trying to connect, http://pastebin.com/5MV3tWJx >

Re: ConcurrentSkipListMap performance in the MemStore

See this J: http://www.cloudera.com/blog/2011/03/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-3/ St.Ack On Mon, Jun 13, 2011 at 10:58 PM, Jason Rutherglen wrote: > Is there an issue with garbage generated from using CSLM in the MemStore? >

Re: Please welcome our newest committer, Ted Yu!

Congratulations Ted! On Tue, Jun 14, 2011 at 4:01 AM, Ted Yu wrote: > I am thankful for this opportunity to better serve community. > Currently I have patches for 3927 and 3904 which I think are close to being > ready. > Please kindly review them. > > Thanks > On Tue, Jun 14, 2011 at 4:49 AM, Sta

Build failed in Jenkins: hbase-0.90 #203

See Changes: [todd] HBASE-3916. Fix the default bind address of ThriftServer to be wildcard instead of localhost. Contributed by Li Pi -- [...truncated 2864 lines...] Tests run: 42, Failures: 0, Error

Build failed in Jenkins: HBase-TRUNK #1969

See Changes: [stack] HBASE-451 Remove HTableDescriptor from HRegionInfo [stack] HBASE-3963 Schedule all log-spliiting at startup all at once -- reverse test for wals with no server (was wrong) --

Re: ConcurrentSkipListMap performance in the MemStore

I found this paper and implementation of a "Snap Tree" that could perhaps be interesting, though the linear iteration performance isn't as good as CSLM. http://ppl.stanford.edu/papers/ppopp207-bronson.pdf https://github.com/nbronson/snaptree On Mon, Jun 13, 2011 at 10:58 PM, Jason Rutherglen wr

Re: Please welcome our newest committer, Ted Yu!

Congrats and welcome aboard Ted! On Tue, Jun 14, 2011 at 12:31 AM, Ted Yu wrote: > I am thankful for this opportunity to better serve community. > Currently I have patches for 3927 and 3904 which I think are close to being > ready. > Please kindly review them. > > Thanks > On Tue, Jun 14, 2011 a