On Fri, Oct 17, 2014 at 6:54 PM, Howard Chu <[email protected]> wrote: > Luke Kenneth Casson Leighton wrote: >> >> howard, et al, hi, >> >> i notice that LMDB stores data in an "append only" manner (from the >> Q&A). i was thinking of recommending that in python the buffers be >> handed around with *direct* and persistent pointers to the data within >> the LMDB shm/memory-map. >> >> the reason that i believe this will be successful is because the >> circumstances are rather unusual: once data is entered (added) it is >> *never* overwritten, updated, changed, or modified. there simply is >> no need [in this application]. so there will only ever be insertion >> (once and only once) or controlled and monitored deletion (by age of >> record). >> >> under these circumstances is it perfectly reasonable to pass around >> the pointers to the buffers returned from value look-ups >> *indefinitely*? if so that would be awesome. > > > Any time a new value is inserted into an existing page, that page's address > will change due to COW. Any time an old page gets sufficiently old that no > other txn points to it, it becomes eligible for reuse - thus completely > unrelated data could get onto it. So unless you're keeping very close tabs > on your application's key+data sizes and know how full a page got, you can't > rely on particular addresses staying valid.
ok. that makes sense. a pity, though, because it would be reaally cool to refer to data by buffer address forever. COW means it's beyond tiblmdb's control, but hey, i tried :) l.
