Re: TreeImpl.isRemove performance problem and possible workaround

2012-11-07 Thread Thomas Mueller
Hi, Yes, I see. My patch is really just a quick hack (my idea was to avoid creating many HashMaps), but I'm not sure if it will always work correctly. Regards, Thomas On 11/7/12 4:02 PM, "Michael Dürig" wrote: > > >On 7.11.12 14:54, Thomas Mueller wrote: >> Hi, >> >> I think this has been

Re: TreeImpl.isRemove performance problem and possible workaround

2012-11-07 Thread Michael Dürig
On 7.11.12 14:54, Thomas Mueller wrote: Hi, I think this has been discussed before (but I'm not sure): there is a performance problem in TreeImpl.isRemoved(). I think I found a temporary workaround: See also the fixme tag in the code: FIXME rely on underlying mechanism to determine whethe

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Thomas Mueller
Hi, >Generational garbage collection should work pretty well for this case. >If the blob store can keep track of all blobs added since revision X, >it needs to only go through the diff from that revision the latest >ones to determine which of those blobs can be removed early. > >Since most extra b

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Thomas Mueller
Hi, >the format of references to binaries is documented in the MicroKernel >java doc, >see "Retention Policy for Binaries" [0]. Thanks! I didn't know it's already documented. Regards, Thomas

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Michael Dürig
On 7.11.12 10:37, Michael Dürig wrote: the format of references to binaries is documented in the MicroKernel java doc, see "Retention Policy for Binaries" [0]. Oh... missed that ;-) Thanks for clarifying. Oak core does currently not take this into account and uses a different format for refer

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Jukka Zitting
Hi, On Wed, Nov 7, 2012 at 9:51 AM, Thomas Mueller wrote: > Any other ideas? Generational garbage collection should work pretty well for this case. If the blob store can keep track of all blobs added since revision X, it needs to only go through the diff from that revision the latest ones to det

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Michael Dürig
On 7.11.12 10:24, Stefan Guggisberg wrote: On Wed, Nov 7, 2012 at 10:52 AM, Michael Dürig wrote: On 7.11.12 9:48, Thomas Mueller wrote: Hi, Didn't we talk once about defining a format for blob id references, so that a value of the format "bin:{blobId}" (or similar) is reference? This

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Stefan Guggisberg
On Wed, Nov 7, 2012 at 10:52 AM, Michael Dürig wrote: > > > On 7.11.12 9:48, Thomas Mueller wrote: >> >> Hi, >> >> Didn't we talk once about defining a format for blob id references, so >> that a value of the format "bin:{blobId}" (or similar) is reference? > > > This is exactly the problem I want

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Michael Dürig
On 7.11.12 9:48, Thomas Mueller wrote: Hi, Didn't we talk once about defining a format for blob id references, so that a value of the format "bin:{blobId}" (or similar) is reference? This is exactly the problem I wanted to pinpoint. There is a conceptual leak here: in order for the Microker

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Thomas Mueller
Hi, Didn't we talk once about defining a format for blob id references, so that a value of the format "bin:{blobId}" (or similar) is reference? Regards, Thomas On 11/7/12 10:17 AM, "Michael Dürig" wrote: > >On a related note: how does the garbage collector even find out whether >a binary is

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Michael Dürig
On a related note: how does the garbage collector even find out whether a binary is "referenced"? That is, on the Microkernel level, what does it actually mean for a binary to be referenced? Michael On 6.11.12 18:45, Michael Marth wrote: this might be a weird question from the leftfield, bu

Re: [MongoMK] BlobStore garbage collection

2012-11-07 Thread Thomas Mueller
Hi, >are we actually sure that the existing data store concept is worth the >trouble? No, we can't really be sure. There are multiple concepts, one is using a (cryptographic) hash, an other is to use garbage collection. As for "garbage collection" versus "delete blob when deleting the referencin