On Thursday 29 May 2008 01:04, Daniel Cheng wrote: > On Wed, May 28, 2008 at 7:04 PM, Matthew Toseland > <toad at amphibian.dyndns.org> wrote: > > On Wednesday 28 May 2008 01:22, Daniel Cheng wrote: > >> On 5/28/08, Matthew Toseland <toad at amphibian.dyndns.org> wrote: > >> > On Sunday 25 May 2008 12:46, Daniel Cheng wrote: > >> > > On Sat, May 24, 2008 at 7:40 AM, Matthew Toseland > >> > > <toad at amphibian.dyndns.org> wrote: > >> > > > On Friday 23 May 2008 15:20, j16sdiz at freenetproject.org wrote: > >> > > >> Author: j16sdiz > >> > > >> Date: 2008-05-23 14:20:31 +0000 (Fri, 23 May 2008) > >> > > >> New Revision: 20061 > >> > > >> > >> > > >> Modified: > >> > > >> > >> > > > > >> > > > branches/saltedhashstore/freenet/src/freenet/store/SaltedHashFreenetStore.java > >> > > >> Log: > >> > > >> use bloom filter > >> > > > > >> > > > Can we be 100% sure of not adding the same key twice? The write code > > takes > >> > a > >> > > > lock, then checks every place it could be, then writes it, and updates > > the > >> > > > bloom filter, so we should never add the same key twice, right? > >> > > > > >> > > > >> > > Unless two threads trying to put the same entry at the same time. > >> > > >> > Locking... > >> > >> In the current implementation, the lock is not held long enough to > >> prevent this. Updating the bloom filter is slow, as it always fsync() > >> after each update. I don't think holding a lock that long is a good > >> idea. > > > > Can't you do the fsync outside the lock? > > That's what we are doing: > 1 check if the store already exist > 2 write to bloom filter > 3 fsync > 4 write the store > > If we want to prevent writing the bloom filter twice for the same key, > the lock have to be held until the block is written to the store. > If we swap step 3 and 4, there may be false negative if it crash in between
You need to hold the lock while actually writing to the bloom filter: if you have two threads operating on different bits within the same byte, and there is no locking, one of them will undo the other's changes. That is assuming you always read a byte and write a byte... Do we do this? If there is a crash we will get false negatives anyway, won't we? Or do we put the bit before we write the entry? Maybe we're okay then? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20080529/942d6553/attachment.pgp>
