On Wed, Jun 13, 2012 at 9:01 AM, Michal Novotny <[email protected]> wrote: > On 06/11/2012 11:42 PM, Nick Hurley wrote: >> >> -Keep some sort of hash of contents per entry (similar to how Chrome >> does things) >> *This could conceivably allow us to only delete entries that are known >> corrupt >> *This doesn't (necessarily) require any wild changes to the current >> on-disk format (it could be kept as a metadata field on all entries) >> *At first glance, the implementation seems relatively straightforward >> (adding the standard caveats here about how nothing is ever nearly as >> simple as one thinks, etc) > > The main drawback of this approach is that it would probably affect the > performance. Without doing any major change in how we open/read/write cache > entries we would need to check the content when opening the entry. This > means, we would need to read and hash up to 50MB (this is the default > max_entry_size) even if we just want to check entry's metadata.
Yes, I had thought of this when I was listing the options, not sure why I didn't list it (I know I intended to). >> -Use some sort of journaling and/or append-only structure to store the >> cache, with some markers for consistency per entry (or some other >> subdivision of the data) >> *The consistency markers are still undetermined >> *There are a lot of open implementations of filesystems using these >> kinds of structures, so lots of good reference material >> *I have a sneaking suspicion that even with lots of good reference >> material, this would not exactly be a simple implementation > > I was looking into this option about a year ago and I couldn't come up with > any journaling solution (for our current data structures) that wouldn't use > fsync. And fsync is a no-go option... You make a good point about fsyncing. However, I had envisioned this as something that would involve changing our current data structures, which then may or may not get us a no-fsync solution. You might have a better idea as to whether or not that's possible, given that you've already looked into journaling at least a little bit. _______________________________________________ dev-tech-network mailing list [email protected] https://lists.mozilla.org/listinfo/dev-tech-network
