Hi, On Nov 15, 2007 11:07 AM, Thomas Mueller <[EMAIL PROTECTED]> wrote: > A few remarks about the current NGP prototype: > > java.nio.ByteBuffer (FileJournal / MediumRecord) > Just curious, did you run some tests, is it faster than using > RandomAccessFile.read?
I'm planning to... It would be really nice to map the entire journal file (or the part that already exists when a repository is started) into memory for direct access. In theory this should reduce the amount of copying taking place, but it's still unclear whether that happens in practice. > FileChannel.map > My experience with memory mapped files is: it is fast, but > problematic. The biggest problem (for me) is: there is no way to unmap > a file. That means it is not possible to truncate, rename, or delete a > file that was once mapped. See: > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4724038 > > Journal: > "A record is never modified or removed once it has been added to a > journal.". I agree, if there is a mechanism to removing old journal > files. The way I see it, during normal operation we only need to be able to append data to the file, so this shouldn't be a problem. At some points the journal file needs to be vacuumed to save space, but that operation would typically produce a new file that's then used to replace the old journal. I'm not yet sure how often and at which points such vacuuming needs to take place. Also, I have an experimental C version of the journal and tree code. The idea behind that is possibly to implement a mod_dav module to enable direct WebDAV access to the tree structure stored in the journal. The C implementation could also eventually become a highly optimized JNI backend for an NGP implementation, in which case we'd also have much more control over memory mapping (of course at the expense of operating system dependencies). BR, Jukka Zitting
