On Wed, 22 Mar 2006, John Stanton wrote:
>Our approach to byte order independence was fairly simple, and worked
>well with a mmap'd index. It involved keeping the just word pointers in
>a local byte ordered block if the machine were a different Endian. The
>overhead was next to insignificant. Our indices were all byte order
>independent.
The biggest problem I can envisage is you'd have to implement a lots of
small mmap'd regions, as you can't guarantee the file will fit in
the process memory map. This has performance issues for the kernel, both
in tracking the number of regions, and potentially the management of the
regions (think MMU overhead, TLB shoot-down etc.) Against IO, this
overhead is probably small, but it does add another level of complexity in
the source.
Also, on platforms such as HP-UX, the mmap cache and the regular block
cache are not synched, so having the same file mmap'ed and open as a file
descriptor can cause problems. Another reason why HP-UX sucks, I suppose.
Finally, locking is file descriptor based. You'd have to keep the file
descriptor around, both for the locking, and to increase the file size.
>
>Avoiding buffer shadowing seemed to be one big win, and the other was
>letting the OS VM management take control.
Which OS was this, BTW?
>JS
>
>Nathan Kurz wrote:
>> On Wed, Mar 22, 2006 at 10:41:23AM +1100, John Stanton wrote:
>>
>>>The mmap'd index was about three times faster than when it
>>>used a LRU paged cache.
>>
>>
>> I looked fairly closely into the possibility of using mmap for the
>> SQLite btree backend, and realized that it would be quite difficult.
>> Because the SQLite file format is host byte-order independent, it's
>> almost impossible to use mmap without a separate cache. If one were
>> to give up on the cross-platform portability, I think one could get a
>> significant speedup on large file access, but it would be necessary to
>> write/adapt the entire backend rather than just making small changes.
>>
>> Nathan Kurz
>> [EMAIL PROTECTED]
>
>
--
/"\
\ / ASCII RIBBON CAMPAIGN - AGAINST HTML MAIL
X - AGAINST MS ATTACHMENTS
/ \
