Ulrich Windl wrote: >>>> "Vitaly Zuevsky" <vitaly.zuev...@gmail.com> schrieb am 24.05.2021 um 22:06 >>>> in > Nachricht <068e01d750d8$400a64e0$c01f2ea0$@gmail.com>: >>> Vitaly Zuevsky wrote: >>>> Hi >>>> >>>> MAP_LOCK flag to mmap() on LMDB readers should populate page cache >>>> from the file being mapped and preclude those pages from eviction. I >>>> was wondering if that was ever tested, especially with frequent writes >>>> to the underlying file. And is there a standard way (a compile option) >>>> to include that flag without modifying source code directly? I see >>>> quite a few major faults with perf stat, and I am looking if they could be >>> avoided. >>>> >>>> I am new to LMDB and I guess this discussion may have taken place >>>> before, if you could point me out to the right place.. >>> >>> No. In general it's a bad idea to interfere with the OS's paging decisions. >> In >>> particular, on a machine with plenty of RAM, pageouts aren't an issue >>> anyway. >>> On a system where memory is tight, page faults will be unavoidable, so the >>> question is what other memory are you going to lose if you force the LMDB >>> pages to stay locked in? >> >> Memory pressure isn't my problem. I see 2-15 major faults a second in the >> event loop of the reader thread, which can effectively stall the loop for >> anything between 1-400ms (I measure iowait jiffies of the pid). I understand >> the faults result from the writer thread, actively modifying the file mmaped >> by the reader. I assume writes to the file invalidate corresponding pages of >> the reader, and those invalid pages only get updated (via major faults) on >> first access by the reader. If not locking, perhaps madvise/willneed >> combination could help, what you think? > > > I don't know the internals of LMDB, but writing to a file that has mapped > region seems to be a poor idea. > Why not write the region?
On any modern OS with unified buffer cache, there is no difference. (OpenBSD is notable in its lack of a unified buffer cache, can't think of any other such deficient OSs these days.) > AFAIR HP-UX really didn't like that. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
