I would be interested to test it with zfs, it might even be harmful On Mon, Jul 10, 2017 at 11:55 PM, Michael Barker <mike...@gmail.com> wrote:
> From our own experience with journaling the benefits of pre-touching files > depends on the file system used. When ended up using XFS and threw away > all of our pre-touching code as it had no benefit there. I would recommend > testing with a wide variety of file systems. > > Mike. > > On 11 July 2017 at 07:11, Vitaly Davidovich <vita...@gmail.com> wrote: > >> A few suggestions: >> >> 1) have you tried just reading the data in the prefaulting code, instead >> of dirtying it with a dummy write? Since this is a disk backed mapping, it >> should page fault and map the underlying file data (rather than mapping to >> a zero page, e.g.). At a high rate of dirtying, this will amplify the >> number of dirty pages and will add writeback pressure. Are you seeing >> writeback during the tests by the way? How much disk io is happening? Are >> the cores you're running on spending significant %iowait time? >> >> 2) have you tried playing with mmap and/or madvise hints? E.g. might be >> interesting to prefault the mapping with MAP_POPULATE, turn off your manual >> prefaulting, and see tail latency. Or madvise a sequential hint and see >> what happens with kernels own readahead/prefault heuristics. Some of this >> may not apply to OS X, I'm not that familiar with it. >> >> 3) as others have mentioned, try doing this benchmarking on Linux with >> something like perf and see where the counters end up between the different >> experiments. >> >> >> On Mon, Jul 10, 2017 at 2:41 PM Roman Leventov <leventov...@gmail.com> >> wrote: >> >>> On 10 July 2017 at 13:21, Martin Thompson <mjpt...@gmail.com> wrote: >>> >>>> Are you running recent MacOS with APFS which supports sparse files? >>>> >>> >>> If this is of any interest, my OS version is >>> >>> Darwin MacBook-Pro.local 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 >>> 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64 >>> >>> (i. e. very recent) >>> >>> When considering an algorithm for pre-touch ahead it is often useful to >>>> consider rate to determine who far ahead you should be pre-touching. >>>> >>> >>> Logically the higher the rate the longer pre-touch chunk should be, but >>> when I did pretouch of the whole 32 MB buffers (instead of 1 MB chunks) >>> pauses were even worse. >>> >>> >>>> When pre-touching then it can be better with a positional write of a >>>> byte, i.e. use https://docs.oracle.com/javase >>>> /7/docs/api/java/nio/channels/FileChannel.html#write(java. >>>> nio.ByteBuffer,%20long) which should map to pwrite() on Linux. This >>>> will be a safepoint whereas the mapped buffer write will not be. >>>> >>> >>> Thanks, will try >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "mechanical-sympathy" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to mechanical-sympathy+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> Sent from my phone >> >> -- >> You received this message because you are subscribed to the Google Groups >> "mechanical-sympathy" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to mechanical-sympathy+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "mechanical-sympathy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to mechanical-sympathy+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Studying for the Turing test -- You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group. To unsubscribe from this group and stop receiving emails from it, send an email to mechanical-sympathy+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
