Re: fadvise DONTNEED implementation (or lack thereof)
On Mon, 15 Nov 2010 16:28:32 +0900 (JST), KOSAKI Motohiro kosaki.motoh...@jp.fujitsu.com wrote: Who can make rsync like io pattern test suite? a code change is easy. but to comfirm justification is more harder work. I'm afraid I don't have time to work up any code. I would be happy to try the patch with my backup use-case though. I'll just have to think of an objective way of measuring the result. - Ben -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: fadvise DONTNEED implementation (or lack thereof)
On 11/14/2010 12:20 AM, Ben Gamari wrote: On Sun, 14 Nov 2010 14:09:29 +0900 (JST), KOSAKI Motohirokosaki.motoh...@jp.fujitsu.com wrote: Because we have an alternative solution already. please try memcgroup :) Alright, fair enough. It still seems like there are many cases where fadvise seems more appropriate, but memcg should at least satisfy my personal needs so I'll shut up now. Thanks! - Ben Could someone expand on this a little? The there are no users of this feature argument is indeed a silly one. I've only wanted the ability to perform i/o without poisoning the cache since oh, 10 or more years ago at least. It really hurts my users since they are all direct login interactive db app users. No load balancing web interface can hide the fact when a box goes to a crawl. How would one use memcgroup to prevent a backup or other large file operation from wiping out the cache with used-once garbage? (note for rsync in particular, how does this help rsync on other platforms?) -- bkw -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: fadvise DONTNEED implementation (or lack thereof)
On Tue, 9 Nov 2010 16:28:02 +0900 (JST), KOSAKI Motohiro kosaki.motoh...@jp.fujitsu.com wrote: So, I don't think application developers will use fadvise() aggressively because we don't have a cross platform agreement of a fadvice behavior. I strongly disagree. For a long time I have been trying to resolve interactivity issues caused by my rsync-based backup script. Many kernel developers have said that there is nothing the kernel can do without more information from user-space (e.g. cgroups, madvise). While cgroups help, the fix is round-about at best and requires configuration where really none should be necessary. The easiest solution for everyone involved would be for rsync to use FADV_DONTNEED. The behavior doesn't need to be perfectly consistent between platforms for the flag to be useful so long as each implementation does something sane to help use-once access patterns. People seem to mention frequently that there are no users of FADV_DONTNEED and therefore we don't need to implement it. It seems like this is ignoring an obvious catch-22. Currently rsync has no fadvise support at all, since using[1] the implemented hints to get the desired effect is far too complicated^M^M^M^Mhacky to be considered merge-worthy. Considering the number of Google hits returned for fadvise, I wouldn't be surprised if there were countless other projects with this same difficulty. We want to be able to tell the kernel about our useage patterns, but the kernel won't listen. Cheers, - Ben [1] http://insights.oetiker.ch/linux/fadvise.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
