On Wed, Jul 05, 2017 at 08:36:46AM +0200, Michal Hocko wrote: > PROT_NONE would explicitly fault but we would simply > run over this mapping too easily and who knows what might end up below > it. So to me the guard gap does its job here.
I tend to think that applications that implement their own stack guard using PROT_NONE also assume that they will never perfom unchecked stack allocations larger than their own guard, thus the condition above should never happen. Otherwise they're bogus and/or vulnerable by design and it is their responsibility to fix it. Thus maybe if that helps we could even relax some of the stack guard checks as soon as we meet a PROT_NONE area, allowing VMAs to be tightly packed if the application knows what it's doing. That wouldn't solve the libreoffice issue though, given the lower page is RWX. Willy