On Tue, Oct 22, 2019 at 3:14 PM Rafael J. Wysocki <raf...@kernel.org> wrote: > > On Tue, Oct 22, 2019 at 11:26 PM Luigi Semenzato <semenz...@google.com> wrote: > > > > Thank you for the quick reply! > > > > On Tue, Oct 22, 2019 at 1:57 PM Rafael J. Wysocki <raf...@kernel.org> wrote: > > > > > > On Tue, Oct 22, 2019 at 10:09 PM Luigi Semenzato <semenz...@google.com> > > > wrote: > > > > > > > > Following a thread in linux-pm > > > > (https://marc.info/?l=linux-mm&m=157012300901871) I have some issues > > > > that may be of general interest. > > > > > > > > 1. To the best of my knowledge, Linux hibernation is guaranteed to > > > > fail if more than 1/2 of total RAM is in use (for instance, by > > > > anonymous pages). My knowledge is based on evidence, experiments, > > > > code inspection, the thread above, and a comment in > > > > Documentation/swsusp.txt, copied here: > > > > > > So I use it on a regular basis (i.e. every day) on a system that often > > > has over 50% or RAM in use and it all works. > > > > > > I also know about other people using it on a regular basis. > > > > > > For all of these users, it is usable. > > > > > > > "Instead, we load the image into unused memory and then atomically > > > > copy it back to it original location. This implies, of course, a > > > > maximum image size of half the amount of memory." > > > > > > That isn't right any more. An image that is loaded during resume can, > > > in fact, be larger than 50% of RAM. An image that is created during > > > hibernation, however, cannot. > > > > Sorry, I don't understand this. Are you saying that, for instance, > > you can resume a 30 GB image on a 32 GB device, but that image could > > only have been created on a 64 GB device? > > Had it been possible to create images larger than 50% of memory during > hibernation, it would have been possible to load them during resume as > well. > > The resume code doesn't have a 50% of RAM limitation, the image > creation code does.
Thanks a lot for the clarifications. It is possible that you and I have different definitions of "working in general". My main issue ia that I would like image creation (i.e. entering hibernation) to work with >50% of RAM in use, and I am extrapolating that other people would like that too. I can see that there are many uses where this is not needed though, especially if you mostly care about resume. > > > > > 2. There's no simple/general workaround. Rafael suggested on the > > > > thread "Whatever doesn't fit into 50% of RAM needs to be swapped out > > > > before hibernation". This is a good suggestion: I am actually close > > > > to achieving this using memcgroups, but it's a fair amount of work, > > > > and a fairly special case. Not everybody uses memcgroups, and I don't > > > > know of other reliable ways of forcing swap from user level. > > > > > > I don't need to do anything like that. > > > > Again, I don't understand. Why did you make that suggestion then? > > > > > hibernate_preallocate_memory() manages to free a sufficient amount of > > > memory on my system every time. > > > > Unfortunately this doesn't work for me. I may have described a simple > > experiment: on a 4GB device, create two large processes like this: > > > > dd if=/dev/zero bs=1100M count=1 | sleep infinity & > > dd if=/dev/zero bs=1100M count=1 | sleep infinity & > > > > so that more than 50% of TotalMem is used for anonymous pages. Then > > echo disk > /sys/power/state fails with ENOMEM. > > I guess hibernate_preallocate_memory() is not able to free enough > memory for itself in that case. > > > Is this supposed to work? > > Yes, it is, in general. > > > Maybe I am doing something wrong? > > Hibernation works before I create the dd processes. After I force > > some of those pages to a separate swap device, hibernation works too, > > so those pages aren't mlocked or anything. > > It looks like you are doing something that is not covered by > hibernate_preallocate_memory(). > > > > > 3. A feature that works only when 1/2 of total RAM can be allocated > > > > is, in my opinion, not usable, except possibly under special > > > > circumstances, such as mine. Most of the available articles and > > > > documentation do not mention this important fact (but for the excerpt > > > > I mentioned, which is not in a prominent position). > > > > > > It can be used with over 1/2 of RAM allocated and that is quite easy > > > to demonstrate. > > > > > > Honestly, I'm not sure what your problem is really. > > > > I apologize if I am doing something stupid and I should know better > > before I waste other people's time. I have been trying to explain > > these issues as best as I can. I have a reproducible failure. I'll > > be happy to provide any additional detail. > > Simply put, hibernation, as implemented today, needs to allocate over > 50% of RAM (or at least as much as to be able to copy all of the > non-free pages) for image creation. If it cannot do that, it will > fail and you know how to prevent it from allocating enough memory in a > reproducible way. AFAICS that's a situation in which every attempt to > allocate 50% of memory for any other purpose will fail as well. > > Frankly, you are first to report this problem, so it arguably is not > common. It looks like hibernate_preallocate_memory() may be improved > to cover that case, but then the question is how much more complicated > it will have to become for this purpose and whether or not that's > worth pursuing. Right. I was hoping to discuss that. Is it easier to do in the kernel what I am trying to do at user level, i.e. force swap of excess pages (possibly to a separate device or partition) so that enough pages are freed up to make hibernate_preallocate_memory always succeed? I started reading the swap code, but it is entangled with page reclaim and I haven't seen a simple solution, neither do I know if there is one and how long it would take to find it, or code around it. (However I haven't looked yet at how it works when memcgroup limits are lowered---that may give me good ideas). Thanks! > > > > > > > > Two questions then: > > > > > > > > A. Should the documentation be changed to reflect this fact more > > > > clearly? I feel that the current situation is a disservice to the > > > > user community. > > > > > > Propose changes. > > > > Sure, after we resolve the above questions. > > > > > > B. Would it be worthwhile to improve the hibernation code to remove > > > > this limitation? Is this of interest to anybody (other than me)? > > > > > > Again, propose specific changes.
