On Dec 9, 2019, at 12:54 PM, Richard Damon <rich...@damon-family.org> wrote: > > But without virtual memory, many applications combinations that work > acceptably now would just fail to run at all.
You don’t even have to get into swapping to find such cases. I once ran a headless app on a small cloud VPS that would run the system out of memory because it was linked to some rather large GUI libraries, which provided some small but essential bits of functionality. Because those libraries were written to assume they were running on multi-gigabyte desktop computers with VMM, they did things that were simply ill-considered on my 256 MB VPS, which was hosted on tech without VMM. If my VPS provider had used a hosting technology that allowed for swap space, most of those libraries’ pages could have been swapped out, solving the problem. Instead, I ended up having to upgrade to a 512 MB plan just to give that program some scratch space to play with. > Virtual memory itself isn’t the issue. Well, every layer of indirection has a cost. The question then becomes, what’s the cost of the alternatives? To take my GUI library case again, I could have rebuilt my app statically so that only the necessary library functions were linked into my program instead of having the whole .so mapped into my process's VM space, but that then means I need to relink the program every time the OS updates that library. I can pay that cost at least three different ways: 1. Download the sources again on each library upgrade, build from source, install, and remove the build and source trees. 2. Keep the sources around, build, install, and “make clean” on each library upgrade, paying extra for the disk space to hold the sources, but saving some on bandwidth and disk I/O from not needing to repeatedly unpack tarballs. 3. Keep the objects around as well, paying more for disk to hold the built but unlinked binaries in order to save some CPU on each relink. TANSTAAFL. You don’t get to not pay, you only get to choose *where* you pay. > the Linux system to my understanding doesn’t have an easy call to just start > up a brand new process with parameters from you Whether that’s true depends on your definitions. I’d say that system() and posix_spawn() are easy calls for starting brand new processes. However, these calls may be implemented in terms of fork() or similar, so we must continue down the rabbit hole… > a process will fork itself, creating two identical copies of itself, one will > continue, and the other will exec the new process, replacing itself with the > desired process. The act of forking SHOULD allocated all the virtual memory > for the copy of the process, but that will take a bit of time. You’re describing fork() before the mid-1980s, roughly. > Because most of the time, all that memory is just going to be released in a > couple of instructions, it made sense to just postpone the actual allocation > until it was actually used (which it likely wasn’t). It’s better to describe what happens as copy-on-write rather than anything being “postponed.” Modern fork() uses the more powerful VMM features of CPUs to mark the forked process’s pages as CoW so that they’re shared between the two children until one tries to change them. At that point, both processes get an independent copy of the changed page. In the case of the fork()/exec() pattern, most pages never do get copied, since they’re almost immediately released by the exec(). Thus, the cost is a bit of setup and tear-down that strictly speaking didn’t need to happen. It’s tiny. > the system allowed itself to overcommit memory Which is fine as long as you don’t run the system into swapping and you keep a bit of swap space around. You don’t run into serious problems under that condition until you run the system wholly out of swap, causing all of the bills to come due at once. > If the system was changed to not allow over committing, then forking would be > slower which hits all of the standard system routines. You could still have fast forking without overcommitting, you’d just pay the cost in unreachable RAM. If I have 4 GB of RAM in the system, and the kernel takes 1 GB of that, I start a 2.5 GB user space process, and my process forks itself with the intent of starting an 0.1 GB process, that fork would have to fail if overcommitting weren’t allowed. _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
