On 11/7/06, Dennis Schaffer <[EMAIL PROTECTED]> wrote:
My Linux guy has asked a question (one of many) I can't answer about Linux memory/swap storage hierarchy and how VM affects that.
And I thought the paper on Linux swapping that I am working on would answer every question one might ask... ;-) And to avoid confusion like we had last week - this is how I believe CP works, which may or may not be what you would make if you designed a new operating system.
Within Linux (forget VM for a moment), Linux tries to keep active pages in memory. When Linux completely consumes memory, it migrates what it considers to be inactive pages to swap to free memory. When it needs pages from swap, it migrates them back to memory, sometimes resulting in other pages being migrated to swap.
Yes, but do remember that "lately referenced" is not all that counts when Linux selects pages to be freed up by swapping out the contents.
Using the IND USER EXP command and my performance monitor (assuming I'm interpreting them both correctly), I can see that both Linux machine virtual storage and swap/vdisk are spread across all three of the VM storage hierarchies.
For the true evaluation and sizing you also need to look at paging rates at each of those boundaries. Your performance monitor provides the necessary details. I am trying to come up with a consistent way to use the numbers for sizing, but there's a lot of exceptions.
Now for the question: Since Linux has moved what it considers inactive pages to swap/vdisk, why would VM keep any of that machine's vdisk pages in central storage in a VM-central-storage-constrained environment? I would expect that VM would migrate all of a Linux machine's vdisk pages to xstor/disk before it would migrate any of that machine's virtual storage pages away to xstor/pagdisk. The only exception I can think of would be that VM might migrate vdisk pages from pagdisk/xstor to cstor as part of Linux swap migration back to its memory.
1) Selection of pages in real memory is not LRU but WAG. So there may not be a good reason why one page is selected to be paged out and the other is not. 2) You would need how pages are freed. Many installations find the Linux servers mostly in-queue so CP will find little relief in going after pages of idle virtual machines. In that case CP ends up taking pages from active virtual machines, and iirc at that point it does not matter anymore whether the pages are part of virtual machine main memory or a vdisk. 3) When the virtual machine references a page that is already out on disk, CP will bring in the entire swap set (which for a DAT-on virtual machine is not always what you would look for) Normally, you'd want the virtual machine small enough that it is not being paged. But there is part of the virtual machine inactive that you do want to have paged out (like portions of the kernel that you don't use, tables for parts that are not in use). So for the virtual machine primary address space you mainly watch the page-in rate. Since the page selection is WAG, some page-in from expanded storage is hard to avoid if you want to let CP pick some pages to page out. But at least you should be concerned when the virtual machine is constantly paging in from DASD (because the two LRU algorithms on top of each other make it bad). For the vdisk you have the same instrumentation. Again, you want the vdisk small enough that it is not being paged from DASD by VM (since the typical reference pattern is write-once, read-once that would mean each swap causes a page-in by CP, which is not what we want). If your vdisk is small enough the references will be frequent enough to keep it in. And since a small vdisk may not be enough to satisfy Linux' virtual memory needs, we create a hierarchy in swap devices as well.
And, another question: when VM decides to migrate pages from expanded storage to dasd, does it choose vdisk pages before virtual storage pages?
My understanding is that this is LRU only. Rob -- Rob van der Heij Velocity Software, Inc http://velocitysoftware.com/
