Re: z/VM page space
On Tue, 14 Jun 2011 11:04:25 -0700, Schuh, Richard rsc...@visa.com wrot e: A good decision, probably not a difficult one, by Mr. Holder and friends . Untangling that can of worms should not be a high priority use of development $$. Regards, Richard Schuh There are, of course, other reasons one might want this (or similar) function, such as supporting paging devices with dramatically different response times and bandwidths - load balancing new outbound writes which we already do) isn't really sufficient for such a scenario - in order to prevent the fastest devices from filling up with the oldest data over time, a migration function would be required to move those now older pages off to slower devices (since they're not being re-referenced, you really don't want them taking up space on the faster devices). Also, the real design issue is that there's simply not enough real estate to maintain backward pointers up the structures in order to be able to easily determine, from a paging slot (4K record) on a paging volume back up to the owner of the page it contains - the storage overhead becomes astronomical rather quickly, and you don't want to be spending lots of storage representing things that are themselves paged out. So the only way to find the pages is to run all of the translation structures for all virtual storage from the top down scanning for them (this is indeed how expanded storage migration works, which is why ATTACH XSTORE often takes so long). One further comment on the idea of bootlegging the migration function by touching user pages - it's not just user pages that are out there, it's quite likely that the target paging volume also contains some pageable CP owned pages (especially PGMBKs), some of which are simply not touchable by any action from a guest. So even if you could figure out which users had pages on the volume, it would still likely be impossible to get all of the pages off of it. Anyway, there are long standing requirements open for the drain migrate function, we know it would be useful, but thus far, there have always seemed to be more pressing needs. Bill Holder, Senior Software Engineer IBM z/VM Development, Memory Management, Endicott, NY Phone: 607-429-3640
Re: z/VM page space
On 6/14/11 12:03 PM, David Boyes dbo...@sinenomine.net wrote: I suspect that it would be possible to tweak the page device list in real storage to remove the volume in question, run the CP page list in real core and force a page in/page out sequence for pages on the volume in question, but There Lie Big Nasty Dragons. There's a lot of interlocking pieces and CP integrity is at stake, so it would not be simple code. Hmm... This might not be as hard as I thought. If the volume is already drained for new traffic (probably a safety prereq for this to happen), then determining pages on those volumes and forcing page-in/page-out is probably safer than previously thought. CP can maintain the appropriate interlocks if we force CP to fault the page in, touch the page in a way that forces CP to consider the page as dirty, and then allow the normal page-out logic to determine the new location. It would cause a fairly large burst in paging activity while running, but that's probably not an big issue for the few times this would get activated. Scratch, scratch, scratch... There's an itch in here somewhere. -- db
Re: z/VM page space
An absolute prereq, not just a safety one. If it is not already draining, then there would be nothing to prevent new pages from being written on the device. It will still be somewhat tricky because the page may already be in storage, in use by a virtual machine. Also, it may be part of a DCSS, in which case, only your private copy (resulting from your changing the page) will be paged out by your activity; others will still be referencing the original page. Regards, Richard Schuh This might not be as hard as I thought. If the volume is already drained for new traffic (probably a safety prereq for this to happen), then determining pages on those volumes and forcing page-in/page-out is probably safer than previously thought. CP can maintain the appropriate interlocks if we force CP to fault the page in, touch the page in a way that forces CP to consider the page as dirty, and then allow the normal page-out logic to determine the new location. It would cause a fairly large burst in paging activity while running, but that's probably not an big issue for the few times this would get activated. Scratch, scratch, scratch... There's an itch in here somewhere. -- db
Re: z/VM page space
On 6/14/11 1:20 PM, Schuh, Richard rsc...@visa.com wrote: An absolute prereq, not just a safety one. If it is not already draining, then there would be nothing to prevent new pages from being written on the device. Yeah. Playing with it on the whiteboard indicated that. Probably the only safe way to get a CP interlock in place that would prevent new pages being written. It will still be somewhat tricky because the page may already be in storage, in use by a virtual machine. Also, it may be part of a DCSS, in which case, only your private copy (resulting from your changing the page) will be paged out by your activity; others will still be referencing the original page. I think that's why you'd have to force CP to dirty the page rather than doing it inside a virtual machine, especially with CP's own pages potentially written at startup. Probably only doable from inside CP itself, or at minimum, via manipulate of real storage rather than virtual storage. There Be Dragons. Another project for my Copious Spare Time. Not. -- db
Re: z/VM page space
On Tuesday, 06/14/2011 at 01:30 EDT, David Boyes dbo...@sinenomine.net wrote: I think that's why you'd have to force CP to dirty the page rather than doing it inside a virtual machine, especially with CP's own pages potentially written at startup. Probably only doable from inside CP itself, or at minimum, via manipulate of real storage rather than virtual storage. There Be Dragons. Another project for my Copious Spare Time. Not. And now you know why DRAIN MIGRATE doesn't exist. :-) As Bill Holder, z/VM Memory Master, alludes, there are no data structures in CP that index the contents of paging volumes. You would have to traverse every users' memory management data structures to find references to page slots on the drained volume. In short, Eeeww. Alan Altmark z/VM and Linux on System z Consultant IBM System Lab Services and Training ibm.com/systems/services/labservices office: 607.429.3323 mobile; 607.321.7556 alan_altm...@us.ibm.com IBM Endicott
Re: z/VM page space
On 6/14/11 1:45 PM, Alan Altmark alan_altm...@us.ibm.com wrote: And now you know why DRAIN MIGRATE doesn't exist. :-) Although SNAPDUMP does something awfully similar in concept -- there's probably some thinking that could be borrowed there. Doing the same kind of system suspend might be the route to building that table; again, it's probably not going to be used that often so a few seconds of freeze while we figure this stuff out might be OK. Beats a full IPL, anyway. As Bill Holder, z/VM Memory Master, alludes, there are no data structures in CP that index the contents of paging volumes. You would have to traverse every users' memory management data structures to find references to page slots on the drained volume. In short, Eeeww. Hmph. You guys let it out the door w/o spaying it properly...8-) Eeew, indeed. -- db
Re: z/VM page space
A good decision, probably not a difficult one, by Mr. Holder and friends. Untangling that can of worms should not be a high priority use of development $$. Regards, Richard Schuh -Original Message- From: The IBM z/VM Operating System [mailto:IBMVM@LISTSERV.UARK.EDU] On Behalf Of Alan Altmark Sent: Tuesday, June 14, 2011 10:45 AM To: IBMVM@LISTSERV.UARK.EDU Subject: Re: z/VM page space On Tuesday, 06/14/2011 at 01:30 EDT, David Boyes dbo...@sinenomine.net wrote: I think that's why you'd have to force CP to dirty the page rather than doing it inside a virtual machine, especially with CP's own pages potentially written at startup. Probably only doable from inside CP itself, or at minimum, via manipulate of real storage rather than virtual storage. There Be Dragons. Another project for my Copious Spare Time. Not. And now you know why DRAIN MIGRATE doesn't exist. :-) As Bill Holder, z/VM Memory Master, alludes, there are no data structures in CP that index the contents of paging volumes. You would have to traverse every users' memory management data structures to find references to page slots on the drained volume. In short, Eeeww. Alan Altmark z/VM and Linux on System z Consultant IBM System Lab Services and Training ibm.com/systems/services/labservices office: 607.429.3323 mobile; 607.321.7556 alan_altm...@us.ibm.com IBM Endicott
Re: z/VM page space
Agree. Those who have HA needs have more than one VM system to which to move the workload. Marcy -Original Message- From: The IBM z/VM Operating System [mailto:IBMVM@LISTSERV.UARK.EDU] On Behalf Of Schuh, Richard Sent: Tuesday, June 14, 2011 11:04 AM To: IBMVM@LISTSERV.UARK.EDU Subject: Re: [IBMVM] z/VM page space A good decision, probably not a difficult one, by Mr. Holder and friends. Untangling that can of worms should not be a high priority use of development $$. Regards, Richard Schuh
