Re: Policy-based memory allocations (was Re: RFC: 1.97 roadmap)
=20 - Low memory heap (useful to move code off kern/i386/pc/startup.S)= =2E =20 Originally I thought of a path relocator32-relocator users-mm relocator32 is ready for next round of review but is untested. Now I= think about it mm patch isn't actually dependent on relocator32, jus= t you won't get some features (as loading big initrds and removal of os_area_size/os_area_addr fields) before relocator32 is used by all loaders. I will adjust mm patch to this and add .(text|data|bss)-lowmem section support. =20 I don't understand, what is the relation between relocator in loaders= and low memory heap? =20 Actually low memory heap is a special case of policy based allocation.= My design is: I have up to 4 different policies (can be more by modifying defines but has to be hardcoded for performance reasons and multiple of 4 for alignment reasons) Every region knows which allocator it has to use together with which policy. Current allocators: -Skip. Don't use this region with given policy -First. Try to allocate as low as possible -Last. Try to allocate as high as possible -Second. Allocate second free chunk from region. It's what is used cur= rently. The idea behind that design is that often loaders need a big continuous chunk of memory so if loaders get memory from bottom and the rest takes memory from top we're likely to have a chunk of necessary size available. =20 But available memory is several orders of magnitude bigger than the lar= gest block a loader will need. So is this really an issue? =20 Resume from hibernation needs a lot of memory in a single chunk --=20 Regards Vladimir '=CF=86-coder/phcoder' Serbinenko signature.asc Description: OpenPGP digital signature ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [RFT] Re: Policy-based memory allocations (was Re: RFC: 1.97 roadmap)
On Sun, Aug 16, 2009 at 09:35:37PM +0200, Vladimir 'phcoder' Serbinenko wrote: Actually I don't understand why it was proposed to include it in 1.97 at all - it changes memory management, bugs in it are likely to be critical and benefit is only smaller core. If it was up to me I would just postpone it. But it's possible to put it in 1.97 if someone tests it enough. Well, I was expecting that this wouldn't have to change anything on how the normal heap works. The heap we have now doesn't add low memory to its arena, so in theory it's possible for another mechanism which reuses the same functions to use a separate region that doesn't disrupt normal heap operation in any way. This solves the problem with bloat in startup.S, which is the reason I proposed lowmem heap as a feature for 1.97. But I agree that approaches that change internal heap structures shouldn't be pushed to 1.97. -- Robert Millan The DRM opt-in fallacy: Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
[RFT] Re: Policy-based memory allocations (was Re: RFC: 1.97 roadmap)
I have this thingie available at my git branches mm and mm+move but it needs more testing. Unless it's tested enough it should be postponed and not included in 1.97. Actually I don't understand why it was proposed to include it in 1.97 at all - it changes memory management, bugs in it are likely to be critical and benefit is only smaller core. If it was up to me I would just postpone it. But it's possible to put it in 1.97 if someone tests it enough. Personally I think nested partition patch should go in 1.97 since it's more tested and provides useful feature namely: (together with 2 smaller patches) ability to boot solaris. On Thu, Aug 13, 2009 at 11:01 PM, Vladimir 'phcoder' Serbinenkophco...@gmail.com wrote: But available memory is several orders of magnitude bigger than the largest block a loader will need. So is this really an issue? It's not always the case. Two examples 1) Solaris. At least some distributions of solaris use a big (70 MiB compressed, around 200 MiB compressed) initrd which has to be loaded as multiboot module in a single chunk. This puts biggest needed chunk in the same order of magnitude as RAM available on some smaller systems. 2) XNU hibernating. It requires booter to load hibernating image in a single chunk. Even though it's compressed it can easily be 50% of total RAM or more -- Robert Millan The DRM opt-in fallacy: Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel -- Regards Vladimir 'phcoder' Serbinenko Personal git repository: http://repo.or.cz/w/grub2/phcoder.git -- Regards Vladimir 'phcoder' Serbinenko Personal git repository: http://repo.or.cz/w/grub2/phcoder.git ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: Policy-based memory allocations (was Re: RFC: 1.97 roadmap)
On Wed, Aug 12, 2009 at 02:45:19PM +0200, Vladimir 'phcoder' Serbinenko wrote: - Low memory heap (useful to move code off kern/i386/pc/startup.S). Originally I thought of a path relocator32-relocator users-mm relocator32 is ready for next round of review but is untested. Now I think about it mm patch isn't actually dependent on relocator32, just you won't get some features (as loading big initrds and removal of os_area_size/os_area_addr fields) before relocator32 is used by all loaders. I will adjust mm patch to this and add .(text|data|bss)-lowmem section support. I don't understand, what is the relation between relocator in loaders and low memory heap? Actually low memory heap is a special case of policy based allocation. My design is: I have up to 4 different policies (can be more by modifying defines but has to be hardcoded for performance reasons and multiple of 4 for alignment reasons) Every region knows which allocator it has to use together with which policy. Current allocators: -Skip. Don't use this region with given policy -First. Try to allocate as low as possible -Last. Try to allocate as high as possible -Second. Allocate second free chunk from region. It's what is used currently. The idea behind that design is that often loaders need a big continuous chunk of memory so if loaders get memory from bottom and the rest takes memory from top we're likely to have a chunk of necessary size available. But available memory is several orders of magnitude bigger than the largest block a loader will need. So is this really an issue? -- Robert Millan The DRM opt-in fallacy: Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: Policy-based memory allocations (was Re: RFC: 1.97 roadmap)
But available memory is several orders of magnitude bigger than the largest block a loader will need. So is this really an issue? It's not always the case. Two examples 1) Solaris. At least some distributions of solaris use a big (70 MiB compressed, around 200 MiB compressed) initrd which has to be loaded as multiboot module in a single chunk. This puts biggest needed chunk in the same order of magnitude as RAM available on some smaller systems. 2) XNU hibernating. It requires booter to load hibernating image in a single chunk. Even though it's compressed it can easily be 50% of total RAM or more -- Robert Millan The DRM opt-in fallacy: Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel -- Regards Vladimir 'phcoder' Serbinenko Personal git repository: http://repo.or.cz/w/grub2/phcoder.git ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Policy-based memory allocations (was Re: RFC: 1.97 roadmap)
- Low memory heap (useful to move code off kern/i386/pc/startup.S). Originally I thought of a path relocator32-relocator users-mm relocator32 is ready for next round of review but is untested. Now I think about it mm patch isn't actually dependent on relocator32, just you won't get some features (as loading big initrds and removal of os_area_size/os_area_addr fields) before relocator32 is used by all loaders. I will adjust mm patch to this and add .(text|data|bss)-lowmem section support. I don't understand, what is the relation between relocator in loaders and low memory heap? Actually low memory heap is a special case of policy based allocation. My design is: I have up to 4 different policies (can be more by modifying defines but has to be hardcoded for performance reasons and multiple of 4 for alignment reasons) Every region knows which allocator it has to use together with which policy. Current allocators: -Skip. Don't use this region with given policy -First. Try to allocate as low as possible -Last. Try to allocate as high as possible -Second. Allocate second free chunk from region. It's what is used currently. The idea behind that design is that often loaders need a big continuous chunk of memory so if loaders get memory from bottom and the rest takes memory from top we're likely to have a chunk of necessary size available. To take advantage of this design kernel area (first 3/4 of memory which aren't added to heap) has to be eliminated. For this to happen all loaders have to use relocator. However I can make patches without need of relocator (w/o eliminating kernel area). Just you won't get all the advantages of policy-based allocations I'll need to catch up with the lowmem heap discussion. What's the approach? What about savedefault? Which savedefault way you prefer? I think it would be good to have. But I haven't followed on the savedefault discussion, I just know it would build upon the existing envfile support. Bigger overhauls like the fancy menu I started splitting Collin's patches and actually only quite few need to go to the parts already present in grub2. Perhaps 1.97 can be brought to a state when gfxmenu can be compiled externally? Depends on how intrusive are those changes :-) I'll present them and I'm ok if they are postponed. -- Robert Millan The DRM opt-in fallacy: Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel -- Regards Vladimir 'phcoder' Serbinenko Personal git repository: http://repo.or.cz/w/grub2/phcoder.git ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
