RE: vmalloc size
> Now this also means that increasing vmalloc inadvertently reduces lowmem. > Why is this designed such a way? It may or may not depending on the amount of physical memory and the size of the vmalloc space. vmalloc space will normally increase vmalloc_end, which won't reduce lowmem. If the end can't advance any further, then I believe that the start can be reduced. This will reduce lowmem, if the lowmem overlaps with vmalloc memory. Okay! got that! VMALLOC_END is a define, it is decided staticly when compile. but VMALLOC_START is a dynamic value, which is calculated when running. VMALLOC_START depends on the "vmalloc=XXX" parameter which is passed by bootloader(u-boot). so if you specify a large vmalloc size in bootloader, for example "vmalloc=512M", the VMALLOC_START will be reduced accordingly. the rough formula is as below: 1. define VMALLOC_END = 0xF000 2. pass in parameter "vmalloc=256M" in u-boot. 3. in kernel SW, a global variable VMALLOC_MIN = VMALLOC_END - vmalloc_size = 0xF000 - 256M = 0xE000 4. calculate total size of "mem=" which is passed by u-boot, for example total memory size is 1GB. 5. global variable high_memory = VMALLOC_MIN = 0xE000, so lowmem is from 0xC000 ~ 0xE000, totoal 512MB, and total high memory is 512MB. 6. VMALLOC_START = high_memory + 8MB = 0xE080. at last , the vmalloc region will be 0xE080 ~ 0xF000 ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmalloc size
On Tue, Jun 26, 2012 at 12:41 AM, Dave Hylands wrote: > Hi Subbu, > > On Mon, Jun 25, 2012 at 7:45 PM, Subramaniam Appadodharana > wrote: > ...snip... > >> However, if you call vmalloc and lets suppose that vmalloc just > >> happens to return 0xE000. The physical address of the first page > >> might be 0xD2345000. > >> > >> What's important is that the physical pages which back up the vmalloc > >> area all come from the kernel direct mapped area. They won't ever be > >> backed by pages from high-memory. So the physical addresses will all > >> be in the range 0x4000 thru 0x5FFF. > >> > > This is the one that I had a completely wrong understanding of!!! > > I understand from the above statement that the vmalloc'ed virtual > address > > will _ALWAYS_ correspond to a physical address from the lowmem region! > I > > was under the impression that the carved out region for the vmalloc, > > is the one that would back any vmalloc'ed virtual address, which is > > absolutely wrong by what you are saying. > > Actually, I was the one that was wrong. vmalloc pages can come from > low or highmem. > Thanks for clarifying! Not a problem. > > > Now this also means that increasing vmalloc inadvertently reduces lowmem. > > Why is this designed such a way? > > It may or may not depending on the amount of physical memory and the > size of the vmalloc space. > > vmalloc space will normally increase vmalloc_end, which won't reduce > lowmem. > If the end can't advance any further, then I believe that the start > can be reduced. This will reduce lowmem, if the lowmem overlaps with > vmalloc memory. > > Okay! got that! > > Essentially, the idea that we increase vmalloc is because we expect more > > memory to be consumed via vmalloc > > calls, and hence we might need more physical address backing. But > increasing > > vmalloc decreases low mem, which would also mean that we have less > backing. > > Am I missing something here too :)? > > No - that was my mistake. > > No problem again. Thanks for correcting it! > -- > Dave Hylands > Shuswap, BC, Canada > http://www.davehylands.com > ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmalloc size
Hi Subbu, On Mon, Jun 25, 2012 at 7:45 PM, Subramaniam Appadodharana wrote: ...snip... >> However, if you call vmalloc and lets suppose that vmalloc just >> happens to return 0xE000. The physical address of the first page >> might be 0xD2345000. >> >> What's important is that the physical pages which back up the vmalloc >> area all come from the kernel direct mapped area. They won't ever be >> backed by pages from high-memory. So the physical addresses will all >> be in the range 0x4000 thru 0x5FFF. >> > This is the one that I had a completely wrong understanding of!!! > I understand from the above statement that the vmalloc'ed virtual address > will _ALWAYS_ correspond to a physical address from the lowmem region! I > was under the impression that the carved out region for the vmalloc, > is the one that would back any vmalloc'ed virtual address, which is > absolutely wrong by what you are saying. Actually, I was the one that was wrong. vmalloc pages can come from low or highmem. > Now this also means that increasing vmalloc inadvertently reduces lowmem. > Why is this designed such a way? It may or may not depending on the amount of physical memory and the size of the vmalloc space. vmalloc space will normally increase vmalloc_end, which won't reduce lowmem. If the end can't advance any further, then I believe that the start can be reduced. This will reduce lowmem, if the lowmem overlaps with vmalloc memory. > Essentially, the idea that we increase vmalloc is because we expect more > memory to be consumed via vmalloc > calls, and hence we might need more physical address backing. But increasing > vmalloc decreases low mem, which would also mean that we have less backing. > Am I missing something here too :)? No - that was my mistake. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmalloc size
Hi, On Mon, Jun 25, 2012 at 9:01 PM, 卜弋天 wrote: > >> Suppose your machine has 1 GB of RAM with a physical addresses of >> 0x4000 thru 0x7FFF >> >> Let's suppose that PHYS_OFFSET = 0xC000 >> Let's suppose that VMALLOC_START is set to 0xE000 and VMALLOC_END >> is at 0xEFFF (256 Mb) >> > is this PAGE_OFFSET or PHYS_OFFSET? PAGE_OFFSET is a virtual address, and PHYS_OFFSET is a physical address, so you are correct. That should have been PAGE_OFFSET. >> So what we have is that memory from 0xC000 thru 0xDFFF (512 >> Mb) is kernel direct mapped memory and maps to physical addresses >> 0x4000 thru 0x5FFF >> >> Physical memory from 0x6000 thru 0x7FFF (512Mb) is considered >> high-mem. For the most part this is only usable by user-space. From >> kernel space, you need to use the kmap/kunmap functions to even access >> the memory. >> > i don't think high-memory is only used by user-space. > vmalloc() funciton will try to allocate physical memory from high_memory > first, if there is no high_memory, then allocate from normal region. That also seems to be correct. The code in vmalloc that calls the underlying page allocator passes in __GFP_HIGHMEM, so it will try to use highmem for vmalloc'd pages. >> However, if you call vmalloc and lets suppose that vmalloc just >> happens to return 0xE000. The physical address of the first page >> might be 0xD2345000. >> > suppose vmalloc() return 0xE000, which is the VMALLOC_START, then the > physical memory address > should be 0x6000 thru 0x7FFF in your example. > > >> What's important is that the physical pages which back up the vmalloc >> area all come from the kernel direct mapped area. They won't ever be >> backed by pages from high-memory. So the physical addresses will all >> be in the range 0x4000 thru 0x5FFF. >> > what do you mean "back up", maybe i have some misunderstanding. > in my opnion, vmalloc area should map to physical address 0x6000 thru > 0x7FFF first, > if this resion is exhausted, then normal area will be allocated. By backed-by, I meant the associated physical page. I was incorrect in saying that only lowmem pages would be used. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
RE: vmalloc size
> Suppose your machine has 1 GB of RAM with a physical addresses of > 0x4000 thru 0x7FFF > > Let's suppose that PHYS_OFFSET = 0xC000 > Let's suppose that VMALLOC_START is set to 0xE000 and VMALLOC_END > is at 0xEFFF (256 Mb) > is this PAGE_OFFSET or PHYS_OFFSET? > So what we have is that memory from 0xC000 thru 0xDFFF (512 > Mb) is kernel direct mapped memory and maps to physical addresses > 0x4000 thru 0x5FFF > > Physical memory from 0x6000 thru 0x7FFF (512Mb) is considered > high-mem. For the most part this is only usable by user-space. From > kernel space, you need to use the kmap/kunmap functions to even access > the memory. > i don't think high-memory is only used by user-space. vmalloc() funciton will try to allocate physical memory from high_memory first, if there is no high_memory, then allocate from normal region. > However, if you call vmalloc and lets suppose that vmalloc just > happens to return 0xE000. The physical address of the first page > might be 0xD2345000. > suppose vmalloc() return 0xE000, which is the VMALLOC_START, then the physical memory address should be 0x6000 thru 0x7FFF in your example. > What's important is that the physical pages which back up the vmalloc > area all come from the kernel direct mapped area. They won't ever be > backed by pages from high-memory. So the physical addresses will all > be in the range 0x4000 thru 0x5FFF. > what do you mean "back up", maybe i have some misunderstanding. in my opnion, vmalloc area should map to physical address 0x6000 thru 0x7FFF first, if this resion is exhausted, then normal area will be allocated. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmalloc size
Hi Dave, Thank you for taking your time to provide this example. This is indeed very insightful. On Mon, Jun 25, 2012 at 7:41 PM, Dave Hylands wrote: > Hi Subbu, > > ...snip,,, > >> __pa only works on kernel direct addresses. > >> > >> __pa doesn't work on the addresses from vmalloc > >> > >> Using __pa on VMALLOC_START or VMALLOC_END doesn't really make sense. > >> If there were any physical memory there, it would be highmem. > >> > > Okay, but my intention was to just print out the pa for the VMALLOC_START > > and END, just to know where they are. > > And I thought they will work. no? > > You'll get some numbers, but they're meaningless. > > Let's take a concrete example (using ARM mappings) > > Suppose your machine has 1 GB of RAM with a physical addresses of > 0x4000 thru 0x7FFF > > Let's suppose that PHYS_OFFSET = 0xC000 > Let's suppose that VMALLOC_START is set to 0xE000 and VMALLOC_END > is at 0xEFFF (256 Mb) > > Agreed! > So what we have is that memory from 0xC000 thru 0xDFFF (512 > Mb) is kernel direct mapped memory and maps to physical addresses > 0x4000 thru 0x5FFF > > Agreed > Physical memory from 0x6000 thru 0x7FFF (512Mb) is considered > high-mem. For the most part this is only usable by user-space. From > kernel space, you need to use the kmap/kunmap functions to even access > the memory. > > Agreed > If you use __pa(VMALLOC_START) you'll ALWAYS (with the above example) > get 0xE000. > Calling __pa(VMALLOC_END) would return 0xEFFF which is some > address from the middle of high-memory, essentially a meaningless > number. > > Agreed this one too!!! > However, if you call vmalloc and lets suppose that vmalloc just > happens to return 0xE000. The physical address of the first page > might be 0xD2345000. > > What's important is that the physical pages which back up the vmalloc > area all come from the kernel direct mapped area. They won't ever be > backed by pages from high-memory. So the physical addresses will all > be in the range 0x4000 thru 0x5FFF. > > This is the one that I had a completely wrong understanding of!!! I understand from the above statement that the vmalloc'ed virtual address will _ALWAYS_ correspond to a physical address from the lowmem region! I was under the impression that the carved out region for the vmalloc, is the one that would back any vmalloc'ed virtual address, which is absolutely wrong by what you are saying. Now this also means that increasing vmalloc inadvertently reduces lowmem. Why is this designed such a way? Essentially, the idea that we increase vmalloc is because we expect more memory to be consumed via vmalloc calls, and hence we might need more physical address backing. But increasing vmalloc decreases low mem, which would also mean that we have less backing. Am I missing something here too :)? You can use vmalloc_to_pfn to return the PFN of the page which is > backing a vmalloc'd virtual address. (if the physical address of > 0xE000 was 0xD2345000, then the pfn would be 0xD2345, assuming 4K > pages). > Agreed on this too! Thanks again for your time and patience. > > -- > Dave Hylands > Shuswap, BC, Canada > http://www.davehylands.com > ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmalloc size
Hi Subbu, ...snip,,, >> __pa only works on kernel direct addresses. >> >> __pa doesn't work on the addresses from vmalloc >> >> Using __pa on VMALLOC_START or VMALLOC_END doesn't really make sense. >> If there were any physical memory there, it would be highmem. >> > Okay, but my intention was to just print out the pa for the VMALLOC_START > and END, just to know where they are. > And I thought they will work. no? You'll get some numbers, but they're meaningless. Let's take a concrete example (using ARM mappings) Suppose your machine has 1 GB of RAM with a physical addresses of 0x4000 thru 0x7FFF Let's suppose that PHYS_OFFSET = 0xC000 Let's suppose that VMALLOC_START is set to 0xE000 and VMALLOC_END is at 0xEFFF (256 Mb) So what we have is that memory from 0xC000 thru 0xDFFF (512 Mb) is kernel direct mapped memory and maps to physical addresses 0x4000 thru 0x5FFF Physical memory from 0x6000 thru 0x7FFF (512Mb) is considered high-mem. For the most part this is only usable by user-space. From kernel space, you need to use the kmap/kunmap functions to even access the memory. If you use __pa(VMALLOC_START) you'll ALWAYS (with the above example) get 0xE000. Calling __pa(VMALLOC_END) would return 0xEFFF which is some address from the middle of high-memory, essentially a meaningless number. However, if you call vmalloc and lets suppose that vmalloc just happens to return 0xE000. The physical address of the first page might be 0xD2345000. What's important is that the physical pages which back up the vmalloc area all come from the kernel direct mapped area. They won't ever be backed by pages from high-memory. So the physical addresses will all be in the range 0x4000 thru 0x5FFF. You can use vmalloc_to_pfn to return the PFN of the page which is backing a vmalloc'd virtual address. (if the physical address of 0xE000 was 0xD2345000, then the pfn would be 0xD2345, assuming 4K pages). -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmalloc size
On Mon, Jun 25, 2012 at 3:00 PM, Dave Hylands wrote: > Hi Subbu, > > On Mon, Jun 25, 2012 at 11:18 AM, Subramaniam Appadodharana > wrote: > > > > > > On Mon, Jun 25, 2012 at 1:00 PM, Mulyadi Santosa < > mulyadi.sant...@gmail.com> > > wrote: > >> > >> Hi :) > >> > >> On Sun, Jun 24, 2012 at 1:32 AM, Subramaniam Appadodharana > >> wrote: > >> > Hi All, > >> > I am trying to understand how vmalloc memory is reserved in kernel. In > >> > 3.4+ > >> > latest kernel, the default vmalloc size is 240MB. > >> > >> vmalloc reserved address range, you meant? > >> > >> >Is this a carveout from > >> > the 1GiB memory that kernel has? > >> > >> Yup > >> > >> > In other words can I do a __pa(VMALLOC_START) or __pa(VMALLOC_END) > >> > or __pa(highmemory) irrespective of the vmalloc size that I request, > say > >> > doing vmalloc=1G? > >> > >> uhmnot surevmalloc-ed is not physically contigous. Also, it is > >> set up far after identity mapping setup. So, the value you get from > >> __pa() IMHO would be likely invalid or has no meaning. > >> > > I thought that, as log as we know that the address is within the 1GiB, > we > > could get the > > pa of the virtual address using __pa(). Is this not the case? > > __pa only works on kernel direct addresses. > > __pa doesn't work on the addresses from vmalloc > > Using __pa on VMALLOC_START or VMALLOC_END doesn't really make sense. > If there were any physical memory there, it would be highmem. > > Okay, but my intention was to just print out the pa for the VMALLOC_START and END, just to know where they are. And I thought they will work. no? __pa should only be used on memory from PAGE_OFFSET through to (high_memory > - 1) > > -- > Dave Hylands > Shuswap, BC, Canada > http://www.davehylands.com > ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmalloc size
Hi Subbu, On Mon, Jun 25, 2012 at 11:18 AM, Subramaniam Appadodharana wrote: > > > On Mon, Jun 25, 2012 at 1:00 PM, Mulyadi Santosa > wrote: >> >> Hi :) >> >> On Sun, Jun 24, 2012 at 1:32 AM, Subramaniam Appadodharana >> wrote: >> > Hi All, >> > I am trying to understand how vmalloc memory is reserved in kernel. In >> > 3.4+ >> > latest kernel, the default vmalloc size is 240MB. >> >> vmalloc reserved address range, you meant? >> >> >Is this a carveout from >> > the 1GiB memory that kernel has? >> >> Yup >> >> > In other words can I do a __pa(VMALLOC_START) or __pa(VMALLOC_END) >> > or __pa(highmemory) irrespective of the vmalloc size that I request, say >> > doing vmalloc=1G? >> >> uhmnot surevmalloc-ed is not physically contigous. Also, it is >> set up far after identity mapping setup. So, the value you get from >> __pa() IMHO would be likely invalid or has no meaning. >> > I thought that, as log as we know that the address is within the 1GiB, we > could get the > pa of the virtual address using __pa(). Is this not the case? __pa only works on kernel direct addresses. __pa doesn't work on the addresses from vmalloc Using __pa on VMALLOC_START or VMALLOC_END doesn't really make sense. If there were any physical memory there, it would be highmem. __pa should only be used on memory from PAGE_OFFSET through to (high_memory - 1) -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmalloc size
On Mon, Jun 25, 2012 at 1:00 PM, Mulyadi Santosa wrote: > Hi :) > > On Sun, Jun 24, 2012 at 1:32 AM, Subramaniam Appadodharana > wrote: > > Hi All, > > I am trying to understand how vmalloc memory is reserved in kernel. In > 3.4+ > > latest kernel, the default vmalloc size is 240MB. > > vmalloc reserved address range, you meant? > > >Is this a carveout from > > the 1GiB memory that kernel has? > > Yup > > > In other words can I do a __pa(VMALLOC_START) or __pa(VMALLOC_END) > > or __pa(highmemory) irrespective of the vmalloc size that I request, say > > doing vmalloc=1G? > > uhmnot surevmalloc-ed is not physically contigous. Also, it is > set up far after identity mapping setup. So, the value you get from > __pa() IMHO would be likely invalid or has no meaning. > > I thought that, as log as we know that the address is within the 1GiB, we could get the pa of the virtual address using __pa(). Is this not the case? > > -- > regards, > > Mulyadi Santosa > Freelance Linux trainer and consultant > > blog: the-hydra.blogspot.com > training: mulyaditraining.blogspot.com > ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: vmalloc size
Hi :) On Sun, Jun 24, 2012 at 1:32 AM, Subramaniam Appadodharana wrote: > Hi All, > I am trying to understand how vmalloc memory is reserved in kernel. In 3.4+ > latest kernel, the default vmalloc size is 240MB. vmalloc reserved address range, you meant? >Is this a carveout from > the 1GiB memory that kernel has? Yup > In other words can I do a __pa(VMALLOC_START) or __pa(VMALLOC_END) > or __pa(highmemory) irrespective of the vmalloc size that I request, say > doing vmalloc=1G? uhmnot surevmalloc-ed is not physically contigous. Also, it is set up far after identity mapping setup. So, the value you get from __pa() IMHO would be likely invalid or has no meaning. -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
vmalloc size
Hi All, I am trying to understand how vmalloc memory is reserved in kernel. In 3.4+ latest kernel, the default vmalloc size is 240MB. Is this a carveout from the 1GiB memory that kernel has? In other words can I do a __pa(VMALLOC_START) or __pa(VMALLOC_END) or __pa(highmemory) irrespective of the vmalloc size that I request, say doing vmalloc=1G? Thank you and Regards Subbu ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
