Re: Kernel memory
On Tue, 17 May 2016 12:41:44 +0530, Ronit Halder said: > Hi, > > Where in the memory kernel is located in the boot time? During which exact phase of the boot, and does it actually matter? And physical or virtual address? (Hint: If you're not the bootstrap that unpacks the compressed kernel into memory, or the code that sets up the ASLR code, or are trying to work around an issue with a hardware device that insists on taking certain physical addresses for itself, you probably don't really care where the kernel is) pgpA8PmnyxbRA.pgp Description: PGP signature ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel memory leak
On Saturday 28 June 2014 12:29 PM, Santhosh Kumar wrote: Is there a way to trace the allocations of memory from different buckets of kmalloc ? You could try out kmemleak - https://www.kernel.org/doc/Documentation/kmemleak.txt ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel Memory management
The answer from manty is totally wrong. I suggest that you put real question here. 在 2013年11月18日，15:17，manty kuma mantyk...@gmail.com 写道： Here is an interesting question(not mine) in SO related to Kernel memory management. Most of the points are my questions aswell. It needs ex[ert comments. could we try to answer questions posted there. http://stackoverflow.com/questions/20041212/does-virtual-memory-area-struct-only-comes-into-picture-when-there-is-a-page-fau ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel Memory
Hi, I am a kernel newbie too, Kernel memory is not pagable, because kernel itself is responsible for paging. See this discussion, http://kerneltrap.org/node/6404 Paging happens for regular processes, i.e each process memory is divided into a page of certain size(4kb in Linux), so it can swap for another page that might be needed for that particular moment, it usually replaces an page that have not been used for long time(see LRU,internal fragmentation). Kernel is a process too, just like anyother process, but it differs from others because it directly talks to the hardware and also it is the one which takes care of paging and LRU algo's. So it does not make much sense, for a kernel memory to be pageable, because, if it removes the page that contains that has the paging algorithm, then there is no way to come back, you want to retrieve a page from disk_swap_area, but you cannot, because the page that contains code to retrieve the instructions are paged :)... This is not just for paging, since kernel controls everything, it is not advisable to put its own code in swap area(see, virtual memory, virtual memory = physical memory + swap space). BTW, i am a kernel newbie too, this is my basic understanding, please feel free to correct it, if I am wrong On Thu, Jun 21, 2012 at 8:27 AM, Vijay Chauhan kernel.vi...@gmail.comwrote: Hello, I am newbie. It has been said kernel memory is not pageable What does it mean? There is no concept of kernel virtual address? Any simple explanation will help me to udnerstand. Thanks, Vijay ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel Memory
Hey Vijay I am a newbie too. Just sharing what I could go through. It is said that Kernel or atleast a part of kernel needs to be non paged for fast interrupt access etc as pinned memory Wiki says Pinned/Locked/Fixed pages Operating systems have memory areas that are pinned (never swapped to secondary storage). For example, interrupthttp://en.wikipedia.org/wiki/Interruptmechanisms rely on an array of pointers to their handlers, such as I/O http://en.wikipedia.org/wiki/I/O completion and page faulthttp://en.wikipedia.org/wiki/Page_fault. If the pages containing these pointers or the code that they invoke were pageable, interrupt-handling would become far more complex and time-consuming, particularly in the case of page fault interrupts. Hence, some part of the page table structures is not pageable. Some pages may be pinned for short periods of time, others may be pinned for long periods of time, and still others may need to be permanently pinned. For example: - The paging supervisor code and drivers for secondary storage devices on which pages reside must be permanently pinned, as otherwise paging wouldn't even work because the necessary code wouldn't be available. - Timing-dependent components may be pinned to avoid variable paging delays. - Data buffers http://en.wikipedia.org/wiki/Data_buffer that are accessed directly by peripheral devices that use direct memory accesshttp://en.wikipedia.org/wiki/Direct_memory_accessor I/O channels http://en.wikipedia.org/wiki/I/O_channel must reside in pinned pages while the I/O operation is in progress because such devices and the buses http://en.wikipedia.org/wiki/Bus_%28computing%29 to which they are attached expect to find data buffers located at physical memory addresses; regardless of whether the bus has a memory management unit for I/O http://en.wikipedia.org/wiki/IOMMU, transfers cannot be stopped if a page fault occurs and then restarted when the page fault has been processed. There are other two discussion thread which say kernel is non-pageable and now due to growing kernel Data structures it is allowed http://kerneltrap.org/node/6404 http://kerneltrap.org/node/8206 Regards Kishore On Thu, Jun 21, 2012 at 5:57 PM, Vijay Chauhan kernel.vi...@gmail.comwrote: Hello, I am newbie. It has been said kernel memory is not pageable What does it mean? There is no concept of kernel virtual address? Any simple explanation will help me to udnerstand. Thanks, Vijay ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel Memory
Hello Vijay, On Thu, Jun 21, 2012 at 5:57 PM, Vijay Chauhan kernel.vi...@gmail.com wrote: Hello, I am newbie. It has been said kernel memory is not pageable What does it mean? There is no concept of kernel virtual address? You might have heard about 3G/1G split. This 1GB is the virtual address of the kernel. And whenever kernel try to access any address in this range(for ARM architecture it is 0xC000 - 0x), there should not be any page fault. That means MMU should be able to convert your virtual address to physical. This 1GB contains your IO address, RAM address. Paging is a mechanism OS uses to pull the data(in pagesizes) to and fro between system RAM and secondary memory. Kernel memory is not pageable. This means memory allocated for the kernel will not be pagged out. If you try to access any memory in kernel with out creating page tables(this can be done by ioremap) you will end up in OOPS. The main reason of kernel not being swapable or pageable is as follows. Think this way. What will happen if we have paged out that portion of the logic which decides what to do when a page fault occurs? Who will take care of the page fault then? But if a user program hit a page fault(ie accessed address is not in main memory), kernel will load the page from secondary memory if it is a valid address. And if the address accesses is illegal, kernel kill the user application(Segmentation fault). Thanks, Arun Any simple explanation will help me to udnerstand. Thanks, Vijay ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Kernel Memory
Vijay Chauhan kernel.vi...@gmail.com writes: Hello, I am newbie. It has been said kernel memory is not pageable What does it mean? There is no concept of kernel virtual address? Yes. Kernel works on static adress space. Any simple explanation will help me to udnerstand. I'm not sure if you want to understand how kernel manages memory for its internal DS. If its the case you should read the following documents. 1) Read the chapter 8 of Linux Device driver 3rd edition. http://lwn.net/Kernel/LDD3/ 2) To understand slab allocator read the following papers by bonwick a) 94 paper describing slab allocator: http://static.usenix.org/publications/library/proceedings/bos94/full_papers/bonwick.a b) Its followup in 2001 http://static.usenix.org/event/usenix01/full_papers/bonwick/bonwick_html/index.html 3) These should be enough. But if you want to know detailed architecture of how virtual memory manager work you should read Gorman's book on Linux virtual memory manager. Its a free pdf. can be found here: http://ptgmedia.pearsoncmg.com/images/0131453483/downloads/gorman_book.pdf Happy hacking. Cheers aft Thanks, Vijay ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: kernel memory allocation
Hi, On Thu, Dec 22, 2011 at 6:33 PM, J.Hwan Kim frog1...@gmail.com wrote: Hi, everyone How can I allocated contiguous kernel memory over 128MB ? When I use _get_free_pages() function, it returns error. I guess the memory size is greater than the amount which the function can allocate. You can use bootmem to reserve the memory at boot time, and then use the bootmem allocator to alloocate from that reserved memory/ There is also some new code called CMA (Contiguous Memory Allocator) which hasn't hit the mainline yet, but it could also be used. See: http://lwn.net/Articles/468044/ for further details. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: kernel memory allocation
Hi. From [1] I'm lead to believe the only difference between the regular kmalloc() arguments and those of kmalloc_node() is the one specifying which node you want to allocate the memory on. Aside from the third argument, which seems to be related with NUMA (with which I never worked on kernel-context), I would suggest you to read [2]. If on the other hand you only want to allocate memory locally, maybe you should use kmalloc() or one of its variants [3]. Cheers. [1] - http://www.kernel.org/doc/htmldocs/kernel-api/API-kmalloc-node.html [2] - http://www.kernel.org/doc/htmldocs/kernel-api/API-kcalloc.html [3] - http://www.kernel.org/doc/htmldocs/kernel-api/mm.html#id408507 --- João Eduardo Luís gpg key: 477C26E5 from pool.keyserver.eu On Jun 3, 2011, at 7:07 PM, Amirali Shambayati wrote: Hello all, I just want to allocate memory for a struct instantiation. Would anyone guide me what arguments I should pass to kmalloc_node? Regards, -- Amirali Shambayati Bachelor Student Computer Engineering Department Sharif University of Technology Tehran, Iran ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies PGP.sig Description: This is a digitally signed message part ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: kernel memory allocation
Hi. In future replies, please CC the list. Does it panic in during the kmalloc, or afterwards? Are you checking if 'newBun' is NULL? I may be missing something obvious in that code, but unless you are out of memory or with some past corruption, I don't think that should panic the kernel. In any case, the panic trace should help. Cheers. --- João Eduardo Luís gpg key: 477C26E5 from pool.keyserver.eu On Jun 3, 2011, at 7:50 PM, Amirali Shambayati wrote: thanks for your guidance. I just want to allocate memory for a struct. I use this: struct bundle* newBun; newBun = kmalloc(sizeof(*newBun), GFP_KERNEL); but it goes to panic state. 2011/6/3 João Eduardo Luís jecl...@gmail.com Hi. From [1] I'm lead to believe the only difference between the regular kmalloc() arguments and those of kmalloc_node() is the one specifying which node you want to allocate the memory on. Aside from the third argument, which seems to be related with NUMA (with which I never worked on kernel-context), I would suggest you to read [2]. If on the other hand you only want to allocate memory locally, maybe you should use kmalloc() or one of its variants [3]. Cheers. [1] - http://www.kernel.org/doc/htmldocs/kernel-api/API-kmalloc-node.html [2] - http://www.kernel.org/doc/htmldocs/kernel-api/API-kcalloc.html [3] - http://www.kernel.org/doc/htmldocs/kernel-api/mm.html#id408507 --- João Eduardo Luís gpg key: 477C26E5 from pool.keyserver.eu On Jun 3, 2011, at 7:07 PM, Amirali Shambayati wrote: Hello all, I just want to allocate memory for a struct instantiation. Would anyone guide me what arguments I should pass to kmalloc_node? Regards, -- Amirali Shambayati Bachelor Student Computer Engineering Department Sharif University of Technology Tehran, Iran ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies -- Amirali Shambayati Bachelor Student Computer Engineering Department Sharif University of Technology Tehran, Iran PGP.sig Description: This is a digitally signed message part ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: kernel memory allocation
Hello, Once again, I would pretty much enjoy if you CC'ed the list on replies. On Jun 3, 2011, at 8:49 PM, Amirali Shambayati wrote: I set breakpoint before kmalloc. panic happens after kmalloc. 2011/6/4 Amirali Shambayati amirali.shambay...@gmail.com ofcourse it's null before allocate it. I make an instantiation of it. then I want to allocate memory for it.What else I should do? I meant, are you checking if 'newBun' is NULL _AFTER_ the kmalloc? The kmalloc call will return NULL instead of a pointer to newly allocated memory whenever an error occurs. If by any chance you are not checking if 'newBun' is NULL after the kmalloc, then you should do it. Also, if the panic happens _after_ the kmalloc, and not _during_ the kmalloc, I would say there's a slim chance of kmalloc being the cause of the panic. --- João Eduardo Luís gpg key: 477C26E5 from pool.keyserver.eu PGP.sig Description: This is a digitally signed message part ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
