Re: A question regarding GIT
Hi! > The problem seems more like you are trying to git clone a http:// address, i > am not sure, but you may need to find a git address for the same. Git can clone from http:// and rsync:// as well. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: A question regarding GIT
Hi! On 22:33 Thu 18 Nov , Rahul Ramasubramanian wrote: > Hi , > I had a query regarding git usage > i am trying to clone a specific branch from a repo. I am following the > instructions from this site > http://www.linux-arm.org/LinuxKernel/LinuxAndroidPlatform( > patching the kernel subsection) > But when i try to clone > http://linux-arm.org/git?p=linux-2.6-stable.git;a=summary > using the following command > http://linux-arm.org/linux-2.6-stable.git > I dont find the required branch ( it is named 2.6.33-armdroid) . The > git branch shows only 1 branch. You have to use "git branch -a" to see remote branches. Use "git checkout remotes/origin/2.6.33-armdroid" to switch to the armdroid branch. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: A question regarding GIT
> On Fri, Nov 19, 2010 at 9:03 AM, Rahul Ramasubramanian > wrote: > > Hi , > > I had a query regarding git usage > > i am trying to clone a specific branch from a repo. I am following the > > instructions from this site > > http://www.linux-arm.org/LinuxKernel/LinuxAndroidPlatform( > > patching the kernel subsection) > > But when i try to clone > > http://linux-arm.org/git?p=linux-2.6-stable.git;a=summary > > using the following command > > http://linux-arm.org/linux-2.6-stable.git > > I dont find the required branch ( it is named 2.6.33-armdroid) . The > > git branch shows only 1 branch. > > There seems to be another project called armdroid , but that does not > > have the sources , it just has few patches and prebuilt images. > The problem seems more like you are trying to git clone a http:// address, i am not sure, but you may need to find a git address for the same.
Re: A question regarding GIT
This question might be OT here! :) More suited for GIT discussion forum. To see remote branches, try 'git branch -r' not just 'git branch'. For checking out a specific branch, use 'git checkout -b HTH, -mandeep On Fri, Nov 19, 2010 at 9:03 AM, Rahul Ramasubramanian wrote: > Hi , > I had a query regarding git usage > i am trying to clone a specific branch from a repo. I am following the > instructions from this site > http://www.linux-arm.org/LinuxKernel/LinuxAndroidPlatform ( > patching the kernel subsection) > But when i try to clone > http://linux-arm.org/git?p=linux-2.6-stable.git;a=summary > using the following command > http://linux-arm.org/linux-2.6-stable.git > I dont find the required branch ( it is named 2.6.33-armdroid) . The > git branch shows only 1 branch. > There seems to be another project called armdroid , but that does not > have the sources , it just has few patches and prebuilt images. > > So my question is , how do i clone the correct project and then switch > to the correct branch. > Hope my question is clear. > > Thanks in advance for the replies > Regards > Rahul > > -- > To unsubscribe from this list: send an email with > "unsubscribe kernelnewbies" to ecar...@nl.linux.org > Please read the FAQ at http://kernelnewbies.org/FAQ > > -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
RE: A question
[Resending, I'm not sure where the previous email is lost.] >From: kernelnewbies-bou...@nl.linux.org >[mailto:kernelnewbies-bou...@nl.linux.org] On Behalf Of hiren panchasara >Sent: Thursday, September 02, 2010 12:28 PM >To: Prabhu nath >Cc: kernelnewbies@nl.linux.org; techtrain...@tataelxsi.co.in >Subject: Re: A question > >Thanks a lot Prabhu for great explanation, comments in-line: >On Wed, Sep 1, 2010 at 10:02 PM, Prabhu nath wrote: >Interrupt context is a loaded term. It is used for both top-half and >bottom-half processing. >I need some more reading, will do that :) > [Nilesh] Also request you to keep the subject line quite specific. :) > >Linux defines two terms in the top-half, one is Interrupt handler (IH) and >Interrupt Service Routine (ISR). >Interrupt handler is a standard kernel code that is executed by the processor >once the interrupt is generated. To abstract it, this generic code is executed >for all the generated interrupts. This, in general saves the present processor >context and paves way for the execution of ISR. >ISR is a interrupt service routine written by a device driver programmer to >handle the interrupt of a device. ISR is mainly responsible to verify the >status register of a device to find out the cause of the interrupt and act >accordingly. > >Both IH and ISR will not have a context of its own. Interrupts generated by a >hardware device is asynchronous. i.e. it can be generated at any point in time. >When such a interrupt is generated, the current execution is interrupted and >the execution control is transferred to IH and ISR will be executed. Thus we >can infer that both IH and ISR will be executing in an anonymous context. >I am struggling to find meaning of interrupt context now. > [Nilesh] Context refers to several stuffs that play role in execution . the registers, the stack etc. etc. http://book.chinaunix.net/special/ebook/Linux_Kernel_Development/0672327201/ch06lev1sec5.html > > >For Eg. Consider there are 3 tasks T2, T3. Suppose T2 invoked a read system >call to read data from the secondary media (say hard disk), driver of the hard >disk will initiate the device to gather the data and put task T2 in the wait >queue and invokes scheduler. Now, scheduler picks up say T3. When processor is >executing task T3, there was interrupt generated by the harddisk controller >indicating that data is ready. At this point IH and ISR is executed. IH and >ISR of this interrupt has no relevance to task T3, but still it is executed in >the context of T3. Hence we say that IH and ISR will always be executed in the >anonymous context. >Wow, didnt know this. So IH and ISR wont execute in interrupt context? What is >interrupt context then? and if they execute in context of T3, does it have any >side-effect on processing of T3? > [Nilesh] May be I am not able to completely interpret what is written above. AFAIK, when the task is scheduled out in order to handle interrupt, the stack used is different for the interrupt (current pointer still points to the task_struct of the task, but that.s irrelevant for interrupt). Also the registers are stored (context switch?) when the process is scheduled out. I am not sure how it should run task T3 context. Could someone please clarify? > > >Most of the bottom half execution happens the same way, i.e. it will be >executing in the anonymous context unless the job of bottom half is relegated >to respective kernel threads. In the latter case the bottom half execution >will have its own context. > >For more details about top-half and bottom half, refer ULK 3rd edtion by >Bovet, Chapter 4 >Would read it. > >Thanks again for great insights. > >Regards, >Prabhu Thanks, Nilesh
Re: A question
Hi.. On Thu, Sep 2, 2010 at 14:11, John Mahoney wrote: > I agree. I was trying to show an example of the minimal amount of > work which must be accomplished by an interrupt handler with > interrupts disabled before the work can be passed to a process > context. I also think the threaded interrupts is a cool concept. > > I apologize if this made the discussion more confusing. I don't think it becomes more confusing...it was enriching the value of this discussion :) After all, it's there in the mainline kernel, so we should better know it :) -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: A question
On Thu, Sep 2, 2010 at 2:36 AM, Mulyadi Santosa wrote: > On Thu, Sep 2, 2010 at 04:28, John Mahoney wrote: >> I had read an article[1] a while back about a push in the real-time >> kernel branch to make interrupt handlers scheduleable, but the article > > IIRC that's threaded interrupt handler...however, there is "basic > part" that must be done in atomic style. > I agree. I was trying to show an example of the minimal amount of work which must be accomplished by an interrupt handler with interrupts disabled before the work can be passed to a process context. I also think the threaded interrupts is a cool concept. I apologize if this made the discussion more confusing. -- John -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: A question
Thanks a lot Prabhu for great explanation, comments in-line: On Wed, Sep 1, 2010 at 10:02 PM, Prabhu nath wrote: > Interrupt context is a loaded term. It is used for both top-half and > bottom-half processing. > I need some more reading, will do that :) > Linux defines two terms in the top-half, one is Interrupt handler (IH) and > Interrupt Service Routine (ISR). > Interrupt handler is a standard kernel code that is executed by the > processor once the interrupt is generated. To abstract it, this generic code > is executed for all the generated interrupts. This, in general saves the > present processor context and paves way for the execution of ISR. > ISR is a interrupt service routine written by a device driver programmer to > handle the interrupt of a device. ISR is mainly responsible to verify the > status register of a device to find out the cause of the interrupt and act > accordingly. > > Both IH and ISR will not have a context of its own. Interrupts generated by > a hardware device is asynchronous. i.e. it can be generated at any point in > time. > When such a interrupt is generated, the current execution is interrupted > and the execution control is transferred to IH and ISR will be executed. > Thus we can infer that both IH and ISR will be executing in an anonymous > context. > I am struggling to find meaning of interrupt context now. > > For Eg. Consider there are 3 tasks T2, T3. Suppose T2 invoked a read system > call to read data from the secondary media (say hard disk), driver of the > hard disk will initiate the device to gather the data and put task T2 in the > wait queue and invokes scheduler. Now, scheduler picks up say T3. When > processor is executing task T3, there was interrupt generated by the > harddisk controller indicating that data is ready. At this point IH and ISR > is executed. IH and ISR of this interrupt has no relevance to task T3, but > still it is executed in the context of T3. Hence we say that IH and ISR will > always be executed in the anonymous context. > Wow, didnt know this. So IH and ISR wont execute in interrupt context? What is interrupt context then? and if they execute in context of T3, does it have any side-effect on processing of T3? > > Most of the bottom half execution happens the same way, i.e. it will be > executing in the anonymous context unless the job of bottom half is > relegated to respective kernel threads. In the latter case the bottom half > execution will have its own context. > > For more details about top-half and bottom half, refer ULK 3rd edtion by > Bovet, Chapter 4 > Would read it. Thanks again for great insights. > > Regards, > Prabhu > > > > On Wed, Sep 1, 2010 at 9:08 PM, Hiren Panchasara < > hiren.panchas...@gmail.com> wrote: > >> Process context is schedulable but interrupt context is not. Why and how? >> Any examples I can look at? >> >> Thanks. >> >> >> >> -- >> To unsubscribe from this list: send an email with >> "unsubscribe kernelnewbies" to ecar...@nl.linux.org >> Please read the FAQ at http://kernelnewbies.org/FAQ >> >> >
Re: A question
On Thu, Sep 2, 2010 at 04:28, John Mahoney wrote: > I had read an article[1] a while back about a push in the real-time > kernel branch to make interrupt handlers scheduleable, but the article IIRC that's threaded interrupt handler...however, there is "basic part" that must be done in atomic style. Threaded handler is introduced to push more part of interrupt handler into schedulable context. I guess the real goal is reduce latency and maximize throughput (although maybe sacrifice responsivenee in certain degree). -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: A question
Interrupt context is a loaded term. It is used for both top-half and bottom-half processing. Linux defines two terms in the top-half, one is Interrupt handler (IH) and Interrupt Service Routine (ISR). Interrupt handler is a standard kernel code that is executed by the processor once the interrupt is generated. To abstract it, this generic code is executed for all the generated interrupts. This, in general saves the present processor context and paves way for the execution of ISR. ISR is a interrupt service routine written by a device driver programmer to handle the interrupt of a device. ISR is mainly responsible to verify the status register of a device to find out the cause of the interrupt and act accordingly. Both IH and ISR will not have a context of its own. Interrupts generated by a hardware device is asynchronous. i.e. it can be generated at any point in time. When such a interrupt is generated, the current execution is interrupted and the execution control is transferred to IH and ISR will be executed. Thus we can infer that both IH and ISR will be executing in an anonymous context. For Eg. Consider there are 3 tasks T2, T3. Suppose T2 invoked a read system call to read data from the secondary media (say hard disk), driver of the hard disk will initiate the device to gather the data and put task T2 in the wait queue and invokes scheduler. Now, scheduler picks up say T3. When processor is executing task T3, there was interrupt generated by the harddisk controller indicating that data is ready. At this point IH and ISR is executed. IH and ISR of this interrupt has no relevance to task T3, but still it is executed in the context of T3. Hence we say that IH and ISR will always be executed in the anonymous context. Most of the bottom half execution happens the same way, i.e. it will be executing in the anonymous context unless the job of bottom half is relegated to respective kernel threads. In the latter case the bottom half execution will have its own context. For more details about top-half and bottom half, refer ULK 3rd edtion by Bovet, Chapter 4 Regards, Prabhu On Wed, Sep 1, 2010 at 9:08 PM, Hiren Panchasara wrote: > Process context is schedulable but interrupt context is not. Why and how? > Any examples I can look at? > > Thanks. > > > > -- > To unsubscribe from this list: send an email with > "unsubscribe kernelnewbies" to ecar...@nl.linux.org > Please read the FAQ at http://kernelnewbies.org/FAQ > >
Re: A question
I had read an article[1] a while back about a push in the real-time kernel branch to make interrupt handlers scheduleable, but the article also does a good job of explaining the context of interrupt handlers currently. I do not know the state of this, but at the time found it interesting. [1]http://lwn.net/Articles/302043/ -- John -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: A question
On 9/1/2010 9:45 PM, Tayade, Nilesh wrote: -Original Message- From: kernelnewbies-bou...@nl.linux.org [mailto:kernelnewbies- bou...@nl.linux.org] On Behalf Of Hiren Panchasara Sent: Wednesday, September 01, 2010 9:08 PM To: kernelnewbies@nl.linux.org Subject: A question Process context is schedulable but interrupt context is not. Why and how? Interrupt line can be shared by multiple interrupts or to avoid delay in handling subsequent interrupts on single line - the handler has to be quick enough. So we avoid scheduling out or sleeping inside the interrupt handlers. Adding... Scheduling blocks. And if interrupt handler is blocked ( scheduled ) it will lead to freeze. Thanks. Any examples I can look at? -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
RE: A question
> -Original Message- > From: kernelnewbies-bou...@nl.linux.org [mailto:kernelnewbies- > bou...@nl.linux.org] On Behalf Of Hiren Panchasara > Sent: Wednesday, September 01, 2010 9:08 PM > To: kernelnewbies@nl.linux.org > Subject: A question > > Process context is schedulable but interrupt context is not. Why and > how? Interrupt line can be shared by multiple interrupts or to avoid delay in handling subsequent interrupts on single line - the handler has to be quick enough. So we avoid scheduling out or sleeping inside the interrupt handlers. > Any examples I can look at? -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecar...@nl.linux.org Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question of mmap() of files into memory
Peter Teoh wrote: read() vs mmap().mmap() takes a page fault with every single byte of memory access, if the mmap() ptr is byte ptr, then it seemingly should be slower than read() As suggested by the name PAGE fault, it faults in a page at a time. This means it will take a page fault for every 4kB of data on x86 systems. Page fault VS memory copy ?? which is more overhead ?? I think Rik can answer this question. It depends on what you do. Page faults are not exactly cheap, but neither is copying around the data. If you access the file data just once, read() or write() are likely to be faster. If you access the data over and over again, mmap is likely to be faster. -- All rights reversed. -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question of mmap() of files into memory
On Mon, Nov 24, 2008 at 12:17 PM, MinChan Kim <[EMAIL PROTECTED]> wrote: > On Mon, Nov 24, 2008 at 11:58 AM, Peter Teoh <[EMAIL PROTECTED]> wrote: >> On Mon, Nov 24, 2008 at 7:29 AM, MinChan Kim <[EMAIL PROTECTED]> wrote: >>> I'm so late :) >>> >>> On Sun, Nov 23, 2008 at 4:17 PM, Wang Yu <[EMAIL PROTECTED]> wrote: On Sun, Nov 23, 2008 at 2:23 AM, Rik van Riel <[EMAIL PROTECTED]> wrote: > > Peter Teoh wrote: >> >> when a process mmap() a section of a file into its own process memory, >> the process memory will maintain a copy of the data of that section of >> the file. > > No, it does not maintain a copy. > > It mmaps the page cache pages into its own address space. According to your explanation, the flow is physical file(on disk)-->Page Cache(on memory, but in kernel space)-->Process Memory(on memory, but in user space). Is it? I am not sure > >>> >>> Yes. When kernel find no page mapping, it allocate new page in page >>> cache and copy from on-disk page to new page, then map the new page to >>> user space address. >>> so, Never duplication. >>> >> so...does there exists duplicated buffering? (one in kernel - >> pagecache, and one in userspace - for mmap() content of the file in >> process memory) > > No, there is no such double buffering. But what is the difference? Why linux do it? >>> >>> In case of read system call except O_DIRECT, It's duplication between >>> user buffer and page cache. >>> Read/write system call abstract page to file. so you always need user >>> buffer. >>> Let think. If you want read some data, First of all you need some >>> space which is user buffer. >>> Mmap system call abstract page to memory. so you can handle file as >>> memory operation without user buffer that mean It don't have >>> duplication overhead. >> >> I see. So u are saying that read() will duplicate buffer between >> userspace (user buffer) and kernel (pagecache), but for mmap() >> operation, since there is no duplication, all access from user process >> will immediate trigger a context switch, to read in the data from >> ring0, right? > > yeb. More exactly, to read in the data from file fault in page fault handler. > >> Since u normally read in blocks of data using read(), and access data >> byte-wise with mmap()'s pointer, so read() is much more efficient, as >> it trigger much lesser context switches than mmap()'s way of pointer >> accessing kernel memory? >> >> Performance-wise, mmap() will perform worst off than read()? > > I am not sure which case is better. > That's because read operation have a overhead memory copy from kernel to user. > But it occur less page fault than mmap. but on-demand readahead or > hint function can increase performance in case of sequential read > read() vs mmap().mmap() takes a page fault with every single byte of memory access, if the mmap() ptr is byte ptr, then it seemingly should be slower than read() and a possible similar experience is here: http://lkml.org/lkml/2008/1/14/517 > Page fault VS memory copy ?? which is more overhead ?? > I think Rik can answer this question. > -- Regards, Peter Teoh Ernest Hemingway - "Never mistake motion for action." -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question of mmap() of files into memory
On Mon, Nov 24, 2008 at 11:58 AM, Peter Teoh <[EMAIL PROTECTED]> wrote: > On Mon, Nov 24, 2008 at 7:29 AM, MinChan Kim <[EMAIL PROTECTED]> wrote: >> I'm so late :) >> >> On Sun, Nov 23, 2008 at 4:17 PM, Wang Yu <[EMAIL PROTECTED]> wrote: >>> >>> >>> On Sun, Nov 23, 2008 at 2:23 AM, Rik van Riel <[EMAIL PROTECTED]> wrote: Peter Teoh wrote: > > when a process mmap() a section of a file into its own process memory, > the process memory will maintain a copy of the data of that section of > the file. No, it does not maintain a copy. It mmaps the page cache pages into its own address space. >>> >>> >>>According to your explanation, the flow is physical file(on disk)-->Page >>> Cache(on memory, but in kernel space)-->Process Memory(on memory, but in >>> user space). Is it? I am not sure >> >> Yes. When kernel find no page mapping, it allocate new page in page >> cache and copy from on-disk page to new page, then map the new page to >> user space address. >> so, Never duplication. >> > so...does there exists duplicated buffering? (one in kernel - > pagecache, and one in userspace - for mmap() content of the file in > process memory) No, there is no such double buffering. >>> >>>But what is the difference? Why linux do it? >> >> In case of read system call except O_DIRECT, It's duplication between >> user buffer and page cache. >> Read/write system call abstract page to file. so you always need user buffer. >> Let think. If you want read some data, First of all you need some >> space which is user buffer. >> Mmap system call abstract page to memory. so you can handle file as >> memory operation without user buffer that mean It don't have >> duplication overhead. > > I see. So u are saying that read() will duplicate buffer between > userspace (user buffer) and kernel (pagecache), but for mmap() > operation, since there is no duplication, all access from user process > will immediate trigger a context switch, to read in the data from > ring0, right? yeb. More exactly, to read in the data from file fault in page fault handler. > Since u normally read in blocks of data using read(), and access data > byte-wise with mmap()'s pointer, so read() is much more efficient, as > it trigger much lesser context switches than mmap()'s way of pointer > accessing kernel memory? > > Performance-wise, mmap() will perform worst off than read()? I am not sure which case is better. That's because read operation have a overhead memory copy from kernel to user. But it occur less page fault than mmap. but on-demand readahead or hint function can increase performance in case of sequential read Page fault VS memory copy ?? which is more overhead ?? I think Rik can answer this question. > Thanks. > > > -- > Regards, > Peter Teoh > > Ernest Hemingway - "Never mistake motion for action." > -- Kinds regards, MinChan Kim -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question of mmap() of files into memory
On Mon, Nov 24, 2008 at 7:29 AM, MinChan Kim <[EMAIL PROTECTED]> wrote: > I'm so late :) > > On Sun, Nov 23, 2008 at 4:17 PM, Wang Yu <[EMAIL PROTECTED]> wrote: >> >> >> On Sun, Nov 23, 2008 at 2:23 AM, Rik van Riel <[EMAIL PROTECTED]> wrote: >>> >>> Peter Teoh wrote: when a process mmap() a section of a file into its own process memory, the process memory will maintain a copy of the data of that section of the file. >>> >>> No, it does not maintain a copy. >>> >>> It mmaps the page cache pages into its own address space. >> >> >>According to your explanation, the flow is physical file(on disk)-->Page >> Cache(on memory, but in kernel space)-->Process Memory(on memory, but in >> user space). Is it? I am not sure >>> > > Yes. When kernel find no page mapping, it allocate new page in page > cache and copy from on-disk page to new page, then map the new page to > user space address. > so, Never duplication. > so...does there exists duplicated buffering? (one in kernel - pagecache, and one in userspace - for mmap() content of the file in process memory) >>> >>> No, there is no such double buffering. >> >>But what is the difference? Why linux do it? > > In case of read system call except O_DIRECT, It's duplication between > user buffer and page cache. > Read/write system call abstract page to file. so you always need user buffer. > Let think. If you want read some data, First of all you need some > space which is user buffer. > Mmap system call abstract page to memory. so you can handle file as > memory operation without user buffer that mean It don't have > duplication overhead. I see. So u are saying that read() will duplicate buffer between userspace (user buffer) and kernel (pagecache), but for mmap() operation, since there is no duplication, all access from user process will immediate trigger a context switch, to read in the data from ring0, right? Since u normally read in blocks of data using read(), and access data byte-wise with mmap()'s pointer, so read() is much more efficient, as it trigger much lesser context switches than mmap()'s way of pointer accessing kernel memory? Performance-wise, mmap() will perform worst off than read()? Thanks. -- Regards, Peter Teoh Ernest Hemingway - "Never mistake motion for action." -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question of mmap() of files into memory
Oh As to mmap, it is used to map a file to a process address space in the result that several processes can communicate with each other using this shared file memory. The process can access this file just as accessing the memory not using read/write system call. On Mon, Nov 24, 2008 at 10:58 AM, Peter Teoh <[EMAIL PROTECTED]>wrote: > On Mon, Nov 24, 2008 at 7:29 AM, MinChan Kim <[EMAIL PROTECTED]> > wrote: > > I'm so late :) > > > > On Sun, Nov 23, 2008 at 4:17 PM, Wang Yu <[EMAIL PROTECTED]> wrote: > >> > >> > >> On Sun, Nov 23, 2008 at 2:23 AM, Rik van Riel <[EMAIL PROTECTED]> wrote: > >>> > >>> Peter Teoh wrote: > > when a process mmap() a section of a file into its own process memory, > the process memory will maintain a copy of the data of that section of > the file. > >>> > >>> No, it does not maintain a copy. > >>> > >>> It mmaps the page cache pages into its own address space. > >> > >> > >>According to your explanation, the flow is physical file(on > disk)-->Page > >> Cache(on memory, but in kernel space)-->Process Memory(on memory, but > in > >> user space). Is it? I am not sure > >>> > > > > Yes. When kernel find no page mapping, it allocate new page in page > > cache and copy from on-disk page to new page, then map the new page to > > user space address. > > so, Never duplication. > > > so...does there exists duplicated buffering? (one in kernel - > pagecache, and one in userspace - for mmap() content of the file in > process memory) > >>> > >>> No, there is no such double buffering. > >> > >>But what is the difference? Why linux do it? > > > > In case of read system call except O_DIRECT, It's duplication between > > user buffer and page cache. > > Read/write system call abstract page to file. so you always need user > buffer. > > Let think. If you want read some data, First of all you need some > > space which is user buffer. > > Mmap system call abstract page to memory. so you can handle file as > > memory operation without user buffer that mean It don't have > > duplication overhead. > > I see. So u are saying that read() will duplicate buffer between > userspace (user buffer) and kernel (pagecache), but for mmap() > operation, since there is no duplication, all access from user process > will immediate trigger a context switch, to read in the data from > ring0, right? > > Since u normally read in blocks of data using read(), and access data > byte-wise with mmap()'s pointer, so read() is much more efficient, as > it trigger much lesser context switches than mmap()'s way of pointer > accessing kernel memory? > > Performance-wise, mmap() will perform worst off than read()? > > Thanks. > > > -- > Regards, > Peter Teoh > > Ernest Hemingway - "Never mistake motion for action." > -- National Research Center for Intelligent Computing Systems Institute of Computing Technology, Chinese Academy of Sciences
Re: a question of mmap() of files into memory
I'm so late :) On Sun, Nov 23, 2008 at 4:17 PM, Wang Yu <[EMAIL PROTECTED]> wrote: > > > On Sun, Nov 23, 2008 at 2:23 AM, Rik van Riel <[EMAIL PROTECTED]> wrote: >> >> Peter Teoh wrote: >>> >>> when a process mmap() a section of a file into its own process memory, >>> the process memory will maintain a copy of the data of that section of >>> the file. >> >> No, it does not maintain a copy. >> >> It mmaps the page cache pages into its own address space. > > >According to your explanation, the flow is physical file(on disk)-->Page > Cache(on memory, but in kernel space)-->Process Memory(on memory, but in > user space). Is it? I am not sure >> Yes. When kernel find no page mapping, it allocate new page in page cache and copy from on-disk page to new page, then map the new page to user space address. so, Never duplication. >>> so...does there exists duplicated buffering? (one in kernel - >>> pagecache, and one in userspace - for mmap() content of the file in >>> process memory) >> >> No, there is no such double buffering. > >But what is the difference? Why linux do it? In case of read system call except O_DIRECT, It's duplication between user buffer and page cache. Read/write system call abstract page to file. so you always need user buffer. Let think. If you want read some data, First of all you need some space which is user buffer. Mmap system call abstract page to memory. so you can handle file as memory operation without user buffer that mean It don't have duplication overhead. >> >> >> -- >> All rights reversed. >> >> -- >> To unsubscribe from this list: send an email with >> "unsubscribe kernelnewbies" to [EMAIL PROTECTED] >> Please read the FAQ at http://kernelnewbies.org/FAQ >> > > > > -- > National Research Center for Intelligent Computing Systems > Institute of Computing Technology, Chinese Academy of Sciences > -- Kinds regards, MinChan Kim -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question of mmap() of files into memory
On Sun, Nov 23, 2008 at 2:23 AM, Rik van Riel <[EMAIL PROTECTED]> wrote: > Peter Teoh wrote: > >> when a process mmap() a section of a file into its own process memory, >> the process memory will maintain a copy of the data of that section of >> the file. >> > > No, it does not maintain a copy. > > It mmaps the page cache pages into its own address space. > According to your explanation, the flow is physical file(on disk)-->Page Cache(on memory, but in kernel space)-->Process Memory(on memory, but in user space). Is it? I am not sure > > > so...does there exists duplicated buffering? (one in kernel - >> pagecache, and one in userspace - for mmap() content of the file in >> process memory) >> > > No, there is no such double buffering. But what is the difference? Why linux do it? > > > -- > All rights reversed. > > > -- > To unsubscribe from this list: send an email with > "unsubscribe kernelnewbies" to [EMAIL PROTECTED] > Please read the FAQ at http://kernelnewbies.org/FAQ > > -- National Research Center for Intelligent Computing Systems Institute of Computing Technology, Chinese Academy of Sciences
Re: a question of mmap() of files into memory
Peter Teoh wrote: when a process mmap() a section of a file into its own process memory, the process memory will maintain a copy of the data of that section of the file. No, it does not maintain a copy. It mmaps the page cache pages into its own address space. so...does there exists duplicated buffering? (one in kernel - pagecache, and one in userspace - for mmap() content of the file in process memory) No, there is no such double buffering. -- All rights reversed. -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question of mmap() of files into memory
Thank you Michael. I am forwarding the email to another person - MinChan, not sure if he knows the answer or not? Hello MinChan, do you know the answer to my question? Thanks. -- Forwarded message -- From: Michael Blizek <[EMAIL PROTECTED]> Date: Sat, Nov 22, 2008 at 4:53 PM Subject: Re: a question of mmap() of files into memory To: Peter Teoh <[EMAIL PROTECTED]> Cc: kernelnewbies Hi! On 15:49 Sat 22 Nov , Peter Teoh wrote: > when a process mmap() a section of a file into its own process memory, > the process memory will maintain a copy of the data of that section of > the file. > > but then this is pagecache which cache the physically opened file, right? > > so...does there exists duplicated buffering? (one in kernel - > pagecache, and one in userspace - for mmap() content of the file in > process memory) > > Correct? So so...then it looks like a lot of duplicated stuff . When multiple programs load the same libraries into memory, they are mmap()ed and shared. I cannot tell you exactly, but I do not think that the files exist in an mmap() space and in the page cache. However, I have heared that readahead and page reclaim is differently. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com -- Regards, Peter Teoh -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question of mmap() of files into memory
Hi! On 15:49 Sat 22 Nov , Peter Teoh wrote: > when a process mmap() a section of a file into its own process memory, > the process memory will maintain a copy of the data of that section of > the file. > > but then this is pagecache which cache the physically opened file, right? > > so...does there exists duplicated buffering? (one in kernel - > pagecache, and one in userspace - for mmap() content of the file in > process memory) > > Correct? So so...then it looks like a lot of duplicated stuff . When multiple programs load the same libraries into memory, they are mmap()ed and shared. I cannot tell you exactly, but I do not think that the files exist in an mmap() space and in the page cache. However, I have heared that readahead and page reclaim is differently. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: A question about switch_to macro
Got it, many thanks ;-)
On Fri, Aug 8, 2008 at 2:53 AM, Johannes Weiner <[EMAIL PROTECTED]> wrote:
> Hi,
>
> ZelluX <[EMAIL PROTECTED]> writes:
>
> > In Linux Kernel 2.6.11, the switch_to macro is defined as follows(on x86
> > platform)
> >
> > 15#define switch_to(prev,next,last) do {
> \
> > 16unsigned long esi,edi;
> \
> > 17asm volatile("pushfl\n\t"
> \
> > 18 "pushl %%ebp\n\t"
> \
> > 19 "movl %%esp,%0\n\t"/* save ESP */
> \
> > 20 "movl %5,%%esp\n\t"/* restore ESP */
> \
> > 21 "movl $1f,%1\n\t" /* save EIP */
> \
> > 22 "pushl %6\n\t" /* restore EIP */
> \
> > 23 "jmp __switch_to\n"
> \
> > 24 "1:\t"
> \
> > 25 "popl %%ebp\n\t"
> \
> > 26 "popfl"
> \
> > 27 :"=m" (prev->thread.esp),"=m" (prev->thread.eip),
> \
> > 28 "=a" (last),"=S" (esi),"=D" (edi)
> \
> > 29 :"m" (next->thread.esp),"m" (next->thread.eip),
> \
> > 30 "2" (prev), "d" (next));
> \
> > 31} while (0)
> >
> > In the middle of the code there's a movl instruction
> > 21 "movl $1f,%1\n\t" /* save EIP
> > */ \
> >
> > which saves the address labeled 1 in prev->thread.eip as ULK points out.
> >
> > My question is why there's an f after $1? Is there any specification to
> > this grammar?
>
> It specifies that the address referred to is further down the code
> sequence. If it would be before the movl, it'd be 1b.
>
>Hannes
>
Re: A question about switch_to macro
Hi,
ZelluX <[EMAIL PROTECTED]> writes:
> In Linux Kernel 2.6.11, the switch_to macro is defined as follows(on x86
> platform)
>
> 15#define switch_to(prev,next,last) do { \
> 16unsigned long esi,edi; \
> 17asm volatile("pushfl\n\t" \
> 18 "pushl %%ebp\n\t" \
> 19 "movl %%esp,%0\n\t"/* save ESP */ \
> 20 "movl %5,%%esp\n\t"/* restore ESP */ \
> 21 "movl $1f,%1\n\t" /* save EIP */ \
> 22 "pushl %6\n\t" /* restore EIP */ \
> 23 "jmp __switch_to\n"\
> 24 "1:\t" \
> 25 "popl %%ebp\n\t" \
> 26 "popfl"\
> 27 :"=m" (prev->thread.esp),"=m" (prev->thread.eip), \
> 28 "=a" (last),"=S" (esi),"=D" (edi) \
> 29 :"m" (next->thread.esp),"m" (next->thread.eip),\
> 30 "2" (prev), "d" (next)); \
> 31} while (0)
>
> In the middle of the code there's a movl instruction
> 21 "movl $1f,%1\n\t" /* save EIP
> */ \
>
> which saves the address labeled 1 in prev->thread.eip as ULK points out.
>
> My question is why there's an f after $1? Is there any specification to
> this grammar?
It specifies that the address referred to is further down the code
sequence. If it would be before the movl, it'd be 1b.
Hannes
--
To unsubscribe from this list: send an email with
"unsubscribe kernelnewbies" to [EMAIL PROTECTED]
Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question on cr3
Hi Peter! On Sun, Jun 29, 2008 at 10:56 PM, Peter Teoh <[EMAIL PROTECTED]> wrote: > Thank you Mulyadi for your answer, appreciate very much You're welcome... > Ah...yeswhich means that every process context switch the cr3 will > change. But per-threads inside the process, nothing happened. yes,you got it. > > I can see the pagetable creation per process in kernel/fork.c - > essentially those pgd alloc stuff. Correct? Thanks. IIRC yes. you must pay attention that COW (Copy On Write) mechanism is applied here, thus PTE is created in "lazy style"it will be created once write or read happens. As long as the related pages aren't referenced, their PTEs won't be created. Folksfeel free to CMIIW. regards, Mulyadi. -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question on cr3
Thank you Mulyadi for your answer, appreciate very much On 6/28/08, Mulyadi Santosa <[EMAIL PROTECTED]> wrote: > Hi ! > > On 6/27/08, Peter Teoh <[EMAIL PROTECTED]> wrote: > > Is it true that the values of CR3 never change throughout the kernel > > life time? Ie, it points to based of the pagetableso unless the > > pagetable is migrated it is not necessary to change. > > > It change whenever there is context switching between processes. Since > threads which belong to same parent share PGD, CR3 stays still if a > thread change to another thread which belong to the same parent (thus > share same tgid) > Ah...yeswhich means that every process context switch the cr3 will change. But per-threads inside the process, nothing happened. I can see the pagetable creation per process in kernel/fork.c - essentially those pgd alloc stuff. Correct? Thanks. > > > And when the kernel switch to the userspace , each different process > > should have its own per-process pagetable...so in this case the CR3 > > will change its value per-process, correct? > > > CR3 stays the same when there is a switch between user space to kernel > space or vice versa (if there is no preemption in between...of > course). > -- Regards, Peter Teoh -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: a question on cr3
Hi ! On 6/27/08, Peter Teoh <[EMAIL PROTECTED]> wrote: > Is it true that the values of CR3 never change throughout the kernel > life time? Ie, it points to based of the pagetableso unless the > pagetable is migrated it is not necessary to change. It change whenever there is context switching between processes. Since threads which belong to same parent share PGD, CR3 stays still if a thread change to another thread which belong to the same parent (thus share same tgid) > And when the kernel switch to the userspace , each different process > should have its own per-process pagetable...so in this case the CR3 > will change its value per-process, correct? CR3 stays the same when there is a switch between user space to kernel space or vice versa (if there is no preemption in between...of course). regards, Mulyadi. -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
Re: A question about the implementation of routing subsystem in Linux kernel
Hi! On 19:22 Fri 08 Feb , Zheng Da wrote: > Recently, I try to understand the implementation of the routing subsystem. > It's so complex, and I'm quite confused by so many different structures. > As my understanding about the four structures fib_node, fib_alias, fib_info, > and fib_nh: > fib_node can be used to identify a subnet or a host. > fib_alias is used to represent a route. > fib_info contains the real routing information. > fib_nh contains the information of the next hop. > I can understand fib_node can have a few fib_alias objects because Linux > supports multipath. For the same subnet, there may exist several routes to > it. > Several fib_alias objects share one fib_info, it's still reasonable because > different routes may share the same parameter values of an existing fib_info > object. > But why one fib_info can have several fib_nh objects? That means a route may > have several next hops, but it seems unreasonable. > Can someone show me an example to help me understand it? > Thank you in advance. What happens when multiple interfaces have an address in the same subnet? This is just a guess, I'm not so deeply involved in IP routing. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.homelinux.net -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to [EMAIL PROTECTED] Please read the FAQ at http://kernelnewbies.org/FAQ
