Re: [Lustre-discuss] Read ahead / prefetching
Hi Andrea, How can I know for sure that my Lustre's deployment has this feature or not? Is it available in version 1.6.x or its only for 1.8.x? Thanks, Arifa. -Original Message- From: Andreas Dilger [mailto:andreas.dil...@oracle.com] Sent: Friday, July 30, 2010 1:32 AM To: Arifa Nisar Cc: lustre-discuss@lists.lustre.org Subject: Re: [Lustre-discuss] Read ahead / prefetching If the size of the read calls is 1MB then after 2 reads at 4MB strides the 3rd read would trigger readahead initially for the next stride (1MB+1MB). If the same stride pattern continues the readahead size will grow an additional 1MB each time up to the limit. Cheers, Andreas On 2010-07-29, at 22:53, Arifa Nisar a-ni...@u.northwestern.edu wrote: Consider a case where stripe count = 4, stripe size = 1MB and all I/O accesses are aligned with the stripe boundary. If a client reads every 4th full stripe (accessing same OST always) then one stripe will be readahead from that OST in every call? Thanks, Arifa. -Original Message- From: Andreas Dilger [mailto:andreas.dil...@oracle.com] Sent: Thursday, July 29, 2010 11:41 PM To: Arifa Nisar Cc: lustre-discuss@lists.lustre.org Subject: Re: [Lustre-discuss] Read ahead / prefetching On 2010-07-29, at 14:02, Arifa Nisar wrote: Thanks for the response. If an access pattern triggers page aligned strided access then how many pages will be readahead? Are there some environment variables which may give insight about readahead behavior? Because Lustre is tuned for RPC-sized IO, it will always readahead at least a full RPC at a time (by default 1MB), unless the application is reading larger chunks than this, then it reads ahead in units of the IO size aligned to RPC-sized boundaries. -Original Message- From: Andreas Dilger [mailto:andreas.dil...@oracle.com] Sent: Wednesday, July 28, 2010 5:43 PM To: Arifa Nisar Cc: lustre-discuss@lists.lustre.org Subject: Re: [Lustre-discuss] Read ahead / prefetching On 2010-07-28, at 10:43, Arifa Nisar wrote: I am trying to educate myself on prefetching/readahead algorithm for Lustre's read. For a starter I only have two simple questions. 1 - Does Lustre detect linear or random I/O pattern or it always triggers readahead? The current algorithm detects sequential and strided read patterns. 2 - If readahead is triggered, how many pages are read in addition to what is necessary? It is variable based on how long the same pattern was repeating and the number of pages per request. I would be happy to have you look at the strided readahead detection code more closely, since I suspect it would not correctly detect stride patterns with odd byte size and offset vs. stride patterns with full-page size and offset. We disussed a while ago to rewrite the readahead code to be layered better on the client, but haven't had time to work on it yet. Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc. Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] Read ahead / prefetching
Thanks for the response. If an access pattern triggers page aligned strided access then how many pages will be readahead? Are there some environment variables which may give insight about readahead behavior? Arifa. -Original Message- From: Andreas Dilger [mailto:andreas.dil...@oracle.com] Sent: Wednesday, July 28, 2010 5:43 PM To: Arifa Nisar Cc: lustre-discuss@lists.lustre.org Subject: Re: [Lustre-discuss] Read ahead / prefetching On 2010-07-28, at 10:43, Arifa Nisar wrote: I am trying to educate myself on prefetching/readahead algorithm for Lustre's read. For a starter I only have two simple questions. 1 - Does Lustre detect linear or random I/O pattern or it always triggers readahead? The current algorithm detects sequential and strided read patterns. 2 - If readahead is triggered, how many pages are read in addition to what is necessary? It is variable based on how long the same pattern was repeating and the number of pages per request. I would be happy to have you look at the strided readahead detection code more closely, since I suspect it would not correctly detect stride patterns with odd byte size and offset vs. stride patterns with full-page size and offset. We disussed a while ago to rewrite the readahead code to be layered better on the client, but haven't had time to work on it yet. Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Re: [Lustre-discuss] Read ahead / prefetching
Consider a case where stripe count = 4, stripe size = 1MB and all I/O accesses are aligned with the stripe boundary. If a client reads every 4th full stripe (accessing same OST always) then one stripe will be readahead from that OST in every call? Thanks, Arifa. -Original Message- From: Andreas Dilger [mailto:andreas.dil...@oracle.com] Sent: Thursday, July 29, 2010 11:41 PM To: Arifa Nisar Cc: lustre-discuss@lists.lustre.org Subject: Re: [Lustre-discuss] Read ahead / prefetching On 2010-07-29, at 14:02, Arifa Nisar wrote: Thanks for the response. If an access pattern triggers page aligned strided access then how many pages will be readahead? Are there some environment variables which may give insight about readahead behavior? Because Lustre is tuned for RPC-sized IO, it will always readahead at least a full RPC at a time (by default 1MB), unless the application is reading larger chunks than this, then it reads ahead in units of the IO size aligned to RPC-sized boundaries. -Original Message- From: Andreas Dilger [mailto:andreas.dil...@oracle.com] Sent: Wednesday, July 28, 2010 5:43 PM To: Arifa Nisar Cc: lustre-discuss@lists.lustre.org Subject: Re: [Lustre-discuss] Read ahead / prefetching On 2010-07-28, at 10:43, Arifa Nisar wrote: I am trying to educate myself on prefetching/readahead algorithm for Lustre's read. For a starter I only have two simple questions. 1 - Does Lustre detect linear or random I/O pattern or it always triggers readahead? The current algorithm detects sequential and strided read patterns. 2 - If readahead is triggered, how many pages are read in addition to what is necessary? It is variable based on how long the same pattern was repeating and the number of pages per request. I would be happy to have you look at the strided readahead detection code more closely, since I suspect it would not correctly detect stride patterns with odd byte size and offset vs. stride patterns with full-page size and offset. We disussed a while ago to rewrite the readahead code to be layered better on the client, but haven't had time to work on it yet. Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc. Cheers, Andreas -- Andreas Dilger Lustre Technical Lead Oracle Corporation Canada Inc. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] Read ahead / prefetching
Hi, I am trying to educate myself on prefetching/readahead algorithm for Lustre's read. For a starter I only have two simple questions. 1 - Does Lustre detect linear or random I/O pattern or it always triggers readahead? 2 - If readahead is triggered, how many pages are read in addition to what is necessary? Thanks, Arifa. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] Write optimizations on OSS
Hi, I have been running some write only benchmarks on Lustre with following two versions at two different machines. lustre: 1.6.6 build: 1.6.6-1969123118-PRISTINE-.usr.src.linux-2.6.18-92.1.10.el5_lustre.1.6.6 smp-perfctr Configuration: 1 OSS for 11 OST. lustre: 1.6.5 build: 1.6.5-2.1.50HDB_PS13A-rsrel-REL_2_1_50-19575-2009-11-13-18:58 Configuration: 1 OSS for 2 OST I have noticed that if contiguous data is written to the OSTs under one OSS then some performance benefit is achieved. Is there any caching, aggregation or other optimization implemented in OSS for write accesses? I understand that there is read caching but I could not get much information about write optimization on OSS. Thanks, Arifa. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] Extent Based Locking Implementation
Hi, I have a question regarding implementation of server-based extent locking at Lustre. I have a situation where two processes are concurrently accessing one I/O server for writing one stripe at a time. Both of the processes are writing alternate stripes stored on that server. I want to understand how extent based locking protocol will work in this situation? I understand first process will be given lock to all the stripes. What will happen when second process sends a lock request? Will I/O server revoke all the (unused/ un asked for) locks back from processes 0, or it will revoke the locks to the required stripe(s) only? Please explain if P0 and P1 requests locks to stripes S0 . S8 in this order. P0S0 P1S1 P0S2 P1S3 P0S4 P1S5 P0S6 P1S7 P0S8 Does algorithm remains same if number of processes increases beyond two? Thanks, Arifa. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
[Lustre-discuss] Extent Based Locking Implementation
Hi, I have a question regarding implementation of server-based extent locking at Lustre. I have a situation where two processes are concurrently accessing one I/O server for writing one stripe at a time. Both of the processes are writing alternate stripes stored on that server. I want to understand how extent based locking protocol will work in this situation? I understand first process will be given lock to all the stripes. What will happen when second process sends a lock request? Will I/O server revoke all the (unused/ un asked for) locks back from processes 0, or it will revoke the locks to the required stripe(s) only? Please explain if P0 and P1 requests locks to stripes S0 . S8 in this order. P0S0 P1S1 P0S2 P1S3 P0S4 P1S5 P0S6 P1S7 P0S8 Does algorithm remains same if number of processes increases beyond two? Thanks, Arifa. ___ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
