Hi Elwyn, Thanks for all of the review and help effort. Besides the inexplicable typos, you unearthed some issues which would give an experienced reader pause.
Tom > On Dec 2, 2014, at 8:41 AM, Elwyn Davies <[email protected]> wrote: > > Hi, David. > > Excellent. I think some suitable combination of your words below would fill > the bill. > > The point about the very large directories is well taken - I hadn't grokked > that previously. > > I guess the update should go in s15.26.5 after para 1: > > Something like: > > ----------------- > > As there is no way for the client to indicate that a cookie value once > received, will not be subsequently used, server implementations should avoid > schemes that allocate memory corresponding to a returned cookie. Such > allocation can be avoided if the server bases cookie values on a value such > as the offset within the directory where the scan is to be resumed. > > Cookies generated by such techniques should be designed to remain valid > despite modification of the associated directory. If a server were to > invalidate a cookie because of a directory modification, READDIR's of large > directories might never finish. > > If a directory is deleted after the client has carried out one or more > READDIR operations on the directory, the cookies returned will become invalid > but the server does not need to be concerned as the directory file handle > used previously would have become stale and would be reported as such on > subsequent READDIR operations. The server would not need to check the cookie > verifier in this case. > > However, certain re-organization operations on a directory (including > directory compaction) may invalidate READDDIR cookies previously given out. > When such a situation occurs, the server should modify the cookie verifier so > as to accept cookies which would otherwise no longer be valid. > Done, with Dave’s suggested tweak. > ------------------- > > Aside: handling reorganization without creating state sounds challenging but > life wouldn't be interesting without challenges! :-) > > A couple of nits that got missed from yesterday: > > s5.4, 2nd bullet labelled '1.': s/common to all object within given file > system./common to all objects within the given file system./ Done > > s9.1.3, last sentence: s/an can be treated as later./and can be treated as > later./ > ^^^^ > Done > > With something like the above, I think we are done. > > Congratulations! I hope this gets to be an RFC RSN. > > Cheers, > Elwyn > > > > > > On 02/12/14 14:56, David Noveck wrote: >> Hence, NFS has no equivalent of the "directory stream" referred to >> in the Unix manual pages. Furthermore you don't 'open' a directory >> in NFSv4, you just give it the filehandle. >> >> >> Correct. NFSv4 just took over the NFSv3 approach to reading directories >> and NFSv3 doesn't support open/close at all. >> >> However the lack of a closedir equivalent means that the server has >> no idea when the client has finished with the READDIR cookie(s). >> >> Right. >> >> I think it would be helpful to give some advice on what the server >> should do to avoid a build up of probably useless state. >> >> how about: >> >> As there is no way for the client to indicate that a cookie value >> once received, will not be subsequently used, serer implementations >> should avoid schemes that allocate memory corresponding to a >> returned cookie. Such allocation can be avoided if the server bases >> cookie values on a value such as the offset within the directory >> wher the scan is to be resumed. >> >> >> Given the usual usage of READDIR, I would expect that once a call >> has returned eof<-TRUE, the client will in almost all cases be done >> and the state for the (client,directory) pair can be discarded. >> >> >> Except there is no state for client,directory pair. The spec does not >> mandate that such state be maintained, and i don't know of server that >> maintain that sort of state. NFSv4. servers follow the NFSv3 practice, >> where client state/per se/ did not exist. >> >> However, there might be cases where the call has to be rerun due to >> comms failure I suppose, so maybe after a number of calls at eof? >> Presumably there also ought to be some sort of timeout (like if the >> leases expire??) I have no idea what current implementations do here. >> >> None of these situations arise since there is no per--client readdir state. >> >> >> >> Crafting a couple of sentences to mention these three items (deletion, >> modification and stale state) would IMO be helpful. It would certainly >> have helped me!. >> >> suggestions: >> >> * *deletion* >> >> cookies return by READDIR become invalid when the directory is >> removed but the server is not obliged for check for this situation >> or check the cookie verifier since the directory handle will become >> stale. >> >> * *modification* >> >> Cookies generated by such techniques should reaiin valid despite >> modification of the associated directory. If a server were to >> invalidate a cookie because of a directory modification, READDIR's >> of large directories might never finish. >> >> * *stale state* >> >> I think it is clear from the text above that there is no state to >> become stale. Not sure what else you can say. >> >> * *directory reorganization* >> >> certain re-organization operation on a directory (including >> directory compaction) may invalidate READDDIR cookies >> previously given out. when these situation ocur, the server should >> modify the cookie verifier to accepting cookies which are no longer >> valid. >> >> >> On Tue, Dec 2, 2014 at 9:00 AM, Elwyn Davies <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi. >> >> Sorry to keep harping on about READDIR, but... >> >> Tom's comment about opendir/closedir at the end of the mail nbelow >> obviously triggered something in my brain overnight, 'cos I woke up >> realizing that I wasn't correctly understanding the semantics here. >> >> Please bear with while I parade my naivete :-( >> >> Having refreshed my memory on the semantics of reading directories >> in Unix and checked back on NFSv4, I realized that NFSv4 doesn't >> have the equivalents of opendir and closedir. >> >> Hence, NFS has no equivalent of the "directory stream" referred to >> in the Unix manual pages. Furthermore you don't 'open' a directory >> in NFSv4, you just give it the filehandle. >> >> So, the case of the directory getting deleted under your feet >> between READDIR calls presumably just results in a NFS4ERR_STALE >> error on the next READDIR call and the issue of whether the cookie >> is valid is moot. It is then up to the client to map this into local >> OS behaviour. >> >> This means that only the directory modification case is relevant to >> the cookie validity issue, and I understand the comment that David >> was making yesterday more clearly now. >> >> However the lack of a closedir equivalent means that the server has >> no idea when the client has finished with the READDIR cookie(s). I >> think it would be helpful to give some advice on what the server >> should do to avoid a build up of probably useless state. >> >> Given the usual usage of READDIR, I would expect that once a call >> has returned eof<-TRUE, the client will in almost all cases be done >> and the state for the (client,directory) pair can be discarded. >> However, there might be cases where the call has to be rerun due to >> comms failure I suppose, so maybe after a number of calls at eof? >> Presumably there also ought to be some sort of timeout (like if the >> leases expire??) I have no idea what current implementations do here. >> >> Crafting a couple of sentences to mention these three items >> (deletion, modification and stale state) would IMO be helpful. It >> would certainly have helped me! >> >> Regards, >> Elwyn >> >> On 02/12/14 00:56, Tom Haynes wrote: >> >> >> On Dec 1, 2014, at 11:54 AM, Elwyn Davies >> <[email protected] <mailto:[email protected]>> wrote: >> >> Hi, David. >> >> Thanks for the response. >> >> A couple of comments below... >> >> Regards, >> Elwyn >> >> On 01/12/14 18:50, David Noveck wrote: >> >> My view (remains) that if the cookie is not tied to >> the change >> attribute of the directory, this makes life >> difficult for both the >> server (what does it send if the data has been >> changed and how does >> it manage maintaining checkpoints -- where a >> readdir finished --, in >> particular if the checkpoint was on an entry that >> has been deleted), >> and the client. >> >> >> The cookie is not tied to the change attribute of the >> directory. >> >> The purpose of the cookie is ensuring that you do a >> READDIR that >> encompasses multiple requests: >> >> * entries that exist during the entire period of the >> multi-requuest >> directory read appear exactly once in the result. >> * entries that exists during some part of the period >> of the >> multi-request directory read appear at most once >> in the result. >> >> >> Basically implementations scan through the directory as >> it exists on >> disk and if there is a created entry, the client may or >> may not see it, >> depending on where the the new entry happens to be put, >> with respect to >> the ongoing scan. As far as deleted entries, the >> implementation only >> has to be able to move on to the next entry if it >> encouneters an entry >> marked as deleted. >> >> >> OK, so the server has to be clever rather than telling the >> client to restart the readdir -- I'm not sure this is the >> right trade-off since the client has to cope with being told >> the cached cookie is useless (as per Trond's input below) >> but I guess implementations work this way already and it >> isn't my call. In that case I suggest that the last >> sentence of s15.26.4, para 9, is modified to tell server >> implementers a bit more clearly what they have to do. >> Suggestion: >> OLD: >> Ideally, the cookie value should not change if the >> directory is >> modified since the client may be caching these values. >> NEW: >> The server SHOULD try to accept cookie values issued >> with READDIR >> responses even if the directory has been modified >> between the >> READDIR calls but MAY return NFS4ERR_NOT_VALID if this >> is not >> possible as might be the case if the server has rebooted >> in the >> interim. >> >> >> I’ve taken this change. >> >> >> One nasty thought occurs to me regarding the idea that a >> client caches >> the cookie value(s): Is the server supposed to maintain the >> whole >> sequence of cookies for a sequence of READDIRs on a single >> directory so >> the client can reissue some of the READDIR calls or is it >> fair enough for the server to throw them away after the >> client has used them once? And how long should the server >> hang onto cookie state in normal operation? >> >> >> No, it does not have to save them. >> >> But the cookies are pretty persistent in the first place. I.e., >> use some property of either the inode or the dirent to generate >> the cookie. >> >> As such, the cookie state does not need to be held onto. >> >> And for the life, it actually needs to be “stable” between the >> opendir() and closedir() call on the client. While we are safe >> on a client reboot, a client could end up holding onto the DIR >> pointer for a large indeterminate time. >> >> _______________________________________________ Gen-art mailing list [email protected] https://www.ietf.org/mailman/listinfo/gen-art
