Hi, David.
Excellent. I think some suitable combination of your words below would
fill the bill.
The point about the very large directories is well taken - I hadn't
grokked that previously.
I guess the update should go in s15.26.5 after para 1:
Something like:
-----------------
As there is no way for the client to indicate that a cookie value once
received, will not be subsequently used, server implementations should
avoid schemes that allocate memory corresponding to a returned cookie.
Such allocation can be avoided if the server bases cookie values on a
value such as the offset within the directory where the scan is to be
resumed.
Cookies generated by such techniques should be designed to remain valid
despite modification of the associated directory. If a server were to
invalidate a cookie because of a directory modification, READDIR's of
large directories might never finish.
If a directory is deleted after the client has carried out one or more
READDIR operations on the directory, the cookies returned will become
invalid but the server does not need to be concerned as the directory
file handle used previously would have become stale and would be
reported as such on subsequent READDIR operations. The server would not
need to check the cookie verifier in this case.
However, certain re-organization operations on a directory (including
directory compaction) may invalidate READDDIR cookies previously given
out. When such a situation occurs, the server should modify the cookie
verifier so as to accept cookies which would otherwise no longer be valid.
-------------------
Aside: handling reorganization without creating state sounds challenging
but life wouldn't be interesting without challenges! :-)
A couple of nits that got missed from yesterday:
s5.4, 2nd bullet labelled '1.': s/common to all object within given file
system./common to all objects within the given file system./
s9.1.3, last sentence: s/an can be treated as later./and can be treated
as later./
^^^^
With something like the above, I think we are done.
Congratulations! I hope this gets to be an RFC RSN.
Cheers,
Elwyn
On 02/12/14 14:56, David Noveck wrote:
Hence, NFS has no equivalent of the "directory stream" referred to
in the Unix manual pages. Furthermore you don't 'open' a directory
in NFSv4, you just give it the filehandle.
Correct. NFSv4 just took over the NFSv3 approach to reading directories
and NFSv3 doesn't support open/close at all.
However the lack of a closedir equivalent means that the server has
no idea when the client has finished with the READDIR cookie(s).
Right.
I think it would be helpful to give some advice on what the server
should do to avoid a build up of probably useless state.
how about:
As there is no way for the client to indicate that a cookie value
once received, will not be subsequently used, serer implementations
should avoid schemes that allocate memory corresponding to a
returned cookie. Such allocation can be avoided if the server bases
cookie values on a value such as the offset within the directory
wher the scan is to be resumed.
Given the usual usage of READDIR, I would expect that once a call
has returned eof<-TRUE, the client will in almost all cases be done
and the state for the (client,directory) pair can be discarded.
Except there is no state for client,directory pair. The spec does not
mandate that such state be maintained, and i don't know of server that
maintain that sort of state. NFSv4. servers follow the NFSv3 practice,
where client state/per se/ did not exist.
However, there might be cases where the call has to be rerun due to
comms failure I suppose, so maybe after a number of calls at eof?
Presumably there also ought to be some sort of timeout (like if the
leases expire??) I have no idea what current implementations do here.
None of these situations arise since there is no per--client readdir state.
Crafting a couple of sentences to mention these three items (deletion,
modification and stale state) would IMO be helpful. It would certainly
have helped me!.
suggestions:
* *deletion*
cookies return by READDIR become invalid when the directory is
removed but the server is not obliged for check for this situation
or check the cookie verifier since the directory handle will become
stale.
* *modification*
Cookies generated by such techniques should reaiin valid despite
modification of the associated directory. If a server were to
invalidate a cookie because of a directory modification, READDIR's
of large directories might never finish.
* *stale state*
I think it is clear from the text above that there is no state to
become stale. Not sure what else you can say.
* *directory reorganization*
certain re-organization operation on a directory (including
directory compaction) may invalidate READDDIR cookies
previously given out. when these situation ocur, the server should
modify the cookie verifier to accepting cookies which are no longer
valid.
On Tue, Dec 2, 2014 at 9:00 AM, Elwyn Davies <[email protected]
<mailto:[email protected]>> wrote:
Hi.
Sorry to keep harping on about READDIR, but...
Tom's comment about opendir/closedir at the end of the mail nbelow
obviously triggered something in my brain overnight, 'cos I woke up
realizing that I wasn't correctly understanding the semantics here.
Please bear with while I parade my naivete :-(
Having refreshed my memory on the semantics of reading directories
in Unix and checked back on NFSv4, I realized that NFSv4 doesn't
have the equivalents of opendir and closedir.
Hence, NFS has no equivalent of the "directory stream" referred to
in the Unix manual pages. Furthermore you don't 'open' a directory
in NFSv4, you just give it the filehandle.
So, the case of the directory getting deleted under your feet
between READDIR calls presumably just results in a NFS4ERR_STALE
error on the next READDIR call and the issue of whether the cookie
is valid is moot. It is then up to the client to map this into local
OS behaviour.
This means that only the directory modification case is relevant to
the cookie validity issue, and I understand the comment that David
was making yesterday more clearly now.
However the lack of a closedir equivalent means that the server has
no idea when the client has finished with the READDIR cookie(s). I
think it would be helpful to give some advice on what the server
should do to avoid a build up of probably useless state.
Given the usual usage of READDIR, I would expect that once a call
has returned eof<-TRUE, the client will in almost all cases be done
and the state for the (client,directory) pair can be discarded.
However, there might be cases where the call has to be rerun due to
comms failure I suppose, so maybe after a number of calls at eof?
Presumably there also ought to be some sort of timeout (like if the
leases expire??) I have no idea what current implementations do here.
Crafting a couple of sentences to mention these three items
(deletion, modification and stale state) would IMO be helpful. It
would certainly have helped me!
Regards,
Elwyn
On 02/12/14 00:56, Tom Haynes wrote:
On Dec 1, 2014, at 11:54 AM, Elwyn Davies
<[email protected] <mailto:[email protected]>> wrote:
Hi, David.
Thanks for the response.
A couple of comments below...
Regards,
Elwyn
On 01/12/14 18:50, David Noveck wrote:
My view (remains) that if the cookie is not tied to
the change
attribute of the directory, this makes life
difficult for both the
server (what does it send if the data has been
changed and how does
it manage maintaining checkpoints -- where a
readdir finished --, in
particular if the checkpoint was on an entry that
has been deleted),
and the client.
The cookie is not tied to the change attribute of the
directory.
The purpose of the cookie is ensuring that you do a
READDIR that
encompasses multiple requests:
* entries that exist during the entire period of the
multi-requuest
directory read appear exactly once in the result.
* entries that exists during some part of the period
of the
multi-request directory read appear at most once
in the result.
Basically implementations scan through the directory as
it exists on
disk and if there is a created entry, the client may or
may not see it,
depending on where the the new entry happens to be put,
with respect to
the ongoing scan. As far as deleted entries, the
implementation only
has to be able to move on to the next entry if it
encouneters an entry
marked as deleted.
OK, so the server has to be clever rather than telling the
client to restart the readdir -- I'm not sure this is the
right trade-off since the client has to cope with being told
the cached cookie is useless (as per Trond's input below)
but I guess implementations work this way already and it
isn't my call. In that case I suggest that the last
sentence of s15.26.4, para 9, is modified to tell server
implementers a bit more clearly what they have to do.
Suggestion:
OLD:
Ideally, the cookie value should not change if the
directory is
modified since the client may be caching these values.
NEW:
The server SHOULD try to accept cookie values issued
with READDIR
responses even if the directory has been modified
between the
READDIR calls but MAY return NFS4ERR_NOT_VALID if this
is not
possible as might be the case if the server has rebooted
in the
interim.
I’ve taken this change.
One nasty thought occurs to me regarding the idea that a
client caches
the cookie value(s): Is the server supposed to maintain the
whole
sequence of cookies for a sequence of READDIRs on a single
directory so
the client can reissue some of the READDIR calls or is it
fair enough for the server to throw them away after the
client has used them once? And how long should the server
hang onto cookie state in normal operation?
No, it does not have to save them.
But the cookies are pretty persistent in the first place. I.e.,
use some property of either the inode or the dirent to generate
the cookie.
As such, the cookie state does not need to be held onto.
And for the life, it actually needs to be “stable” between the
opendir() and closedir() call on the client. While we are safe
on a client reboot, a client could end up holding onto the DIR
pointer for a large indeterminate time.
_______________________________________________
Gen-art mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/gen-art
