Yonik Seeley wrote:
> If it will happen so rarely, make it simpler and go directly for
> segments_(N-1)... (treat it like your previous plan if segments_N.done
> hadn't been written yet).
Yes, true, we could just fall back to the prior segments_(N-1) file in
this case. Though that means the reader will likely just hit an
IOException trying to load the segments (since a commit is "in
process") and then I'd have to re-retry against segments_N.
You need to fall back in any case... (remember the writer crashing
scenario).
Reusing the fallback logic makes the code simpler in a case that will
almost never happen.
It's really just a question of if you put in extra retry logic or not.
Yes, but with the separate file (segments_N.done) approach this is
just an file exists check. In other words, the reader does a dir
listing, finds the most recent segments_N for which there also exists
a segments_N.done and uses that.
But you're right I could re-use the retry logic I already have now,
but instead of retrying forwards (N+1), retry backwards specifically
when hitting an IOException when reading the contents of the segments
file. I don't think it's that hard to implement ... my biggest worry
is whether filesystem caching (for the likes of NFS) will cause
delays, ie does NFS "cache" (under its own timeout policy) the fact
that the file was N bytes when last it was opened? I would rather
have a solution that doesn't rely on the caching policies of the
filesystem.
I will try both approaches & report back!
I've been using NFS as my "proxy" for "least common denominator"
I think that's a safe bet ;-)
NFS v2 or v3?
Good question -- so far only v3. I will test v2 as well. So many
filesystems!
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
