Re: [fossil-users] Minor Issues with Fossil 2.5
It's not at all that I think I'm the expert you've summoned in your
last post on this thread. As you've been running your own high-traffic
"althttpd" server for many years, maybe you should explain the
Internet to us.
I was doing some (limited) research about RFCs, recommendations from
the Apache web server documentation, and blog posts by web browser
developers. It's often stated that "If-None-Match" should be given
precedence over "If-Modified-Since", but I haven't come across any
"hard facts".
I've created a small PHP script to test HTTP caching with ETags.
= etags.php =
4) $m=0;
$lmt=1519776000; // Wed, 28 Feb 2018 00:00:00 GMT
$etag='caffee'; // fixed
// handle If-Modified-Since
if (($m==1 || $m>=2 && !isset($_SERVER['HTTP_IF_NONE_MATCH'])) &&
isset($_SERVER['HTTP_IF_MODIFIED_SINCE']) &&
strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE'])==$lmt) {
header('Not Modified',true,304);
exit;
}
// handle If-None-Match
if ($m>=3 && isset($_SERVER['HTTP_IF_NONE_MATCH']) &&
$_SERVER['HTTP_IF_NONE_MATCH']=='"'.$etag.'"') {
header('Not Modified',true,304);
exit;
}
// generate new data
if ($m==2)
$etag='--'; // none
else if ($m!=4)
$etag=uniqid(); // random
if ($m!=2)
header('ETag: "'.$etag.'"');
header('Last-Modified: '.gmdate('D, d M Y H:i:s',$lmt).' GMT'); // fixed
header('Cache-Control: must-revalidate, private');
header('Content-Type: text/plain; charset=utf-8');
echo
"TEST HTTP CONDITIONAL REQUEST PRECEDENCE\n".
"\n\n".
"• Always reply with fixed \"Last-Modified\" date header\n".
"• Reply with random, fixed, or no ETag (see query parameters)\n\n".
"• IF_MODIFIED_SINCE: always cache hit (send \"304/Not Modified\")\n".
"• IF_NONE_MATCH: cache hit on ETag match, cache miss otherwise\n";
echo
"\nQuery Parameters (Mode):\n\n".
"• ?m=0: random ETag, no caching (default)\n".
"• ?m=1: random ETag, favor IF_MODIFIED_SINCE (cache hit)\n".
"• ?m=2: omit ETag, favor IF_MODIFIED_SINCE (cache hit)\n".
"• ?m=3: random ETag, favor IF_NONE_MATCH (cache miss)\n".
"• ?m=4: fixed ETag, favor IF_NONE_MATCH (cache hit)\n";
echo "\nScript State:\n\n";
echo "• Mode: $m\n";
echo "• Date: ".gmdate('D, d M Y H:i:s')." GMT\n";
echo "• ETag: $etag\n";
echo "\nRequest Headers:\n\n";
foreach ($_SERVER as $k=>$v)
if (substr($k,0,5)=='HTTP_') echo "• $k: $v\n";
echo "\nResponse Headers:\n\n";
foreach (headers_list() as $k=>$v)
echo "• $v\n";
?>
= etags.php =
The script always responds with a constant "Last-Modified" header, but
generates a random ETag on each run (there's also other options,
explained in the script output).
The idea is that whenever the ETag → "If-None-Match" route is taken,
the script output in the web browser is also updated, so it's obvious
that the client preferred "If-None-Match" over "If-Modified-Since".
The script should be reloaded with F5 and Ctrl+F5 (to disable caching)
to see the effects.
The clients I've tested all favor ETags over "If-Modified-Since", even
Internet Explorer 8 on Windows XP, so no harm here. But of course
"wget -N" without the extra Perl wrapper to add ETag support favors
"If-Modified-Since".
I see that client-server clock skews, and the low resolution (1
second) of the "Last-Modified" timestamp, even more so in conjunction
with the --mtime option to set arbitrary timestamps for unversioned
files, may cause havoc.
But maybe the above script is helpful to test more "exotic" clients on
the way to the decision to support ETags AND "Last-Modified" vs. ETags
only.
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
On 2/25/18, Florian Balmer wrote: > As far as I remember, I've come across the recommendation to combine > ETags and Last-Modified headers, so the client could pick > If-None-Match or If-Modified-Since to validate its cached content. > > And, it's already there, and works like a charm! Yes. But because of the issue described previously, I wonder if I shouldn't drop support for If-Modified-Since? Do we know that all clients that support both If-None-Match and If-Modified-Since will always choose If-None-Match if it is available from the server? In that case, it might be acceptable to leave in support for If-Modified-Since. But if the presence of If-Modified-Since might cause some clients to use it in place of If-None-Match, then we should take it out. Does anybody know? Is there any reader on this list who as parsed the text of the relevant RFCs and knows the answer? Can you brief me, and prehaps give a link to the appropriate standard? At the very least, I should probably fix Fossil so that never sends a Last-Modified reply header (used by If-Modified-Since) unless there is also an ETag header. That would prevent incorrect answers on any client ETag capable and which always prioritizes ETag over Last-Modified. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
D. Richard Hipp:
> Consider this sequence of operations:
>
> (1) User A does a "uv push"
> (2) User B does a "wget -N" against the uv.
> (3) User C does a "uv push" of different content.
>
> If all of (1), (2), and (3) happen during the same second and if
> unversioned content honors the If-Modified-Since header, then
> subsequent attempts by user B to do "wget -N" will continue to
> retrieve the obsolete version of the unversioned content that was
> uploaded by user A. User B will never see the more recent content
> uploaded by user C.
Yes, I agree.
It's the same situation as if multiple users were sharing files on a
web server space not under version control, with uploads by FTP ("uv
push") and downloads by HTTP ("wget -N").
This may work well for "inert" files, requiring small temporary local
adaptations not worth archiving, but bothersome if overwritten by an
accidental "wget". However, "mission-critical" and "fast-changing"
files, for which the risk of loosing content is unacceptable, had
better be put under version control.
As far as I remember, I've come across the recommendation to combine
ETags and Last-Modified headers, so the client could pick
If-None-Match or If-Modified-Since to validate its cached content.
And, it's already there, and works like a charm!
Thank you very much for all your efforts you're putting into this
project, this is really appreciated!
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
On 2/23/18, Florian Balmer wrote: > > 2. There's no "Last-modified" HTTP header when downloading unversioned > files through the /uv web page, causing undesired network traffic for > `wget -N' scripting scenarios, Consider this sequence of operations: (1) User A does a "uv push" (2) User B does a "wget -N" against the uv. (3) User C does a "uv push" of different content. If all of (1), (2), and (3) happen during the same second and if unversioned content honors the If-Modified-Since header, then subsequent attempts by user B to do "wget -N" will continue to retrieve the obsolete version of the unversioned content that was uploaded by user A. User B will never see the more recent content uploaded by user C. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
On Sun, Feb 25, 2018 at 4:11 AM, wrote: > > Date: Sun, 25 Feb 2018 10:11:04 +0100 > From: Florian Balmer Subject: Re: [fossil-users] Minor Issues with Fossil 2.5 > > I think that the "Last-Modified" header is much easier to handle, as > it boils down to parsing a GMT date/time string. And, as already > mentioned, clobbering of locally-modified files can be avoided with > the timestamp mechanism, this is not possible with an ETag alone. > The ETag is supposed to only change when the content changes Therefore, if the Etag you have is not the same as the current ETag on the server - for the same resource - then the content is different. Admittedly, if the server's copy of the resource is changed to an older version, the ETag will be different, as well. So, if making you always need the newest version, then modification time is needed. But if different is acceptable, then Etag will do that. ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
Ron: > Someone already made an ETag wrapper for wget: > https://www.w3.org/2001/12/rubyrdf/pack/tests/scutter/wget.pl Thanks for the interesting link. > There is no requirement or specification for the value of an ETag, > could use the Fossil artifact hash as the ETag value. Then a script > trying to fetch the file can generate the value from the current copy > of the file. I agree, but with the current implementation, at least the ingredient is hard to guess for a client script trying to generate a Fossil ETag. I think that the "Last-Modified" header is much easier to handle, as it boils down to parsing a GMT date/time string. And, as already mentioned, clobbering of locally-modified files can be avoided with the timestamp mechanism, this is not possible with an ETag alone. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
Thinking about HTTP caching twice, the following comes to my mind: A command line download tool, that unlike a web browser does not keep a cache of content and associated ETags, won't be able to calculate the ETag for a file it is going to request from a Fossil web server. ETags are not simple file checksums, and Fossil seems to use the following information to generate them: /// So command line download tools can only rely on the "Last-Modified" header. And, as already mentioned, I do see additional benefit from receiving file timestamps over the web. This is in contrast to check-out timestamps, which Fossil does not preserve, likely so that build tools like `make' know what to do. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
> Is the new ETag mechanism sufficient for your purposes? That's a great addition, thanks! However, my simple scripting tools don't support ETags, i.e. the `wget -N' example (latest version 1.19.4) mentioned earlier still carries out two full downloads of the Fossil source code archive. With the "Last-Modified" header, the timestamp of the downloaded unversioned file can be set correctly, and locally updated unversioned files waiting to be synchronized with the server would not be clobbered by accidental runs of the download script, for example, but this would fail with ETags (if I get it right, ETags can only say "different", whereas "Last-Modified" can say "newer"). Even the most basic scripting solutions, like "WinHttp.WinHttpRequest.5.1" used with WSH, are able to return the "Last-Modified" header, whereas implementing ETag support in these cases requires "less-recreational" scripting ;) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
On 2/23/18, Florian Balmer wrote: > > 2. There's no "Last-modified" HTTP header when downloading unversioned > files through the /uv web page, causing undesired network traffic for > `wget -N' scripting scenarios, for example, and maybe also with web > spiders (if they are enabled). I would like to suggest sending the > "Last-modified" HTTP header corresponding to the timestamp of the > requested unversioned file. There is still no Last-Modified: header. But I did add ETag: on replies and compare against If-None-Match: on requests, so that unnecessary /uv downloads can be avoided. Is the new ETag mechanism sufficient for your purposes? -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
> That's because autofocus causes CSS flashes on some web-browsers.
> If you can suggest a reasonable workaround, that would be great.
Oh, I'm sorry I've missed that.
The only other way I know is by JavaScript, something like:
window.onload = function() {
document.getElementById("u").focus();
};
Nowadays, the faster "DOM ready" event may be preferred over
`window.onload', but finding a cross-browser solution here always
seems somewhat cumbersome to me, and the latter seems quick enough for
simple web pages.
However, I'm not sure what are "CSS flashes", and whether they may
appear with the JavaScript method, as well.
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
On 2/23/18, Florian Balmer wrote: > 1. On the Login/Logout web page, the User ID input field used to have > the keyboard "autofocus". That's because autofocus causes CSS flashes on some web-browsers. It was removed with check-in https://www.fossil-scm.org/fossil/info/20e680aac75471da. If you can suggest a reasonable workaround, that would be great. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Minor Issues with Fossil 2.5
I've discovered a few minor issues after updating to Fossil 2.5. As I have skipped version 2.4, I'm not sure when they were really introduced. I'd appreciate if they could be considered to be fixed in case there's some nearby work, say like an occasional appendectomy ;) 1. On the Login/Logout web page, the User ID input field used to have the keyboard "autofocus". With Fossil 2.5, the input field is no longer focused, but has to be activated by hitting Tab repeatedly (cycling through all the links in the top navigation bar, first), or by using the mouse. I would like to suggest restoring the old "autofocus" behaviour. 2. There's no "Last-modified" HTTP header when downloading unversioned files through the /uv web page, causing undesired network traffic for `wget -N' scripting scenarios, for example, and maybe also with web spiders (if they are enabled). I would like to suggest sending the "Last-modified" HTTP header corresponding to the timestamp of the requested unversioned file. Example: wget -N http://fossil-scm.org/index.html/uv/fossil-src-2.5.tar.gz Running this command twice should only download the file once, if the timestamp had not changed in between the commands. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
