Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Sorry I'm late to fixing this. I reinstalled the rewrite rules on the server and I configured nginx to use the errors pages that Tim already created. We should be covered for most problems, but let me know if we need to do more. Thanks! Ihor Radchenko writes: > If someone else is willing to allocate time and look into this issue, > feel free to reply here. We can then arrange access to the relevant > configs. -- Bastien
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Tim Cross writes: > If you or others want to look at the 404 page issue, please do so. For > right now, I'll just do a basic 404 page which Bastien can install > easily. If you or others come up with something more sophisticated, it > can be used to replace the simple basic version, otherwise we can > revisit this later if necessary. +1 Having a simple 404 page is already an improvement. There is no urgent need to prioritize further complexities just yet. At least, there is no need to distract Tim from doing much more important job of other WORG improvements. If someone else is willing to allocate time and look into this issue, feel free to reply here. We can then arrange access to the relevant configs. Best, Ihor
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Max Nikulin writes: > On 05/07/2022 09:54, Tim Cross wrote: >> Max Nikulin writes: >>> Bastien. Re: Possible bug report: URL capitalization in online manual. Mon, >>> 10 Feb 2020 >>> 07:48:21 +0100 https://list.orgmode.org/874kvzdjka@gnu.org/ I've now installed the rewrite rules on the server and all these links are redirecting correctly. Given the amount of Org documentation links living out there, that's really a good idea. > > Tim, is there any chance that Bastien forgot to send you some included file > for nginx > configuration? However, the site ignores this file even it exists somewhere > on filesystem. > anything is possible. In a 'standard' nginx configuration, you would typically have all the config related to s specific site in one file. However, that is just convention, not a rule. >>> Tim Cross: At this point, my vote is to just do a basic updated 404 page that points to the index.html page for orgmode.org >>> >>> For the manual and for the guide directories I still consider links to the >>> table of >>> contents (or even full table of contents) on 404 pages as a better variant. >> I don't really care what specific page is linked to - just that we keep >> it simple and have a link for an initial quick fix. Only reason I >> mentioned the index.html page is that it has links to everything - worg, >> git, manual, mail list etc. It is possible that people get to 404 via >> links other than to the manual. > > I do not have ready to use snippet, but I am sure it should be possible to > set specific > 404 pages for particular directories like the manual and the guide. Of > course, these > pages, besides links to table of contents, may have another link to > index.html or a > navigation header shared with other pages on the site. Many things are possible. However, I'm trying very hard not to be overly distracted by this issue, which I think is fairly insignificant. My objective is to do as little as possible at this time because I want to get back to cleanup of worg. It is quite possible that work will also have some influence on the rest of the org web space. If you or others want to look at the 404 page issue, please do so. For right now, I'll just do a basic 404 page which Bastien can install easily. If you or others come up with something more sophisticated, it can be used to replace the simple basic version, otherwise we can revisit this later if necessary.
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
On 05/07/2022 09:54, Tim Cross wrote: Max Nikulin writes: Bastien. Re: Possible bug report: URL capitalization in online manual. Mon, 10 Feb 2020 07:48:21 +0100 https://list.orgmode.org/874kvzdjka@gnu.org/ I've now installed the rewrite rules on the server and all these links are redirecting correctly. Given the amount of Org documentation links living out there, that's really a good idea. Tim, is there any chance that Bastien forgot to send you some included file for nginx configuration? However, the site ignores this file even it exists somewhere on filesystem. Tim Cross: At this point, my vote is to just do a basic updated 404 page that points to the index.html page for orgmode.org For the manual and for the guide directories I still consider links to the table of contents (or even full table of contents) on 404 pages as a better variant. I don't really care what specific page is linked to - just that we keep it simple and have a link for an initial quick fix. Only reason I mentioned the index.html page is that it has links to everything - worg, git, manual, mail list etc. It is possible that people get to 404 via links other than to the manual. I do not have ready to use snippet, but I am sure it should be possible to set specific 404 pages for particular directories like the manual and the guide. Of course, these pages, besides links to table of contents, may have another link to index.html or a navigation header shared with other pages on the site.
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Max Nikulin writes: > On 05/07/2022 05:37, Tim Cross wrote: >> Max Nikulin writes: >>> >>> However it seems, Bastien earlier configured a set of rewrite rules mapping >>> old file names >>> with more lower case letters to new ones. In my opinion it is the best >>> option and it >>> should be restored. List of files may be committed to git (either Org or >>> site) to detect >>> changes later and to add new mappings when some file disappears. >> Are you sure about that. There is nothing along these lines in the nginx >> config file that I can see. Also, my reading following that thread you >> provided earlier was that Bastien thought the overhead to manage such >> lists was too high? > > Bastien. Re: Possible bug report: URL capitalization in online manual. Mon, > 10 Feb 2020 > 07:48:21 +0100 https://list.orgmode.org/874kvzdjka@gnu.org/ >> I've now installed the rewrite rules on the server and all these links >> are redirecting correctly. Given the amount of Org documentation links >> living out there, that's really a good idea. > > I did not check it that time, so I can not be really sure. The rules might be > lost during > migration to deployment using SourceHut. My expectation is "301 Moved > Permanently" > redirection for obsolete file names. > > I do not think, a hundred of redirection rules gives significant overhead. I > attributed > Bastien's "too much" to maintaining documentation for several versions of > Org. A part of > original problem (if I get it correctly) was mix of old files survived from > earlier > versions of the manual and current ones. Certainly removing of outdated files > was a proper > step. > > Tim Cross: >> At this point, my vote is to just do a basic updated 404 page that >> points to the index.html page for orgmode.org > > For the manual and for the guide directories I still consider links to the > table of > contents (or even full table of contents) on 404 pages as a better variant. I don't really care what specific page is linked to - just that we keep it simple and have a link for an initial quick fix. Only reason I mentioned the index.html page is that it has links to everything - worg, git, manual, mail list etc. It is possible that people get to 404 via links other than to the manual. If you or someone else wants to do something more sophisticated, I'd say go for it. In the meantime, I will see if I can put together a default 404 page. However, I think only Bastien has the necessary access to install it.
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
On 05/07/2022 05:37, Tim Cross wrote: Max Nikulin writes: However it seems, Bastien earlier configured a set of rewrite rules mapping old file names with more lower case letters to new ones. In my opinion it is the best option and it should be restored. List of files may be committed to git (either Org or site) to detect changes later and to add new mappings when some file disappears. Are you sure about that. There is nothing along these lines in the nginx config file that I can see. Also, my reading following that thread you provided earlier was that Bastien thought the overhead to manage such lists was too high? Bastien. Re: Possible bug report: URL capitalization in online manual. Mon, 10 Feb 2020 07:48:21 +0100 https://list.orgmode.org/874kvzdjka@gnu.org/ I've now installed the rewrite rules on the server and all these links are redirecting correctly. Given the amount of Org documentation links living out there, that's really a good idea. I did not check it that time, so I can not be really sure. The rules might be lost during migration to deployment using SourceHut. My expectation is "301 Moved Permanently" redirection for obsolete file names. I do not think, a hundred of redirection rules gives significant overhead. I attributed Bastien's "too much" to maintaining documentation for several versions of Org. A part of original problem (if I get it correctly) was mix of old files survived from earlier versions of the manual and current ones. Certainly removing of outdated files was a proper step. Tim Cross: At this point, my vote is to just do a basic updated 404 page that points to the index.html page for orgmode.org For the manual and for the guide directories I still consider links to the table of contents (or even full table of contents) on 404 pages as a better variant.
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Max Nikulin writes: > > However it seems, Bastien earlier configured a set of rewrite rules mapping > old file names > with more lower case letters to new ones. In my opinion it is the best option > and it > should be restored. List of files may be committed to git (either Org or > site) to detect > changes later and to add new mappings when some file disappears. Are you sure about that. There is nothing along these lines in the nginx config file that I can see. Also, my reading following that thread you provided earlier was that Bastien thought the overhead to manage such lists was too high? I guess we need to wait for Bastien to indicate his preference. I find things a little challenging as there seems to be a fair bit of outdated/conflicting information scattered through various files/notes. At this point, my vote is to just do a basic updated 404 page that points to the index.html page for orgmode.org so that users have something to follow and not a big ugly 404 Not Found. The rest we can consider later.
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
On 04/07/2022 19:58, Ihor Radchenko wrote: If JS is the only option it is still better to have it and fall back to more generic page if JS is not supported. Of course, given that the JS code is not going to be too complex and not going to require too much maintenance. It is possible to perform a kind of fuzzy matching on the server, but I am unsure if Bastien wouldn't mind a dynamic 404 page. The manual and the short guide may have their own 404 pages explaining that some page is not available any more and offering table of contents. However it seems, Bastien earlier configured a set of rewrite rules mapping old file names with more lower case letters to new ones. In my opinion it is the best option and it should be restored. List of files may be committed to git (either Org or site) to detect changes later and to add new mappings when some file disappears.
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Tim Cross writes: >> This sounds like a good idea. >> I am not sure if it is feasible, but the 404 page could also provide >> suggestions based on similar existing links. I have something like >> https://list.orgmode.org/orgmode/83tu89b7pr@gnu.org/ in mind. > > Sorry, I don't understand the relevance of that link (it seems to be to > a discussion about GC size?). The link I provided is pointing to a thread, which is not in Org ML. Yet, list.orgmode.org is able to detect messages in other mailing lists :) For me, the page in the link looks like: Message-ID: <83tu89b7pr@gnu.org> found in another inbox: https://yhetil.org/emacs-devel/83tu89b7pr@gnu.org/ Perhaps try an external site: https://marc.info/?i=83tu89b7pr@gnu.org https://www.mail-archive.com/search?l=mid&q=83tu89b7pr@gnu.org nntp://news.gmane.io/83tu89b7pr@gnu.org https://lists.debian.org/msgid-search/83tu89b7pr@gnu.org https://docs.FreeBSD.org/cgi/mid.cgi?db=mid&id=83tu89b7pr@gnu.org https://www.w3.org/mid/83tu89b7pr@gnu.org http://www.postgresql.org/message-id/83tu89b7pr@gnu.org https://lists.debconf.org/cgi-lurker/keyword.cgi?doc-url=/lurker&format=en.html&query=id:83tu89b7pr@gnu.org > Yes, you should be able to use JS to examine the URL which failed and > transform it by making the first character of each word in the url > filename upper case. Essentially do what was proposed with the rewrite > rules, but which wouldn't have the same performance hit as it would only > be triggered when incorrect URLs were entered and wold run in the browser. > > The only downside is it wouldn't work with non JS based browsers (like > eww). If the server had support for server side scripting (perl, ruby, > etc) it could be done so that it would work with all browsers by doing > the rendering server side. If JS is the only option it is still better to have it and fall back to more generic page if JS is not supported. Of course, given that the JS code is not going to be too complex and not going to require too much maintenance. Best, Ihor
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Ihor Radchenko writes: > Tim Cross writes: > >> I do have an alternative suggestion which may help. Given that the >> 'broken' URLs are actually from external links to old documentation >> which has been removed, what we could do is create a more informative >> 404 page. Once users are on the 'real' site, the case issue does not >> exist. It is only a problem due to outdated URLs on external sites like >> stack overflow. Instead of just saying 404 Not Found, the page could say >> the requested URL was not found and is likely a link to old outdated doc >> documentation. The page could include a link to the main orgmode >> page. This would be a fairly simple 'fix' that would improve user >> experience to some degree. > > This sounds like a good idea. > I am not sure if it is feasible, but the 404 page could also provide > suggestions based on similar existing links. I have something like > https://list.orgmode.org/orgmode/83tu89b7pr@gnu.org/ in mind. > Sorry, I don't understand the relevance of that link (it seems to be to a discussion about GC size?). Yes, you should be able to use JS to examine the URL which failed and transform it by making the first character of each word in the url filename upper case. Essentially do what was proposed with the rewrite rules, but which wouldn't have the same performance hit as it would only be triggered when incorrect URLs were entered and wold run in the browser. The only downside is it wouldn't work with non JS based browsers (like eww). If the server had support for server side scripting (perl, ruby, etc) it could be done so that it would work with all browsers by doing the rendering server side.
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Tim Cross writes: > I do have an alternative suggestion which may help. Given that the > 'broken' URLs are actually from external links to old documentation > which has been removed, what we could do is create a more informative > 404 page. Once users are on the 'real' site, the case issue does not > exist. It is only a problem due to outdated URLs on external sites like > stack overflow. Instead of just saying 404 Not Found, the page could say > the requested URL was not found and is likely a link to old outdated doc > documentation. The page could include a link to the main orgmode > page. This would be a fairly simple 'fix' that would improve user > experience to some degree. This sounds like a good idea. I am not sure if it is feasible, but the 404 page could also provide suggestions based on similar existing links. I have something like https://list.orgmode.org/orgmode/83tu89b7pr@gnu.org/ in mind. Best, Ihor
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Max Nikulin writes: >> > I noticed that the Org documentation server gives 404 Not Found for a > large number of links published all over the internet because (1) it > parses URLs case-sensitively and (2) the case has changed at some point. > I stumble upon such 404s errors daily. See, for instance, the link to > Column Groups documentation in this Stack Overflow answer: > > https://stackoverflow.com/a/8570307/1306956. > > On 03/07/2022 21:01, Ihor Radchenko wrote: >> Then, I am also CCing Bastien. >> As an idea, the "fix" can be creating symlinks to alternative file >> names. But we first need to figure out the difference between old/new >> naming schemes. > > Previous discussion of the issue: > > Ori. Re: Possible bug report: URL capitalization in online manual. Sun, 9 Feb > 2020 > 13:49:04 -0500 > https://list.orgmode.org/cacycjqx3xrjsctfemtnup6hwzge1m83kgnc9atyt_lupgk6...@mail.gmail.com/ Thanks Max, that helped fill in some of the blanks and confirms some of what I assumed. Seems that the change to letter case was a deliberate change, as was removal of the old pages without upper case letters following the initial uppercase letter in the filename. Making the server case insensitive is not straight-forward. Essentially, we would need to add a regular expression based rewrite rule OR we would need to add copies or perhaps setup symbolic links. While all possible, it will add to maintenance overhead and will likely be a source for future bugs. I do have an alternative suggestion which may help. Given that the 'broken' URLs are actually from external links to old documentation which has been removed, what we could do is create a more informative 404 page. Once users are on the 'real' site, the case issue does not exist. It is only a problem due to outdated URLs on external sites like stack overflow. Instead of just saying 404 Not Found, the page could say the requested URL was not found and is likely a link to old outdated doc documentation. The page could include a link to the main orgmode page. This would be a fairly simple 'fix' that would improve user experience to some degree.
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
I noticed that the Org documentation server gives 404 Not Found for a large number of links published all over the internet because (1) it parses URLs case-sensitively and (2) the case has changed at some point. I stumble upon such 404s errors daily. See, for instance, the link to Column Groups documentation in this Stack Overflow answer: https://stackoverflow.com/a/8570307/1306956. On 03/07/2022 21:01, Ihor Radchenko wrote: Then, I am also CCing Bastien. As an idea, the "fix" can be creating symlinks to alternative file names. But we first need to figure out the difference between old/new naming schemes. Previous discussion of the issue: Ori. Re: Possible bug report: URL capitalization in online manual. Sun, 9 Feb 2020 13:49:04 -0500 https://list.orgmode.org/cacycjqx3xrjsctfemtnup6hwzge1m83kgnc9atyt_lupgk6...@mail.gmail.com/
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Tim Cross writes: >>> I noticed that the Org documentation server gives 404 Not Found for a >>> large number of links published all over the internet because (1) it >>> parses URLs case-sensitively and (2) the case has changed at some point. >>> I stumble upon such 404s errors daily. See, for instance, the link to >>> Column Groups documentation in this Stack Overflow answer: >>> >>> https://stackoverflow.com/a/8570307/1306956. >> ... >> Thanks for the heads up! >> I am CCing Tim Cross. He is currently doing some work on our server. He >> is more likely to know how to fix this. > > Well I'm not sure what is the correct 'fix'. As far as I am aware, the > file part of URI are case sensitive and always have been. The web server > does not have a simple configuration switch to turn on case insensitive > URLs. If we want the web server to treat them as case insensitive, then > we have to add costly regular expression rewrite rules to the URLs sent to the > server. We would also need to verify this won't break anything - it is > possible some links depend on the server treating URLs as case > sensitive. > ... > BTW this is not something I have the access to change as it would > involve changes to the actual web server configuration. It isn't > something which can be 'fixed' at the Org level. Then, I am also CCing Bastien. As an idea, the "fix" can be creating symlinks to alternative file names. But we first need to figure out the difference between old/new naming schemes. > It isn't clear to me what the chain of events have been here. Is it that > we changed the case on our server or is it simply people have put out > link which arfe wrong? > > Is this the result of some change in the way the HTML manual is > generated or a change in how org generates HTML or is this just the > result of people posting the wrong URL? I have a suspicion that it might have something to do with texinfo versions. AFAIU, Org manual pages are generated using makeinfo and it is makeinfo that it responsible for the file name generation. (see https://www.gnu.org/software/texinfo/manual/texinfo/texinfo.html#HTML-Splitting) Probably, the naming scheme or some configuration variables have changed in different versions of texinfo. Best, Ihor
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Ihor Radchenko writes: > Rudolf Adamkovič writes: > >> I noticed that the Org documentation server gives 404 Not Found for a >> large number of links published all over the internet because (1) it >> parses URLs case-sensitively and (2) the case has changed at some point. >> I stumble upon such 404s errors daily. See, for instance, the link to >> Column Groups documentation in this Stack Overflow answer: >> >> https://stackoverflow.com/a/8570307/1306956. >> >> I know that Org does not compete for popularity, but for the people >> getting familiar with it, this issue might cause a lot of headaches for >> no good reason. > > Thanks for the heads up! > I am CCing Tim Cross. He is currently doing some work on our server. He > is more likely to know how to fix this. > Well I'm not sure what is the correct 'fix'. As far as I am aware, the file part of URI are case sensitive and always have been. The web server does not have a simple configuration switch to turn on case insensitive URLs. If we want the web server to treat them as case insensitive, then we have to add costly regular expression rewrite rules to the URLs sent to the server. We would also need to verify this won't break anything - it is possible some links depend on the server treating URLs as case sensitive. It isn't clear to me what the chain of events have been here. Is it that we changed the case on our server or is it simply people have put out link which arfe wrong? Is this the result of some change in the way the HTML manual is generated or a change in how org generates HTML or is this just the result of people posting the wrong URL? BTW this is not something I have the access to change as it would involve changes to the actual web server configuration. It isn't something which can be 'fixed' at the Org level.
Re: [BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Rudolf Adamkovič writes: > I noticed that the Org documentation server gives 404 Not Found for a > large number of links published all over the internet because (1) it > parses URLs case-sensitively and (2) the case has changed at some point. > I stumble upon such 404s errors daily. See, for instance, the link to > Column Groups documentation in this Stack Overflow answer: > > https://stackoverflow.com/a/8570307/1306956. > > I know that Org does not compete for popularity, but for the people > getting familiar with it, this issue might cause a lot of headaches for > no good reason. Thanks for the heads up! I am CCing Tim Cross. He is currently doing some work on our server. He is more likely to know how to fix this. Best, Ihor
[BUG] The documentation webserver gives 404s [9.5.4 (release_9.5.4-3-g6dc785 @ )]
Hello again! I noticed that the Org documentation server gives 404 Not Found for a large number of links published all over the internet because (1) it parses URLs case-sensitively and (2) the case has changed at some point. I stumble upon such 404s errors daily. See, for instance, the link to Column Groups documentation in this Stack Overflow answer: https://stackoverflow.com/a/8570307/1306956. I know that Org does not compete for popularity, but for the people getting familiar with it, this issue might cause a lot of headaches for no good reason. Rudy Emacs : GNU Emacs 29.0.50 (build 20, aarch64-apple-darwin21.5.0, NS appkit-2113.50 Version 12.4 (Build 21F79)) of 2022-07-02 Package: Org mode version 9.5.4 (release_9.5.4-3-g6dc785 @ ) -- "Logic is a science of the necessary laws of thought, without which no employment of the understanding and the reason takes place." -- Immanuel Kant, 1785 Rudolf Adamkovič [he/him] Studenohorská 25 84103 Bratislava Slovakia