bug#26302: Deploying the i18n’d web site

2020-07-13 Thread pelzflorian (Florian Pelz) 
On Mon, Jul 13, 2020 at 04:48:00PM +0200, pelzflorian (Florian Pelz) wrote:
> On Mon, Jul 13, 2020 at 03:22:36PM +0200, Ludovic Courtès wrote:
> > I’m happy to press the red button on berlin when we feel ready.  Perhaps
> > we can synchronize on IRC in case things go wrong.
> > 
> > Thanks,
> > Ludo’.
> 
> I am currently at my grandma’s and unable to be on IRC most of the time.
> I will try to watch IRC now.
> I believe the berlin last patch works and suggest reverting otherwise.
> 
> Regards,
> Florian

No sorry, I would prefer not to do synchronous communication in the near future.

I am confident in my latest patch with explicit redirects though.

Regards,
Florian

bug#26302: Deploying the i18n’d web site

2020-07-13 Thread pelzflorian (Florian Pelz) 
On Mon, Jul 13, 2020 at 03:22:36PM +0200, Ludovic Courtès wrote:
> Looks like we’re pretty much ready, no?

I think we are.

> I’m happy to press the red button on berlin when we feel ready.  Perhaps
> we can synchronize on IRC in case things go wrong.
> 
> Thanks,
> Ludo’.

I am currently at my grandma’s and unable to be on IRC most of the time.
I will try to watch IRC now.
I believe the berlin last patch works and suggest reverting otherwise.

Regards,
Florian

bug#26302: Deploying the i18n’d web site

2020-07-13 Thread Ludovic Courtès 
Hey!

"pelzflorian (Florian Pelz)"  skribis:

> On Thu, Jul 09, 2020 at 05:56:43PM +0100, Christopher Baines wrote:
>> Thanks for your continued time working on this Florian. I've made a
>> little bit of progress now, I've taken the wip-i18n branch, applied the
>> patch attached to this email and deployed that at [1].
>> 
>> 1: http://guix-website-test.cbaines.net/
>> 
>> This isn't a close test of the configuration for berlin, but might come
>> in useful when testing the NGinx configuration.
>> 
>> Thanks,
>> 
>> Chris
>
> This is a nice replica but with all the same issues.  Have you made
> improvements that should be added to my last patch for
> guix-maintenance before it is deployed?

Looks like we’re pretty much ready, no?

I’m happy to press the red button on berlin when we feel ready.  Perhaps
we can synchronize on IRC in case things go wrong.

Thanks,
Ludo’.

bug#26302: Deploying the i18n’d web site

2020-07-12 Thread pelzflorian (Florian Pelz) 
On Thu, Jul 09, 2020 at 03:09:57PM +0200, Ludovic Courtès wrote:
> "pelzflorian (Florian Pelz)"  skribis:
> > Still failing:
> >
> > http://guix.gnu.org/graphics
> >
> > http://guix.gnu.org/blog/2013/back-from-the-european-lisp-symposium
> >
> > worked before wip-i18n but stopped working.  Hrm.
> 
> What does nginx’s error.log file say?

I forgot to try.  error.log contains a 404 message:

2020/07/12 08:09:14 [error] 321#0: *1 open() "/srv/guix.gnu.org/graphics" 
failed (2: No such file or directory), client: 172.28.112.1, server: 
guix.gnu.org, request: "GET /graphics HTTP/1.1", host: "guix.gnu.org"
2020/07/12 08:09:14 [error] 321#0: *1 open() "/srv/guix.gnu.org/favicon.ico" 
failed (2: No such file or directory), client: 172.28.112.1, server: 
guix.gnu.org, request: "GET /favicon.ico HTTP/1.1", host: "guix.gnu.org"

Note that only directories /srv/guix.gnu.org/de/graphics and
/srv/guix.gnu.org/en/graphics exist and no longer
/srv/guix.gnu.org/graphics.

I don’t have ideas for rewriting, because URLs like
 should not become
.  One could make /en/ go to the
/srv/guix.gnu.org directory, but then 
would always be in English.  Or for old URLs use my last proposed patch:

On Fri, Jul 10, 2020 at 07:28:08PM +0200, pelzflorian (Florian Pelz) wrote:
> The patch introduces a long list of explicit redirects for all URLs
> not ending in a slash (except for 
> URLs, they are too many).

And for new URLs never use  without a slash
at the end but only .

Regards,
Florian

bug#26302: Deploying the i18n’d web site

2020-07-10 Thread pelzflorian (Florian Pelz) 
On Thu, Jul 09, 2020 at 05:56:43PM +0100, Christopher Baines wrote:
> Thanks for your continued time working on this Florian. I've made a
> little bit of progress now, I've taken the wip-i18n branch, applied the
> patch attached to this email and deployed that at [1].
> 
> 1: http://guix-website-test.cbaines.net/
> 
> This isn't a close test of the configuration for berlin, but might come
> in useful when testing the NGinx configuration.
> 
> Thanks,
> 
> Chris

This is a nice replica but with all the same issues.  Have you made
improvements that should be added to my last patch for
guix-maintenance before it is deployed?

Regards,
Florian

bug#26302: Deploying the i18n’d web site

2020-07-10 Thread pelzflorian (Florian Pelz) 
Find attached a desperate patch for guix-maintenance that works around
all listed issues, perhaps not in a nice way.

On Thu, Jul 09, 2020 at 04:48:43PM +0200, pelzflorian (Florian Pelz) wrote:
> With what I currently have
> redirection explodes
> 
> http://guix.gnu.org/manual/html_node/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/html_node
> 
> !! I think this happened too back then.  I have not investigated this yet.
>

This is fixed by redirecting not to relative paths,

(redirect "/manual/html_node/Substitutes.html" 
"../en/html_node/Substitutes.html")

but to absolute paths

(redirect "/manual/html_node/Substitutes.html" 
"/manual/en/html_node/Substitutes.html")

I think this issue existed before any of my i18n changes.


> Previously when visiting
> 
> http://guix.gnu.org/graphics
> 
> then nginx too looked up the index file
> 
> http://guix.gnu.org/graphics/index.html
> 
> This broke.  “rewrite (.*)/$ $1/index.html;” had not fixed it.
> 
> !! I do not know what to do about it.


The patch introduces a long list of explicit redirects for all URLs
not ending in a slash (except for 
URLs, they are too many).  This is an unmaintainable solution if we
want to keep using URLs not ending in a slash.  If we don’t want that,
then if you agree all is ready, please deploy the i18n’d site by
applying this patch to guix-maintenance and shortly thereafter
merge/rebase the guix-artworks wip-i18n branch (shortly because
redirects won’t work in the meantime).

Regards,
Florian
>From 2506c25468b3e2bd07b3a4b6f0d8b21173ecd65e Mon Sep 17 00:00:00 2001
From: Florian Pelz 
Date: Fri, 10 Jul 2020 19:07:32 +0200
Subject: [PATCH 1/2] berlin: Redirect to localized website by browser language
 settings.

* hydra/nginx/berlin.scm (%nginx-configuration): Load required nginx
dynamic module.
(%extra-content): Set $lang variable with it.
(guix.gnu.org-locations): Redirect html URLs according to $lang.  Use
absolute target URLs.  Add redirects for URLs that do not end in a
slash.  Fix a redirect for the Power Management Services manual entry.
(%berlin-servers): Rewrite to index files.
(redirect): Give redirect locations highest priority.
---
 hydra/nginx/berlin.scm | 735 -
 1 file changed, 510 insertions(+), 225 deletions(-)

diff --git a/hydra/nginx/berlin.scm b/hydra/nginx/berlin.scm
index 8c90eb1..e461b5b 100644
--- a/hydra/nginx/berlin.scm
+++ b/hydra/nginx/berlin.scm
@@ -14,7 +14,7 @@
 
 (define (redirect old new)
   (nginx-location-configuration
-   (uri old)
+   (uri (string-append "= " old)) ;= means highest priority
(body (list (string-append "return 301 " new ";\n")
 
 (define (publish-locations url)
@@ -196,235 +196,505 @@ PUBLISH-URL."
;; available at gnu.org/s/guix--e.g.,
;; .
(redirect "/news/feed.xml" "/feeds/blog.atom")
-   (redirect "/news/porting-guix-and-guixsd.html" 
"/blog/2015/porting-guix-and-guixsd")
-   (redirect "/news/gnu-guix-welcomes-three-students-for-gsoc.html" 
"/blog/2015/gnu-guix-welcomes-three-students-for-gsoc")
-   (redirect "/news/gnu-guix-recruits-for-gsoc.html" 
"/blog/2015/gnu-guix-recruits-for-gsoc")
-   (redirect "/news/one-week-to-fosdem.html" "/blog/2014/one-week-to-fosdem")
-   (redirect "/news/gnu-dmd-02-released.html" "/blog/2014/gnu-dmd-02-released")
-   (redirect "/news/emacs-as-a-general-purpose-package-manager.html" 
"/blog/2014/emacs-as-a-general-purpose-package-manager")
-   (redirect "/news/join-gnu-guix-for-gsoc-2017.html" 
"/blog/2017/join-gnu-guix-for-gsoc-2017")
-   (redirect "/news/gnu-guix-05-released.html" 
"/blog/2013/gnu-guix-05-released")
-   (redirect "/news/guix-at-the-2014-gnu-hackers-meeting.html" 
"/blog/2014/guix-at-the-2014-gnu-hackers-meeting")
-   (redirect "/news/state-of-aarch64-on-guix.html" 
"/blog/2017/state-of-aarch64-on-guix")
-   (redirect "/news/coming-events.html" "/blog/2017/coming-events")
-   (redirect "/news/gnu-dmd-01-released.html" "/blog/2013/gnu-dmd-01-released")
-   (redirect "/news/announcing-guix-hpc.html" "/blog/2017/announcing-guix-hpc")
-   (redirect "/news/gnu-guix-looks-for-gsoc-students.html" 
"/blog/2014/gnu-guix-looks-for-gsoc-students")
-   (redirect "/news/guix-at-the-european-lisp-symposium.html" 
"/blog/2013/guix-at-the-european-lisp-symposium")
-   (redirect "/news/gnu-guix-08-released.html" 
"/blog/2014/gnu-guix-08-released")
-   (redirect "/news/gnu-guix-090-released.html" 
"/blog/2015/gnu-guix-090-released")
-   (redirect "/news/index.html" "/blog/")
-   (redirect "/news/gnu-guix-welcomes-four-students-for-gsoc.html" 
"/blog/2016/gnu-guix-welcomes-four-students-for-gsoc")
-   (redirect "/news/gnu-guix-081-released.html" 
"/blog/2015/gnu-guix-081-released")
-   (redirect "/news/timely-delivery-of-security-updates.html" 
"/blog/2016/timely-delivery-of-security-updates")
-   (redirect "/news/guix-at-openbio-codefest-2014.html"

bug#26302: Deploying the i18n’d web site

2020-07-09 Thread Christopher Baines 

pelzflorian (Florian Pelz)  writes:

> Sorry, I forgot to address the patch tracker.  I wrote some hours ago:
>
> On Thu, Apr 09, 2020 at 07:31:04PM +0200, pelzflorian (Florian Pelz) wrote:
>> On Thu, Apr 09, 2020 at 05:27:05PM +0200, Ludovic Courtès wrote:
>> > Hi Florian,
>> >
>> > "pelzflorian (Florian Pelz)"  skribis:
>> >
>> > > +   (redirect "/news/porting-guix-and-guixsd.html" 
>> > > "/$lang/blog/2015/porting-guix-and-guixsd")
>> >
>> > Does that mean that /blog/2015/porting-guix-and-guixsd (without /LANG)
>> > will _not_ redirect to /LANG/blog/2015/porting-guix-and-guixsd?
>> >
>> > It’s important that all the current URL, without /LANG, remain valid.
>>
>> With the new test VM, not all is working.
>>
>> /news/porting-guix-and-guixsd.html fails (code 404).
>>
>> /blog/2015/porting-guix-and-guixsd fails (code 404).
>>
>> /blog/2015/porting-guix-and-guixsd fails (404).
>>
>> But /blog/2015/porting-guix-and-guixsd/ works fine.
>>
>> Well this is difficult to figure out.
>>
>> Regards,
>> Florian
>
> An update:
>
> The attached patch for berlin serves more but not all URLs
> properly when testing on a VM.
>
> I found one problem; the nginx locations for redirecting old URLs can
> be given a higher priority via specifying = before the location path.
>
> I am sorry for neglecting this for so long until Christopher Baines
> offered to help a few days ago.  Now I too started investigating
> myself again.

Thanks for your continued time working on this Florian. I've made a
little bit of progress now, I've taken the wip-i18n branch, applied the
patch attached to this email and deployed that at [1].

1: http://guix-website-test.cbaines.net/

This isn't a close test of the configuration for berlin, but might come
in useful when testing the NGinx configuration.

Thanks,

Chris


signature.asc
Description: PGP signature

bug#26302: Deploying the i18n’d web site

2020-07-09 Thread pelzflorian (Florian Pelz) 
The trouble is that I do not have an understanding in what order nginx
tries which redirections/rewrites.  An understanding is needed instead
of investigating dead ends and 3rd party nginx modules.

What I have done a while ago (the berlin patch for guix-maintenance
from my last e-mail contains this):

To redirect accesses only to HTML files I had added

(nginx-location-configuration
 (uri "~ (.html|.htm)$")
 (body (list "try_files $uri /$lang/$uri /$lang/$uri/index.html =404;")))

However, this does not match when nginx redirects URLs like

http://guix.gnu.org/graphics/

to the index file

http://guix.gnu.org/graphics/index.html


For this reason I had added

rewrite (.*)/$ $1/index.html;

Then it matched.  But:

> > Still failing:
> >
> > http://guix.gnu.org/graphics
> >
> > http://guix.gnu.org/blog/2013/back-from-the-european-lisp-symposium
> >
> > worked before wip-i18n but stopped working.  Hrm.

Previously when visiting

http://guix.gnu.org/graphics

then nginx too looked up the index file

http://guix.gnu.org/graphics/index.html

This broke.  “rewrite (.*)/$ $1/index.html;” had not fixed it.

!! I do not know what to do about it.



My last change addressed this:

On Thu, Jul 09, 2020 at 03:09:57PM +0200, Ludovic Courtès wrote:
> "pelzflorian (Florian Pelz)"  skribis:
> > I found one problem; the nginx locations for redirecting old URLs can
> > be given a higher priority via specifying = before the location path.
> 
> One thing that bit me in the past is that regex locations have higher
> precedence that other locations, IIRC.

Yes, I think this is what happened, the

(nginx-location-configuration
 (uri "~ (.html|.htm)$")
 (body (list "try_files $uri /$lang/$uri /$lang/$uri/index.html =404;")))

was run before the

location /news/gnu-dmd-01-released.html {
  return 301 /blog/2013/gnu-dmd-01-released;
}

and therefore no return was performed.


Changing it to

location = /news/gnu-dmd-01-released.html {
  return 301 /$lang/blog/2013/gnu-dmd-01-released;
}

with = in my last described attempt fixed this.  Because the location
uri does not end in a slash, using = does not make a difference when
matching, but gives higher priority.


> > I cleared the browser cache, restarted nscd and tested these URLs
> > (with a changed /etc/hosts file pointing guix.gnu.org to the VM):
> 
> I guess you could check with “wget -v -O /dev/null” or similar, so you
> can be sure there’s no client cache interfering.

This is a good idea.  In the past I had thought things work when in
reality all was broken and it was just cached.



> If you don’t have the manual at hand, you can just make sure you’re
> getting the expected redirect, even if the end result is 404.

You are right, trying to build the manual was pointless.

> > Still failing:
> >
> > http://guix.gnu.org/graphics
> >
> > http://guix.gnu.org/blog/2013/back-from-the-european-lisp-symposium
> >
> > worked before wip-i18n but stopped working.  Hrm.
> What does nginx’s error.log file say?

I can only check later, I have deleted my VM because texinfo for
building the manual consumed too much disk space.




> > http://guix.gnu.org/manual/html_node/Power-management-Services.html
> 

The URL should have been

http://guix.gnu.org/manual/html_node/Power-Management-Services.html

with capital M.  But the old config has the wrong URL as well I think.


I have made some wrong changes since my last mail.  Will go back and
rebuild the VM from my last mail now.  With what I currently have
redirection explodes

http://guix.gnu.org/manual/html_node/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/html_node

!! I think this happened too back then.  I have not investigated this yet.




> I’d be happy to go ahead and deploy this so maybe let’s see and hammer
> down those remaining issues and then we can profit!  Let us know how we
> can help!
> 
> Thanks,
> Ludo’.

A solution for the two problems I marked with !! might be important.

Other than that, I would be very happy if first the berlin patch to
guix-maintenance and then after that the wip-i18n branch finally would
go to master.

Regards,
Florian

bug#26302: Deploying the i18n’d web site

2020-07-09 Thread Ludovic Courtès 
Hi Florian,

"pelzflorian (Florian Pelz)"  skribis:

> The attached patch for berlin serves more but not all URLs
> properly when testing on a VM.
>
> I found one problem; the nginx locations for redirecting old URLs can
> be given a higher priority via specifying = before the location path.

One thing that bit me in the past is that regex locations have higher
precedence that other locations, IIRC.

> I am sorry for neglecting this for so long until
> Christopher Baines offered to help a few days ago.  Now I too started
> investigating myself again.
>
> I cleared the browser cache, restarted nscd and tested these URLs
> (with a changed /etc/hosts file pointing guix.gnu.org to the VM):

I guess you could check with “wget -v -O /dev/null” or similar, so you
can be sure there’s no client cache interfering.

> Still failing:
>
> http://guix.gnu.org/graphics
>
> http://guix.gnu.org/blog/2013/back-from-the-european-lisp-symposium
>
> worked before wip-i18n but stopped working.  Hrm.

What does nginx’s error.log file say?

> These seem to fail but I could not properly build the manual yet:
>
> http://guix.gnu.org/manual/en/html_node/Miscellaneous-Services.html
>
> http://guix.gnu.org/manual/html_node/Power-management-Services.html

If you don’t have the manual at hand, you can just make sure you’re
getting the expected redirect, even if the end result is 404.

> The rest looks good:
>
> http://guix.gnu.org/news/timely-delivery-of-security-updates.html
>
> http://guix.gnu.org/security/
>
> http://guix.gnu.org/blog/2016/back-from-the-gnu-hackers-meeting-2016/
>
> http://guix.gnu.org/en/blog/2017/back-from-fosdem-2017
>
> http://guix.gnu.org/de/blog/2016/back-from-gbcuw-2016/
>
> works.
>
> http://guix.gnu.org/news/coming-events
>
> http://guix.gnu.org/news
>
> never worked, so it’s OK that these URLs don’t work.

Sounds good.

> http://guix.gnu.org/news/
>
> This redirect now works but did not work before wip-i18n (??).

Nice.

I’d be happy to go ahead and deploy this so maybe let’s see and hammer
down those remaining issues and then we can profit!  Let us know how we
can help!

Thanks,
Ludo’.

bug#26302: Deploying the i18n’d web site

2020-07-05 Thread pelzflorian (Florian Pelz) 
Sorry, I forgot to address the patch tracker.  I wrote some hours ago:

On Thu, Apr 09, 2020 at 07:31:04PM +0200, pelzflorian (Florian Pelz) wrote:
> On Thu, Apr 09, 2020 at 05:27:05PM +0200, Ludovic Courtès wrote:
> > Hi Florian,
> > 
> > "pelzflorian (Florian Pelz)"  skribis:
> > 
> > > +   (redirect "/news/porting-guix-and-guixsd.html" 
> > > "/$lang/blog/2015/porting-guix-and-guixsd")
> > 
> > Does that mean that /blog/2015/porting-guix-and-guixsd (without /LANG)
> > will _not_ redirect to /LANG/blog/2015/porting-guix-and-guixsd?
> > 
> > It’s important that all the current URL, without /LANG, remain valid.
> 
> With the new test VM, not all is working.
> 
> /news/porting-guix-and-guixsd.html fails (code 404).
> 
> /blog/2015/porting-guix-and-guixsd fails (code 404).
> 
> /blog/2015/porting-guix-and-guixsd fails (404).
> 
> But /blog/2015/porting-guix-and-guixsd/ works fine.
> 
> Well this is difficult to figure out.
> 
> Regards,
> Florian

An update:

The attached patch for berlin serves more but not all URLs
properly when testing on a VM.

I found one problem; the nginx locations for redirecting old URLs can
be given a higher priority via specifying = before the location path.

I am sorry for neglecting this for so long until
Christopher Baines offered to help a few days ago.  Now I too started
investigating myself again.

I cleared the browser cache, restarted nscd and tested these URLs
(with a changed /etc/hosts file pointing guix.gnu.org to the VM):

Still failing:

http://guix.gnu.org/graphics

http://guix.gnu.org/blog/2013/back-from-the-european-lisp-symposium

worked before wip-i18n but stopped working.  Hrm.

These seem to fail but I could not properly build the manual yet:

http://guix.gnu.org/manual/en/html_node/Miscellaneous-Services.html

http://guix.gnu.org/manual/html_node/Power-management-Services.html


The rest looks good:

http://guix.gnu.org/news/timely-delivery-of-security-updates.html

http://guix.gnu.org/security/

http://guix.gnu.org/blog/2016/back-from-the-gnu-hackers-meeting-2016/

http://guix.gnu.org/en/blog/2017/back-from-fosdem-2017

http://guix.gnu.org/de/blog/2016/back-from-gbcuw-2016/

works.

http://guix.gnu.org/news/coming-events

http://guix.gnu.org/news

never worked, so it’s OK that these URLs don’t work.

http://guix.gnu.org/news/

This redirect now works but did not work before wip-i18n (??).


I will continue to investigate.

Regards,
Florian
>From 59a82c819ff987a598f14d87ea7524f6c4451276 Mon Sep 17 00:00:00 2001
From: Florian Pelz 
Date: Sat, 4 Jul 2020 21:14:19 +0200
Subject: [PATCH 1/2] berlin: Redirect to localized website by browser language
 settings.

* hydra/nginx/berlin.scm (%nginx-configuration): Load required nginx
dynamic module.
(%extra-content): Set $lang variable with it.
(guix.gnu.org-locations): Redirect html URLs.
(%berlin-servers): Rewrite to index files.
(redirect): Give redirect locations highest priority.
---
 hydra/nginx/berlin.scm | 149 +++--
 1 file changed, 82 insertions(+), 67 deletions(-)

diff --git a/hydra/nginx/berlin.scm b/hydra/nginx/berlin.scm
index 8c90eb1..aa0a857 100644
--- a/hydra/nginx/berlin.scm
+++ b/hydra/nginx/berlin.scm
@@ -14,7 +14,7 @@
 
 (define (redirect old new)
   (nginx-location-configuration
-   (uri old)
+   (uri (string-append "= " old)) ;= means highest priority
(body (list (string-append "return 301 " new ";\n")
 
 (define (publish-locations url)
@@ -196,72 +196,72 @@ PUBLISH-URL."
;; available at gnu.org/s/guix--e.g.,
;; .
(redirect "/news/feed.xml" "/feeds/blog.atom")
-   (redirect "/news/porting-guix-and-guixsd.html" 
"/blog/2015/porting-guix-and-guixsd")
-   (redirect "/news/gnu-guix-welcomes-three-students-for-gsoc.html" 
"/blog/2015/gnu-guix-welcomes-three-students-for-gsoc")
-   (redirect "/news/gnu-guix-recruits-for-gsoc.html" 
"/blog/2015/gnu-guix-recruits-for-gsoc")
-   (redirect "/news/one-week-to-fosdem.html" "/blog/2014/one-week-to-fosdem")
-   (redirect "/news/gnu-dmd-02-released.html" "/blog/2014/gnu-dmd-02-released")
-   (redirect "/news/emacs-as-a-general-purpose-package-manager.html" 
"/blog/2014/emacs-as-a-general-purpose-package-manager")
-   (redirect "/news/join-gnu-guix-for-gsoc-2017.html" 
"/blog/2017/join-gnu-guix-for-gsoc-2017")
-   (redirect "/news/gnu-guix-05-released.html" 
"/blog/2013/gnu-guix-05-released")
-   (redirect "/news/guix-at-the-2014-gnu-hackers-meeting.html" 
"/blog/2014/guix-at-the-2014-gnu-hackers-meeting")
-   (redirect "/news/state-of-aarch64-on-guix.html" 
"/blog/2017/state-of-aarch64-on-guix")
-   (redirect "/news/coming-events.html" "/blog/2017/coming-events")
-   (redirect "/news/gnu-dmd-01-released.html" "/blog/2013/gnu-dmd-01-released")
-   (redirect "/news/announcing-guix-hpc.html" "/blog/2017/announcing-guix-hpc")
-   (redirect "/news/gnu-guix-looks-for-gsoc-students.html" 
"/blog/2014/gnu-guix-looks-for-gsoc-students")
-   (redirect

bug#26302: Deploying the i18n’d web site

2020-07-01 Thread Christopher Baines 

pelzflorian (Florian Pelz)  writes:

> On Thu, Apr 09, 2020 at 08:58:14PM +0200, Bengt Richter wrote:
>> Wondering, could dnsmasq help?
>
> I am already using dnsmasq to access a virtual machine of the machine
> hosting the Guix website so  can replace the
> Guix website while old URLs keep working.

...

> I am confused by all the ways to rewrite, redirect and try_files in
> nginx.  I would be happy if others with more knowledge could help.

Hey Florian,

I'm wondering what the current state of the Guix website translation
work is?

I can see the translated website is available at [1], but maybe not with
the full NGinx configuration needed. I also managed to use haunt
locally, but I noticed there were some merge conflicts between the
wip-i18n and master branch.

1: http://guix.gnu.org/.i18n/en/

I can try and help with the NGinx stuff, I've muddled through
configuring NGinx in the past. I think it might be useful to test
without the /.i18n prefix, maybe berlin can be changed to serve the site
at wip-i18n.guix.gnu.org. I also have a server where I might try
deploying the translated website for testing.

Thanks,

Chris


signature.asc
Description: PGP signature

bug#26302: Deploying the i18n’d web site

2020-04-09 Thread pelzflorian (Florian Pelz) 
On Thu, Apr 09, 2020 at 08:58:14PM +0200, Bengt Richter wrote:
> Wondering, could dnsmasq help?

I am already using dnsmasq to access a virtual machine of the machine
hosting the Guix website so  can replace the
Guix website while old URLs keep working.

https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/berlin.scm.

I configured dnsmasq with NetworkManager.

(network-manager-service-type config =>
  (network-manager-configuration
(inherit config)
(dns "dnsmasq")
(vpn-plugins (list network-manager-openconnect)))

Also I made the network manager configuration as described in the Guix
manual.

I made the attached modifications to berlin.scm so I can test without
the VM starting to build the world.

Then I changed my /etc/hosts

127.0.0.1 localhost florianmacbook
::1   localhost florianmacbook
172.28.112.244 guix.gnu.org guix.gnu.org

Then I made the changes from my previous patch, except those
nginx-server-configuration that I removed because I cannot test TLS.

I compile the VM using

~/git/maintenance$ GUILE_LOAD_PATH=$GUILE_LOAD_PATH:$(pwd)/modules guix system 
vm-image --image-size=5G --fallback 
/home/florian/git/maintenance/hydra/berlin.scm

(More than --image-size=5G to be safe.)  I copy the resulting image
and name it berlin1.img, make it writable and run it via

qemu-system-x86_64 -enable-kvm berlin1.img -m 2048 -nic 
tap,ifname=tap0,script=no,downscript=no

When running the virtual machine, I log in as root and do “herd stop
mcron” so the VM does not get overloaded building all kinds of things
I do not need.  I run the command from “herd schedule mcron” to build
the guix.gnu.org website manually once.

I am confused by all the ways to rewrite, redirect and try_files in
nginx.  I would be happy if others with more knowledge could help.

Regards,
Florian
>From b914a165a2e9b9b7bd7760cec09a531c70611141 Mon Sep 17 00:00:00 2001
From: Florian Pelz 
Date: Wed, 8 Apr 2020 00:44:18 +0200
Subject: [PATCH 1/2] [various changes for local testing]

---
 hydra/berlin.scm|  80 +
 hydra/modules/sysadmin/services.scm |  39 +
 hydra/nginx/berlin.scm  | 241 +---
 3 files changed, 7 insertions(+), 353 deletions(-)

diff --git a/hydra/berlin.scm b/hydra/berlin.scm
index ee828d8..529f4f0 100644
--- a/hydra/berlin.scm
+++ b/hydra/berlin.scm
@@ -31,7 +31,7 @@
  (ssh-public-key (local-file "keys/ssh/nckx.pub")
 
 ;;; XXX: relative includes don't seem to work with Guile 3.0
-(include "/root/maintenance/hydra/nginx/berlin.scm")
+(include "/home/florian/git/maintenance/hydra/nginx/berlin.scm")
 
 
 ;;;
@@ -116,29 +116,9 @@ Happy hacking!\n"))
  ;; The root file system resides on just a single
  ;; disk, no RAID :-/
  (file-system
-   (device (file-system-label "my-root"))
+   (device "/dev/sda")
(mount-point "/")
(type "ext4"))
- ;; This is a large external storage array
- ;; connected via 2 HBA cards.  We only mount it
- ;; through one of the HBA cards.  We would need
- ;; to use multipathd otherwise.
- (file-system
-   (device (uuid "a6455b66-59d2-40bd-bddb-0c572bb62a2f"))
-   (mount-point "/gnu")
-   (type "ext4"))
- ;; Bind mount cache to large external storage.
- (file-system
-   (device "/gnu/cache")
-   (mount-point "/var/cache")
-   (flags '(bind-mount))
-   (type "none"))
- ;; Access root file system without bind mounts.
- (file-system
-   (device "/")
-   (mount-point "/mnt/root-fs")
-   (flags '(bind-mount))
-   (type "none"))
  %base-file-systems))
 
   ;; Local admin account for MDC maintenance.
@@ -164,52 +144,12 @@ Happy hacking!\n"))
  activation-service-type
  %copy-kernel-and-initrd)
 
- ;; Connection to the DMZ for public access
- ;; This is a 10G port.
- (static-networking-service "enp129s0f0"
-"141.80.181.40"
-#:netmask "255.255.255.0"
-#:gateway "141.80.181.1")
- ;; Connection to build nodes
- (static-networking-service "enp129s0f1"
-"141.80.167.131"
-#:netmask "255.255.255.192")
-
+ (service dhcp-client-service-type)
  ;; Allow login over serial console.
  (agetty-service (agetty-configuration
   (tty "ttyS0")
   (baud-rate

bug#26302: Deploying the i18n’d web site

2020-04-09 Thread Bengt Richter 
Hi Florian,

On +2020-04-09 19:31:02 +0200, pelzflorian (Florian Pelz) wrote:
> On Thu, Apr 09, 2020 at 05:27:05PM +0200, Ludovic Courtès wrote:
> > Hi Florian,
> > 
> > "pelzflorian (Florian Pelz)"  skribis:
> > 
> > > +   (redirect "/news/porting-guix-and-guixsd.html" 
> > > "/$lang/blog/2015/porting-guix-and-guixsd")
> > 
> > Does that mean that /blog/2015/porting-guix-and-guixsd (without /LANG)
> > will _not_ redirect to /LANG/blog/2015/porting-guix-and-guixsd?
> > 
> > It’s important that all the current URL, without /LANG, remain valid.
> 
> With the new test VM, not all is working.
> 
> /news/porting-guix-and-guixsd.html fails (code 404).
> 
> /blog/2015/porting-guix-and-guixsd fails (code 404).
> 
> /blog/2015/porting-guix-and-guixsd fails (404).
> 
> But /blog/2015/porting-guix-and-guixsd/ works fine.
> 
> Well this is difficult to figure out.
>

Wondering, could dnsmasq help?

> Regards,
> Florian
> 
> 
> 

-- 
Regards,
Bengt Richter

bug#26302: Deploying the i18n’d web site

2020-04-09 Thread pelzflorian (Florian Pelz) 
On Thu, Apr 09, 2020 at 05:27:05PM +0200, Ludovic Courtès wrote:
> Hi Florian,
> 
> "pelzflorian (Florian Pelz)"  skribis:
> 
> > +   (redirect "/news/porting-guix-and-guixsd.html" 
> > "/$lang/blog/2015/porting-guix-and-guixsd")
> 
> Does that mean that /blog/2015/porting-guix-and-guixsd (without /LANG)
> will _not_ redirect to /LANG/blog/2015/porting-guix-and-guixsd?
> 
> It’s important that all the current URL, without /LANG, remain valid.

With the new test VM, not all is working.

/news/porting-guix-and-guixsd.html fails (code 404).

/blog/2015/porting-guix-and-guixsd fails (code 404).

/blog/2015/porting-guix-and-guixsd fails (404).

But /blog/2015/porting-guix-and-guixsd/ works fine.

Well this is difficult to figure out.

Regards,
Florian

bug#26302: Deploying the i18n’d web site

2020-04-09 Thread Ludovic Courtès 
Hi Florian,

"pelzflorian (Florian Pelz)"  skribis:

> +   (redirect "/news/porting-guix-and-guixsd.html" 
> "/$lang/blog/2015/porting-guix-and-guixsd")

Does that mean that /blog/2015/porting-guix-and-guixsd (without /LANG)
will _not_ redirect to /LANG/blog/2015/porting-guix-and-guixsd?

It’s important that all the current URL, without /LANG, remain valid.

Ludo’.

bug#26302: Deploying the i18n’d web site

2020-04-09 Thread pelzflorian (Florian Pelz) 
On Thu, Apr 09, 2020 at 05:21:46AM +0200, pelzflorian (Florian Pelz) wrote:
> but I could not yet test building the
> website i.e. if the redirects really work correctly.

It seems redirects broke like

http://guix.gnu.org/news/gnu-dmd-01-released.html

on my test VM. :/

bug#26302: Deploying the i18n’d web site

2020-04-08 Thread pelzflorian (Florian Pelz) 
On Wed, Apr 08, 2020 at 12:02:25AM +0200, pelzflorian (Florian Pelz) wrote:
> set_from_accept_language should better be a global directive I think.
> 

No, I was wrong.  set_from_accept_language must only appear once, but
not in a global directive.  I put it in %extra-content for the
nginx-configuration now.  It seems the attached patch works (after
removing the rottlog service), but I could not yet test building the
website i.e. if the redirects really work correctly.  It is so slow.

Then again, better break the website before the release than after (if
it does not work).  I cannot test tls and certbot anyway.

Regards,
Florian
diff --git a/hydra/nginx/berlin.scm b/hydra/nginx/berlin.scm
index 303fd35..a57640e 100644
--- a/hydra/nginx/berlin.scm
+++ b/hydra/nginx/berlin.scm
@@ -461,20 +461,24 @@ PUBLISH-URL."
 (body (list "alias /srv/guix.gnu.org-i18n;")))
 
(git-http-nginx-location-configuration
 (git-http-configuration))
 
;; For Hurd bootstrap binaries.
(nginx-location-configuration
 (uri "/guix")
 (body (list "root /var/www;")))
 
+   (nginx-location-configuration
+(uri "~ (.html|.htm)$")
+(body (list "try_files /$lang/$uri $uri =404;")))
+
(nginx-location-configuration  ;certbot
 (uri "/.well-known")
 (body (list "root /var/www;")
 
 (define guix.info-locations
   (list (nginx-location-configuration ;certbot
  (uri "~ ^/\\.well-known")
  (body (list "root /var/www;")))
 (nginx-location-configuration
  (uri "~ /(.*)")
@@ -507,31 +511,33 @@ PUBLISH-URL."
  (list
   "access_log /var/log/nginx/bootstrappable.access.log;")))
 
(nginx-server-configuration
 (listen '("80"))
 (server-name '("guix.gnu.org"))
 (root "/srv/guix.gnu.org")
 (locations guix.gnu.org-locations)
 (raw-content
  (list
+  "rewrite (.*)/$ $1/index.html;"
   "access_log /var/log/nginx/guix-info.access.log;")))
 
(nginx-server-configuration
 (listen '("80"))
 (server-name '("guix.info"
"www.guix.info"))
 (locations guix.info-locations)
 (raw-content
  (append
   %tls-settings
   (list
+   "rewrite (.*)/$ $1/index.html;"
"access_log /var/log/nginx/guix-info.https.access.log;"
 
(nginx-server-configuration
 (listen '("80"))
 (server-name '("issues.guix.info"
"issues.guix.gnu.org"))
 (root "/home/rekado/mumi/")
 (locations
  (list (nginx-location-configuration ;certbot
 (uri "/.well-known")
@@ -614,33 +620,35 @@ PUBLISH-URL."
 (listen '("443 ssl"))
 (server-name '("guix.info"
"www.guix.info"))
 (ssl-certificate (le "guix.info"))
 (ssl-certificate-key (le "guix.info" 'key))
 (locations guix.info-locations)
 (raw-content
  (append
   %tls-settings
   (list
+   "rewrite (.*)/$ $1/index.html;"
"access_log /var/log/nginx/guix-info.https.access.log;"
 
(nginx-server-configuration
 (listen '("443 ssl"))
 (server-name '("guix.gnu.org"))
 (ssl-certificate (le "guix.gnu.org"))
 (ssl-certificate-key (le "guix.gnu.org" 'key))
 (root "/srv/guix.gnu.org")
 (locations guix.gnu.org-locations)
 (raw-content
  (append
   %tls-settings
   (list
+   "rewrite (.*)/$ $1/index.html;"
"access_log /var/log/nginx/guix-gnu-org.https.access.log;"
 
(nginx-server-configuration
 (listen '("443 ssl"))
 (server-name '("issues.guix.info"))
 (ssl-certificate (le "issues.guix.info"))
 (ssl-certificate-key (le "issues.guix.info" 'key))
 (root "/home/rekado/mumi/")
 (locations
  (list (nginx-location-configuration
@@ -710,20 +718,22 @@ PUBLISH-URL."
"proxy_send_timeout  600;"
"proxy_read_timeout  600;"
"send_timeout600;"
"access_log /var/log/nginx/workflows-guix-info.https.access.log;"))
 
 (define %extra-content
   (list
"default_type  application/octet-stream;"
"sendfileon;"
 
+   "set_from_accept_language $lang en de;"
+
;; Maximum chunk size to send.  Partly this is a workaround for
;; , but also the nginx docs mention that
;; "Without the limit, one fast connection may seize the worker
;; process entirely."
;;  
"sendfile_max_chunk 1m;"
 
"keepalive_timeout  65;"
 
;; Use HTTP 1.1 to talk to the backend so we benefit from keep-alive
@@ -768,20 +778,25 @@ PUBLISH-URL."
"proxy_connect_timeout 7s;"
"proxy_read_timeout 10s;"
"proxy_send_timeout 10s;"
 
;; Cache timeouts for a little while to avoid increasing pressure.
"proxy_cache_valid 504 30s;"))
 
 (define %nginx-configuration
   (nginx-configuration
(server-blocks %berlin-servers)
+   (modules
+(list
+ ;; Module to redirect users to the

bug#26302: Deploying the i18n’d web site

2020-04-07 Thread pelzflorian (Florian Pelz) 
On Tue, Apr 07, 2020 at 11:18:56PM +0200, Ludovic Courtès wrote:
> I haven’t tested it though.  I’m a bit concerned about the rewrite rule,
> because there are bits that should not be rewritten, like:
> 
>   /manual (must not be: /LANG/manual, at least for now)
>   /guix-refcard.pdf
>   …
> 
> Will there be rewritten with the config below?

These should be OK.  Because it uses try_files, both /en/manual and
/manual should be checked.  guix-refcard.pdf does not end in .html and
therefore is not rewritten.

set_from_accept_language should better be a global directive I think.


> I know you’re already busy with the installer, Florian, so maybe we can
> postpone that after the release, unless someone else champions to test it!
> 
> Thanks,
> Ludo’.
> 

I will test, but I cannot test if certbot and tls work.  But of course
the translated website can wait.

Regards,
Florian

bug#26302: Deploying the i18n’d web site

2020-04-07 Thread Ludovic Courtès 
Hi Florian,

"pelzflorian (Florian Pelz)"  skribis:

>> What would it take to complete the i18n’d web site deployment?  It would
>> be great to get that done by the time of the next release.
>>
>
> set_from_accept_language $lang en de;
>
> would better be a global directive.  en comes first so it is the
> default.
>
> Otherwise it’s the changes you quoted.  Testing is difficult though.
> I tested without https and with some modifications to make berlin.scm
> work in a virtual machine.

I had to slightly adjust your changes, leading to the patch below.

I haven’t tested it though.  I’m a bit concerned about the rewrite rule,
because there are bits that should not be rewritten, like:

  /manual (must not be: /LANG/manual, at least for now)
  /guix-refcard.pdf
  …

Will there be rewritten with the config below?

I know you’re already busy with the installer, Florian, so maybe we can
postpone that after the release, unless someone else champions to test it!

Thanks,
Ludo’.

diff --git a/hydra/nginx/berlin.scm b/hydra/nginx/berlin.scm
index 303fd35..7e329fc 100644
--- a/hydra/nginx/berlin.scm
+++ b/hydra/nginx/berlin.scm
@@ -468,6 +468,10 @@ PUBLISH-URL."
 (uri "/guix")
 (body (list "root /var/www;")))
 
+   (nginx-location-configuration
+(uri "~ (.html|.htm)$")
+(body (list "try_files /$lang/$uri $uri =404;")))
+
(nginx-location-configuration  ;certbot
 (uri "/.well-known")
 (body (list "root /var/www;")
@@ -514,6 +518,8 @@ PUBLISH-URL."
 (locations guix.gnu.org-locations)
 (raw-content
  (list
+  "rewrite (.*)/$ $1/index.html;"
+  "set_from_accept_language $lang en de;"
   "access_log /var/log/nginx/guix-info.access.log;")))
 
(nginx-server-configuration
@@ -525,6 +531,8 @@ PUBLISH-URL."
  (append
   %tls-settings
   (list
+   "rewrite (.*)/$ $1/index.html;"
+   "set_from_accept_language $lang en de;"
"access_log /var/log/nginx/guix-info.https.access.log;"
 
(nginx-server-configuration
@@ -621,6 +629,8 @@ PUBLISH-URL."
  (append
   %tls-settings
   (list
+   "rewrite (.*)/$ $1/index.html;"
+   "set_from_accept_language $lang en de;"
"access_log /var/log/nginx/guix-info.https.access.log;"
 
(nginx-server-configuration
@@ -634,6 +644,8 @@ PUBLISH-URL."
  (append
   %tls-settings
   (list
+   "rewrite (.*)/$ $1/index.html;"
+   "set_from_accept_language $lang en de;"
"access_log /var/log/nginx/guix-gnu-org.https.access.log;"
 
(nginx-server-configuration
@@ -775,6 +787,11 @@ PUBLISH-URL."
 (define %nginx-configuration
   (nginx-configuration
(server-blocks %berlin-servers)
+   (modules
+(list
+ ;; Module to redirect users to the localized pages of their choice.
+ (file-append nginx-accept-language-module
+  "/etc/nginx/modules/ngx_http_accept_language_module.so")))
(global-directives
 ;; This is a 72-core machine, but let's not use all of them for nginx.
 '((worker_processes . 16)

bug#26302: Deploying the i18n’d web site

2020-03-25 Thread pelzflorian (Florian Pelz) 
On Thu, Mar 26, 2020 at 12:21:00AM +0100, pelzflorian (Florian Pelz) wrote:
> I will try rebasing wip-i18n to current master now.

Done.

bug#26302: Deploying the i18n’d web site

2020-03-25 Thread pelzflorian (Florian Pelz) 
On Wed, Mar 25, 2020 at 06:33:46PM +0100, Ludovic Courtès wrote:
> Hi Florian,
> 
> We dropped the ball on this, but it looks like we were close to the
> finish line.
> 
> What would it take to complete the i18n’d web site deployment?  It would
> be great to get that done by the time of the next release.
>

set_from_accept_language $lang en de;

would better be a global directive.  en comes first so it is the
default.

Otherwise it’s the changes you quoted.  Testing is difficult though.
I tested without https and with some modifications to make berlin.scm
work in a virtual machine.

I will try rebasing wip-i18n to current master now.

Regards,
Florian

bug#26302: Deploying the i18n’d web site

2020-03-25 Thread Ludovic Courtès 
Hi Florian,

We dropped the ball on this, but it looks like we were close to the
finish line.

What would it take to complete the i18n’d web site deployment?  It would
be great to get that done by the time of the next release.

Thanks!

Ludo’.

"pelzflorian (Florian Pelz)"  skribis:

> First on serving the internationalized website:
>
> The following changes to maintenance:berlin redirect requests for old
> URLs properly *only for non-HTTPS* in local testing on a vm-image.  I
> assume the same will work when added for the HTTPS location, though
> perhaps the line "set_from_accept_language $lang en de;" cannot or
> need not be duplicated for both non-HTTPS and HTTPS.  I hope the
> changes are compatible with the manual and cookbook URLs.  The changes
> would better be tested more but I do not know how.
>
>
> diff --git a/hydra/nginx/berlin.scm b/hydra/nginx/berlin.scm
> index 2947759..8b66ba7 100644
> --- a/hydra/nginx/berlin.scm
> +++ b/hydra/nginx/berlin.scm
> @@ -468,6 +468,10 @@ PUBLISH-URL."
>  (uri "/guix")
>  (body (list "root /var/www;")))
>  
> +   (nginx-location-configuration
> +(uri "~ (.html|.htm)$")
> +(body (list "try_files /$lang/$uri $uri =404;")))
> +
> (nginx-location-configuration  ;certbot
>  (uri "/.well-known")
>  (body (list "root /var/www;")
> @@ -505,6 +509,9 @@ PUBLISH-URL."
>  (root "/home/rekado/bootstrappable.org")
>  (raw-content
>   (list
> +  "rewrite (.*)/$ $1/index.html;"
> +  ;; put en first so it is the default:
> +  "set_from_accept_language $lang en de;"
>"access_log /var/log/nginx/bootstrappable.access.log;")))
>  
> (nginx-server-configuration
> @@ -767,5 +774,11 @@ PUBLISH-URL."
>  (define %nginx-configuration
>(nginx-configuration
> (server-blocks %berlin-servers)
> +   (modules
> +(list
> + ;; We need this module for redirecting users to the localized
> + ;; website of their choice.
> + (file-append nginx-accept-language-module "\
> +/etc/nginx/modules/ngx_http_accept_language_module.so")))
> (extra-content
>  (string-join %extra-content "\n"
>
>
>
> I attach a complete patch that can only be used for testing on a local
> VM.  For testing, I perform the following steps:
>
> cd ~/git/maintenance/hydra/
> GUILE_LOAD_PATH=$(readlink -f 
> ~/git/maintenance/hydra/modules):$GUILE_LOAD_PATH guix system vm-image 
> --image-size=14G berlin.scm --fallback
> cp /gnu/store/mm000wdzkzrvalg09jxk0y6nhi9c4iai-qemu-image berlin1.img
> guix gc -D /gnu/store/mm000wdzkzrvalg09jxk0y6nhi9c4iai-qemu-image
> chmod +w berlin1.img
> qemu-system-x86_64 -enable-kvm berlin1.img -m 2048 -nic 
> tap,ifname=tap0,script=no,downscript=no
>
> Note that I use NetworkManager with dnsmasq for a tap0 host-to-guest
> network connection as specified in the Guix manual.
>
> ---
>
> Second, unrelated to serving the website:
>
> I also noticed that Microsoft Internet Explorer 11 cannot display the
> new accessible dropdowns on https://guix.gnu.org/.i18n/en.  Do we
> care?  Perhaps the use of CSS z-index causes the problems.  The
> previous dropdowns used display: none; which made the dropdowns
> not-keyboard navigable.
>
> Regards,
> Florian