Re: certbot service experience

[Chris Marusich]

Christopher Allan Webber  writes:

> [-bug]
>
> Chris Marusich writes:
>
>> Christopher Allan Webber  writes:
>>
>>>  - I was surprised that I was prompted for an email while doing guix
>>>system reconfigure
>>
>> That does seem odd.  Why were you prompted for an email address?  Can
>> that be fixed somehow?
>
> It's the certbot initial-setup script firing off here.  The email
> address is given so you can be notified for security updates, etc.
> Maybe recovery?  Don't remember about that last bit.

I see.  Having never used certbot before myself, I don't think I can
really give useful advice here.  It just seems a little strange to
require user interaction during the reconfigure. One would think that
one could provide any necessary information declaratively in the
operating system configuration file.

>
>>>   2) Enable the certbot-service-type (and mcron-service-type if you
>>>  haven't already):
>>>
>>>(service certbot-service-type
>>> (certbot-configuration
>>>  ;; Replace these with your own domain and web root
>>>  (hosts '("test.activitypub.rocks"))
>>>  (webroot "/srv/activitypub.rocks/site/")))
>>>;; if you don't have an mcron service already
>>>(service mcron-service-type)
>>
>> Where is the certbot-service-type defined?  I couldn't find it in the
>> master branch.  Also, why is mcron required?  I don't know much about
>> LetsEncrypt, but I thought certbot was a one-time thing that you do
>> manually...  Why is it a "service" here?
>
> It's not in the master branch is why.  I was looking at the
> wip-git-https branch. :)  I think it's also in the wip-potluck branch.
>

That explains it!  :)

>
> It's extending the mcron service so it can install an auto-update rule
> for you, which is pretty cool!


I see.  I would have thought that if service A extends service B, and
one defines service A in the operating system configuration file,
service B gets pulled in automatically.  Is that not true?

>>>   3) Okay hopefully that went successfully!  It should say.  Assuming it
>>>  did, *now* we can add the keys appropriately to the nginx config.
>>>
>>>(service nginx-service-type
>>> (nginx-configuration
>>>  (server-blocks
>>>   (list
>>>(nginx-server-configuration
>>> ;; Again, adjust to your site
>>> (server-name '("test.activitypub.rocks"))
>>> (root "/srv/activitypub.rocks/site/")
>>> (ssl-certificate
>>>  
>>> "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
>>> (ssl-certificate-key
>>>  
>>> "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))
>>>
>>>  Reconfigure and cross your fingers!
>>>
>>>   4) At this point I was surprised that it seemed like nginx should have
>>>  been working with https since everything was in place, but I
>>>  couldn't access it from my browser over https.  Frustrated, I
>>>  restarted the server.
>>>
>>>  And then it worked! :)
>>>
>>> So, this involved reconfiguring, reconfiguring, reconfiguring, and then
>>> a restart, then it worked for me.  (Well, plus a few reconfigures where
>>> nothing worked at all because I broke things of course. ;))  I wonder if
>>> that can be improved?
>>
>> I wonder if it is possible to define a custom service which orchestrates
>> the execution of nginx and certbot in the way you require, so that you
>> can define it all in one place, at once, without needing to reconfigure
>> multiple times?
>
> It might be, I dunno!  Maybe in starting the service, if it sees that
> the keys have not been generated yet, it pulls up nginx temporarily just
> to do the registration with the nginx that doesn't have the keys in it,
> does the generation of the keys and verifies them with letsencrypt, then
> pulls up the proper nginx at last.
>
> I don't know how easy/feasible this is.

It would be nice if we could just declare what we want in the operating
system configuration file, and get it without having to run multiple
reconfigures or interact with the user.  But hey, anything that works is
a step in the right direction!

-- 
Chris


signature.asc
Description: PGP signature

Re: Doc: reference to guix-devel for submitting patches

[Chris Marusich]

Petter  writes:

> Hi Guix,
>
> I quite recently realized I sent a couple of patches to guix-devel, whereas
> they should now go to guix-patches. I use the online manual and the current
> state is that there's a couple of references to guix-devel for submitting
> patches.
>
> https://www.gnu.org/software/guix/manual/html_node/Submitting-Patches.html#Submitting-Patches
>>We welcome contributions in the form of patches as produced by git 
>>format-patch
>> sent to the mailing list
> Where "mailing list" is mailto:guix-devel@gnu.org. This is updated in the
> repository however, so I assume the online manual will reflect this shortly.

In the current master branch, this has been changed to "guix-patches".
Perhaps somebody needs to update the website, too?  The website is
only updated periodically, so sometimes it is out of date.

>
>
> But here,
> https://www.gnu.org/software/guix/manual/html_node/Contributing.html#Contributing
> it says
>> Please get in touch with us on guix-devel@gnu.org and #guix on the Freenode
>> IRC network. We welcome ideas, bug reports, patches, and anything that may be
>> helpful to the project.
>

Yes, this wording is unclear.  It could be interpreted as an invitation
to send bug reports and patches to guix-devel@, which is not quite
right.

On related note, I see there is a section on submitting patches ((guix)
Submitting Patches), but there is no similar, dedicated, section on
submitting bugs (i.e., to bug-guix).

>
> And here,
> https://www.gnu.org/software/guix/manual/html_node/Packaging-Guidelines.html#Packaging-Guidelines
> the user is referred to the Contributing node.
>> Once your package builds correctly, please send us a patch (see 
>> Contributing).
>
> I think at least one of the latter two nodes could use an update to better
> inform the soon-to-be-contributor that patches should go to guix-patches. Not
> sure what the better fix is though.
>
> What do you think?
>

Seems to me like we should clarify the wording in the "Contributing"
section.  And maybe we should add a "Submitting Bug Reports"
sub-section, similar to the "Submitting Patches" one.

Do you think you could try your hand at writing this?

-- 
Chris


signature.asc
Description: PGP signature

Re: certbot service experience

[Christopher Allan Webber]

[-bug]

Chris Marusich writes:

> Christopher Allan Webber  writes:
>
>>  - I was surprised that I was prompted for an email while doing guix
>>system reconfigure
>
> That does seem odd.  Why were you prompted for an email address?  Can
> that be fixed somehow?

It's the certbot initial-setup script firing off here.  The email
address is given so you can be notified for security updates, etc.
Maybe recovery?  Don't remember about that last bit.

>>   2) Enable the certbot-service-type (and mcron-service-type if you
>>  haven't already):
>>
>>(service certbot-service-type
>> (certbot-configuration
>>  ;; Replace these with your own domain and web root
>>  (hosts '("test.activitypub.rocks"))
>>  (webroot "/srv/activitypub.rocks/site/")))
>>;; if you don't have an mcron service already
>>(service mcron-service-type)
>
> Where is the certbot-service-type defined?  I couldn't find it in the
> master branch.  Also, why is mcron required?  I don't know much about
> LetsEncrypt, but I thought certbot was a one-time thing that you do
> manually...  Why is it a "service" here?

It's not in the master branch is why.  I was looking at the
wip-git-https branch. :)  I think it's also in the wip-potluck branch.

It's extending the mcron service so it can install an auto-update rule
for you, which is pretty cool!

>>   3) Okay hopefully that went successfully!  It should say.  Assuming it
>>  did, *now* we can add the keys appropriately to the nginx config.
>>
>>(service nginx-service-type
>> (nginx-configuration
>>  (server-blocks
>>   (list
>>(nginx-server-configuration
>> ;; Again, adjust to your site
>> (server-name '("test.activitypub.rocks"))
>> (root "/srv/activitypub.rocks/site/")
>> (ssl-certificate
>>  
>> "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
>> (ssl-certificate-key
>>  
>> "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))
>>
>>  Reconfigure and cross your fingers!
>>
>>   4) At this point I was surprised that it seemed like nginx should have
>>  been working with https since everything was in place, but I
>>  couldn't access it from my browser over https.  Frustrated, I
>>  restarted the server.
>>
>>  And then it worked! :)
>>
>> So, this involved reconfiguring, reconfiguring, reconfiguring, and then
>> a restart, then it worked for me.  (Well, plus a few reconfigures where
>> nothing worked at all because I broke things of course. ;))  I wonder if
>> that can be improved?
>
> I wonder if it is possible to define a custom service which orchestrates
> the execution of nginx and certbot in the way you require, so that you
> can define it all in one place, at once, without needing to reconfigure
> multiple times?

It might be, I dunno!  Maybe in starting the service, if it sees that
the keys have not been generated yet, it pulls up nginx temporarily just
to do the registration with the nginx that doesn't have the keys in it,
does the generation of the keys and verifies them with letsencrypt, then
pulls up the proper nginx at last.

I don't know how easy/feasible this is.

>> That said, it's still really exciting to be able to describe these
>> things declaratively, and to have Guix take care of keeping things
>> renewed for me. :)  Excited to have this landing, and to be that much
>> closer to doing server deployment with GuixSD!
>
> Pretty cool!  Thanks for sharing your experience.  It's always neat to
> read about how people are using the system.

Yeah!  I'm excited to finally move to hosting my servers with GuixSD at
last, after ages of talking about it. ;)

Re: certbot service experience

[Chris Marusich]

Christopher Allan Webber  writes:

>  - I was surprised that I was prompted for an email while doing guix
>system reconfigure

That does seem odd.  Why were you prompted for an email address?  Can
that be fixed somehow?

>   2) Enable the certbot-service-type (and mcron-service-type if you
>  haven't already):
>
>(service certbot-service-type
> (certbot-configuration
>  ;; Replace these with your own domain and web root
>  (hosts '("test.activitypub.rocks"))
>  (webroot "/srv/activitypub.rocks/site/")))
>;; if you don't have an mcron service already
>(service mcron-service-type)

Where is the certbot-service-type defined?  I couldn't find it in the
master branch.  Also, why is mcron required?  I don't know much about
LetsEncrypt, but I thought certbot was a one-time thing that you do
manually...  Why is it a "service" here?

>   3) Okay hopefully that went successfully!  It should say.  Assuming it
>  did, *now* we can add the keys appropriately to the nginx config.
>
>(service nginx-service-type
> (nginx-configuration
>  (server-blocks
>   (list
>(nginx-server-configuration
> ;; Again, adjust to your site
> (server-name '("test.activitypub.rocks"))
> (root "/srv/activitypub.rocks/site/")
> (ssl-certificate
>  
> "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
> (ssl-certificate-key
>  
> "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))
>
>  Reconfigure and cross your fingers!
>
>   4) At this point I was surprised that it seemed like nginx should have
>  been working with https since everything was in place, but I
>  couldn't access it from my browser over https.  Frustrated, I
>  restarted the server.
>
>  And then it worked! :)
>
> So, this involved reconfiguring, reconfiguring, reconfiguring, and then
> a restart, then it worked for me.  (Well, plus a few reconfigures where
> nothing worked at all because I broke things of course. ;))  I wonder if
> that can be improved?

I wonder if it is possible to define a custom service which orchestrates
the execution of nginx and certbot in the way you require, so that you
can define it all in one place, at once, without needing to reconfigure
multiple times?

> That said, it's still really exciting to be able to describe these
> things declaratively, and to have Guix take care of keeping things
> renewed for me. :)  Excited to have this landing, and to be that much
> closer to doing server deployment with GuixSD!

Pretty cool!  Thanks for sharing your experience.  It's always neat to
read about how people are using the system.

-- 
Chris


signature.asc
Description: PGP signature

certbot service experience

[Christopher Allan Webber]

I'm crossposting this to guix-devel, even though it's in reply to
guix-patches bug #26685 adding the cerbot service, because I think it's
more about my experiences with workflow and less about what might affect
that specific bug.  (If you reply on guix-devel, maybe remove the
debbugs address.)

Andy Wingo writes:

> Attached patch adds a certbot service extending nginx.  Only question
> is, how to ensure that nginx is runing when the certbot activation runs?
> In practice I bet this races so it's not a big issue but if there's a
> way to require nginx before activation, that would be nice.

So I got a server up and running using certbot and this.  Yay!  It went
pretty well, and I'm happy about it now that it's running, though I did
run into some bumps (which is okay, I ran into bumps doing this on
Debian too, I just think now's a good time to reflect on the experience
and see if we can do anything better... plus maybe it's useful to
explore for someone else too).  I figured I'd document what the process
was like for me.  Here's my observations:

 - I was surprised that I was prompted for an email while doing guix
   system reconfigure (and in being asked if I was willing to supply
   that info to the EFF so they can put me on their mailing lists, which
   I'm not against but was surprising to encounter in a reconfigure).
   IIRC this is the first time anything like that has happened.  I
   generally anticipate such a reconfigure to be "prompt-less".
   I wonder if there should be an email field we can provide, so that it
   doesn't do this prompt?  Granted, it's for the first time run only.
   Maybe it's okay to do things as they are, but we should document it?
   I dunno?

 - Not specifically something to do with this package, but nginx wasn't
   coming up at a few points (btw, default nginx configuration won't
   work, because it points at certificate filepaths that we don't
   "automatically" put in place).  When nginx doesn't come up, there's
   pretty much no information as to why; nothing in the shepherd logs,
   nothing useful from shepherd itself ("shepherd: Service nginx could
   not be started."), and nothing in the nginx logs either.

 - Getting the certbot stuff to work basically involved three stages:

   1) Enable nginx without cert certbot, and point the root somewhere
  which will be the same place you'll set certbot-configuration's
  webroot at in the next step.  This looks something like the
  following:

(service nginx-service-type
  (nginx-configuration
   (server-blocks
(list
 (nginx-server-configuration
  ;; Replace these with your own domain and web root
  (server-name '("test.activitypub.rocks"))
  (root "/srv/activitypub.rocks/site/")
  ;; Note that you have to disable https and any certificates
  ;; esp because our nginx config defaults to pointing at keys
  ;; that probably aren't there.  Without doing this nginx
  ;; will die mysteriously.
  (https-port #f)
  (ssl-certificate #f)
  (ssl-certificate-key #f))

  (You can maybe do this in the same phase as the next step but
  there's a risk of a race condition I think.)

  Anyway now do "guix system reconfigure" and nginx should come up
  and serve your stuff *without* https.

  2) Enable the certbot-service-type (and mcron-service-type if you
 haven't already):

   (service certbot-service-type
(certbot-configuration
 ;; Replace these with your own domain and web root
 (hosts '("test.activitypub.rocks"))
 (webroot "/srv/activitypub.rocks/site/")))
   ;; if you don't have an mcron service already
   (service mcron-service-type)

 Okay, with those added, run "guix system reconfigure" again.
 You'll be prompted for your email address and about whether or not
 you want to join the EFF mailing lists, so watch for that.

  3) Okay hopefully that went successfully!  It should say.  Assuming it
 did, *now* we can add the keys appropriately to the nginx config.

   (service nginx-service-type
(nginx-configuration
 (server-blocks
  (list
   (nginx-server-configuration
;; Again, adjust to your site
(server-name '("test.activitypub.rocks"))
(root "/srv/activitypub.rocks/site/")
(ssl-certificate
 
"/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
(ssl-certificate-key
 
"/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))

 Reconfigure and cross your fingers!

  4) At this point I was surprised that it seemed like nginx should have
 been working with https since everything was in place, but I
 couldn't access it