Thien-Thi Nguyen wrote: > I confirm that it is working on the client side now, as well. > My guess is that i was too hasty in retrying, and didn't give > savannah enough time to process the new key.
Hmm... I hate intermittent failures! :-( At this time there should be no delay between updating an ssh key in the Savannah web UI and being able to use it across the vcs and download servers with ssh. The ssh key is stored in an SQL database and upon storing it is immediately available with an SQL query which is used for ssh in our configuration. In the development of Savannah (and the SourceForge it is based upon) a lot of actions were done through cron running every hour or every half hour or whatever. One thing happens one place and then after a bit the cron job runs and a reaction happens elsewhere. That paradigm of action and cron task later reaction is so pervasive that it is easy to think that everything just needs time to settle. (I would like to get rid of all of those cases...) And ssh keys used to be that way until about ten years ago as far as I can tell from the historical record. Take a look at the last paragraph at that SshAccess page. file:///home/bob/src/savannah-stuff/administration/sviki/SshAccess.html (Savannah admin info: specifically, in /etc/ssh/authorized_keys/USERNAME on the subhosts; a cron job sv_authorized_keys runs on each vm.) Well... That isn't true anymore. But it says that it used to be true. I saw that while looking at it for your incident and have it queued up to make a change to that doc to update that bit but haven't had a chance to do it yet. Some time ago not sure when or who but things were changed to use the MySQL database directly to hold ssh keys. That has many advantages. Not the least of which being that the web UI can store the changes to the database and then the new keys are immediately available to the other systems that need them for ssh access. Both ssh keys and also libnss for user account data. They are separate processes but both using the SQL database over the network. The libnss SQL module for user account data has worked pretty well. It does have a dependency on the network between the front facing system with ssh needing it and the SQL database system that is shared among the collection. Usually that is okay. But if the network has a problem then reports will be logged. libnss-mysql: mysql_query failed: Lost connection to MySQL server during query, trying again (2) I'll say that I probably never saw that happen for several years and then in the last few months have suddenly started to see this quite frequently in the logs. (I use logcheck to scan the logs continuously and send alerts upon unusual events.) Most often this happens in the middle of the US nighttime (say 3am or so but really at any time) when no one, admins or volunteers, are doing anything. And therefore I have this idea that there must be network maintenance happening in the datacenter in those hours which causes transient network glitches. I am sure this affects our overseas free software friends the worst since it is happening during their daylight times. I feel bad about that. But then whenever we look at the problem later no trouble is ever found. I expect this to be causing transient glitches in using the vcs services. Upon retrying I expect things to work fine. It isn't a problem of something changing on the systems because as you have heard our problem is that we need to upgrade them and are working on doing that, on other systems, and so nothing has changed on the vcs systems. Also this happens to both vcs0 and vcs1 both and they each have different OS versions. Therefore I am pretty sure this particular problem is due to external influences. > In any case, many thanks again for your help resolving things. > Happy hacking! I am happy things have been resolved for you! Happy Hacking! Bob
signature.asc
Description: PGP signature