[Cloud] [Cloud-announce] (Another) toolforge outage coming Thursday

2023-04-03 Thread Andrew Bogott
On Thursday we will be migrating most toolforge databases to a new server. Thiswill take place on Thursday at 17:00UTC. During this window ToolsDBwill be read-only and most tools that rely on writing to the database will fail. This migration should take about an hour but unexpected side-effect

[Cloud] [Cloud-announce] Re: Two toolforge outages coming next week, Monday and Thursday

2023-04-03 Thread Andrew Bogott
Today's migration is complete, and the new server seems to be performing adequately. All volumes are now read/write, and most tools should be running again. Taavi is in the process of restarting most kubernetes worker nodes which will take another 60 minutes or so. If your tools and jobs are s