GitLab downtime
Hi all, Today I'm going to be working on migrating our GitLab instance to a new machine. Starting in likely an hour there will be a few hours of downtime while the migration proceeds. I will send an email when the downtime begins and concludes. Apologies for the late notice! Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Brief GitLab downtime
Hi everyone, I'll be rebooting gitlab.haskell.org for a short upgrade. Shouldn't be more than a few minutes. As usual, I'll email when things are back up. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
Ben Gamari writes: > Hi all, > > Today I'm going to be working on migrating our GitLab instance to a new > machine. Starting in likely an hour there will be a few hours of > downtime while the migration proceeds. I will send an email when the > downtime begins and concludes. > Hi all, A brief update: The GitLab downtime has not yet started as the migration preparation has taken a bit longer than expected. As it is getting late, I will be postponing the downtime until tomorrow evening. Sorry for the inconvenience! Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
Ben Gamari writes: > Ben Gamari writes: > >> Hi all, >> >> Today I'm going to be working on migrating our GitLab instance to a new >> machine. Starting in likely an hour there will be a few hours of >> downtime while the migration proceeds. I will send an email when the >> downtime begins and concludes. >> > Hi all, > > A brief update: The GitLab downtime has not yet started as the migration > preparation has taken a bit longer than expected. As it is getting late, > I will be postponing the downtime until tomorrow evening. > > Sorry for the inconvenience! Hi all, As noted yesterday I will be continue the GitLab migration starting in around 45 minutes. Do expect a bit of outage starting at that time. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
GitLab appears to be back up. However, access via ssh tells me @@@ @ WARNING: POSSIBLE DNS SPOOFING DETECTED! @ @@@ The ED25519 host key for gitlab.haskell.org has changed, and the key for the corresponding IP address 147.75.105.243 is unknown. This could either mean that DNS SPOOFING is happening or the IP address for the host and its host key have changed at the same time. @@@ @WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! This kind of error seems in keeping with the upgrade you've done, but I want explicit reassurance that I should proceed in the face of this error. Thanks! Richard > On May 31, 2021, at 4:23 PM, Ben Gamari wrote: > > Ben Gamari mailto:b...@well-typed.com>> writes: > >> Ben Gamari writes: >> >>> Hi all, >>> >>> Today I'm going to be working on migrating our GitLab instance to a new >>> machine. Starting in likely an hour there will be a few hours of >>> downtime while the migration proceeds. I will send an email when the >>> downtime begins and concludes. >>> >> Hi all, >> >> A brief update: The GitLab downtime has not yet started as the migration >> preparation has taken a bit longer than expected. As it is getting late, >> I will be postponing the downtime until tomorrow evening. >> >> Sorry for the inconvenience! > > Hi all, > > As noted yesterday I will be continue the GitLab migration starting in > around 45 minutes. Do expect a bit of outage starting at that time. > > Cheers, > > - Ben > ___ > ghc-devs mailing list > ghc-devs@haskell.org <mailto:ghc-devs@haskell.org> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > <http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs> ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
Richard Eisenberg writes: > GitLab appears to be back up. However, access via ssh tells me > > @@@ > @ WARNING: POSSIBLE DNS SPOOFING DETECTED! @ > @@@ > The ED25519 host key for gitlab.haskell.org has changed, > and the key for the corresponding IP address 147.75.105.243 > is unknown. This could either mean that > DNS SPOOFING is happening or the IP address for the host > and its host key have changed at the same time. > @@@ > @WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ > @@@ > IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! > > This kind of error seems in keeping with the upgrade you've done, but > I want explicit reassurance that I should proceed in the face of this > error. > Thanks for letting my know, Richard! You should now find that the new box has the same SSH host key as the previous box. Let me know if you see any other unexpected differences. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
Ben Gamari writes: > Hi all, > Hi all, I believe gitlab.haskell.org should be back up at this point. There are still a few ancillary services (e.g. grafana.gitlab.haskell.org) that I haven't yet validated but every critical should be functional. Do let me know if you find anything amiss. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
It is currently down with a 502 error. - Vlad > On 1 Jun 2021, at 04:01, Ben Gamari wrote: > > Hi all, > > I believe gitlab.haskell.org should be back up at this point. There are > still a few ancillary services (e.g. grafana.gitlab.haskell.org) that I > haven't yet validated but every critical should be functional. Do let me > know if you find anything amiss. > > Cheers, > > - Ben > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
Seems to be serving again now On Tue, Jun 1, 2021 at 4:13 PM Vladislav Zavialov wrote: > It is currently down with a 502 error. > > - Vlad > > > On 1 Jun 2021, at 04:01, Ben Gamari wrote: > > > > Hi all, > > > > I believe gitlab.haskell.org should be back up at this point. There are > > still a few ancillary services (e.g. grafana.gitlab.haskell.org) that I > > haven't yet validated but every critical should be functional. Do let me > > know if you find anything amiss. > > > > Cheers, > > > > - Ben > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- Shayne Fletcher ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
I restarted the service but somethings still appear to be broken. On Tue, Jun 1, 2021 at 9:16 AM Shayne Fletcher wrote: > > Seems to be serving again now > > On Tue, Jun 1, 2021 at 4:13 PM Vladislav Zavialov > wrote: >> >> It is currently down with a 502 error. >> >> - Vlad >> >> > On 1 Jun 2021, at 04:01, Ben Gamari wrote: >> > >> > Hi all, >> > >> > I believe gitlab.haskell.org should be back up at this point. There are >> > still a few ancillary services (e.g. grafana.gitlab.haskell.org) that I >> > haven't yet validated but every critical should be functional. Do let me >> > know if you find anything amiss. >> > >> > Cheers, >> > >> > - Ben >> > ___ >> > ghc-devs mailing list >> > ghc-devs@haskell.org >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > -- > Shayne Fletcher > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
502 again. Giorgio On Tue, 1 Jun 2021 at 10:20, Matthew Pickering wrote: > > I restarted the service but somethings still appear to be broken. > > On Tue, Jun 1, 2021 at 9:16 AM Shayne Fletcher > wrote: > > > > Seems to be serving again now > > > > On Tue, Jun 1, 2021 at 4:13 PM Vladislav Zavialov > > wrote: > >> > >> It is currently down with a 502 error. > >> > >> - Vlad > >> > >> > On 1 Jun 2021, at 04:01, Ben Gamari wrote: > >> > > >> > Hi all, > >> > > >> > I believe gitlab.haskell.org should be back up at this point. There are > >> > still a few ancillary services (e.g. grafana.gitlab.haskell.org) that I > >> > haven't yet validated but every critical should be functional. Do let me > >> > know if you find anything amiss. > >> > > >> > Cheers, > >> > > >> > - Ben > >> > ___ > >> > ghc-devs mailing list > >> > ghc-devs@haskell.org > >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >> > >> ___ > >> ghc-devs mailing list > >> ghc-devs@haskell.org > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > > > > > > > > -- > > Shayne Fletcher > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: GitLab downtime
It looks like this is the result of the recurrence of a nixpkgs issue. I have applied a potential workaround. Cheers, - Ben On June 2, 2021 12:26:26 AM EDT, Giorgio Marinelli wrote: >502 again. > > >Giorgio > > >On Tue, 1 Jun 2021 at 10:20, Matthew Pickering > wrote: >> >> I restarted the service but somethings still appear to be broken. >> >> On Tue, Jun 1, 2021 at 9:16 AM Shayne Fletcher >> wrote: >> > >> > Seems to be serving again now >> > >> > On Tue, Jun 1, 2021 at 4:13 PM Vladislav Zavialov > wrote: >> >> >> >> It is currently down with a 502 error. >> >> >> >> - Vlad >> >> >> >> > On 1 Jun 2021, at 04:01, Ben Gamari wrote: >> >> > >> >> > Hi all, >> >> > >> >> > I believe gitlab.haskell.org should be back up at this point. >There are >> >> > still a few ancillary services (e.g. grafana.gitlab.haskell.org) >that I >> >> > haven't yet validated but every critical should be functional. >Do let me >> >> > know if you find anything amiss. >> >> > >> >> > Cheers, >> >> > >> >> > - Ben >> >> > ___ >> >> > ghc-devs mailing list >> >> > ghc-devs@haskell.org >> >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> >> >> ___ >> >> ghc-devs mailing list >> >> ghc-devs@haskell.org >> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > >> > >> > >> > -- >> > Shayne Fletcher >> > ___ >> > ghc-devs mailing list >> > ghc-devs@haskell.org >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >___ >ghc-devs mailing list >ghc-devs@haskell.org >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- Sent from my Android device with K-9 Mail. Please excuse my brevity.___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Brief GitLab downtime
Ben Gamari writes: > Hi everyone, > > I'll be rebooting gitlab.haskell.org for a short upgrade. Shouldn't be > more than a few minutes. As usual, I'll email when things are back up. > Unfortunately due to an operating system bug [1] this upgrade has resulted in down-time. I'm working on resolving the issue but it may be a while longer. Cheers, - Ben [1] https://github.com/NixOS/nixpkgs/issues/69360#issuecomment-558823357 signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Brief GitLab downtime
Ben Gamari writes: > Hi everyone, > > I'll be rebooting gitlab.haskell.org for a short upgrade. Shouldn't be > more than a few minutes. As usual, I'll email when things are back up. > Service has been restored. Many apologies for the unexpected downtime. Hopefully things should now be in better shape to avoid this sort of problem in the future. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Brief GitLab downtime
Ben Gamari writes: > Ben Gamari writes: > >> Hi everyone, >> >> I'll be rebooting gitlab.haskell.org for a short upgrade. Shouldn't be >> more than a few minutes. As usual, I'll email when things are back up. >> > Service has been restored. Many apologies for the unexpected downtime. > Hopefully things should now be in better shape to avoid this sort of problem > in the future. > There may be one more brief outage in a few minutes while a support engineer does a hardware check on our server. However, this should be fairly short-lived. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
