Re: [Wikitech-l] Phabricator upgrade window next Wednesday 00:00 UTC

2019-12-05 Thread Daniel Zahn
It turned out BIOS settings on phab1001 were such that disks were running
under "legacy IDE" mode and write cache was turned off.

This was really bad for performance. Changing that in BIOS unfortunately
also means complete data loss.

So we had to go back to phab1003 yesterday, fix phab1001, reimage it, rsync
all the data again and finally (today) switch to phab1001 for real.

The good part is that the process to switch is now simplified and there was
basically no downtime for users besides the rarely used git-ssh service,
which is also fixed now.

This is done now.

On Wed, Dec 4, 2019 at 12:14 AM Daniel Zahn  wrote:

> This has happened. The Phabricator production server is now phab1001
> (again) and not phab1003 anymore.
>
> Phabricator now runs on Debian stable (buster) and PHP 7.3.
>
> https://phabricator.wikimedia.org/T238956  shows all the related patches
> and can be used to report issues.
>


-- 
Daniel Zahn 
Operations Engineer
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Phabricator upgrade window next Wednesday 00:00 UTC

2019-12-03 Thread Daniel Zahn
This has happened. The Phabricator production server is now phab1001
(again) and not phab1003 anymore.

Phabricator now runs on Debian stable (buster) and PHP 7.3.

https://phabricator.wikimedia.org/T238956  shows all the related patches
and can be used to report issues.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Phabricator upgrade window next Wednesday 00:00 UTC

2019-11-26 Thread Daniel Zahn
Next Wednesday, December 4th, at 00:00 UTC (4pm San Francisco) there will be
a Phabricator upgrade window that might last up to 2 hours.

While we expect downtime to be much shorter than that Phabricator might be
unreachable in that time frame.

We are switching the production Phabricator server to a buster (Debian
stable) machine.

https://phabricator.wikimedia.org/T238956

-- 
Daniel Zahn 
Operations Engineer
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l