Update: NFS for labs projects is back to read write now. There was an incident in the process of the migration that caused /home in a significant number of labs instances (where /home was not on NFS) to be replaced by an erroneous symlink. We are currently working on restoring all the home directories - it looks like it may take at-least a couple hours. I'll keep the list updated on progress.
On Wed, Jan 18, 2017 at 11:34 AM, Madhumitha Viswanathan < mviswanat...@wikimedia.org> wrote: > Update: this is still in progress. There's some ongoing issues with home > directories on non-nfs /home across Labs (not tools or maps) - we are > working on it, and will update soon. Do feel free to reach out > #wikimedia-labs if you have any questions or concerns. > > On Wed, Jan 18, 2017 at 8:38 AM, Madhumitha Viswanathan < > mviswanat...@wikimedia.org> wrote: > >> Reminder: This is starting soon, in ~22 minutes. >> >> On Tue, Jan 17, 2017 at 12:07 PM, Madhumitha Viswanathan < >> mviswanat...@wikimedia.org> wrote: >> >>> Reminder: This is happening tomorrow, starting 09:00 PST(16:00 UTC). >>> >>> On Wed, Jan 4, 2017 at 1:43 PM, Madhumitha Viswanathan < >>> mviswanat...@wikimedia.org> wrote: >>> >>>> Hello, >>>> >>>> Continuing the storage redundancy and reliability efforts for Labs ( >>>> https://phabricator.wikimedia.org/T126083), the final migration to the >>>> new NFS storage cluster for Labs projects with NFS enabled is upcoming. The >>>> migration is planned to happen 2017-01-18 starting 09:00 PST(16:00 >>>> UTC). This *does not* affect tools, maps or any other projects that don't >>>> have /home or /data/project mounted. The migration window is expected to be >>>> fairly short (<3 hours) - but could last up to 6 hours. >>>> >>>> During the migration, no new data will be written to NFS (/home or >>>> /data/project), but existing data will be accessible in Read-only mode for >>>> the most part. Post migration, any services or jobs that were running on >>>> top of NFS (/home or /data/project) will require manual restarts. Jobs >>>> running on top of /scratch or /public/dumps will be unaffected. I will keep >>>> the lists and #wikimedia-labs updated on progress during and after the >>>> migration. >>>> >>>> The list of labs projects that will be affected in this migration are: >>>> >>>> >>>> - catgraph >>>> >>>> >>>> - account-creation-assistance >>>> >>>> >>>> - contributors >>>> >>>> >>>> - wikidata-topicmaps >>>> >>>> >>>> - sugarcrm >>>> >>>> >>>> - wikidumpparse >>>> >>>> >>>> - video >>>> >>>> >>>> - openstack >>>> >>>> >>>> - testlabs >>>> >>>> >>>> - wikidata-dev >>>> >>>> >>>> - quarry >>>> >>>> >>>> - huggle >>>> >>>> >>>> - editor-engagement >>>> >>>> >>>> - utrs >>>> >>>> >>>> - wmt >>>> >>>> >>>> - cvn >>>> >>>> >>>> - fastcci >>>> >>>> >>>> - toolsbeta >>>> >>>> >>>> - project-proxy >>>> >>>> >>>> - dumps >>>> >>>> >>>> - bots >>>> >>>> >>>> - snuggle >>>> >>>> >>>> - math >>>> >>>> >>>> - wikisource-tools >>>> >>>> >>>> The tracking task on phabricator is here - https://phabricator.wikimedi >>>> a.org/T154336. Let us know if you have any questions or concerns on >>>> the list or on #wikimedia-labs. >>>> >>>> -- >>>> Madhu Viswanathan >>>> Operations Engineer, Wikimedia Labs >>>> >>> >>> >>> >>> -- >>> --Madhu :) >>> >> >> >> >> -- >> --Madhu :) >> > > > > -- > --Madhu :) > -- --Madhu :) _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l