Well, the Sun patch didn't hold us. So we are going to do a gradual move of directories, some data, and the actual thumbnail regeneration over to another server, ms4. How much data we move depends on how fast ms1 can push it over to ms4, currently very slow indeed. Even removing files on ms1 is painfully slow. So...
In order to get ms1 out of its hole, we're turning off uploads once again, so that it can free up space at a reasonable pace. Once we have enough free space to make ZFS happy, we'll turn it them back on. We expect that the rest of the migration of data and/or empty directories should take place over the next several days without further outages. Ariel Στις 15-07-2009, ημέρα Τετ, και ώρα 14:28 -0700, ο/η Brion Vibber έγραψε: > Brion Vibber wrote: > > We're applying a Solaris kernel patch for the ZFS performance problem on > > ms1, our main media file server. > > > > While in progress, uploads will be temporarily disabled. Cached images > > should still display, but you may see some missing images in the meantime... > > The kernel patch is applied, the machine's rebooted, and uploads & math > are re-enabled. So far all looks well; this'll keep things running > smoothly while we continue work on rearranging the files (moving thumbs > out to a separate server will simplify things and keep the main > filesystem backup snapshots from filling up with short-lived files, for > instance). > > -- brion > > > _______________________________________________ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
