Quick reminder that the planned tree closure is scheduled for 12:00 EST (one hour from now) and should last approximately two hours. Try will remain open. Updates will be posted to #firefox-ci <https://matrix.to/#/#firefox-ci:mozilla.org> on Matrix and I'll follow-up here when the migration is finished as well.
Please be on the lookout for suspicious failures, especially around: 1. Missing artifacts. 2. Failure to decompress artifacts. Please ping the #firefox-ci <https://matrix.to/#/#firefox-ci:mozilla.org> channel on Matrix if you find any issues. Thanks, Andrew On Fri, Jun 23, 2023 at 2:43 PM Andrew Halberstadt <[email protected]> wrote: > *Update:* Firefox CI artifact migration and planned tree closure is moved > to July 12th @ 12:00 EST. > > The community migration was completed successfully, however a problem > related to how S3 and GCS handle gzip compressed files came up which will > likely cause some issues in Firefox CI (read below for details). Since June > 28th is preceding a long weekend followed by the Firefox 115 release, we've > decided to delay the migration until July 12th, when we'll be under no time > pressure to fix fallout. > > > > *Issue Details* > It turns out that S3 serves the file with whatever encoding it was > uploaded with no matter what. So if you request a gzip compressed artifact, > S3 will serve you that gzip compressed artifact. GCS on the other hand, > pays attention to the Accept-Encoding > <https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding> > header. So if you request a gzip compressed artifact, GCS will serve you > the gzip compressed artifact if you specify `Accept-Encoding: gzip`, but > otherwise helpfully decompress the artifact (as the lack of the > `Accept-Encoding` header indicates you can't handle gzipped files). This > means that after the transition, places that were expecting downloaded > artifacts to be gzipped, may instead be decompressed. > > We'll use the extra time to perform extra testing and validation to try > and root out issues ahead of time. However, it is not possible to find > every place that downloads artifacts from Taskcluster. If you maintain any > tasks or tools that download gzipped Taskcluster artifacts and want to > avoid any disruption, please ensure you are passing the `Accept-Encoding: > gzip` header in your GET requests. > > - Andrew > > > > On Tue, Jun 20, 2023 at 12:32 PM Andrew Halberstadt <[email protected]> > wrote: > >> Hey everyone, >> >> We're planning a tree closure to migrate Firefox CI's artifact storage >> backend from Amazon S3 to Google Cloud Storage: >> >> *When:* June 28th @ 12:00 EST >> *Duration:* 2hrs (or longer if we need to rollback) >> *What to expect:* >> Try will remain open. While we anticipate the transition should be mostly >> seamless, you may notice some recent artifacts will be missing for a short >> period of time following the cutover (but they will be synced over soon) >> and some unlucky tasks may fail with an exception and need to be rerun. >> >> We'll also be migrating the Community Taskcluster instance tomorrow in >> order to detect any issues that arise beforehand. If you notice missing >> artifacts or other issues, please reach out on #firefox-ci >> <https://matrix.to/#/%23firefox-ci:mozilla.org> (for issues with the >> Firefox CI instance) or #taskcluster >> <https://matrix.to/#/%23taskcluster:mozilla.org> (for issues with the >> Community instance). >> >> Big thanks to Eric Maydeck for all the prerequisite work to make the >> migration possible! >> >> Please reach out if you have any questions or concerns. >> Cheers, >> Andrew >> > -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-platform/CAAJAz%2B5cF9eiCKqXVSZy2VXTyhHdv2GSGewfTHKURTfH_3hd4g%40mail.gmail.com.
