I have just posted a new draft RFC on changing the thumbnail storage and caching pipeline for Wikimedia projects [0].
This RFC was started as part of the Multimedia tech debt project. Aaron, Faidon and I would like to move discussion forward on the issue of the thumbnail caching layer in general and specifically would like to find consensus on a reasonable change to address some of the shortfalls of the current system in place for upload.wikimedia.org and related wikis. The core problem has been discussed on list previously [1], namely that scaled media files (aka thumbnails) take up a lot of space in the storage servers. This RFC proposes following a course of action proposed in that prior discussion [2] to customize the Varnish hashing configuration for thumbs such that a single HTCP purge would invalidate all size variations for a given file. It attempts to document some of the benefits and drawbacks of that proposal as well as present a few related variations. Any and all feedback from this group would be greatly appreciated in vetting the core idea and determining reasonable next steps towards a viable long term solution. [0]: https://www.mediawiki.org/wiki/Requests_for_comment/Simplify_Thumbnail_Cache [1]: http://wikimedia.7.x6.nabble.com/scaled-media-thumbs-as-temporary-files-not-stored-forever-tt4984340.html [2]: http://wikimedia.7.x6.nabble.com/scaled-media-thumbs-as-temporary-files-not-stored-forever-tt4984340.html#a4987922 Bryan -- Bryan Davis Wikimedia Foundation <bd...@wikimedia.org> [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID irc: bd808 v:415.839.6885 x6855 _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
