I would recommend serving content directly whether it is on primary storage or in archive. Sending a 301 redirect may or may not work, depending on a particular client. Arguably, using a redirect exposes aspects of internal storage architecture that don't need exposing.
The download server could be implemented to function similarly to how memory systems work. A single interface, with different levels of storage. The client doesn't know if they are hitting L1, L2 or whatever. Hope this is helpful. - Konstantin From: cross-project-issues-dev-boun...@eclipse.org [mailto:cross-project-issues-dev-boun...@eclipse.org] On Behalf Of Denis Roy Sent: Tuesday, June 19, 2012 1:36 PM To: cross-project-issues-dev@eclipse.org Subject: Re: [cross-project-issues-dev] List of files on Eclipse Mirror servers On 06/19/2012 03:40 PM, Konstantin Komissarchik wrote: Currently, only the mirror selection script is able to delegate to archive.eclipse.org, so auto-archiving would break those repositories. That gives me an idea. We currently use a small PHP script to handle 404's on download.eclipse.org. If the request comes from Eclipse or a robot, we send a small 404 response. If it's from a browser, we send the stunningly beautiful page you see here: http://downlaod.eclipse.org/khasjkas We could alter the script to return a 301redirect if the 404 file in question is indeed found on archive.eclipse.org. This means in theory, you _could_ move files and p2 repos to the archives without any breakage at all. Or even better, the PHP script could just send the file's contents down the wire with a 200 response, no redirects. I would be concerned about the increased CPU load, since download.e.o currently handles about 6 million 404's a day. But it would likely be a very elegant solution. Denis If there are resources available to work on this, it would be much better if download.eclipse.org served as central gateway to both the primary download server and the archive. Then one can implement all sort of archiving policies without involving projects or affecting end users referencing these URLs. You can auto-archive based on age, you can auto-archive based on demand, heck you can even pick a goal for how much to mirror and have a heuristic that shuffles between primary and archive to meet that goal. - Konstantin From: cross-project-issues-dev-boun...@eclipse.org [mailto:cross-project-issues-dev-boun...@eclipse.org] On Behalf Of Wayne Beaton Sent: Tuesday, June 19, 2012 12:15 PM To: cross-project-issues-dev@eclipse.org Subject: Re: [cross-project-issues-dev] List of files on Eclipse Mirror servers Does it make sense to consider moving files that are old--say three or more years old--to the archive server automatically? Wayne On 06/19/2012 10:33 AM, Denis Roy wrote: To help you clean up older builds and files from the Eclipse download server, I've compiled a list of files that are actually sync'ed to our mirrors. http://download.eclipse.org/technology/phoenix/download.eclipse.org-filelist .txt.gz I each project to look at the file list, and either remove files that are no longer needed, or move older releases[1] to the archives. Thanks, Denis [1] http://wiki.eclipse.org/IT_Infrastructure_Doc#Downloads
_______________________________________________ cross-project-issues-dev mailing list cross-project-issues-dev@eclipse.org https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
