On Sat, Jan 30, 2016 at 2:15 AM, Sébastien Delafond <sdelaf...@gmail.com> wrote: > On 2016-01-07, Grant Murphy <grantcmur...@gmail.com> wrote: >> - The file has not had whitespace removed (will reduce file size by >> about 5mb) > > I've just pushed a change that does two things: > > * uses the most compact JSON representation; measured against the > "stable-only" DB, it cuts down the download size around 30%, from > 3.6Mo to 2.5Mo. > > * removes the hash sorting altogether; this brings the execution > time from 3.7s to 2.5s ("stable-only" once again), which should > hopefully help with the load on security-tracker. > > Those are ballpark figures, of course, and we'll need to see how it > behaves against the full DB.
Cool thanks. I'm currently getting a 404 from this URL - https://security-tracker.debian.org/tracker/data/json is this related? > >> - The file is not compressed (gzip will reduce __total__ file size >> to ~2mb) > > Not sure why that stream isn't gzipped, will look into that. > That would be awesome. I haven't had a chance yet to dig into the code but try to look into it also. >> - It is generated per request >> - The 'If-Modified-Since' header is not honored so the complete file >> will need to be re-downloaded every time to check if it has >> changed. > > Good points. I'm not too sure about the technical implications related > to the integration of those two features, but your contribution would > in any case be most welcome :) > > Cheers, > > --Seb >