Hi, On Tue, Apr 13, 2021 at 11:27 AM Lucas Nussbaum <lu...@debian.org> wrote: > > From the UDD point of view, I would very much prefer to get a full dump > something I can import every few hours, than having to deal with a > stream of updates or with querying a per-package API.
Since few users ever need *all* data, would it make sense to re-conceive UDD as a "query broker" to help people get the data they actually need? > Currently the full import (that runs twice a day) takes about 10 minutes The power of COPY. The Lintian website currently takes 12 hours to import a single run across the archive in 42 bulk UPSERTS via JSON (but will eventually cease to generate data that way). Kind regards Felix Lechner