brion added a comment.
Pulling individual data items out of large lists; pulling relevant columns in order to sum them; pulling or updating a small number of cells during editing; sub setting a large data set to graph the subset; sub setting a large data set to perform operations on it Ina Lua module. For a concrete example: say I have a data set that lists the voting breakdown during US primary elections for every county. That's roughly 3000 rows, each with data points for each candidate. Not too awful perhaps, so let's make a bigger table. Population of every US census place for every 10-year census since 1790. That's probably a lot. Now add more columns for various breakdown information. Subset queries on a large data set would allow for reading in data relevant to a map area, or a particular place, or a particular breakdown column, or a particular year range, or some combination of all these things, having had to pull in only the column list and do some sort of index scan on the key rows. The alternative is to break down the giant table into lots of small tables, and have the client code in a Lua module or whatever 'stitch' together the multiple data sets when needing to cross boundaries. You might say that no one should store that much data in one table, but I'm pretty sure people will push the limits of a system like this. :) TASK DETAIL https://phabricator.wikimedia.org/T120452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Yurik, brion Cc: Jdforrester-PERSONAL, Jdforrester-WMF, brion, ThurnerRupert, intracer, TerraCodes, Pokefan95, gerritbot, -jem-, Bawolff, MZMcBride, Alkamid, Milimetric, Thryduulf, JEumerus, MarkTraceur, Yurik, Matanya, ekkis, matmarex, Lydia_Pintscher, Aklapper, Steinsplitter, StudiesWorld, DannyH, Riley_Huntley, D3r1ck01, Izno, JAllemandou, Wikidata-bugs, aude, El_Grafo, Ricordisamoa, Shizhao, Fabrice_Florin, Mbch331, Jay8g, Krenair, jeremyb _______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
