[Wikidata-bugs] [Maniphest] [Commented On] T72246: provide incremental JSON dumps for Wikidata
awight added a comment. I have a side project that would benefit from daily JSON dumps. Happy to look into providing this if there's anyone else who cares?TASK DETAILhttps://phabricator.wikimedia.org/T72246EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: JanZerebecki, awightCc: awight, ValterVB, Wikidata-bugs, JanZerebecki, MZMcBride, Magnus, Aklapper, Lydia_Pintscher, hoo, D3r1ck01, Izno, aude, Svick, Mbch331, jeremyb___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T72246: provide incremental JSON dumps for Wikidata
Magnus added a comment. I believe I originally asked for this, but current WDQ wouldn't use these anymore, and SPARQL replacements are on the way. In case I would have been the only customer, this task could be closed now. TASK DETAIL https://phabricator.wikimedia.org/T72246 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign . EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Magnus Cc: Wikidata-bugs, JanZerebecki, MZMcBride, Magnus, Aklapper, Lydia_Pintscher, hoo, aude, Svick, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T72246: provide incremental JSON dumps for Wikidata
hoo added a comment. This would probably be implemented like this: Have a script that dumps all entity ids that have been changed since the last incremental dump. Then just dump all entities on that list. The first script is not yet implemented, but that shouldn't be to hard. Potential shortcomings of this (that may or may not also apply to the other incremental dumps, I have no idea): Deletions and merges (that turn things into redirects) wouldn't show up that way. TASK DETAIL https://phabricator.wikimedia.org/T72246 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign . EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hoo Cc: Magnus, Lydia_Pintscher, hoo, Wikidata-bugs, aude ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
