[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

amy_rc closed this task as "Resolved". amy_rc added a comment. Thanks. We'll send a short note to the Wikidata mailing list and add it to the next weekly summary. \o/ TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

ArielGlenn added a comment. These look fine to me from today, and I've done all the buster-side testing so that's ok too. Closing this! Ah, do we want to anounce it anywhere though? Maybe I won't close it pending that answer. Places it could be announced: xmldatadumps-l, wikitech-l, research

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

Lydia_Pintscher reassigned this task from hoo to ArielGlenn. Lydia_Pintscher added a subscriber: hoo. Lydia_Pintscher moved this task from Test (Verification) to Done on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. Lydia_Pintscher added a comment. This seems to be looking good

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

ArielGlenn added a comment. I am doing some prep work before I try to test this on buster. Getting close! TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hoo, ArielGlenn Cc: noarave, ArielGlenn

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

noarave moved this task from Stalled/Waiting to Test (Verification) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. noarave added a comment. This seemed to be ready for verification, as it has already ran a couple of times: https://dumps.wikimedia.org/wikidatawiki/entities/

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

ArielGlenn added a comment. All set. We should check on these again in the middle of next week, as the run starts on Monday at ridiculous-o-clock when we are all sleeping. TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/p

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

gerritbot added a comment. Change 659945 **merged** by ArielGlenn: [operations/puppet@production] Update minimum expected file size for lexeme JSON dumps https://gerrit.wikimedia.org/r/659945 TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricat

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hoo, gerritbot Cc: ArielGlenn, Lucas_Werkmeister_WMDE, Lydia_Pintscher, WMDE-leszek, Pablo-WMDE, Alter-

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

gerritbot added a comment. Change 659945 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [operations/puppet@production] Update minimum expected file size for lexeme JSON dumps https://gerrit.wikimedia.org/r/659945 TASK DETAIL https://

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

Lucas_Werkmeister_WMDE added a comment. …or maybe use the file size that the error message //already tells us// as a guideline 🤦 TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hoo, Lucas_Werk

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

Lucas_Werkmeister_WMDE added a comment. Ah, thanks. Looks like the threshold was last adjusted in June 2019 . I guess for the inaugural lexeme JSON dumps it would be best to skip the check completely? And then use the first dum

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

ArielGlenn added a comment. In T264883#6786811 , @Lucas_Werkmeister_WMDE wrote: > Are you sure they ran? That directory only contains RDF dumps as far as I can tell (Turtle and NTriples), we’ve been generating those for a while (compare

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

Lucas_Werkmeister_WMDE added a comment. Are you sure they ran? That directory only contains RDF dumps as far as I can tell (Turtle and NTriples), we’ve been generating those for a while (compare 20210122 with 20201218

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

ArielGlenn added a comment. These ran and are available at https://dumps.wikimedia.org/other/wikibase/wikidatawiki/20210122/ How do they look? TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

Maintenance_bot removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hoo, Maintenance_bot Cc: Lucas_Werkmeister_WMDE, Lydia_Pintscher, WMDE-leszek, Pablo-WMDE, Akuc

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

Lucas_Werkmeister_WMDE added a comment. Dump generation is currently scheduled for Wednesday (3:15 UTC, just like the non-lexeme JSON dumps are on Monday 3:15 UTC). TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/em

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

gerritbot added a comment. Change 637895 **merged** by ArielGlenn: [operations/puppet@production] Do weekly dumps of Wikidata Lexeme https://gerrit.wikimedia.org/r/637895 TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/setti

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

Lydia_Pintscher moved this task from Stalled/Waiting to To Do (prioritised from top to bottom) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. Lydia_Pintscher added a comment. Moving this back to todo because we need to get this unstuck. TASK DETAIL https://phabricator.wikim

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

hoo added a comment. I still need to test my changes (sadly there are no automated tests for these scripts and it would be excruciatingly hard to write some for the current bash scripts). Once that is done, this is good to go from my side. TASK DETAIL https://phabricator.wikimedia.org/T264

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

gerritbot added a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hoo, gerritbot Cc: WMDE-leszek, Pablo-WMDE, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Iflorez,

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

gerritbot added a comment. Change 637895 had a related patch set uploaded (by Hoo man; owner: Hoo man): [operations/puppet@production] Do weekly dumps of Wikidata Lexeme https://gerrit.wikimedia.org/r/637895 TASK DETAIL https://phabricator.wikimedia.org/T264883 EMAIL PREFERENCES ht

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

hoo claimed this task. hoo moved this task from To Do to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞) board. TASK DETAIL https://phabricator.wikimedia.org/T264883 WORKBOARD https://phabricator.wikimedia.org/project/board/3539/ EMAIL PREFERENCES https://phabricator.wikime

[Wikidata-bugs] [Maniphest] T264883: Prepare deployment of JSON dumps for Lexeme

Pablo-WMDE created this task. Pablo-WMDE added projects: Wikidata, Wikidata Lexicographical data, Wikidata-Campsite (Wikidata-Campsite-Iteration-∞). TASK DESCRIPTION - changes to operations-puppet (modules/snapshot/ in puppet, the bash for the JSON dumps is in `modules/snapshot/files/cron/dump