dcausse added a comment.
There are two dump types involved here: - the full dumps happening once a week and generating a RDF turtle file (puppet: `modules/snapshot/files/cron/dumpcategoriesrdf.sh`) - the daily diffs generating a SPARQL update file (puppet: `modules/snapshot/files/cron/dumpcategoriesrdf-daily.sh`) to know the date boundaries to run the daily dumps the shell scripts relies on timestamps stored in these `last` files. I suspect some erroneous conditions on the timestamps that prevent the daily dump from running. The `thankyou` wiki is pretty new and is probably why it's failing just now. There are a couple of quick fixes that could be made I suppose: - fix `modules/snapshot/files/cron/dumpcategoriesrdf-daily.sh` so that it supports the first run and properly generate a `last` file - on the wdqs side (`wikidata/query/rdf`: `dist/src/script/loadCategoryDaily.sh`) the daily reload should not fail on a missing `last` file - the list of wikis of wikis to dump and import is seeded from the the categories-rdf dblist (`operations/mediawiki-config` repo), this wiki `thankyou` wiki is perhaps not needed here Rewriting all this machinery with a higher level language is a large undertaking but definitely worthwhile as it's causing a lot of pain to debug and is quite fragile currently. TASK DETAIL https://phabricator.wikimedia.org/T261097 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, RKemper, Gehel, Aklapper, lmata, CBogen, Akuckartz, darthmon_wmde, Legado_Shulgin, Nandana, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
