mpopov added a comment.
In T239565#5706854 <https://phabricator.wikimedia.org/T239565#5706854>, @Milimetric wrote: > Yay, I get to work with @mpopov :) Aw, I feel likewise! :D > - how often should this report be updated? I think for the intended purpose a monthly granularity is fine since the check-ins have in the past been quarterly or every 6mo. Even if the query takes like 35 minutes to run on unsqooped data, would it be okay to schedule it to run daily or weekly? > - is it exactly that query? This task mentions "queries" plural, just making sure It's starting to look like the query in T238878#5708511 <https://phabricator.wikimedia.org/T238878#5708511> is the one that should be used? > - given the confusion about deletion (T238878#5706835 <https://phabricator.wikimedia.org/T238878#5706835>), should we also count stuff from the archive table? I don't think deleted files should be counted, no. ---- I think the end result should be, ideally, a daily-granularity data source in Turnilo/Superset having: - total count of files on Commons - total count of files on Commons having structured data (per query in T238878#5708511 <https://phabricator.wikimedia.org/T238878#5708511>) This would enable @Abit & @Ramsey-WMF to track progress of SDC over time in a dashboard as (1) an absolute, and (2) relative % (via post-aggregation in Superset) in Superset (esp. since that also has periodicity like YoY built in, which would be useful for them). Would have to be careful with the auto aggregation, though. The metrics would need to be specified as, like, longMax instead of longSum. @Milimetric: do you have a destination in mind for the reports? I guess the MVP is just a CSV in /srv/published-datasets and we can figure out next steps later so this task's scope doesn't blow up, or do y'all have an easy pipeline/process for running reportupdater and ingesting the output into Druid? TASK DETAIL https://phabricator.wikimedia.org/T239565 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Milimetric, mpopov Cc: Abit, Ramsey-WMF, kzimmerman, Addshore, matthiasmullie, gsingers, Mayakp.wiki, Ladsgroup, nettrom_WMF, Cparle, Nuria, Milimetric, mpopov, 4748kitoko, darthmon_wmde, DannyS712, Nandana, JKSTNK, Akovalyov, Lahi, PDrouin-WMF, Gq86, E1presidente, Anooprao, SandraF_WMF, GoranSMilovanovic, QZanden, Tramullas, Acer, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, Susannaanas, JAllemandou, Jane023, terrrydactyl, Wikidata-bugs, Base, aude, Ricordisamoa, Wesalius, Lydia_Pintscher, Fabrice_Florin, Raymond, Steinsplitter, Mbch331, jeremyb
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
