[Wikidata-bugs] [Maniphest] T362849: [Analytics] Segments of Wikidata's data over time

2024-04-19 Thread mpopov
mpopov added subscribers: AndrewTavis_WMDE, mpopov. mpopov added a comment. @AndrewTavis_WMDE asked me for some thoughts/suggestions here :) I started typing out a DM reply but decided some of this stuff would be good to have on public record. > it's not normal that snap

[Wikidata-bugs] [Maniphest] T348999: Add linter and formatter to wmfdata-python (and link check)

2024-01-18 Thread mpopov
mpopov removed a project: Product-Analytics. TASK DETAIL https://phabricator.wikimedia.org/T348999 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, mpopov Cc: nshahquinn-wmf, xcollazo, Aklapper, AndrewTavis_WMDE

[Wikidata-bugs] [Maniphest] T349531: Add testing framework to wmfdata-python

2024-01-03 Thread mpopov
mpopov removed a project: Product-Analytics. TASK DETAIL https://phabricator.wikimedia.org/T349531 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov Cc: nshahquinn-wmf, xcollazo, Aklapper, AndrewTavis_WMDE, Danny_Benjafield_WMDE, Mohamed

[Wikidata-bugs] [Maniphest] T342111: [Analytics] Find out the size of direct instances of Q13442814 (scholarly article)

2023-07-31 Thread mpopov
mpopov added a comment. > are most people at WMF writing spark pythonically and not with queries? I guess it depends on who you talk to and what they're doing. All of the data scientists/analysts I work with use Spark SQL engine and write HiveQL queries, often because `hive.run

[Wikidata-bugs] [Maniphest] T177358: Metrics for SDoC: translations

2022-08-02 Thread mpopov
mpopov closed subtask T182352: UDF for language detection as "Invalid". TASK DETAIL https://phabricator.wikimedia.org/T177358 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov Cc: RhinosF1, PDrouin-WMF, Aklapper, mpopov, che

[Wikidata-bugs] [Maniphest] T292152: dashboard with daily query service usage not updating

2021-10-01 Thread mpopov
mpopov closed this task as a duplicate of T287381: External referrer & WDQS metrics stopped updating on 2021-04-25. TASK DETAIL https://phabricator.wikimedia.org/T292152 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov Cc: SWakiyama, MPha

[Wikidata-bugs] [Maniphest] T292152: dashboard with daily query service usage not updating

2021-10-01 Thread mpopov
mpopov added a comment. Thanks @MPhamWMF! What Mike and David said is correct. Also, this ticket prompted me to finally add the decommission notice to the dashboard (previously it was only on the homepage). In T292152#7391826 <https://phabricator.wikimedia.org/T292152#7391

[Wikidata-bugs] [Maniphest] [Unassigned] T199016: Count structured data uploads and edits by volunteer-built tools

2020-05-18 Thread mpopov
mpopov removed mpopov as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T199016 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov Cc: mpopov, Ramsey-WMF, Abit, CBogen, darthmon_wmde, Nandana, JKSTNK, Lahi, PDrouin-WMF

[Wikidata-bugs] [Maniphest] [Commented On] T239565: Create reportupdater reports that execute SDC requests

2019-12-09 Thread mpopov
mpopov added a comment. @Abit: it's still not entirely clear which query from T238878 <https://phabricator.wikimedia.org/T238878> @Milimetric should productionize in this ticket. From my conversation with Kate, it seems like your team wants to use the 7.8M number from the Lu

[Wikidata-bugs] [Maniphest] [Commented On] T239565: Create reportupdater reports that execute SDC requests

2019-12-04 Thread mpopov
mpopov added a comment. In T239565#5706854 <https://phabricator.wikimedia.org/T239565#5706854>, @Milimetric wrote: > Yay, I get to work with @mpopov :) Aw, I feel likewise! :D > - how often should this report be updated? I think for the intended purpo

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T238878: Data about how many file pages on Commons contain at least one structured data element

2019-11-22 Thread mpopov
mpopov added subscribers: Mayakp.wiki, daniel, Ladsgroup. mpopov added a comment. I was looking at populateEntityUsage.php <https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/Wikibase/+/814e7a53ab65e6a90f30cb9f066a04b822a76c71/client/maintenance/populateEntityUsage.

[Wikidata-bugs] [Maniphest] [Commented On] T238878: Data about how many file pages on Commons contain at least one structured data element

2019-11-22 Thread mpopov
mpopov added a comment. Here are the missing screenshots: In T238878#5683048 <https://phabricator.wikimedia.org/T238878#5683048>, @Nuria wrote: > The work done by @mpopov (if you are so kind @mpopov > please upload your screenshots) > The wbc_entity_usage table

[Wikidata-bugs] [Maniphest] [Commented On] T213597: [REQUEST] Baselines for structured data on Commons

2019-01-23 Thread mpopov
mpopov added a comment. @Abit @Ramsey-WMF in addition to T213597#4900741, here's the history of that metric with a 7-day rolling average to smooth the daily data a bit: F28004771: 2019-01_checkin.pngTASK DETAILhttps://phabricator.wikimedia.org/T213597EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T213597: [REQUEST] Baselines for structured data on Commons

2019-01-23 Thread mpopov
mpopov added a comment. In T213597#4900903, @Neil_P._Quinn_WMF wrote: True, but its revisions do have revision_is_deleted set, so you've already filtered them out of your query. Huh! Yeah, you're right! Haha, okay so I think what happened was I had checked the summarized_revisions ta

[Wikidata-bugs] [Maniphest] [Commented On] T213597: [REQUEST] Baselines for structured data on Commons

2019-01-22 Thread mpopov
mpopov added a comment. Okay, here are the numbers which were calculated with the following conditions: Using the December 2018 snapshot of MediaWiki History in the Data Lake Only files which have not been deleted are counted Only revisions to the metadata which were not reverted AND which were

[Wikidata-bugs] [Maniphest] [Commented On] T213597: [REQUEST] Baselines for structured data on Commons

2019-01-22 Thread mpopov
mpopov added a comment. In T213597#4893765, @Neil_P._Quinn_WMF wrote: I noticed once big thing: it seems like your counts of file page edits (n_edits_total, n_additions_total, etc.) include the initial edit that creates the pages, so in the end you're getting the proportion of files which

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T213597: [REQUEST] Baselines for structured data on Commons

2019-01-18 Thread mpopov
mpopov added subscribers: chelsyx, Neil_P._Quinn_WMF.mpopov added a comment. Okay, here are the numbers which were calculated with the following conditions: Using the December 2018 snapshot of MediaWiki History in the Data Lake Only files which have not been deleted are counted Only revisions to

[Wikidata-bugs] [Maniphest] [Commented On] T213597: [REQUEST] Baselines for structured data on Commons

2019-01-17 Thread mpopov
mpopov added a comment. Thanks for clarifying! Okay, one more question for @Abit & @Ramsey-WMF just so everyone is on the same page. The statistic you want is: the % of all uploaded files which have had additions to their pages in the first 2 months after upload. No breakdown by file type or

[Wikidata-bugs] [Maniphest] [Commented On] T213597: [REQUEST] Baselines for structured data on Commons

2019-01-16 Thread mpopov
mpopov added a comment. @Ramsey-WMF: hi, I would like to clarify what "metadata" includes. Here's my initial list: every field in the Information template Licensing Categories Or are you referring to the entire page as the metadata? i.e. the whole shebang: F27911262: Screen Sho

[Wikidata-bugs] [Maniphest] [Closed] T204415: Query stats dashboard not updating

2018-10-02 Thread mpopov
mpopov closed this task as "Resolved".mpopov added a comment. All good now :)TASK DETAILhttps://phabricator.wikimedia.org/T204415EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: Jonas, gerritbot, Gehel, mpopov, chelsyx, Aklapper, Addshore,

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T204415: Query stats dashboard not updating

2018-09-28 Thread mpopov
mpopov removed subscribers: mforns, Ottomata, elukey, Nuria.mpopov added a comment. Alright, I wiped all the request counts starting with August 10th (after making a backup) so Golden/Reportupdater is going to start a re-count using the webrequests in the 'text' partition. WDQS stat

[Wikidata-bugs] [Maniphest] [Unblock] T204415: Query stats dashboard not updating

2018-09-27 Thread mpopov
mpopov closed subtask T205441: 'group' parameter in Reportupdater for automatic chgrp of generated reports as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T204415EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: mfor

[Wikidata-bugs] [Maniphest] [Updated] T204415: Query stats dashboard not updating

2018-09-25 Thread mpopov
mpopov added a subscriber: mforns.mpopov added a comment. In T204415#4612751, @Ottomata wrote: Ok, I've added the analytics-search system user to the analytics-search-users group. You should make your script chgrp analytics-search-users after it creates it. Thank you very much, Andrew! T

[Wikidata-bugs] [Maniphest] [Commented On] T204415: Query stats dashboard not updating

2018-09-24 Thread mpopov
mpopov added a comment. @Ottomata @Gehel: I tried editing stat1005:/srv/published-datasets/discovery/metrics/wdqs/basic_usage.tsv but couldn't because the file belongs to group analytics-search, not analytics-search-users which sort of makes sense because of how we have it configured right n

[Wikidata-bugs] [Maniphest] [Commented On] T204415: Query stats dashboard not updating

2018-09-24 Thread mpopov
mpopov added a comment. In T204415#4611729, @Nuria wrote: Assigned to @mpopov Again, our apologies that the data sources are hardcoded like this. As I mentioned on our meeting abetter path to go forward would be using the tags for wdqs to identify the requests: https://github.com/wikimedia

[Wikidata-bugs] [Maniphest] [Updated] T204415: Query stats dashboard not updating

2018-09-24 Thread mpopov
mpopov added a subscriber: Gehel.mpopov added a comment. Thanks for looking into it, @Nuria! And for confirming, @elukey @Ottomata! :) A note for #operations: this is not the first time we've encountered an issue like this. Last year our query for Maps usage stopped working because of part

[Wikidata-bugs] [Maniphest] [Closed] T177358: Metrics for SDoC: translations

2018-04-23 Thread mpopov
mpopov closed this task as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T177358EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: PDrouin-WMF, Aklapper, mpopov, chelsyx, Abit, SandraF_WMF, Ramsey-WMF, Capt_Swing, debt,

[Wikidata-bugs] [Maniphest] [Unblock] T174519: [epic] SDoC: Determine baseline for metrics

2018-04-23 Thread mpopov
mpopov closed subtask T177358: Metrics for SDoC: translations as "Resolved".Herald added a project: Product-Analytics. TASK DETAILhttps://phabricator.wikimedia.org/T174519EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: chelsyx, mpopovCc: Nuria,

[Wikidata-bugs] [Maniphest] [Changed Project Column] T177358: Metrics for SDoC: translations

2017-12-13 Thread mpopov
mpopov moved this task from In progress to Needs review on the Discovery-Analysis (Current work) board.mpopov added a comment. Search query language breakdown note & results at https://github.com/wikimedia-research/SDoC-Initial-Metrics/tree/master/T177358-2TASK DETAILh

[Wikidata-bugs] [Maniphest] [Edited] T177358: Metrics for SDoC: translations

2017-12-13 Thread mpopov
mpopov updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...** [x] How many search queries happen in what languages?...TASK DETAILhttps://phabricator.wikimedia.org/T177358EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: Aklapper

[Wikidata-bugs] [Maniphest] [Claimed] T177358: Metrics for SDoC: translations

2017-12-07 Thread mpopov
mpopov claimed this task.mpopov set the point value for this task to "8". TASK DETAILhttps://phabricator.wikimedia.org/T177358EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: Aklapper, mpopov, chelsyx, Abit, SandraF_WMF, Ramsey-WMF, Capt_S

[Wikidata-bugs] [Maniphest] [Changed Project Column] T177357: Metrics for SDoC: future work of interest (templates and licensing)

2017-11-21 Thread mpopov
mpopov moved this task from Current work to Up Next on the Discovery-Analysis board.mpopov edited projects, added Discovery-Analysis; removed Discovery-Analysis (Current work). TASK DETAILhttps://phabricator.wikimedia.org/T177357WORKBOARDhttps://phabricator.wikimedia.org/project/board/1850/EMAIL

[Wikidata-bugs] [Maniphest] [Changed Project Column] T177357: Metrics for SDoC: future work of interest (templates and licensing)

2017-11-14 Thread mpopov
mpopov moved this task from Needs triage to Current work on the Discovery-Analysis board.mpopov edited projects, added Discovery-Analysis (Current work); removed Discovery-Analysis. TASK DETAILhttps://phabricator.wikimedia.org/T177357WORKBOARDhttps://phabricator.wikimedia.org/project/board/1850

[Wikidata-bugs] [Maniphest] [Commented On] T177354: Metrics for SDoC: look at contributions

2017-10-13 Thread mpopov
mpopov added a comment. @chelsyx do you wanna add your stuff to https://github.com/wikimedia-research/SDoC-Initial-Metrics ?TASK DETAILhttps://phabricator.wikimedia.org/T177354EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: chelsyx, mpopovCc: Aklapper, mpopov

[Wikidata-bugs] [Maniphest] [Changed Project Column] T177356: Metrics for SDoC: look at querying databases

2017-10-13 Thread mpopov
mpopov moved this task from In progress to Done on the Discovery-Analysis (Current work) board.mpopov added a comment. Queries & data uploaded to https://github.com/wikimedia-research/SDoC-Initial-Metrics Moving this into 'Done' as I don't think there's anything lef

[Wikidata-bugs] [Maniphest] [Edited] T177356: Metrics for SDoC: look at querying databases

2017-10-13 Thread mpopov
mpopov updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...** [x] How many people are involved in flagging for deletion/deleting files TASK DETAILhttps://phabricator.wikimedia.org/T177356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To

[Wikidata-bugs] [Maniphest] [Commented On] T177356: Metrics for SDoC: look at querying databases

2017-10-13 Thread mpopov
mpopov added a comment. Growth of number of deleters over time: F10188497: cumulative_deleters.png How many users deleted N-many files: F10188503: deleter_activity.pngTASK DETAILhttps://phabricator.wikimedia.org/T177356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Commented On] T177356: Metrics for SDoC: look at querying databases

2017-10-13 Thread mpopov
mpopov added a comment. Total files uploaded to Commons (as of right now) by extension: mediaextensionuploads audioogg773305 audiooga6180 audioflac6140 audiomid4993 audiowav3512 audioopus410 docspdf354765 docsdjvu60524 imagejpg/jpeg36918799 imagepng2268026 imagesvg1176530 imagetif/tiff807921

[Wikidata-bugs] [Maniphest] [Edited] T177356: Metrics for SDoC: look at querying databases

2017-10-13 Thread mpopov
mpopov updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...* [x] How many: mpegs, pngs, ogg, etc...** [x] Track organic growth rate of uploads (historical trends)...TASK DETAILhttps://phabricator.wikimedia.org/T177356EMAIL PREFERENCEShttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Edited] T177356: Metrics for SDoC: look at querying databases

2017-10-11 Thread mpopov
mpopov updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...** [x] Average time to deletion? * [] How many people are involved in flagging for deletion/deleting files TASK DETAILhttps://phabricator.wikimedia.org/T177356EMAIL PREFERENCEShttps://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Commented On] T177356: Metrics for SDoC: look at querying databases

2017-10-11 Thread mpopov
mpopov added a comment. Time-to-deletion: F10150716: time-to-deletion.png Most copyright-related deletions happen within 1 day of upload across almost all media types, with the exception of 'drawing' (SVGs) A lot of audio files are deleted within 1 minute or 1 week of upload Half of

[Wikidata-bugs] [Maniphest] [Edited] T177356: Metrics for SDoC: look at querying databases

2017-10-11 Thread mpopov
mpopov updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...*** copyright violations (Use case: creation of auto-copyright violation tools) Use case: creation of auto-copyright violation tools*** [[ https://commons.wikimedia.org/wiki/Commons:OTRS | OTRS ]] ** [] ores

[Wikidata-bugs] [Maniphest] [Commented On] T177356: Metrics for SDoC: look at querying databases

2017-10-11 Thread mpopov
mpopov added a comment. Reasons for files deleted in 2017: F10148687: deletion_reasons.pngTASK DETAILhttps://phabricator.wikimedia.org/T177356EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: Aklapper, mpopov, chelsyx, Abit, SandraF_WMF, Ramsey-WMF

[Wikidata-bugs] [Maniphest] [Commented On] T177354: Metrics for SDoC: look at contributions

2017-10-11 Thread mpopov
mpopov added a comment. In T177354#3676545, @chelsyx wrote: Unfortunately, the mediawiki snapshot doesn't has the image table which describes images and other uploaded files. Ah, yeah. I missed the reference to image in your query. But looks like we can use img_timestamp, although those qu

[Wikidata-bugs] [Maniphest] [Commented On] T177354: Metrics for SDoC: look at contributions

2017-10-11 Thread mpopov
mpopov added a comment. In T177354#3675988, @debt wrote: Hey @chelsyx - what time frame does this cover? Jumping in to say this looks like it's from launch of Commons to now. Can we also get a count of how this has changed over the last week and compare that to the last 30 days? It

[Wikidata-bugs] [Maniphest] [Claimed] T177356: Metrics for SDoC: look at querying databases

2017-10-11 Thread mpopov
mpopov moved this task from Backlog to In progress on the Discovery-Analysis (Current work) board.mpopov set the point value for this task to "6".mpopov claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T177356WORKBOARDhttps://phabricator.wikimedia.org/project/board/

[Wikidata-bugs] [Maniphest] [Changed Project Column] T177356: Metrics for SDoC: look at querying databases

2017-10-11 Thread mpopov
mpopov moved this task from Needs triage to Current work on the Discovery-Analysis board.mpopov edited projects, added Discovery-Analysis (Current work); removed Discovery-Analysis. TASK DETAILhttps://phabricator.wikimedia.org/T177356WORKBOARDhttps://phabricator.wikimedia.org/project/board/1850

[Wikidata-bugs] [Maniphest] [Commented On] T149963: Analyze WDQS traffic data to find parallel connection patterns

2016-11-30 Thread mpopov
mpopov added a comment. How many IPs use parallel connections to the WDQS servers? Out of the IPs that do the above, how many have the same/different user agents (hinting at one tool or proxy serving multiple clients)? Of 14K unique IPs observed between Nov 1st and 28th, 1.9K (13.6%) had made

[Wikidata-bugs] [Maniphest] [Commented On] T149963: Analyze WDQS traffic data to find parallel connection patterns

2016-11-30 Thread mpopov
mpopov added a comment. @Smalyshev: still in the process of figuring out the parallel connection aspect but here are some minute-by-minute-over-24-hours graphs/stats you might be interested in that I made in the process of playing with the data F4911654: sparql_median_2.png F4911656

[Wikidata-bugs] [Maniphest] [Changed Project Column] T149963: Analyze WDQS traffic data to find parallel connection patterns

2016-11-28 Thread mpopov
mpopov moved this task from Up Next to Current work on the Discovery-Analysis board.mpopov edited projects, added Discovery-Analysis (Current work); removed Discovery-Analysis. TASK DETAILhttps://phabricator.wikimedia.org/T149963WORKBOARDhttps://phabricator.wikimedia.org/project/board/1850/EMAIL

[Wikidata-bugs] [Maniphest] [Claimed] T149963: Analyze WDQS traffic data to find parallel connection patterns

2016-11-28 Thread mpopov
mpopov claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T149963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: debt, Deskana, chelsyx, mpopov, Gehel, Aklapper, Smalyshev, EBjune, mschwarzer, Avner, D3r1ck01, Jonas, FloNight, Xmlizer

[Wikidata-bugs] [Maniphest] [Commented On] T143762: WDQS: Geographic breakdown of SPARQL queries

2016-09-27 Thread mpopov
mpopov added a comment. Great job! Let's put it up on Commons! :) Use the following licensing & categorization: =={{int:license-header}}== {{WMF-staff-upload|license=cc-by-sa-4.0}} {{Wikimedia trademark}} [[Category:Wikimedia Discovery]] [[Category:Wiki Research]]TASK DE

[Wikidata-bugs] [Maniphest] [Commented On] T143762: WDQS: Geographic breakdown of SPARQL queries

2016-09-13 Thread mpopov
mpopov added a comment. Reviewed copy with minor corrections & suggestions sent back to Chelsy.TASK DETAILhttps://phabricator.wikimedia.org/T143762EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: chelsyx, mpopovCc: Addshore, Aklapper, mpopov, Smalyshev,

[Wikidata-bugs] [Maniphest] [Commented On] T143762: WDQS: Geographic breakdown of SPARQL queries

2016-09-01 Thread mpopov
mpopov added a comment. Reviewed; marked-up copy of the 1st draft sent back to Chelsy. Looking forward to 2nd draft :PTASK DETAILhttps://phabricator.wikimedia.org/T143762EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: chelsyx, mpopovCc: Aklapper, mpopov

[Wikidata-bugs] [Maniphest] [Commented On] T143762: WDQS: Geographic breakdown of SPARQL queries

2016-08-31 Thread mpopov
mpopov added a comment. First draft looks good! I will try to review this as soon as I can :)TASK DETAILhttps://phabricator.wikimedia.org/T143762EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: chelsyx, mpopovCc: Aklapper, mpopov, Smalyshev, debt, mschwarzer

[Wikidata-bugs] [Maniphest] [Edited] T143762: WDQS: Geographic breakdown of SPARQL queries

2016-08-23 Thread mpopov
mpopov edited the task description. (Show Details) EDIT DETAILS...* These articles on [[ https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hive | Hive ]] and [[ https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hive/Queries | Hive queries ]] are good resources. That second one uses

[Wikidata-bugs] [Maniphest] [Created] T143762: WDQS: Geographic breakdown of SPARQL queries

2016-08-23 Thread mpopov
mpopov created this task.mpopov added projects: Discovery-Analysis (Current work), Epic, Wikidata-Query-Service.Herald added projects: Wikidata, Discovery. TASK DESCRIPTIONBackground In T112605, we performed a broad analysis of Wikidata Query Service users and queries. This was almost a year ago

[Wikidata-bugs] [Maniphest] [Claimed] T141135: "median" not working on WDQS dashboards

2016-08-08 Thread mpopov
mpopov claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T141135EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: mpopov, Aklapper, Smalyshev, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer, Izno, jkroll, Wikidata-bugs, Jdouglas

[Wikidata-bugs] [Maniphest] [Updated] T141135: "median" not working on WDQS dashboards

2016-08-08 Thread mpopov
mpopov edited projects, added Discovery-Analysis-Sprint; removed Discovery-Analysis-Backlog. TASK DETAILhttps://phabricator.wikimedia.org/T141135EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: mpopov, Aklapper, Smalyshev, Avner, debt, Gehel

[Wikidata-bugs] [Maniphest] [Commented On] T141135: "median" not working on WDQS dashboards

2016-08-08 Thread mpopov
mpopov added a comment. Done: http://discovery.wmflabs.org/wdqs/TASK DETAILhttps://phabricator.wikimedia.org/T141135EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: mpopov, Aklapper, Smalyshev, Avner, debt, Gehel, D3r1ck01, Jonas, FloNight, Xmlizer

[Wikidata-bugs] [Maniphest] [Commented On] T141135: "median" not working on WDQS dashboards

2016-08-08 Thread mpopov
mpopov added a comment. Forgot to tag this in https://gerrit.wikimedia.org/r/#/c/303582/TASK DETAILhttps://phabricator.wikimedia.org/T141135EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: mpopovCc: mpopov, Aklapper, Smalyshev, Avner, debt, Gehel, D3r1ck01

[Wikidata-bugs] [Maniphest] [Commented On] T111790: Improve Phabricator link on Wikidata Query Service dashboard

2015-09-08 Thread mpopov
mpopov added a comment. They do not. Wikimedia repos on GitHub are simple mirrors of Gerrit. To the point where the version on GitHub says that OliverKeyes committed to it but there's no such user. The patch needs to be submitted to Gerrit. TASK DETAIL https://phabricator.wikimedi

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109361: Create a Wikidata query service usage dashboard

2015-09-02 Thread mpopov
mpopov moved this task to Done on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109361 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov

[Wikidata-bugs] [Maniphest] [Commented On] T109361: Create a Wikidata query service usage dashboard

2015-09-02 Thread mpopov
mpopov added a comment. First version is live at http://searchdata.wmflabs.org/wdqs/ Will chat with Stas soon to clarify/fix any issues with the data/queries. P.S. Also gave the Discovery Dashboards page a bit of a facelift http://searchdata.wmflabs.org/ :D TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109361: Create a Wikidata query service usage dashboard

2015-09-02 Thread mpopov
mpopov moved this task to Stalled/Waiting on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109361 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109361: Create a Wikidata query service usage dashboard

2015-09-02 Thread mpopov
mpopov moved this task to In progress on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109361 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Commented On] T109361: Create a Wikidata query service usage dashboard

2015-09-01 Thread mpopov
mpopov added a comment. Waiting for code review: https://gerrit.wikimedia.org/r/#/c/235365/ TASK DETAIL https://phabricator.wikimedia.org/T109361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov Cc: EBernhardson, mpopov, Ironholds, Aklapper

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109361: Create a Wikidata query service usage dashboard

2015-09-01 Thread mpopov
mpopov moved this task to Stalled/Waiting on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109361 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-09-01 Thread mpopov
mpopov added a comment. Script: https://gerrit.wikimedia.org/r/#/c/235137/1/data_retrieval/wdqs.R Oliver added it to the scheduler and I ran it on the past 40 days to backfill the aggregate dataset that will be up-to-date going forward. TASK DETAIL https://phabricator.wikimedia.org/T109360

[Wikidata-bugs] [Maniphest] [Commented On] T109361: Create a Wikidata query service usage dashboard

2015-09-01 Thread mpopov
mpopov added a comment. Awesome, thank you @EBernhardson :D TASK DETAIL https://phabricator.wikimedia.org/T109361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov Cc: EBernhardson, mpopov, Ironholds, Aklapper, Smalyshev, jkroll, Wikidata

[Wikidata-bugs] [Maniphest] [Commented On] T109361: Create a Wikidata query service usage dashboard

2015-08-31 Thread mpopov
mpopov added a comment. Erik is working on an issue with installing new R packages, especially ones that require version of R newer (e.g. 3.1.2) than what is currently installed (3.0.2). The dashboard is live at http://searchdata.wmflabs.org/wdqs/ but is currently busted because of lack of

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-08-31 Thread mpopov
mpopov moved this task to Done on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109360 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov

[Wikidata-bugs] [Maniphest] [Changed Project Column] T108732: [Task] Train Wikidata people on how to add data/metrics to a Shiny dashboard for Wikidata

2015-08-31 Thread mpopov
mpopov moved this task to Done on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T108732 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov

[Wikidata-bugs] [Maniphest] [Changed Project Column] T108732: [Task] Train Wikidata people on how to add data/metrics to a Shiny dashboard for Wikidata

2015-08-28 Thread mpopov
mpopov moved this task to In progress on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T108732 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Claimed] T108732: [Task] Train Wikidata people on how to add data/metrics to a Shiny dashboard for Wikidata

2015-08-28 Thread mpopov
mpopov claimed this task. mpopov set Story Points to 2. TASK DETAIL https://phabricator.wikimedia.org/T108732 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov Cc: Ironholds_backup, Abraham, Christopher, Lydia_Pintscher, Ironholds, JanZerebecki

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109361: Create a Wikidata query service usage dashboard

2015-08-28 Thread mpopov
mpopov moved this task to Stalled/Waiting on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109361 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T109361: Create a Wikidata query service usage dashboard

2015-08-28 Thread mpopov
mpopov added a comment. Dedicated WDQS dashboard is sitting locally on my computer. Waiting for my request for project to be done so I can push the code out to Gerrit. TASK DETAIL https://phabricator.wikimedia.org/T109361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109361: Create a Wikidata query service usage dashboard

2015-08-28 Thread mpopov
mpopov moved this task to In progress on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109361 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109361: Create a Wikidata query service usage dashboard

2015-08-27 Thread mpopov
mpopov moved this task to Needs review on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109361 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-08-27 Thread mpopov
mpopov moved this task to Needs review on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109360 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Claimed] T109361: Create a Wikidata query service usage dashboard

2015-08-27 Thread mpopov
mpopov claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T109361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: mpopov Cc: mpopov, Ironholds, Aklapper, Smalyshev, jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, JanZerebecki

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109361: Create a Wikidata query service usage dashboard

2015-08-27 Thread mpopov
mpopov moved this task to In progress on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109361 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-08-27 Thread mpopov
mpopov moved this task to In progress on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109360 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-08-26 Thread mpopov
mpopov moved this task to Backlog on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109360 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-08-25 Thread mpopov
mpopov moved this task to In progress on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109360 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-08-25 Thread mpopov
mpopov moved this task to Backlog on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109360 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Commented On] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-08-21 Thread mpopov
mpopov added a comment. Need to transfer the logic from HiveQL query to UDF and then to run the script on previous days to fill in the backlog. TASK DETAIL https://phabricator.wikimedia.org/T109360 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-08-21 Thread mpopov
mpopov moved this task to Stalled/Waiting on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109360 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Changed Project Column] T109360: Create a script to extract request logs for query.wikidata.org for dashboards

2015-08-19 Thread mpopov
mpopov moved this task to In progress on the Discovery-Analysis-Sprint workboard. TASK DETAIL https://phabricator.wikimedia.org/T109360 WORKBOARD https://phabricator.wikimedia.org/project/board/1241/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Updated] T108732: Train Jan Zerebecki of Wikimedia Germany on how to set up a Shiny dashboard for Wikidata

2015-08-12 Thread mpopov
mpopov added a blocking task: T108094: As a project lead, I'd like documentation on how to set up a Shiny dashboard so that I can visualise the project's key performance indicators . TASK DETAIL https://phabricator.wikimedia.org/T108732 EMAIL PREFERENCES https://phabricator.wik