Christopher Johnson (WMDE) has submitted this change and it was merged. Change subject: changes per prototype review adds preliminary owl for metric definitions adds dates and percentage to infoBox reorganizes tabs adds items as a separate graph ......................................................................
changes per prototype review adds preliminary owl for metric definitions adds dates and percentage to infoBox reorganizes tabs adds items as a separate graph Bug: T108404 Change-Id: I89e1bd115ea31ca4e344ae23f6d37f2615380981 --- A assets/content/wikidata_content_items.md M assets/content/wikidata_content_refstmts_other.md M assets/content/wikidata_content_refstmts_wikipedia.md A assets/metrics.owl A data/wikidata_content_items.tsv M server.R M ui.R M utils.R 8 files changed, 309 insertions(+), 40 deletions(-) Approvals: Christopher Johnson (WMDE): Verified; Looks good to me, approved diff --git a/assets/content/wikidata_content_items.md b/assets/content/wikidata_content_items.md new file mode 100644 index 0000000..a29a4e6 --- /dev/null +++ b/assets/content/wikidata_content_items.md @@ -0,0 +1,8 @@ +Wikidata Items +======= + +* Type: Aggregate +* Source: +* Update Interval: +* Reliability: +* Description: diff --git a/assets/content/wikidata_content_refstmts_other.md b/assets/content/wikidata_content_refstmts_other.md index 561614f..af9e300 100644 --- a/assets/content/wikidata_content_refstmts_other.md +++ b/assets/content/wikidata_content_refstmts_other.md @@ -1,4 +1,4 @@ -Statements referenced to other sources +Referenced Statements to other sources ======= * Type: Aggregate diff --git a/assets/content/wikidata_content_refstmts_wikipedia.md b/assets/content/wikidata_content_refstmts_wikipedia.md index c74e91a..26d2e74 100644 --- a/assets/content/wikidata_content_refstmts_wikipedia.md +++ b/assets/content/wikidata_content_refstmts_wikipedia.md @@ -1,4 +1,4 @@ -Statements referenced to Wikipedia +Referenced Statements to Wikipedia ======= * Type: Aggregate diff --git a/assets/metrics.owl b/assets/metrics.owl new file mode 100644 index 0000000..e8bf389 --- /dev/null +++ b/assets/metrics.owl @@ -0,0 +1,174 @@ +<?xml version="1.0"?> + + +<!DOCTYPE rdf:RDF [ + <!ENTITY owl "http://www.w3.org/2002/07/owl#"; > + <!ENTITY xsd "http://www.w3.org/2001/XMLSchema#"; > + <!ENTITY rdfs "http://www.w3.org/2000/01/rdf-schema#"; > + <!ENTITY rdf "http://www.w3.org/1999/02/22-rdf-syntax-ns#"; > +]> + + +<rdf:RDF xmlns="http://wikiba.se/metrics#"; + xml:base="http://wikiba.se/metrics"; + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"; + xmlns:owl="http://www.w3.org/2002/07/owl#"; + xmlns:xsd="http://www.w3.org/2001/XMLSchema#"; + xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#";> + <owl:Ontology rdf:about="http://wikiba.se/metrics#"/> + + + + <!-- + /////////////////////////////////////////////////////////////////////////////////////// + // + // Classes + // + /////////////////////////////////////////////////////////////////////////////////////// + --> + + + + + <!-- http://wikiba.se/metrics#Community_Health --> + + <owl:Class rdf:about="http://wikiba.se/metrics#Community_Health";> + <rdfs:subClassOf rdf:resource="http://wikiba.se/metrics#KPI"/> + </owl:Class> + + + + <!-- http://wikiba.se/metrics#Content --> + + <owl:Class rdf:about="http://wikiba.se/metrics#Content"/> + + + + <!-- http://wikiba.se/metrics#Developer --> + + <owl:Class rdf:about="http://wikiba.se/metrics#Developer"/> + + + + <!-- http://wikiba.se/metrics#Engagement --> + + <owl:Class rdf:about="http://wikiba.se/metrics#Engagement"/> + + + + <!-- http://wikiba.se/metrics#External_Use --> + + <owl:Class rdf:about="http://wikiba.se/metrics#External_Use";> + <rdfs:subClassOf rdf:resource="http://wikiba.se/metrics#KPI"/> + </owl:Class> + + + + <!-- http://wikiba.se/metrics#Internal_Use --> + + <owl:Class rdf:about="http://wikiba.se/metrics#Internal_Use";> + <rdfs:subClassOf rdf:resource="http://wikiba.se/metrics#KPI"/> + </owl:Class> + + + + <!-- http://wikiba.se/metrics#KPI --> + + <owl:Class rdf:about="http://wikiba.se/metrics#KPI"/> + + + + <!-- http://wikiba.se/metrics#Partnerships --> + + <owl:Class rdf:about="http://wikiba.se/metrics#Partnerships";> + <rdfs:subClassOf rdf:resource="http://wikiba.se/metrics#KPI"/> + </owl:Class> + + + + <!-- http://wikiba.se/metrics#Quality --> + + <owl:Class rdf:about="http://wikiba.se/metrics#Quality";> + <rdfs:subClassOf rdf:resource="http://wikiba.se/metrics#KPI"/> + </owl:Class> + + + + <!-- + /////////////////////////////////////////////////////////////////////////////////////// + // + // Individuals + // + /////////////////////////////////////////////////////////////////////////////////////// + --> + + + + + <!-- http://wikiba.se/metrics#percentage_of_statements_with_a_non-Wikimedia_reference_as_of_a_given_month --> + + <owl:NamedIndividual rdf:about="http://wikiba.se/metrics#percentage_of_statements_with_a_non-Wikimedia_reference_as_of_a_given_month";> + <rdf:type rdf:resource="http://wikiba.se/metrics#Quality"/> + <rdfs:comment>Total number is reported every two weeks in https://tools.wmflabs.org/wikidata-todo/stats.php ("Statements referenced to Wikipedia")</rdfs:comment> + </owl:NamedIndividual> + + + + <!-- http://wikiba.se/metrics#number_of_active_editors_who_make_[5,_100]_edits_in_[given_month,_rolling_30_day_window] --> + + <owl:NamedIndividual rdf:about="http://wikiba.se/metrics#number_of_active_editors_who_make_[5,_100]_edits_in_[given_month,_rolling_30_day_window]";> + <rdf:type rdf:resource="http://wikiba.se/metrics#Community_Health"/> + <rdfs:comment>Reported by month in https://stats.wikimedia.org/wikispecial/EN/TablesWikipediaWIKIDATA.htm per longstanding standard definitions. Rolling 30 day window report in https://metrics.wmflabs.org/static/public/dash/ but only for 5 edit threshold</rdfs:comment> + <rdfs:comment></rdfs:comment> + </owl:NamedIndividual> + + + + <!-- http://wikiba.se/metrics#number_of_items/statements_contributed_by_[partnership_A,_..._partnership_Z]_in_a_given_month,_broken_down_by_quality,_edited_statements,_setup_length,_community_onboarding_time,_technical_audit,_size_of_institution,_usage_of_data_after_launch --> + + <owl:NamedIndividual rdf:about="http://wikiba.se/metrics#number_of_items/statements_contributed_by_[partnership_A,_..._partnership_Z]_in_a_given_month,_broken_down_by_quality,_edited_statements,_setup_length,_community_onboarding_time,_technical_audit,_size_of_institution,_usage_of_data_after_launch";> + <rdf:type rdf:resource="http://wikiba.se/metrics#Partnerships"/> + <rdfs:comment>Should be fairly straightforward to track once actual partnerships exist and quality/usage measures mature; see https://commons.wikimedia.org/wiki/Commons:Partnerships for pre-existing models in Wikimedia Commons</rdfs:comment> + </owl:NamedIndividual> + + + + <!-- http://wikiba.se/metrics#number_of_queries_by_[non-browser_app_A,_...,_non-browser_app_Z]_in_[given_month,_rolling_30_day_window] --> + + <owl:NamedIndividual rdf:about="http://wikiba.se/metrics#number_of_queries_by_[non-browser_app_A,_...,_non-browser_app_Z]_in_[given_month,_rolling_30_day_window]";> + <rdf:type rdf:resource="http://wikiba.se/metrics#External_Use"/> + <rdfs:comment>In principle obtainable via Hadoop analysis of logs; application-breakdown can be tricky as we don't require API keys and user-agents may be unhelpful</rdfs:comment> + </owl:NamedIndividual> + + + + <!-- http://wikiba.se/metrics#percentage/number_of_infoboxes_in_Wikipedia_which_use_(1,_...,_n)_statements_from_Wikidata_as_of_a_given_month --> + + <owl:NamedIndividual rdf:about="http://wikiba.se/metrics#percentage/number_of_infoboxes_in_Wikipedia_which_use_(1,_...,_n)_statements_from_Wikidata_as_of_a_given_month"> + <rdf:type rdf:resource="http://wikiba.se/metrics#Internal_Use"/> + <rdfs:comment>Depends on usage tracking support for items, which is being gradually rolled out as of May 2015, see https://phabricator.wikimedia.org/T49288</rdfs:comment> + </owl:NamedIndividual> + + + + <!-- http://wikiba.se/metrics#percentage/number_of_pages_in_Wikimedia_wikis_which_use_[1,_...,_n]_statements_from_Wikidata_as_of_a_given_month --> + + <owl:NamedIndividual rdf:about="http://wikiba.se/metrics#percentage/number_of_pages_in_Wikimedia_wikis_which_use_[1,_...,_n]_statements_from_Wikidata_as_of_a_given_month";> + <rdf:type rdf:resource="http://wikiba.se/metrics#Internal_Use"/> + <rdfs:comment>Depends on usage tracking support for items, which is being gradually rolled out as of May 2015, see https://phabricator.wikimedia.org/T49288</rdfs:comment> + </owl:NamedIndividual> + + + + <!-- http://wikiba.se/metrics#percentage_of_items_with_a_quality_score_according_to_[scoring_algorithm_A,_...,_scoring_algorithm_Z]_higher_than_[0,_...,_1] --> + + <owl:NamedIndividual rdf:about="http://wikiba.se/metrics#percentage_of_items_with_a_quality_score_according_to_[scoring_algorithm_A,_...,_scoring_algorithm_Z]_higher_than_[0,_...,_1]";> + <rdf:type rdf:resource="http://wikiba.se/metrics#Quality"/> + <rdfs:comment>Experimental Wikipedia-centric work ongoing at https://meta.wikimedia.org/wiki/Research:Revision_scoring_as_a_service , Wikidatan User:Ladsgroup is peripherally involved to assess value for fighting spam on Wikidata</rdfs:comment> + </owl:NamedIndividual> +</rdf:RDF> + + + +<!-- Generated by the OWL API (version 3.5.0) http://owlapi.sourceforge.net --> + diff --git a/data/wikidata_content_items.tsv b/data/wikidata_content_items.tsv new file mode 100644 index 0000000..009373b --- /dev/null +++ b/data/wikidata_content_items.tsv @@ -0,0 +1,65 @@ +Date Total Items +2013-02-04 3887755 +2013-02-15 4423079 +2013-02-28 5825626 +2013-03-16 7040595 +2013-03-30 8534183 +2013-04-17 10621495 +2013-05-05 12022530 +2013-05-14 12402098 +2013-05-27 12565377 +2013-06-10 12562722 +2013-06-23 12639014 +2013-07-10 12783651 +2013-07-27 13355707 +2013-08-05 13566707 +2013-08-17 13608804 +2013-08-27 13685369 +2013-09-10 13833267 +2013-09-22 13889820 +2013-10-06 13989956 +2013-10-21 14010471 +2013-11-05 14042236 +2013-11-20 14082962 +2013-12-05 14132136 +2013-12-21 14240430 +2014-01-06 14404065 +2014-01-23 14371860 +2014-02-10 14440962 +2014-02-26 14519234 +2014-03-15 14616935 +2014-03-31 14648218 +2014-04-20 15093996 +2014-05-08 15331176 +2014-05-26 15429932 +2014-06-12 15590480 +2014-07-05 15685743 +2014-07-31 15789885 +2014-08-04 15792256 +2014-08-23 15856289 +2014-09-15 16199115 +2014-09-29 16266835 +2014-10-13 16320152 +2014-10-27 16421663 +2014-11-10 16572031 +2014-11-24 16606415 +2014-12-10 16628776 +2014-12-22 16668610 +2015-01-05 16675727 +2015-01-20 16770028 +2015-02-02 16864651 +2015-02-16 17198893 +2015-03-02 17271174 +2015-03-16 17474239 +2015-03-30 17553149 +2015-04-13 17622509 +2015-04-27 17632496 +2015-05-11 17679971 +2015-05-25 17679766 +2015-06-08 17752335 +2015-06-22 17880164 +2015-07-06 18308081 +2015-07-20 18336598 +2015-08-05 18378250 +2015-08-17 18434725 +2015-09-07 18514196 diff --git a/server.R b/server.R index ab7dcdc..5fdb51a 100644 --- a/server.R +++ b/server.R @@ -5,14 +5,13 @@ get_datasets <- function(){ wikidata_edits <<- download_set("wikidata-edits.tsv") - edits_latest <- cbind("Edits" = safe_tail(wikidata_edits$edits, 2)) - edits_delta <<- diff(edits_latest) - wikidata_pages <<- download_set("wikidata-pages.tsv") - wikidata_properties <<- download_set("wikidata-properties.tsv") wikidata_active_users <<- download_set("wikidata-active-users.tsv") wikidata_social_media <<- download_set("wikidata-social-media.tsv") wikidata_mailing_lists <<-download_set("wikidata_mailing_lists.tsv") wikidata_content_overview <<- download_set("wikidata-content-overview.tsv") + wikidata_pages <<- download_set("wikidata-pages.tsv") + wikidata_content_items <<- download_set("wikidata_content_items.tsv") + wikidata_properties <<- download_set("wikidata-properties.tsv") wikidata_content_refstmts <<-download_set("wikidata-content-refstmts.tsv") wikidata_content_refstmts_wikipedia <<- download_set("wikidata-content-refstmts-wikipedia.tsv") wikidata_content_refstmts_other <<- download_set("wikidata_content_refstmts_other.tsv") @@ -37,29 +36,20 @@ "", "Edits", "Wikidata Edits") }) output$editdelta <- renderInfoBox({ + edits_period <- tail(wikidata_edits$date, 2) + period_last <- format(edits_period[2]) + period_current <- format(edits_period[1]) + edits_latest <- cbind("Edits" = safe_tail(wikidata_edits$edits, 2)) + edits_delta <<- diff(edits_latest) + edits_last_total <- edits_latest[1] + edits_delta_percentage <- percent(edits_delta/edits_last_total) form_edits <- prettyNum(edits_delta, big.mark=",") - infoBox( - "Edit Delta from Last Period", paste0(form_edits), icon = icon("arrow-up"), + box_title <- paste0("Edit Delta") + box_subtitle <- paste0(period_current, " to ", period_last) + box_value <- paste0(form_edits, " - ", edits_delta_percentage) + infoBox(box_title, box_value, box_subtitle, icon = icon("arrow-up"), color = "green" ) - }) - output$wikidata_pages_plot <- renderDygraph({ - make_dygraph(wikidata_pages, - "", "Pages", "Wikidata Pages", legend_name = "pages") - }) - output$wikidata_properties_plot <- renderDygraph({ - wikidata_properties<- xts(wikidata_properties[, -1], wikidata_properties[, 1]) - return(dygraph(wikidata_properties, - main = "Wikidata Properties", - ylab = "Properties") %>% - dyLegend(width = 400, show = "always") %>% - dySeries("V1", label = "properties") %>% - dyOptions(useDataTimezone = TRUE, - labelsKMB = TRUE, - connectSeparatedPoints = TRUE, - strokeWidth = 3, - colors = brewer.pal(max(3, ncol(data)), "Set1")) %>% - dyCSS(css = "./assets/css/custom.css")) }) output$wikidata_active_users_plot <- renderDygraph({ make_dygraph(wikidata_active_users, @@ -87,10 +77,41 @@ strokeWidth = 2, colors = brewer.pal(5, "Set2")[5:1]) %>% dyCSS(css = "./assets/css/custom.css")) }) + output$wikidata_pages_plot <- renderDygraph({ + make_dygraph(wikidata_pages, + "", "Pages", "Wikidata Pages", legend_name = "pages") + }) + output$wikidata_content_items_plot <- renderDygraph({ + wikidata_content_items<- xts(wikidata_content_items[, -1], wikidata_content_items[, 1]) + return(dygraph(wikidata_content_items, + main = "Wikidata Items", + ylab = "Items") %>% + dyLegend(width = 400, show = "always") %>% + dySeries("V1", label = "Items") %>% + dyOptions(useDataTimezone = TRUE, + labelsKMB = TRUE, + strokeWidth = 3, + colors = brewer.pal(max(3, ncol(data)), "Set1")) %>% + dyCSS(css = "./assets/css/custom.css")) + }) + output$wikidata_properties_plot <- renderDygraph({ + wikidata_properties<- xts(wikidata_properties[, -1], wikidata_properties[, 1]) + return(dygraph(wikidata_properties, + main = "Wikidata Properties", + ylab = "Properties") %>% + dyLegend(width = 400, show = "always") %>% + dySeries("V1", label = "properties") %>% + dyOptions(useDataTimezone = TRUE, + labelsKMB = TRUE, + connectSeparatedPoints = TRUE, + strokeWidth = 3, + colors = brewer.pal(max(3, ncol(data)), "Set1")) %>% + dyCSS(css = "./assets/css/custom.css")) + }) output$wikidata_content_overview_plot <- renderDygraph({ wikidata_content_overview<- xts(wikidata_content_overview[, -1], wikidata_content_overview[, 1]) return(dygraph(wikidata_content_overview, - main = "Wikidata Content Overview", + main = "Wikidata References Overview", ylab = "Statements") %>% dyLegend(width = 400, show = "always", labelsDiv = "legend", labelsSeparateLines = TRUE) %>% dyOptions(useDataTimezone = TRUE, @@ -197,7 +218,7 @@ output$wikidata_content_wikilinks_item_plot <- renderDygraph({ wikidata_content_wikilinks_item<- xts(wikidata_content_wikilinks_item[, -1], wikidata_content_wikilinks_item[, 1]) return(dygraph(wikidata_content_wikilinks_item, - main = "Wikilinks per Item", + main = "Wiki(m|p)edia links per item", ylab = "Links") %>% dyLegend(width = 400, show = "always", labelsDiv = "legend_wikilinks_item", labelsSeparateLines = TRUE) %>% dyOptions(useDataTimezone = TRUE, diff --git a/ui.R b/ui.R index cb5f306..186c5ac 100644 --- a/ui.R +++ b/ui.R @@ -15,17 +15,16 @@ sidebarMenu( menuItem(text = "Engagement", menuSubItem(text = "Edits", tabName = "wikidata_edits"), - menuSubItem(text = "Pages", tabName = "wikidata_pages"), - menuSubItem(text = "Properties", tabName = "wikidata_properties"), menuSubItem(text = "Active Users", tabName = "wikidata_active_users"), menuSubItem(text = "Social Media", tabName = "wikidata_social_media"), menuSubItem(text = "Mailing Lists", tabName = "wikidata_mailing_lists")), menuItem(text = "Content", - menuSubItem(text = "Overview", tabName = "wikidata_content_overview"), - menuSubItem(text = "Referenced statements", tabName = "wikidata_content_refstmts"), - menuSubItem(text = "Statement Refs to Wikipedia", tabName = "wikidata_content_refstmts_wikipedia"), - menuSubItem(text = "Statement Refs to Other", tabName = "wikidata_content_refstmts_other"), - menuSubItem(text = "References", tabName = "wikidata_content_references"), + menuSubItem(text = "Pages", tabName = "wikidata_pages"), + menuSubItem(text = "Items", tabName = "wikidata_items"), + menuSubItem(text = "Properties", tabName = "wikidata_properties"), + menuSubItem(text = "References", tabName = "wikidata_content_overview"), + menuSubItem(text = "Referenced Statements by Type", tabName = "wikidata_content_refstmts"), + menuSubItem(text = "References by Type", tabName = "wikidata_content_references"), menuSubItem(text = "Statement ranks", tabName = "wikidata_content_statement_ranks"), menuSubItem(text = "Statements per item", tabName = "wikidata_content_statement_item"), menuSubItem(text = "Labels per item", tabName = "wikidata_content_labels_item"), @@ -46,6 +45,9 @@ tabItem(tabName = "wikidata_pages", dygraphOutput("wikidata_pages_plot"), includeMarkdown("./assets/content/wikidata-pages.md")), + tabItem(tabName = "wikidata_items", + dygraphOutput("wikidata_content_items_plot"), + includeMarkdown("./assets/content/wikidata_content_items.md")), tabItem(tabName = "wikidata_properties", dygraphOutput("wikidata_properties_plot"), includeMarkdown("./assets/content/wikidata-properties.md")), @@ -69,12 +71,10 @@ tabItem(tabName = "wikidata_content_refstmts", dygraphOutput("wikidata_content_refstmts_plot"), htmlOutput("legend_refstmts"), - includeMarkdown("./assets/content/wikidata-content-refstmts.md")), - tabItem(tabName = "wikidata_content_refstmts_wikipedia", + includeMarkdown("./assets/content/wikidata-content-refstmts.md"), dygraphOutput("wikidata_content_refstmts_wikipedia_plot"), htmlOutput("legend_refstmts_wikipedia"), - includeMarkdown("./assets/content/wikidata_content_refstmts_wikipedia.md")), - tabItem(tabName = "wikidata_content_refstmts_other", + includeMarkdown("./assets/content/wikidata_content_refstmts_wikipedia.md"), dygraphOutput("wikidata_content_refstmts_other_plot"), htmlOutput("legend_refstmts_other"), includeMarkdown("./assets/content/wikidata_content_refstmts_other.md")), diff --git a/utils.R b/utils.R index 76d924b..0b83ddf 100644 --- a/utils.R +++ b/utils.R @@ -5,11 +5,12 @@ library(reshape2) library(RColorBrewer) library(ggplot2) +library(scales) library(toOrdinal) library(lubridate) library(magrittr) -data_uri <- "http://wdm-data.wmflabs.org/"; +data_uri <- "http://localhost/"; download_set <- function(file, uri = data_uri){ out <- tryCatch( -- To view, visit https://gerrit.wikimedia.org/r/239855 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I89e1bd115ea31ca4e344ae23f6d37f2615380981 Gerrit-PatchSet: 1 Gerrit-Project: wikidata/analytics/dashboard Gerrit-Branch: master Gerrit-Owner: Christopher Johnson (WMDE) <[email protected]> Gerrit-Reviewer: Christopher Johnson (WMDE) <[email protected]> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
