[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-11-21 Thread Michael 
Michael closed this task as "Resolved".
Michael added a comment.


  The panel is now at https://grafana.wikimedia.org/d/tBp4Uen4k/entityschema
  
  I'm closing this task so that the overall product verification can take place 
in the parent task.

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-31 Thread Manuel 
Manuel added a comment.


  Yay, good news! \o/

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael, Manuel
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-31 Thread Michael 
Michael added a comment.


  PR #174 

 has been merged, and we should expect to see data in Graphite on Thursday 
latest. Then we can create a panel for this on Grafana and finally finish this 
task and the parent :)

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-14 Thread Manuel 
Manuel added a comment.


  Thank you @Michael for investigating the spiders. This made me realize that 
"community run bots" are, of course, classified as spiders here.
  
  > The page views by spiders do not really seem meaningful in that case.
  
  @Lucas_Werkmeister_WMDE: My assessment was wrong. We will definitely need to 
keep measuring spider activity. Let's please ensure that we clarify on the 
board what goes into the spider category: basically everything that is not 
using a classic browser (including community run bots, our own infrastructure, 
and of course classic web crawlers). This might help other PMs looking at the 
dashboard making the same mistake. We could also think of naming the two 
categories more intuitively, e.g. "browser" and "non-browser".
  
  > The ones with user agent seem harmless, but the vast majority does not have 
a user agent. Not sure what is going on there.
  
  I checked: The huge number of hits without defined user agent have all 
originated from only one machine and stopped yesterday. So I guess it was just 
an ill behaved script.

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael, Manuel
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-13 Thread Michael 
Michael added a comment.


  Currently working on 
https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/174

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-13 Thread Maintenance_bot 
Maintenance_bot removed a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael, Maintenance_bot
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331, Jersione, Hellket777, LisafBia6531, 786, Biggs657, Juan90264, 
Alter-paule, Beast1978, Un1tY, Hook696, Kent7301, joker88john, CucyNoiD, 
Gaboe420, Giuliamocci, Cpaulf30, Af420, Bsandipan, Lewizho99, Maathavan, 
Neuronton
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-13 Thread gerritbot 
gerritbot added a comment.


  Change 811979 **merged** by Mforns:
  
  [analytics/refinery@master] Track views of EntitySchema namespaces on Wikidata
  
  https://gerrit.wikimedia.org/r/811979

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael, gerritbot
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Jersione, 
Hellket777, LisafBia6531, Astuthiodit_1, 786, Biggs657, Invadibot, maantietaja, 
Juan90264, Alter-paule, Beast1978, Un1tY, SCIdude, Akuckartz, Hook696, 
Kent7301, pdehaye, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, 
Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Andrawaag, GoranSMilovanovic, QZanden, 
YULdigitalpreservation, LawExplorer, Salgo60, Lewizho99, Maathavan, _jensen, 
rosalieper, Neuronton, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, aude, 
Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-13 Thread Michael 
Michael added a comment.


  Out of curiosity, I had a closer look at the user agents of those spiders for 
the EntitySchema namespace:
  
Mozilla/5.0 (compatible; SeznamBot/4.0-RC1; 
+http://napoveda.seznam.cz/seznambot-intro/)1
yacybot (/global; amd64 Linux 5.4.0-126-generic; java 1.8.0_342; 
America/en) http://yacy.net/bot.html   2
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
3
IABot/2.0 
(+https://meta.wikimedia.org/wiki/InternetArchiveBot/FAQ_for_sysadmins) 
(Checking if link from Wikipedia is broken and needs removal) 57
Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html) 
1
Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)  3
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
2
Twitterbot/1.0  3
-   31776
  
  (Data for 2022-10-11)
  
  The ones with user agent seem harmless, but the vast majority does not have a 
user agent. Not sure what is going on there.

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Jersione, 
Hellket777, LisafBia6531, Astuthiodit_1, 786, Biggs657, Invadibot, maantietaja, 
Juan90264, Alter-paule, Beast1978, Un1tY, SCIdude, Akuckartz, Hook696, 
Kent7301, pdehaye, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, 
Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Andrawaag, GoranSMilovanovic, QZanden, 
YULdigitalpreservation, LawExplorer, Salgo60, Lewizho99, Maathavan, _jensen, 
rosalieper, Neuronton, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, aude, 
Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-12 Thread hoo 
hoo assigned this task to Michael.
hoo moved this task from Todo/Backlog to Peer Review on the Wikidata Dev Team 
(Sprint-∞) board.

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

WORKBOARD
  https://phabricator.wikimedia.org/project/board/6017/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael, hoo
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Jersione, 
Hellket777, LisafBia6531, Astuthiodit_1, 786, Biggs657, Invadibot, maantietaja, 
Juan90264, Alter-paule, Beast1978, Un1tY, SCIdude, Akuckartz, Hook696, 
Kent7301, pdehaye, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, 
Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Andrawaag, GoranSMilovanovic, QZanden, 
YULdigitalpreservation, LawExplorer, Salgo60, Lewizho99, Maathavan, _jensen, 
rosalieper, Neuronton, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, aude, 
Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-10 Thread gerritbot 
gerritbot added a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Jersione, 
Hellket777, LisafBia6531, Astuthiodit_1, 786, Biggs657, Invadibot, maantietaja, 
Juan90264, Alter-paule, Beast1978, Un1tY, SCIdude, Akuckartz, Hook696, 
Kent7301, pdehaye, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, 
Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Andrawaag, GoranSMilovanovic, QZanden, 
YULdigitalpreservation, LawExplorer, Salgo60, Lewizho99, Maathavan, _jensen, 
rosalieper, Neuronton, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, aude, 
Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-10 Thread gerritbot 
gerritbot added a comment.


  Change 811979 had a related patch set uploaded (by Michael Große; author: 
Michael Große):
  
  [analytics/refinery@master] Track views of EntitySchema namespaces on Wikidata
  
  https://gerrit.wikimedia.org/r/811979

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-05 Thread Manuel 
Manuel added a comment.


  Thank you @Lucas_Werkmeister_WMDE for thinking about this!
  
  From what you wrote my assumption would be that "Wikidata users using 
machines" would mainly access Special:EntitySchemaText while "external spiders" 
would mainly/also access EntitySchema:, right? The page views by spiders do not 
really seem meaningful in that case. But to be on the safe side, let's please 
track page views by both users and spiders separately. This way we can confirm 
this assumption (and maybe stop tracking of spiders at some point).

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Manuel
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-05 Thread Lucas_Werkmeister_WMDE 
Lucas_Werkmeister_WMDE set the point value for this task to "5".
Lucas_Werkmeister_WMDE moved this task from Parents/Waiting to Todo/Backlog on 
the Wikidata Dev Team (Sprint-∞) board.
Lucas_Werkmeister_WMDE added a comment.


  > Should we track all page views, only human/user page views, or spider and 
user page views separately? According to the WIP patch description, there are 
about 32000 spider "views" and only 41 user views per day.
  
  @Manuel question for you

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

WORKBOARD
  https://phabricator.wikimedia.org/project/board/6017/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T319380: Track EntitySchema content and talk page views

2022-10-05 Thread Lucas_Werkmeister_WMDE 
Lucas_Werkmeister_WMDE created this task.
Lucas_Werkmeister_WMDE added projects: Shape Expressions, Wikidata Analytics, 
Wikidata, User-Michael, Wikidata Dev Team (Sprint-∞).

TASK DESCRIPTION
  In addition to the requests to Special:EntitySchemaText (implemented as part 
of the parent task), we also want to track page views to EntitySchema: and 
EntitySchema talk: pages.
  
  **Notes:**
  
  - Graphite is not a requirement
  - This can probably be implemented similar to the EntitySchemaText tracking, 
i.e. an HQL query and airflow script in data-engineering/airflow-dags 
; this WIP 
patch  can be 
used as a starting point
  
  **Open questions:**
  
  - Should we track all page views, only human/user page views, or spider and 
user page views separately? According to the WIP patch description, there are 
about 32000 spider "views" and only 41 user views per day.

TASK DETAIL
  https://phabricator.wikimedia.org/T319380

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lydia_Pintscher, Aklapper, Manuel, Michael, ItamarWMDE, 
guergana.tzatchkova, karapayneWMDE, Lucas_Werkmeister_WMDE, Astuthiodit_1, 
Invadibot, maantietaja, SCIdude, Akuckartz, pdehaye, Nandana, Lahi, Gq86, 
Andrawaag, GoranSMilovanovic, QZanden, YULdigitalpreservation, LawExplorer, 
Salgo60, _jensen, rosalieper, Scott_WUaS, MisterSynergy, abian, Wikidata-bugs, 
aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org