[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel added a comment. > Here are the tables that break down the device_family values, @Manuel :) Wow, that's great, thank you! It is even more insightful than I thought!  It seems that with `user_agent_map.os_family = 'Android', iOS or KaiOS` we can now reliably identify mobile

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: JAllemandou, AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: JAllemandou, AndrewTavis_WMDE, Michael, Manuel, Aklapper,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: JAllemandou, AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: JAllemandou, AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: JAllemandou, AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

AndrewTavis_WMDE added a comment. As far as totals for this task are concerned, @Manuel, what I'm getting is the following: **Population for the following is**: All views of `www.wikidata.org` and `m.wikidata.org` inclusively between 01/7/2023 and 31/7/2023 that are further

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

AndrewTavis_WMDE added a comment. @Manuel, I think we can throw out the idea of creating an edits subset of webrequests, sadly :( The following would be where we'd find the various actions that we'd need to collect to define as edits fully: https://www.wikidata.org/w/api.php. We know at the

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

AndrewTavis_WMDE added a comment. Here are the tables that break down the `device_family` values, @Manuel :) As before: **Population for the following is**: All requests to `www.wikidata.org` and `m.wikidata.org` inclusively between 25/7/2023 and 31/7/2023 that are further

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

AndrewTavis_WMDE added a subscriber: JAllemandou. AndrewTavis_WMDE added a comment. And here are the finalized heuristics (@JAllemandou, tagging you as well). The following query is saved as a temporary view as `df_requests_subset`: SELECT user_agent_map.os_family AS os_family,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel added a comment. > I am now not sure anymore if my assumption was right that Tizen is mostly older mobile phones. Wouldn't we then expect a desktop/mobile ratio more similar to Android? Idea: They are likely misidentified by our server and that is why the desktop interface is

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel added a comment. I am now not sure anymore if my assumption was right that Tizen is mostly older mobile phones. Wouldn't we then expect a desktop/mobile ratio more similar to Android? Could you please investigate this combination more (and all others that seem suspicios)? We can

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

AndrewTavis_WMDE added a comment. Here are the values for `Tizen` as well, @Manuel: | access_method | total_requests | percent | | - | -- | --- | | desktop | 10874 | 93.7333 | | mobile web| 727| 6.2667 | | I'd assume

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel added a comment. > Tizen We will use this data to work to get answers to some diversity questions. So it would be important to ensure that we do not systematically exclude non-premium smartphone devices. I know that e.g. Samsung used Tizen on their non-premium smarphones. So

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

AndrewTavis_WMDE added a comment. We'd talked "Tizen" a bit this morning, @Manuel, but let's not focus on it. Did a bit of Wikipedia research and since since 2021 it's mostly in use in Samsung Smart TVs. That leaves us with `Android` and `iOS` for the

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

AndrewTavis_WMDE added a comment. Here are the answers to the three questions we had from the daily, @Manuel: `uri_host LIKE 'wikidata'` as mentioned above: | uri_host | | -- | | test.m.wikidata.org| | query-preview.wikidata.org

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel added a comment. Hi Andrew, thx for the outputs that we're getting for LIKE 'wikidata' ! Wikidata (desktop interface) - 'www.wikidata.org' - 'wikidata.org' (should only be a redirect to the main URI) Wikidata (mobile interface) - 'm.wikidata.org' Test Wikidata

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices

Manuel renamed this task from "[Analytics] Identify access via mobile devices independent of tags " to "[Analytics] Identify access via mobile devices ". Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel added a comment. Ah, I see! This task was only about finding a way to reliably identify all uses (viewing and editing) from mobile devices, independent of their tags. Once we have a reliable way of detecting the user device type, the next goal will be along the lines of what you

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE added a comment. Or am I just jumping to the question in the description and we just want to figure out mobile edits and views over the period? TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE added a comment. I guess I'm confused what the goal here is then 樂 As I understand it we're looking for users who are using the normal desktop UI on a mobile device. For the `wmf.webrequest` table we'd then use: - `user_agent` -> device used via `python-user-agent` and

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel added a comment. Yes, I edited my comment, as I missed this important point: access_method just checks the URL for indications of a mobile UI, so it is about UI and not about device. So it is out of scope here, after all. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE added a comment. My understanding of `access_method` is that it's only related to `user_agent` for mobile apps: > Mobile app requests are identified by the user agent including `WikipediaApp` or `Wikipedia/5.0`. For web requests it's based on the subdomains. TASK

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE added a comment. I've already checked and there are differences between a `python-user-agents` derived device via `user_agents.parse(ua_value).is_mobile` and the `access_method`. Specifically we are getting users where the device from `.is_mobile` is `mobile`, but the

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel added a comment. > Are we trying then the combination of user_agent_var.is_mobile = True and access_method = "desktop" via the access_method column from wmf.webrequest? It is important to keep user devices and Wikidata UI separate here. For now we are only trying to get the

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE added a comment. @Manuel, I've been using python-user-agents and so far it's going ok in so far as the `.ismobile` method seems to be working well. Are we trying then the combination of `user_agent_var.ismobile = True` and

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE added a comment. @Manuel, just a note on using the `wmf.webrequest` table: now that I'm using Spark a bit more and can see the number of steps, it's definitely worth it to try to restrict the data based on the year and month as we're doing. Selecting 30 days over two months

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

AndrewTavis_WMDE updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel added a comment. That was very helpful, thank you, @Michael! TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Michael added a comment. > How do we deal with API consumers that disguise themselves as browsers? I'm not fully sure if that is true across the board, but from a quick look, it seems that API requests from the browser seem to usually have a referrer header set. Also, they usually have

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE, Manuel Cc: AndrewTavis_WMDE, Michael, Manuel, Aklapper, Danny_Benjafield_WMDE,

[Wikidata-bugs] [Maniphest] T336361: [Analytics] Identify access via mobile devices independent of tags

Manuel renamed this task from "[Analytics] Identify access via mobile phones independent of tags " to "[Analytics] Identify access via mobile devices independent of tags ". Manuel updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T336361 EMAIL PREFERENCES