| Addshore added a comment. |
In T191424#4297796, @GoranSMilovanovic wrote:@Lydia_Pintscher @Addshore https://gerrit.wikimedia.org/r/#/c/analytics/wmde/scripts/+/440901/
Q1. Where do you make this SQL calls from? It would help me to understand the workflow.
The scripts are puppetized and run on stat1005 currently as the user "analytics-wmde"
Q2. Where from (and when) do you send the data obtained from SQL to Graphite?
The SQL @ https://github.com/wikimedia/analytics-wmde-scripts/blob/master/src/wikidata/site_stats/sql/select_pages_by_namespace.sql#L6-L8 is used in https://github.com/wikimedia/analytics-wmde-scripts/blob/master/src/wikidata/site_stats/pages_by_namespace.php which makes the query and sends the data to graphite.
Q3. Would you consider switching this and similar event logging things to Big Data (HiveQL from the Data Lake)? Because sooner or later...
Yes, this could easily be moved to the data lake and the mediawiki_history table now, although I might actually be more efficient to do it in sql? (I haven't looked into this)
This isn't really an "event logging" thing.
Q4. How will we track the forms data (see T191424#4236354).
This query can not be modified to track forms as forms are not a top level entity / have no namespace.
If this is schematic - in a sense that I always need to introduce the changes into SQL code, or write new SQL code, in analytics/wmde/scripts - I can do these things from now on. It would be enough if you let me know what exactly do you need to be done on every new occasion.
It will be easier if on occasions such as this you ask the questions and we answer them in tickets rather than always explain every step in tickets as we risk spending time explaining things that are already known.
Cc: Addshore, GoranSMilovanovic, Aklapper, Lydia_Pintscher, Lahi, Gq86, QZanden, LawExplorer, Wikidata-bugs, aude, Darkdadaah, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
