Re: [Analytics] Farewell, Erik!

> > Dario > > > -- > *Dario Taraborelli *Director, Head of Research, Wikimedia Foundation > research.wikimedia.org • nitens.org • @readermeter > <http://twitter.com/readermeter> > ___ > Analytics mailing list > Ana

Re: [Analytics] clicks on red links

you asked for (“number of views” instead of “number of people”, and the longtail gets cut off below 1000 views), but maybe it's close enough to be useful for you? Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies

Re: [Analytics] Daily Pageviews in Dashiki showing no recent data

Hi, just to keep archives happy ... the below issue is now T97588 https://phabricator.wikimedia.org/T97588 . Have fun, Christian On Sat, Apr 18, 2015 at 12:47:20AM +0200, Christian Aistleitner wrote: Hi Analytics dev team, just a heads up that it's a week that the pagecounts-all

Re: [Analytics] Missing media file request counts dataset for April 14, 2015

in the meantime. Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43

Re: [Analytics] Daily Pageviews in Dashiki showing no recent data

://commons.wikimedia.org/wiki/File:Refinery-oozie-overview.png -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria

Re: [Analytics] measuring traffic on mobile web beta

, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81

[Analytics] Balance HDFS now to spare weekend troubles [was: Re: [Cluster] If Analytics Cluster seems slow over the weekend, read this email [aka HDFS balancing]]

that are chained to that). Have fun, Christian P.S.: https://wikitech.wikimedia.org/w/index.php?title=Analytics/Cluster/Hadoop/Administrationoldid=148246#Balancing_HDFS -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz

Re: [Analytics] [Cluster] If Analytics Cluster seems slow over the weekend, read this email [aka HDFS balancing]

Hi, On Sun, Mar 08, 2015 at 10:18:40AM +0100, Christian Aistleitner wrote: But since HDFS is still heavily misbalanced, I am leaving the balancer running for now. Balancing now finished. HDFS is balanced again. I did a short write-up about HDFS Balancing at https://wikitech.wikimedia.org

Re: [Analytics] New pageviews definition only half-QA-ed [was: Re: [Technical] final pageviews QA]

not mean to. Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone

[Analytics] New pageviews definition only half-QA-ed [was: Re: [Technical] final pageviews QA]

accordingly. Have fun, Christian [1] https://git.wikimedia.org/commit/analytics%2Fwebstatscollector.git/7617da88b9fa36dcf3bce593ad9a6da3bf9ec325 -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian

Re: [Analytics] [Technical] inaccuracy in our pageview dump documentation

Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax:+43 7946 / 20 5 81

Re: [Analytics] [Technical] inaccuracy in our pageview dump documentation

Hi, [ just to keep archives happy ] On Tue, Mar 10, 2015 at 05:16:30PM +0100, Christian Aistleitner wrote: After the next rsync (in ~1 hour) the new README should be live. The new README is live now at: http://dumps.wikimedia.org/other/pagecounts-all-sites/README.txt Have fun, Christian

Re: [Analytics] [Cluster] Monitoring the impact Hive jobs have on the Analytics cluster

Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax:+43 7946 / 20 5 81

Re: [Analytics] [Cluster] Monitoring the impact Hive jobs have on the Analytics cluster

the Cluster/Hadoop/Administration page. But sure. Be Bold. Move wherever you seem fit. :-) Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3

Re: [Analytics] [Cluster] If Analytics Cluster seems slow over the weekend, read this email [aka HDFS balancing]

Hi, On Sat, Mar 07, 2015 at 10:47:50AM +0100, Christian Aistleitner wrote: We've been up to 5 unhealthy nodes. Since we're missing their memory, [...] We're back to 0 unhealthy nodes \o/ So the cluster can again use it's full amount of memory. But since HDFS is still heavily misbalanced, I

[Analytics] [Cluster] Monitoring the impact Hive jobs have on the Analytics cluster

. Especially, it allows you to detect if the cluster got stalled, and if it did, it tells you what to do. Have fun, Christian P.S.: The above URL has diagrams! Click the URL! -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz

Re: [Analytics] Duplicate events in the webrequest logs

, pagecounts-all-sites, ... -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5

Re: [Analytics] s1-analytics-slave

://upload.wikimedia.org/wikipedia/commons/b/b0/Geowiki_workflow.png -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria

Re: [Analytics] s1-analytics-slave

/wikipedia/commons/b/b0/Geowiki_workflow.png (The second s1-analytics-slave reference on the top left does not care whether it's db1047 or dbstore[12]002 and can be switched without issues) -- quelltextlich e.U. \\ Christian Aistleitner Companies

Re: [Analytics] Timezone on files?

one might expect. See for example here: https://wikitech.wikimedia.org/w/index.php?title=Analytics/Pagecounts-raw#dumps.wikimedia.org Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz

Re: [Analytics] s1-analytics-slave

/Geowiki_workflow.png (The second s1-analytics-slave reference on the top left does not care whether it's db1047 or dbstore[12]002 and can be switched without issues) -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian

Re: [Analytics] [Ops] webrequest_misc added to Kafka + Hive

. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax:+43 7946 / 20 5

Re: [Analytics] Better way to identify client IP address

that the above links point to are not fully static. While their proxy configuration does not change every other day, they do change every now and then. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian

[Analytics] Renaming wmf.webstats Hive table and /wmf/data/archive/webstats HDFS path

. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax:+43

[Analytics] Discussion about target URL for varnish monitoring requests (Angelsberg/Main_Page - Undefined ?)

/ a discussion is starting on how to detect such monitoring requests and how they might change. Chime in there, or at least watch it's outcome to make sure your scripts keep on detecting/filtering monitoring requests. Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner

Re: [Analytics] Troubleshooting

. This BETWEEN condition on the literal string “timestamp” is never met. Hence, the SELECT gave the empty result set. Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner

Re: [Analytics] Page view generalized filter draft (due Friday, Dec 12th)

-- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81

Re: [Analytics] graphite/statsd stopped collecting deprecation data

seems to have been written by you. [3] I could find neither you nor Ori in IRC, and the service was not doing it's job anyways, and looking at the code, it looked like it's safe to restart. -- quelltextlich e.U. \\ Christian Aistleitner Companies

Re: [Analytics] Where is Flow's data

row in set (0.02 sec) [3] https://phabricator.wikimedia.org/T84990 -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau

Re: [Analytics] Where is Flow's data

Hi, On Fri, Dec 19, 2014 at 12:31:50PM +0100, Christian Aistleitner wrote: We previously could connect to that slave from stat1003, but that did no longer work when I tried just now :-/ That might be related to yesterday's vlan move of stat1003, so I filed T84990 [3] to open up the firewall

[Analytics] Adventures in Clusterland 2014-12-01--2014-12-07

for them. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81

[Analytics] Adventures in Clusterland 2014-12-08--2014-12-14

!), and replication caught up. Restrictions on table naming got set up and are still getting tuned a bit. [1] https://phabricator.wikimedia.org/T74805 -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian

[Analytics] [Resolved] Re: Partly lagging EventLogging database replication

Hi, On Tue, Dec 09, 2014 at 07:55:59PM +0100, Christian Aistleitner wrote: it seems some tables of the EventLogging database are no longer replicating since around 2014-12-09T11:32 [1] on both dbstore1002 and db1047 (s1-analytics-slave). Sorry for the late “all-good-again” message. I

Re: [Analytics] Detecting SSL terminators

Hi, On Thu, Dec 11, 2014 at 06:27:02PM -0500, Oliver Keyes wrote: On Sun, Dec 07, 2014 at 12:59:27PM +0100, Christian Aistleitner wrote: http://config-master.wikimedia.org/pybal/esams/text-https [...] I'm not sure how to interpret the pybal, The exemplary file linked above holds lines

Re: [Analytics] EventLogging data QA

/client-side-events.log or /srv/log/eventlogging/server-side-events.log depending on the kind of event you're looking for. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner

Re: [Analytics] EventLogging data QA

parts on vanadium are UDP, it's better not to saturate the network. So please only run the command if you have to, and only as long as you have to. Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y

Re: [Analytics] EventLogging data QA

-- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax

Re: [Analytics] Analytics Engineering Team Commitments 2014-12-11 -- 2014-12-25

you please make them public? Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria

Re: [Analytics] Detecting SSL terminators

that you're looking at. But I figure you're mostly interested in Hive data (for different streams, there are other methods). More or less the same question got asked on the internal list on Sunday. There I pointed towards pybal: On Sun, Dec 07, 2014 at 12:59:27PM +0100, Christian Aistleitner wrote

[Analytics] Partly lagging EventLogging database replication

. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81

[Analytics] Adventures in Clusterland 2014-11-10--2014-11-16

very silent.) [2] https://wikitech.wikimedia.org/wiki/Incident_documentation/20141113-EventLogging [3] https://phabricator.wikimedia.org/T1200 -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian

[Analytics] Adventures in Clusterland 2014-11-24--2014-11-30

-November/002819.html https://lists.wikimedia.org/pipermail/analytics/2014-December/002821.html https://phabricator.wikimedia.org/T76334 -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner

Re: [Analytics] Writing EventLogging events to database failed on 2014-11-25 between 03:09 and ~midnight

Hi, On Wed, Nov 26, 2014 at 02:35:47AM +0100, Christian Aistleitner wrote: The still rather empty Incident Report is at https://wikitech.wikimedia.org/wiki/Incident_documentation/20141125-EventLogging I'll fill it with more information tomorrow. Done. Backfilling from the logs

[Analytics] Writing EventLogging events to database failed on 2014-11-25 between 03:09 and ~midnight

. Backfilling from the logs is already running, and should also finish tomorrow. Best regards, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ

[Analytics] Backfilling done [was: Re: Round-up of recent EventLogging issues]

Hi, On Fri, Nov 21, 2014 at 02:22:47PM +0100, Christian Aistleitner wrote: For the last three of the four issues, backfilling is still pending, [...] Backfilling the database just finished (without issues). Numbers in the various EventLogging-powered dashboards should jump up again after

Re: [Analytics] Backfilling done [was: Re: Round-up of recent EventLogging issues]

Hi, On Mon, Nov 24, 2014 at 03:32:48PM +0100, Christian Aistleitner wrote: On Fri, Nov 21, 2014 at 02:22:47PM +0100, Christian Aistleitner wrote: For the last three of the four issues, backfilling is still pending, [...] Backfilling the database just finished (without issues). I forgot

Re: [Analytics] Backfilling done [was: Re: Round-up of recent EventLogging issues]

with him off-list. Thanks, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone

[Analytics] Writing EventLogging events to database failed on 2014-11-22 for between ~19:30 and ~21:00

should again hold good numbers for that period. Just wanted to let you know what happened, in case you notice drops in graphs / dashboards that got generated between the incident and now. Sorry for the inconveniences, Christian -- quelltextlich e.U. \\ Christian Aistleitner

Re: [Analytics] Solving Limn - Zero graph descrepancy

://git.wikimedia.org/blob/analytics%2Frefinery.git/fd54d68a586f081cc9a7eb86f04a233f33168de4/oozie%2Fwebstats%2Finsert_hourly_pagecounts%2Finsert_hourly_pagecounts.hql#L47 -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian

Re: [Analytics] Writing EventLogging events to database failed on 2014-11-13 for ~23 minutes between 00:59 and 01:21

Hi, On Thu, Nov 13, 2014 at 03:15:23PM +0100, Christian Aistleitner wrote: The data for that period is not lost, but is available in backup files, waiting to get injected again into the database. Backfilling is done. Huge props to Ori! His completely modular/pluggable EventLogging backend

[Analytics] Adventures in Clusterland 2014-10-27--2014-11-02

work around it started. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946

[Analytics] Adventures in Clusterland 2014-11-03--2014-11-09

under code-review. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20

Re: [Analytics] s1 replica on analytics-store suffering lag of 12 hours

Hi, On Sat, Nov 01, 2014 at 10:24:54PM +0100, Christian Aistleitner wrote: just a quick heads up that the s1 replication lag on analytics-store.eqiad.wmnet (i.e.: dbstore1002), is currently at 15 hours and increasing. slave lag is back to normal. Have fun, Christian -- quelltextlich

Re: [Analytics] [Fixed] s5 replica on analytics-store suffering lag of 12 hours

Hi, On Fri, Oct 31, 2014 at 08:23:19AM +0100, Christian Aistleitner wrote: [ slave log for s5 ] is currently at 17 hours and increasing. I filed RT ticket 8788: https://rt.wikimedia.org/Ticket/Display.html?id=8788 slave lag is back to normal. Have fun, Christian

[Analytics] s1 replica on analytics-store suffering lag of 12 hours

replica instead until the other replica is fixed. Best regards, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293

[Analytics] s5 replica on analytics-store suffering lag of 12 hours

. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax

[Analytics] Adventures in Clusterland 2014-10-13--2014-10-19

implementing, and reviewing this UDF. But it's not yet merged and will require a bit more work. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ

[Analytics] s3 replica on analytics-store suffering lag of 12 hours

issues daily during the last few days, they are all caused by different unrelated issues. So it seems to me that it is an unfortunate coincidence not a general issue. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz

[Analytics] s1 replica on analytics-store suffering lag of 12 hours

. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax:+43 7946

Re: [Analytics] [Fixed] Re: Geowiki no longer showing up-to-date numbers

://stats.wikimedia.org/geowiki-private/global_south_editor_fractions.csv . Best regards, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ

Re: [Analytics] Traffic device breakdown

-- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax

Re: [Analytics] Traffic device breakdown

-tail cut off to have thrown out a big part of requests from Android devices. Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ

Re: [Analytics] [Fixed] Re: Geowiki no longer showing up-to-date numbers

/active_editors_total -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20

Re: [Analytics] [Fixed] Re: Geowiki no longer showing up-to-date numbers

Hi Asaf, On Thu, Oct 16, 2014 at 03:39:41PM +0200, Christian Aistleitner wrote: On Wed, Oct 15, 2014 at 04:43:30PM -0700, Asaf Bartov wrote: http://gp.wmflabs.org/ is apparently not working again. Is this known and worked on? Thanks! Some graphs seem to be working [1] ... I'll see

Re: [Analytics] Mobile page views available

=131275oldid=131269 Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43

Re: [Analytics] [Ops] POODLE SSL 3.0 vulnerability

-20141015.gz filtered to requests that come from an ssl termintator and used ua-parser 1.3.0 to identify User-Agents. Since we're only interested in ratios, a recent 24h period seemed sufficient. -- quelltextlich e.U. \\ Christian Aistleitner Companies

[Analytics] Adventures in Clusterland 2014-09-29--2014-10-05

://lists.wikimedia.org/pipermail/analytics/2014-October/002597.html -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau

Re: [Analytics] Dropping requests for [uU]ndefined from webstatscollector's page view definition

Hi, On Fri, Oct 10, 2014 at 02:25:49AM +0200, quelltextlich e.U. - Christian Aistleitner wrote: On Wed, Oct 08, 2014 at 12:28:38AM +0200, Christian Aistleitner wrote: If you think it's worth keeping them, please let us know soonish. (I'll try to get the changes merged on 2014-10-09

Re: [Analytics] EventLogging replicas suffering lag of 12 hours

Hi, On Sun, Oct 12, 2014 at 11:31:23PM +0200, Christian Aistleitner wrote: just a quick heads up that the replication lag on both * analytics-store.eqiad.wmnet (i.e.: dbstore1002), and The lag on analytics-store.eqiad.wmnet (i.e.: dbstore1002) is back to normal. So if you run queries against

Re: [Analytics] Analytics services reliability question

. For 2014, we're owning only 3 items there. None of them had direct impact on our users. Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email

[Analytics] EventLogging replicas suffering lag of 12 hours

, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81

Re: [Analytics] Dropping requests for [uU]ndefined from webstatscollector's page view definition

Hi, On Wed, Oct 08, 2014 at 12:28:38AM +0200, Christian Aistleitner wrote: If you think it's worth keeping them, please let us know soonish. (I'll try to get the changes merged on 2014-10-09.) The changes are ready, but deployment was postponed to next week :-/ Have fun, Christian

[Analytics] Making webstatscollector stop counting redirects at the HTTP level

for concrete examples. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946

Re: [Analytics] Welcome Marcel Ruiz Forns to the Analytics Development team

stop by Chambers to say hello if you are in the office. -Toby ___ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics -- quelltextlich e.U. \\ Christian Aistleitner

Re: [Analytics] eventlogging largest tables

___ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email

Re: [Analytics] stats.wikimedia.org and datasets.wikimedia.org unavailable

Hi, [ just to keep the archives happy ] On Mon, Oct 06, 2014 at 01:48:28AM +0200, Christian Aistleitner wrote: the machine that hosts stats.wikimedia.org datasets.wikimedia.org is experiencing problems, and hence the above sites are currently unavailable. The sites are up again

Re: [Analytics] Gerrit permissions for group “Analytics” on “analytics/*” projects

Hi, TL;DR: If your projects in analytics/* just stopped allowing you to directly push or your git workflow with them just broke, let me know. On Mon, Apr 21, 2014 at 12:07:33PM +0200, Christian Aistleitner wrote: people from gerrit's “Analytics” group [1] currently hold * Push (including

Re: [Analytics] eventlogging largest tables

. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax:+43 7946 / 20

[Analytics] Adventures in Clusterland 2014-09-15--2014-09-21

Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81 Fax:+43 7946 / 20 5 81

Re: [Analytics] Errors on stats.grok.se

Hi Henrik, On Thu, Sep 25, 2014 at 08:00:37AM +0800, Henrik Abelsson wrote: It should be up and running again now. Thanks for the immediate fix! Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y

[Analytics] Adventures in Clusterland 2014-09-01--2014-09-07

of the partition leader role. We now need to find the source of those write bursts, to see if they are the real issue, or just the symptom of a different issue. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz

Re: [Analytics] Report about pageview issues

. Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5

[Analytics] Adventures in Clusterland 2014-08-18--2014-08-24

place on the same day, but happened later in the day). Since this happened for the first time, we're writing it off as fluke for now. -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner

Re: [Analytics] Purging data EventLogging for SignupExp* and TrackedPageContentSaveComplete schemas

Hi, On Thu, Aug 07, 2014 at 03:59:13PM +0200, Christian Aistleitner wrote: we'll soonish purge the following tables from the EventLogging databases: SignupExpAccountCreationComplete_8539421 SignupExpAccountCreationImpression_8539445 SignupExpCTAButtonClick_8102619

Re: [Analytics] Request for data purge for a select number of schemas

?id=68931 Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43

Re: [Analytics] Replication lag on analytics-store.eqiad.wmnet 12 hours for s1 replicas

Hi Sean, On Thu, Jul 31, 2014 at 12:19:33PM +1000, Sean Pringle wrote: On Wed, Jul 30, 2014 at 4:40 PM, Christian Aistleitner christ...@quelltextlich.at wrote: Lag on log is currently still ~ 16 hours. I'll keep an eye on it. https://bugzilla.wikimedia.org/show_bug.cgi?id=67450

Re: [Analytics] Request for data purge for a select number of schemas

but only used for debugging the database. At least that's what some people (you included) told me. So if there is no extra-special need, I won't do house-keeping on the raw logs. Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies

Re: [Analytics] Replication lag on analytics-store.eqiad.wmnet 12 hours for s1 replicas

fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone: +43 7946 / 20 5 81

[Analytics] Wikimetrics unresponsive

Hi, is seems Wikimetrics is currently suffering load problems and may appear down. The corresponding bug is 68743: https://bugzilla.wikimedia.org/show_bug.cgi?id=68743 Best regards, Christian -- quelltextlich e.U. \\ Christian Aistleitner

Re: [Analytics] Media Viewer User Preference Data

ACK. What a farce that we have a vote and then ignore clear outcomes as 1:4, and even 1:13 if we do not like them :-( Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner

Re: [Analytics] Team's Sprint Commitments

Hi, On Wed, Jul 02, 2014 at 11:00:44AM +0200, Christian Aistleitner wrote: On Tue, Jul 01, 2014 at 09:16:26AM -0700, Kevin Leduc wrote: Our current sprint started Thursday June 26 and ends Tuesday July 8th. [...] http://sb.wmflabs.org/t/analytics-developers/2014-06-26/ Of the originally

Re: [Analytics] Webstatscollector (used for stats.grok.se, stats.wikimedia.org, ...) not counting SSL traffic

. (Note that although those files they are not affected by /this/ issue, they are affected by other issues like January's https://bugzilla.wikimedia.org/show_bug.cgi?id=60315 ) Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner

Re: [Analytics] Fwd: ** PROBLEM alert - tungsten/Throughput of event logging events is CRITICAL **

/show_bug.cgi?id=67463 Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone

Re: [Analytics] s5-analytics-slave databases/tables and dbstore1002 migration [was: Re: s[23467]-analytics-slave getting pointed to analytics-store]

Hi, On Thu, Jun 19, 2014 at 12:15:54AM +0200, Christian Aistleitner wrote: TL;DR: If you use staging.nov13_dewiki_* tables, speak up! On Wed, Jun 04, 2014 at 02:42:11PM +1000, Sean Pringle wrote: What should we do with s5-analytics-slave? [...] [ Some Tables ] Do those tables ring

Re: [Analytics] EventLogging on graphite

was the outcome of this? Have fun, Christian -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau, Austria Phone

Re: [Analytics] Removing EventLogging tables starting in “zz_” or ending in “_1”

Hi, On Mon, Jun 16, 2014 at 07:34:37AM +0200, Christian Aistleitner wrote: On Mon, Jun 16, 2014 at 07:29:44AM +0200, Christian Aistleitner wrote: If no one speaks up until 2016-06-23, [...] Ha :-D *cough* ... until 2014-06-23. Since no one claimed them, the tables „zz_” and „_1” tables

Re: [Analytics] EventLogging's country column are logging unwanted (sensitive) chunks of cookie

Hi, On Wed, Jun 11, 2014 at 05:09:45PM +0200, Christian Aistleitner wrote: Columns for country data in EventLogging tables sometimes not only contain the country code, but also larger chunks of the client cookies, which may put sensitive data into the tables. The corresponding bug

[Analytics] Data quality expectations for different EventLogging data representations

=66528 [5] https://git.wikimedia.org/commitdiff/operations%2Fpuppet.git/05b4027973c59b0a786433f8dae2bd1fe28b614f -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3

Re: [Analytics] stat1001 serving 503s

Hi, On Wed, Jun 18, 2014 at 03:07:05PM +0200, Christian Aistleitner wrote: On Wed, Jun 18, 2014 at 02:24:17PM +0200, Gilles Dubuc wrote: In case I'm the first to notice it, I'm getting 503s consistently on the paths limn normally reads the tsvs from for our stats: http://stat1001

[Analytics] Removing EventLogging tables starting in “zz_” or ending in “_1”

tracked at https://bugzilla.wikimedia.org/show_bug.cgi?id=66649 -- quelltextlich e.U. \\ Christian Aistleitner Companies' registry: 360296y in Linz Christian Aistleitner Kefermarkterstrasze 6a/3 Email: christ...@quelltextlich.at 4293 Gutau

  1   2   >