Re: [Analytics] Better way to identify client IP address

2015-01-27 Thread Ananth RK
On Fri, Jan 23, 2015 at 11:29 PM, Gilles Dubuc gil...@wikimedia.org wrote: IF proxy_ip does not start with 127.0 or 192.168 or 10. or 169.254: There are other special/private IP blocks that you'll probably want filtered. This RFC contains the full list for IPv4:

[Analytics] MariaDB sql_mode

2015-01-27 Thread Sean Pringle
Another topic at MW Summit was using sql_mode to avoid some of MySQL's odd legacy behaviour. They are: https://mariadb.com/kb/en/mariadb/sql_mode/ You can set sql_mode per client connection on analytics-store without affecting anyone else or replication.

[Analytics] Early registration for CSCW 2015 ends January 30th

2015-01-27 Thread Dario Taraborelli
For those of you interested in attending, the early registration deadline is January 30. See also https://meta.wikimedia.org/wiki/Research:CSCW_2015 https://meta.wikimedia.org/wiki/Research:CSCW_2015 — — — CSCW 2015 | March 14-18 | Vancouver, BC, Canada http://cscw.acm.org

[Analytics] analytics-store disk space

2015-01-27 Thread Sean Pringle
Yesterday at the MW Summit I mentioned to Dan/Nuria/Halfak that analytics-store disk was ~20% used. JFTR that was wrong; it's actually closer to 45% used. It's a 6T RAID10 10K array holding S1-7, Eventlogging, Staging, and the Data Warehouse test schema. Still enough space for a good while at

Re: [Analytics] [Ops] webrequest_misc added to Kafka + Hive

2015-01-27 Thread Christian Aistleitner
Hi Faidon, On Mon, Jan 26, 2015 at 04:09:32PM -0800, Andrew Otto wrote: I’ll let qchris respond in more detail, [...] I do not have much further details. Currently, udp2log contains misc (not directly via varnish, but indirectly via nginx) and hence misc logs can be queried live, and they also

[Analytics] Non-MariaDB options

2015-01-27 Thread Sean Pringle
Yesterday at MW Summit there were some non-MariaDB options tossed aorund. They were: More Hadoop usage than current efforts. Druid http://druid.io/ (Dan) RethinkDB http://rethinkdb.com/ (Ori) Hyperdex http://hyperdex.org/ (Sean) TokuMX http://www.tokutek.com/tokumx-for-mongodb/ (Sean) No

Re: [Analytics] Non-MariaDB options

2015-01-27 Thread Andrew Otto
3 beefy ciscos are available for experimentation. :D On Jan 27, 2015, at 09:04, Sean Pringle sprin...@wikimedia.org wrote: Yesterday at MW Summit there were some non-MariaDB options tossed aorund. They were: More Hadoop usage than current efforts. Druid http://druid.io/ (Dan)