Re: [Wikitech-l] Method of Testing DB Queries

2013-10-08 Thread Aaron Schulz 
They often give the same results on smallish wikis, but I wouldn't carry that
over to test wikis unless lots of content and users, logging, other table
data was somehow imported in. For example a tiny user table might make mysql
start INNER JOINs with that table in queries where it would never do that in
production. In my experience development test wikis are often useless for
estimating what query plan will happen in production.

A smallish wiki with 10ks of pages and the full history and the table data
(not just revision/page/*links stuff from dumps) would probably be useful.
I'm not sure where the threshold roughly starts though.



--
View this message in context: 
http://wikimedia.7.x6.nabble.com/Method-of-Testing-DB-Queries-tp5014676p5014679.html
Sent from the Wikipedia Developers mailing list archive at Nabble.com.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Ori Livneh's new role on site performance

2013-10-08 Thread Matthew Flaschen 

On 10/08/2013 02:59 PM, Rob Lanphier wrote:

Hi everyone,

I'm excited to announce that Ori Livneh will be moving into Platform
Engineering as Senior Performance Engineer.  This work is riffing off
of the work that he's done with the Growth (nee E3) team, where a big
part of his job was instrumenting new features to measure their impact
on editor retention.


This is great to hear.  Your work on the growth team has been a key 
backbone to what we're accomplishing.


I really look forward to working more with you on improving performance, 
and other matters.


Matt


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Method of Testing DB Queries

2013-10-08 Thread Tim Starling 
On 09/10/13 13:38, Tyler Romeo wrote:
> Hello,
> 
> So recently I've been trying to find a method where I could test database
> queries on production data. I originally thought Tool Labs might help, but
> I found out due to a bug in MySQL you can't perform EXPLAIN queries in the
> database replicas. And naturally I don't have access to the actual
> production cluster because I'm not on ops.
> 
> My last method to try is going to be downloading the enwiki database dump
> and setting up my own replica. However, this involves hard drive space that
> I don't have.

EXPLAIN on a small wiki will usually give you the same query plan as
EXPLAIN on a large wiki, if the indexes are the same. So you can try
running your EXPLAIN against your local test wiki. But for the most
part, we estimate query performance a priori, based on code review and
what we know about MySQL, rather than measuring performance during
testing.

Note that EXPLAIN can be misleading even if it is run against a
production database. Even actual performance measurement in production
can be misleading, if the test is run against a small table which will
gain rows over time. And measuring execution time doesn't tell you
everything about the performance of the server -- for example, it
doesn't show buffer pool usage.

-- Tim Starling


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Method of Testing DB Queries

2013-10-08 Thread Tyler Romeo 
Hello,

So recently I've been trying to find a method where I could test database
queries on production data. I originally thought Tool Labs might help, but
I found out due to a bug in MySQL you can't perform EXPLAIN queries in the
database replicas. And naturally I don't have access to the actual
production cluster because I'm not on ops.

My last method to try is going to be downloading the enwiki database dump
and setting up my own replica. However, this involves hard drive space that
I don't have.

Basically I want to know if anybody is aware of any other method through
which I can test queries? Thanks in advance.

*-- *
*Tyler Romeo*
Stevens Institute of Technology, Class of 2016
Major in Computer Science
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Ori Livneh's new role on site performance

2013-10-08 Thread MZMcBride 
James Forrester wrote:
>On 8 October 2013 11:59, Rob Lanphier  wrote:
>> I'm excited to announce that Ori Livneh will be moving into Platform
>> Engineering as Senior Performance Engineer.
>
>​Congratulations, Ori; great news, and I look forward to continuing to
>work
>with you in your new rôle!​

This. Congrats!

MZMcBride



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Sign up for the MediaWiki Architecture Summit (deadline October 22)

2013-10-08 Thread Rob Lanphier 
Hi everyone,

Registration for the MediaWiki Architecture Summit is now open, and
will close Tuesday, October 22, 17:00 UTC (10am PDT).  Here is the
essential event information:

January 23-24, 2014
SPUR - 654 Mission Street
San Francisco, CA, USA

Please put your name/information on the following wiki page to sign up:
https://www.mediawiki.org/wiki/Architecture_guidelines/Meetings/Architecture_Summit_2014

NOTE: even if you've previously RSVP'd (having received an early
invitation as we were picking dates), please sign up on the page
above.  We're requesting information that will help shape the event.

The target audience of this event are mainly project maintainers and
other developers involved in Requests for Comment and, in general, the
future evolution of MediaWiki and related software components. All
participants must specify their current involvement in the MediaWiki
project, the RFC(s) that are promoting and the main future topics they
want to discuss.  We are tentatively aiming to have around 50-60
people and we have some budget to sponsor travel for accepted
participants.

Rob

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] GSoC 2013 Summary : Incremental updates for Kiwix

2013-10-08 Thread Kiran Mathew Koshy 
Hello everyone,

This summer, I was working on the project "ZIM incremental updates for
Kiwix"
 as part of GSoC 2013, under my mentors Emmanuel  Engelhart and Tommi
 Mäkitalo


  The tools zimdiff and zimpatch- used for incremental updates to a zim
file- has been created. Zimdiff is used to create a "diff" file between two
versions of a zim file, and zimpatch is used to obtain the final version of
the file using the diff file and the original file. These tools have been
added as classes in the existing zimlib library(part of openzim project).
  Some integration into Kiwix has been done, mostly on the server side.
Some parts of the client- side integration is still left, and I am working
on it.

https://github.com/kiranmathewkoshy/kiwix_mirror
https://github.com/kiranmathewkoshy/openzim

Its been a great experience working with my mentors, and I intend to stick
around for more.
-- 
Kiran Mathew Koshy
Electrical Engineering,
IIT Patna,
Patna,
India.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Tidy - help needed

2013-10-08 Thread Bartosz Dziewoński 

(Tangenially related: I made a proposal to eradicate Tidy some days ago over at 
https://bugzilla.wikimedia.org/show_bug.cgi?id=54617 . I didn't look into 
alternatives much yet – not enough free time – but 
http://www.bioinformatics.org/phplabware/internal_utilities/htmLawed/index.php 
looks promising. The bug is open for taking :) )

--
Matma Rex

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Ori Livneh's new role on site performance

2013-10-08 Thread James Forrester 
On 8 October 2013 11:59, Rob Lanphier  wrote:

> Hi everyone,
>
> I'm excited to announce that Ori Livneh will be moving into Platform
> Engineering as Senior Performance Engineer.
>

​Congratulations, Ori; great news, and I look forward to continuing to work
with you in your new rôle!​

​J.
-- 
James D. Forrester
Product Manager, VisualEditor
Wikimedia Foundation, Inc.

jforres...@wikimedia.org | @jdforrester
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Ori Livneh's new role on site performance

2013-10-08 Thread Rob Lanphier 
Hi everyone,

I'm excited to announce that Ori Livneh will be moving into Platform
Engineering as Senior Performance Engineer.  This work is riffing off
of the work that he's done with the Growth (nee E3) team, where a big
part of his job was instrumenting new features to measure their impact
on editor retention.

In his new role, Ori will be responsible for many aspects of site
performance.  There are a couple of different things one can optimize
for when optimizing site performance: hardware cost or end-user
experience.  The best optimizations do both, but for some
optimizations you need to choose which is more important (e.g. buying
new hardware in order to increase responsiveness).  Ori will mainly be
focusing on the end user experience, and will be doing so in an
end-to-end fashion (even looking at rendering performance on end-user
machines).

Ori's day-to-day tasks will involve instrumenting the site such that
developers can see the impact of their work on site responsiveness.
He'll also work closely with our TechOps team, making recommendations
on site configuration that can have a big impact.  He's also going to
spearhead the investigation and likely implementation of
performance-related technology on our cluster.  There will also be a
lot of smaller changes that it'll make sense for Ori just to tackle
rather than waiting for someone else to do them, so he'll likely spend
a chunk of his time doing that as well.

Ori has done tremendous work in the year and a half he's been here in
bootstrapping the Growth team, and we're really excited to see what he
will achieve in this new role.  Welcome, Ori!

Rob

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Exceptions, return false/null, and other error handling possibilities.

2013-10-08 Thread Bryan Davis 
> Can we add an example of that usage to the status object with a note
> not to follow the "return this in case of error" pattern that you
> might see elsewhere in the code?  It might even be worth a bit of
> refactoring to get rid of the old pattern or people will still keep
> finding it and copying it.

{{done}} [0]


[0]: 
https://gerrit.wikimedia.org/r/#q,Ia98543caaa829cad443abf0f0f5038b3de943ef8,n,z
-- 
Bryan Davis  Wikimedia Foundation
[[m:User:BDavis_(WMF)]]  Sr Software EngineerBoise, ID
irc: bd808v:415.839.6885 x6855

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Fwd: [wikimedia #5912] Upgrade PHP throughout the cluster to 5.3.10-1ubuntu3.8+wmf1 on Thursday 2013-10-08

2013-10-08 Thread Alexandros Kosiaris 
FYI


-- Forwarded message --
From: Core operations via RT 
Date: Tue, Oct 8, 2013 at 12:14 PM
Subject: [wikimedia #5912] Upgrade PHP throughout the cluster to
5.3.10-1ubuntu3.8+wmf1 on Thursday 2013-10-08
To: akosia...@wikimedia.org


Scheduling an upgrade of PHP through the cluster to from 5.3.10-1ubuntu3.6+wmf1
to 5.3.10-1ubuntu3.8+wmf1. The changes are three CVEs

CVE-2013-4635
CVE-2013-4113
CVE-2013-4248

and bug #63055 per RT #5209 (which was not solved due to one more bug)

The packages have been built and tested on beta and test.wikipedia.org and no
problems have arisen. The upgrade is expected to not be noticeable.


--
(ticket has been created)


-- 
Alexandros Kosiaris 

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Jenkins maintenance Oct 9th 8am UTC

2013-10-08 Thread Antoine Musso 
Hello,

Daniel Zahn and I will be rebooting gallium which is the Jenkins master.
The reboot is scheduled on Oct 10th at 8am and we last for a couple
hours while the server is busy fscking the huge disks.

Jobs will not be processed while the server is busy, I will retrigger
them manually.

Sorry for the inconvenience.


-- 
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Exceptions, return false/null, and other error handling possibilities.

2013-10-08 Thread Nikolas Everett 
On Tue, Oct 8, 2013 at 12:15 AM, Tim Starling  wrote:
> On 08/10/13 14:40, Erik Bernhardson wrote:
>> A reviewer should be able to
>> know if the error conditions are properly handled by looking at the new
>> code, not by looking up all the function calls to see what they can
>> possibly return.
>
> This is why the recommended pattern for Status objects is to return a
> Status object unconditionally

Can we add an example of that usage to the status object with a note
not to follow the "return this in case of error" pattern that you
might see elsewhere in the code?  It might even be worth a bit of
refactoring to get rid of the old pattern or people will still keep
finding it and copying it.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Exceptions, return false/null, and other error handling possibilities.

2013-10-08 Thread Antoine Musso 
Le 08/10/13 07:23, Matthew Flaschen a écrit :
> 
> There are really two parts of the status class:
> 
> * Setting error codes of varying severity (e.g. newGood for success,
> newFatal for fatal error, error for a regular error), then letting the
> caller check for them (e.g. hasMessage(), isOK and isGood, if it's *not*
> okay follow-up with getErrorsArray()).  Note that hasMessage uses
> internal error codes *not* user-facing errors, so it's a way to check
> programmatically if a particular type of error occurred.
> 
> * Message formatting with wfMessage and friends; indeed, this whould
> probably not be in the class if it were designed today, under single
> responsibility principle.
> 
> I would investigate if you can use just the first.

Would it make sense to RFC the decoupling of the Status class and get it
split in two parts: Status and StatusFormatter (or whatever).

-- 
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] New! mw.Title reborn

2013-10-08 Thread Antoine Musso 
Le 07/10/13 23:33, Krinkle a écrit :
> Last Friday, the mw.Title rewrite landed in master. Here's a brief summary of 
> the changes.

For those willing to test it out in an environment similar to the
production wikis, you can use the beta cluster:

  http://en.wikipedia.beta.wmflabs.org/

You should be able to clone your gadget there and play with it safely :-]


-- 
Antoine "hashar" Musso


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l