Hi,
The maintenance was scheduled on Monday, for the day after that. We
had only a few hours to plan for it and communicate about it, and I
think we did a pretty good job given the time we had.
The maintenance banner was up for a few hours (not a day) prior to the
maintenance window to give reade
We already get spammed enough with notices, which is one of the
reasons many people hide them permanently via css so they never
intrude again, which would make them pointless for the more
established users, also overkill for what was meant to be (from my
understanding) only a few minutes of downtim
On 26/05/11 17:57, Milos Rancic wrote:
> On 05/25/2011 01:12 PM, Tim Starling wrote:
>> On 25/05/11 18:14, Thomas Morton wrote:
>>> IRC was flooded with people who didn't understand what was going on. And
>>> many didn't believe/understand that it was maintenance... so this is
>>> definitely an are
On 05/26/2011 10:18 AM, Theo10011 wrote:
> There was, it ran for a day. (
> http://meta.wikimedia.org/wiki/Special:CentralNotice)- Generic maintenance
> notice.
So, then it should just last a bit longer (maybe three days if not a
week?) and we would avoid the most of complains.
_
Thomas Morton, 26/05/2011 10:11:
> I'm pretty sure there was a site notice; I recall seeing one anyway :)
For a day: http://meta.wikimedia.org/wiki/Special:CentralNotice
Nemo
___
foundation-l mailing list
foundation-l@lists.wikimedia.org
Unsubscribe: h
There was, it ran for a day. (
http://meta.wikimedia.org/wiki/Special:CentralNotice)- Generic maintenance
notice.
Theo
On Thu, May 26, 2011 at 1:41 PM, Thomas Morton wrote:
> I'm pretty sure there was a site notice; I recall seeing one anyway :)
>
> Tom
>
> On 26 May 2011 09:09, Federico Leva
On 05/26/2011 10:09 AM, Federico Leva (Nemo) wrote:
> Milos Rancic, 26/05/2011 09:57:
>> Site notice for a week before the maintenance would be useful, too. We
>> communicate with our users via web site, not via emails.
>
> A week of pain to signal (and not avoid) an hour of pain? Doesn't look
> l
I'm pretty sure there was a site notice; I recall seeing one anyway :)
Tom
On 26 May 2011 09:09, Federico Leva (Nemo) wrote:
> Milos Rancic, 26/05/2011 09:57:
> > Site notice for a week before the maintenance would be useful, too. We
> > communicate with our users via web site, not via emails.
Milos Rancic, 26/05/2011 09:57:
> Site notice for a week before the maintenance would be useful, too. We
> communicate with our users via web site, not via emails.
A week of pain to signal (and not avoid) an hour of pain? Doesn't look
like a gain.
Nemo
__
On 05/25/2011 01:12 PM, Tim Starling wrote:
> On 25/05/11 18:14, Thomas Morton wrote:
>> IRC was flooded with people who didn't understand what was going on. And
>> many didn't believe/understand that it was maintenance... so this is
>> definitely an area worth improving.
>
> Maybe we can replace
Domas Mituzas wrote:
> FAIL WHALE!
>
> W W W
> WW W W
> '. W
> .-""-._ \ \.--|
> / "-..__) .-'
> | _ /
> \'-.__, .__.,'
> `''._\--'
> V
http://en.wikipedia.org/wiki/User:MZMcBride/Blame_wheel <3
On Tue, May 24, 2011 at 6:32 AM, Thomas Morton
wrote:
> So, just a quick thought for future reference - during maintenance is it
> possible in future to update the error message to explain that maintenance
> is ongoing?
I work with lots of (library) databases, and standard practice for
these serv
Zitat von MZMcBride :
> m...@marcusbuck.org wrote:
>> The sensible reaction (from a person who is involved in the
>> maintenance) would be:
>> "Oh, sorry, we were so much occupied with making the maintenance work
>> as smooth and uninterruptive as possible that we totally didn't think
>> about th
@Tim: Understood, I'll make sure I know this will work first so as not to
generate work for you. My initial idea might not be so workable given the
architecture used (and how Squid handles error codes). I'll roll up some
servers here at work and run some tests.
@Domos; echoing what Risker said...
Hi!
> Domas, why so defensive?
I'm contrarian in this case :)
> unfeasible because of the work involved, but you can probably say that
> without all the combative snark.
Well, as with every downtime, there are way more issues* that end up uncovered
and have to be looked at, and yet largest em
On 26/05/11 00:05, Thomas Morton wrote:
> Is the Squid configuration the foundation employs available publicly
> somewhere (I'm scanning the SVN and not seeing it..)? Because I don't mind
> having a look and filing a specific bugzilla correction with various bits of
> code & changes.
>
> It's abou
The measure is "impact to users", not "revenue losses"...
Access to Wikimedia is ubiquitous - it pops up in other tools, it's linked
from other sites, it's used to track events of interest to the reader, it's
used for work, leisure, projects, and social activities. Imagine people
browsing google.
On 25 May 2011 09:50, Domas Mituzas wrote:
> Oh, by the way, I don't know where you look, but I somewhat missed
> communication about maintenance events ongoing in Google or Microsoft or
> Apple - you think they have none?
> Did you get lots of clarification why your gmail was unreachable?
> Did
Is the Squid configuration the foundation employs available publicly
somewhere (I'm scanning the SVN and not seeing it..)? Because I don't mind
having a look and filing a specific bugzilla correction with various bits of
code & changes.
It's about time I refreshed my Squid knowledge :)
Tom
On 25
Domas, why so defensive? No one accused you of anything or blamed you
for the downtime. The comments suggesting more finely-tuned error
messages weren't critical of you or Tim or the developers in general,
they were just (reasonable) suggestions. Maybe adjusting all the
various error messages in an
On 25/05/11 23:41, FT2 wrote:
> As a non-tech, don't all reads (at least) pass through the squids, so we can
> identify and report in a nice way a lot of connection errors at that point?
>
Maybe it would be possible to identify error messages by their HTTP
response code, and replace the body with
> Wikipedia going down without a temporary explanation page is roughly of the
> same scale as apple.com going down with no explanation, google.com going
> down with no explanation, microsoft.com going down with no explanation, and
> so on.
WHOAH THERE IS QUITE SOME SELF ENTITLEMENT THERE.
Microso
Just conceptualising...
I haven't played with Squid for a while (so am rusty) but the simplest
solution would probably be to catch all PHP errors somewhere in the
Mediawiki code and return a 500 status error code.
Then get Squid to map that to the static error page.
On the other hand throwing a
As a non-tech, don't all reads (at least) pass through the squids, so we can
identify and report in a nice way a lot of connection errors at that point?
FT2
On Wed, May 25, 2011 at 2:18 PM, Tim Starling wrote:
> There are dozens of places where error messages are generated. It's
> not trivial
2011/5/25 Tim Starling :
> On 25/05/11 22:27, Strainu wrote:
>> What I understood from this thread is: if you have a planned
>> maintenance windows between 13 and 14 GMT, it would be appreciated if
>> you could:
>> - create a simple page that says: "We are working on our servers
>> between 13 and 1
Me - no.
Readers who didn't know - yes.
Wikipedia going down without a temporary explanation page is roughly of the
same scale as apple.com going down with no explanation, google.com going
down with no explanation, microsoft.com going down with no explanation, and
so on.
"Top 5 website" means we
On 25/05/11 22:27, Strainu wrote:
> What I understood from this thread is: if you have a planned
> maintenance windows between 13 and 14 GMT, it would be appreciated if
> you could:
> - create a simple page that says: "We are working on our servers
> between 13 and 14 GMT and Wikipedia might be una
Tim,
Great, thanks for that. Seeing as it was me that raise this ;) I guess it's
only right I take up the gauntlet, so will try and find time later to
propose something.
Tom
On 25 May 2011 13:48, Tim Starling wrote:
> On 25/05/11 21:19, MZMcBride wrote:
> > Tim Starling wrote:
> >> Maybe we c
On 25/05/11 21:19, MZMcBride wrote:
> Tim Starling wrote:
>> Maybe we can replace the IRC link in the Squid error message with a
>> link to the WatchMouse page (status.wikimedia.org). That would reduce
>> the IRC flood.
>
> * https://bugzilla.wikimedia.org/show_bug.cgi?id=16043
> * https://bugzill
What I understood from this thread is: if you have a planned
maintenance windows between 13 and 14 GMT, it would be appreciated if
you could:
- create a simple page that says: "We are working on our servers
between 13 and 14 GMT and Wikipedia might be unavailable during that
time"
- replace the usu
On Wed, May 25, 2011 at 10:09 PM, Theo10011 wrote:
> On Wed, May 25, 2011 at 5:31 PM, MZMcBride wrote:
>
>> Theo10011 wrote:
>> > Instead of diverting users to IRC, how about an outage/error page with a
>> > twitter/identi.ca feed with updates from the tech team, or at least a
>> page
>> > with c
On Wed, May 25, 2011 at 5:31 PM, MZMcBride wrote:
> Theo10011 wrote:
> > Instead of diverting users to IRC, how about an outage/error page with a
> > twitter/identi.ca feed with updates from the tech team, or at least a
> page
> > with customized message in case of previously planned outage. Most
Theo10011 wrote:
> Instead of diverting users to IRC, how about an outage/error page with a
> twitter/identi.ca feed with updates from the tech team, or at least a page
> with customized message in case of previously planned outage. Most of the
> tech staff already use Twitter/Identi.ca to update u
On Wed, May 25, 2011 at 4:40 PM, Domas Mituzas wrote:
> Hi!
>
> > That's... completely missing the point. Yes the specific errors faced
> were
> > unexpected or unforseen, BUT they were a* direct result* of the
> maintenance
> > between 13:00 and 14:00. I am simply passing on the feeling of our
>
Maybe we can replace the IRC link in the Squid error message with a
link to the WatchMouse page
@Tim; that seems a good idea.
@Domas, I'm afraid you don't seem to have understood the premise of my
suggestion.. which is fine. But one fallacy is worth responding to:
> You have some annoying users,
m...@marcusbuck.org wrote:
> The sensible reaction (from a person who is involved in the
> maintenance) would be:
> "Oh, sorry, we were so much occupied with making the maintenance work
> as smooth and uninterruptive as possible that we totally didn't think
> about that. We will integrate it into o
Tim Starling wrote:
> Maybe we can replace the IRC link in the Squid error message with a
> link to the WatchMouse page (status.wikimedia.org). That would reduce
> the IRC flood.
* https://bugzilla.wikimedia.org/show_bug.cgi?id=16043
* https://bugzilla.wikimedia.org/show_bug.cgi?id=20079
MZMcBrid
On 25/05/11 18:14, Thomas Morton wrote:
> IRC was flooded with people who didn't understand what was going on. And
> many didn't believe/understand that it was maintenance... so this is
> definitely an area worth improving.
Maybe we can replace the IRC link in the Squid error message with a
link t
Hi!
> That's... completely missing the point. Yes the specific errors faced were
> unexpected or unforseen, BUT they were a* direct result* of the maintenance
> between 13:00 and 14:00. I am simply passing on the feeling of our
> readership; which was that the situation was badly communicated to t
Domas, what are you trying to achieve with your comments on Tom's
suggestions? He just said that if we know that maintenance is done and
could cause outages we should put up an error message that informs the
reader about the maintenance work and tells him not to worry. That's
obviously a go
Tim,
When I originally wrote:
during maintenance is it possible in future to update the error message to
explain that maintenance is ongoing?
That was a bit of a silly moment from me :) I see how that implies
in-maintenance updates.
In fact my suggestion was to update the error message to mentio
> If we knew what would fail to put an appropriate error message there, we'd
probably fix the problem beforehand. :-)
That's... completely missing the point. Yes the specific errors faced were
unexpected or unforseen, BUT they were a* direct result* of the maintenance
between 13:00 and 14:00. I am
Hi!
> The maintenance was planned, downtime was noted as possible. An error
> message that reflects that seems, frankly, a good idea.
There're lots of great ideas around the world, feeding the hungry and curing
the cancer among them.
> The response to what I thought to be a helpful suggestion
The maintenance was planned, downtime was noted as possible. An error
message that reflects that seems, frankly, a good idea.
The response to what I thought to be a helpful suggestion in improving
communication with readership has been... incredibly disappointing. I wish I
hadn't bothered. :( I wa
Hi!
> Huh? The downtime was expected during 13:00 and 14:00 UTC, or at least there
> was an email warning of such things the day before... hardly unplanned or
> unknown.
there's a bit of a difference between maintenance window and expected downtime
during it.
Domas
_
It might be more worthwhile to put downtime status updates on
status.wikimedia.org as a logical page to display the status of the servers,
and link to it from the default error messages.
Given that status.wm.org is an external service, it would hopefully not be
affected by any outages and the Watc
On Wed, May 25, 2011 at 12:09 PM, Thomas Morton
wrote:
> This is the error message that appeared for me (and apparently others):
> http://nomulous.com/blog/wp-content/uploads/2009/09/wikipedia_error.png
I won't continue arguing about whether or not it should say "planned,"
but I do have to say th
Huh? The downtime was expected during 13:00 and 14:00 UTC, or at least there
was an email warning of such things the day before... hardly unplanned or
unknown.
Tom
On 25 May 2011 11:12, Domas Mituzas wrote:
> >
> > As you can see it refers to some unknown error. In this case the
> > maintentanc
>
> As you can see it refers to some unknown error. In this case the
> maintentance was known and* pre-planned* for several days.
technically this was unknown problem :)
> A lot of people were confused by the outage and the error page was unhelpful
> to them. This could have been mitigated simp
> unless, as Tim already addressed, you wanted a developer
assigned to updating the message in real time.
No, definitely not what was being suggested.
This is the error message that appeared for me (and apparently others):
http://nomulous.com/blog/wp-content/uploads/2009/09/wikipedia_error.png
A
On Wed, May 25, 2011 at 11:57 AM, Thomas Morton
wrote:
> That's interesting, what was the wording for the maintenance message? I only
> ever saw the default "our servers are experiencing a technical problem"
> error page.
I could be misremembering, because I honestly didn't care that much,
but I
Austin,
That's interesting, what was the wording for the maintenance message? I only
ever saw the default "our servers are experiencing a technical problem"
error page.
Tom
On 25 May 2011 10:53, Austin Hair wrote:
> On Wed, May 25, 2011 at 9:32 AM, FT2 wrote:
> > I don't get this.
> >
> > Wou
On Wed, May 25, 2011 at 9:32 AM, FT2 wrote:
> I don't get this.
>
> Would it be possible in future, if the sites are unresponsive, or will be
> unresponsive due to planned maintenance, to establish a fallback that simply
> displays an explanatory status message to the public?
Would it have change
In future can I have vanilla and strawberry with that? :)
FT2
On Wed, May 25, 2011 at 9:16 AM, Domas Mituzas wrote:
> In future we will have five nines availability and no downtimes will
> happen.
>
>
___
foundation-l mailing list
foundation-l@lists.w
> priority task being to get the site working again. Maybe at some time
> in the future, we will have enough 24/7 sysadmin manpower that we can
> respond to any unplanned downtime in the way you suggest. But we don't
> have that capability just yet.
In future we will have five nines availability
I think it's reasonable (and indeed standard) to deploy some sort of
"downtime" maintenance error message.
If that requires improving the error handling code to catch a wider variety
of errors and push people to the error message page then I understand the
time issues :).
If the short term soluti
On 25/05/11 17:32, FT2 wrote:
> I don't get this.
>
> Would it be possible in future, if the sites are unresponsive, or will be
> unresponsive due to planned maintenance, to establish a fallback that simply
> displays an explanatory status message to the public?
You mean replace the entire site w
I don't get this.
Would it be possible in future, if the sites are unresponsive, or will be
unresponsive due to planned maintenance, to establish a fallback that simply
displays an explanatory status message to the public?
FT2
On Wed, May 25, 2011 at 8:15 AM, Tim Starling wrote:
> (snip)
> The
On 24/05/11 23:32, Thomas Morton wrote:
> So, just a quick thought for future reference - during maintenance is it
> possible in future to update the error message to explain that maintenance
> is ongoing?
>
> Seeing as how widely WMF projects are used by a non-technical project the
> current MySQ
Speaking of WP downtime, you might be particularly interested in today's
XKCD:
http://xkcd.com/903/
wittylama.com/blog
Peace, love & metadata
On 24 May 2011 21:35, Itzik Edri wrote:
> I totally agree with Thomas.
>
> On Tue, May 24, 2011 at 4:32 PM, Thomas Morton <
> morton.tho...@googlemail.
I totally agree with Thomas.
On Tue, May 24, 2011 at 4:32 PM, Thomas Morton wrote:
> So, just a quick thought for future reference - during maintenance is it
> possible in future to update the error message to explain that maintenance
> is ongoing?
>
> Seeing as how widely WMF projects are used
So, just a quick thought for future reference - during maintenance is it
possible in future to update the error message to explain that maintenance
is ongoing?
Seeing as how widely WMF projects are used by a non-technical project the
current MySQL connection error I am seeing on Commons is just go
Dear all,
The Wikimedia Foundation will be performing network maintenance on
Tuesday, May 24 between 13:00 and 14:00 (UTC) (see other timezones on
timeanddate.com: http://ur1.ca/49cl2 ).
During the maintenance period, you may experience intermittent
connection issues to Wikimedia Foundation websi
63 matches
Mail list logo
