[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Jesse Stay]

Can you share what exactly this will affect?  What symptoms will we see?

Thanks,

Jesse

On Thu, May 7, 2009 at 4:21 PM, Doug Williams  wrote:

> Prepare your apps! The status blog has the details [1]:
>
>> We’re planning a significant database upgrade that will greatly increase
>> system performance. To do this, we will require two planned maintenance
>> windows: Friday, May 8th from 2p-3p Pacific and again on Monday, May 11th
>> from Noon-1p Pacific.
>>
>> We’re taking these maintenance windows earlier in the day because of the
>> extent of the changes involved. We want to make sure that we are able to
>> respond quickly and efficiently to any issues that may arise from the work
>> being performed.
>>
>> We will update [status.twitter.com] with any changes or additional
>> information about this upgrade.
>>
>
> http://status.twitter.com/post/104738920/planned-maintenance-tomorrow-monday
>
> Thanks,
> Doug
> --
>
> Doug Williams
> Twitter Platform Support
> http://twitter.com/dougw
>
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Brian Gilham]

Since the message doesn't make it clear (unless my sleep-deprived mind 
is missing it), will this mean downtime?


Jesse Stay wrote:

Can you share what exactly this will affect?  What symptoms will we see?

Thanks,

Jesse

On Thu, May 7, 2009 at 4:21 PM, Doug Williams > wrote:


Prepare your apps! The status blog has the details [1]:

We’re planning a significant database upgrade that will
greatly increase system performance. To do this, we will
require two planned maintenance windows: Friday, May 8th from
2p-3p Pacific and again on Monday, May 11th from Noon-1p Pacific.

We’re taking these maintenance windows earlier in the day
because of the extent of the changes involved. We want to make
sure that we are able to respond quickly and efficiently to
any issues that may arise from the work being performed.

We will update [status.twitter.com
] with any changes or additional
information about this upgrade.

http://status.twitter.com/post/104738920/planned-maintenance-tomorrow-monday

Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Abraham Williams]

That is usually what "site maintenance" means...

Sent from Dev G1

On May 8, 2009 12:24 AM, "Brian Gilham"  wrote:

 Since the message doesn't make it clear (unless my sleep-deprived mind is
missing it), will this mean downtime?

Jesse Stay wrote: > > Can you share what exactly this will affect?  What
symptoms will we see? > >...

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Nick Arnett]

On Thu, May 7, 2009 at 10:33 PM, Abraham Williams <4bra...@gmail.com> wrote:

> That is usually what "site maintenance" means...
>
In my experience, a maintenance window means that the site could be down for
any or all of that period.  The way Doug wrote it, I'd imagine that they
expect the site will only be down for a short time, but they're reserving an
hour in case it takes longer for unanticipated reasons.

We shall see...

Nick

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Doug Williams]

Hi all.

To clarify: the site will be down. twitter.com and search.twitter.com will
fail to serve content while the back end changes are being made.

Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 7:31 AM, Nick Arnett  wrote:

>
>
> On Thu, May 7, 2009 at 10:33 PM, Abraham Williams <4bra...@gmail.com>wrote:
>
>> That is usually what "site maintenance" means...
>>
> In my experience, a maintenance window means that the site could be down
> for any or all of that period.  The way Doug wrote it, I'd imagine that they
> expect the site will only be down for a short time, but they're reserving an
> hour in case it takes longer for unanticipated reasons.
>
> We shall see...
>
> Nick
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Michael Bailey]

Which includes OAuth verification functionalities, yes?

Michael Bailey
Thought Leader and Serial Entrepreneur
"Cultivating the landscape of the online multimedia community"
Blog: http://www.mobasoft.com
New Service: http://mobatalk.com




Doug Williams wrote:
> Hi all.
>
> To clarify: the site will be down. twitter.com  
> and search.twitter.com  will fail to serve 
> content while the back end changes are being made.
>
> Thanks,
> Doug
> --
>
> Doug Williams
> Twitter Platform Support
> http://twitter.com/dougw
>
>
>
>
> On Fri, May 8, 2009 at 7:31 AM, Nick Arnett  > wrote:
>
>
>
> On Thu, May 7, 2009 at 10:33 PM, Abraham Williams
> <4bra...@gmail.com > wrote:
>
> That is usually what "site maintenance" means...
>
> In my experience, a maintenance window means that the site could
> be down for any or all of that period.  The way Doug wrote it, I'd
> imagine that they expect the site will only be down for a short
> time, but they're reserving an hour in case it takes longer for
> unanticipated reasons.
>
> We shall see...
>
> Nick
>
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Doug Williams]

@Michael: Yes. The OAuth server makes use of the twitter.com web servers as
well as the database backing store which is the focus of the maintenance.

Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 10:00 AM, Michael Bailey wrote:

>  Which includes OAuth verification functionalities, yes?
>
> Michael Bailey
> Thought Leader and Serial Entrepreneur
> "Cultivating the landscape of the online multimedia community"
> Blog: http://www.mobasoft.com
> New Service: http://mobatalk.com
>
>
>
> Doug Williams wrote:
>
> Hi all.
>
> To clarify: the site will be down. twitter.com and search.twitter.com will
> fail to serve content while the back end changes are being made.
>
> Thanks,
> Doug
> --
>
> Doug Williams
> Twitter Platform Support
> http://twitter.com/dougw
>
>
>
>
> On Fri, May 8, 2009 at 7:31 AM, Nick Arnett  wrote:
>
>>
>>
>>  On Thu, May 7, 2009 at 10:33 PM, Abraham Williams <4bra...@gmail.com>wrote:
>>
>>> That is usually what "site maintenance" means...
>>>
>>  In my experience, a maintenance window means that the site could be down
>> for any or all of that period.  The way Doug wrote it, I'd imagine that they
>> expect the site will only be down for a short time, but they're reserving an
>> hour in case it takes longer for unanticipated reasons.
>>
>>  We shall see...
>>
>>  Nick
>>
>
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Chad Etzel]

Periscope down. Preparing to dive!

On Fri, May 8, 2009 at 1:38 PM, Doug Williams  wrote:
> @Michael: Yes. The OAuth server makes use of the twitter.com web servers as
> well as the database backing store which is the focus of the maintenance.
>
> Thanks,
> Doug
> --
>
> Doug Williams
> Twitter Platform Support
> http://twitter.com/dougw
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Michael Bailey]

Thanks for the clarification Doug. Stock up on some RedBull and best of 
luck for a smooth upgrade to the team!

Michael Bailey




Doug Williams wrote:
> @Michael: Yes. The OAuth server makes use of the twitter.com 
>  web servers as well as the database backing store 
> which is the focus of the maintenance.
>
> Thanks,
> Doug
> --
>
> Doug Williams
> Twitter Platform Support
> http://twitter.com/dougw
>
>
>
>
> On Fri, May 8, 2009 at 10:00 AM, Michael Bailey  > wrote:
>
> Which includes OAuth verification functionalities, yes?
>
> Michael Bailey
> Thought Leader and Serial Entrepreneur
> "Cultivating the landscape of the online multimedia community"
> Blog: http://www.mobasoft.com
> New Service: http://mobatalk.com
>
> 
>
>
>
> Doug Williams wrote:
>> Hi all.
>>
>> To clarify: the site will be down. twitter.com
>>  and search.twitter.com
>>  will fail to serve content while the
>> back end changes are being made.
>>
>> Thanks,
>> Doug
>> --
>>
>> Doug Williams
>> Twitter Platform Support
>> http://twitter.com/dougw
>>
>>
>>
>>
>> On Fri, May 8, 2009 at 7:31 AM, Nick Arnett
>> mailto:nick.arn...@gmail.com>> wrote:
>>
>>
>>
>> On Thu, May 7, 2009 at 10:33 PM, Abraham Williams
>> <4bra...@gmail.com > wrote:
>>
>> That is usually what "site maintenance" means...
>>
>> In my experience, a maintenance window means that the site
>> could be down for any or all of that period.  The way Doug
>> wrote it, I'd imagine that they expect the site will only be
>> down for a short time, but they're reserving an hour in case
>> it takes longer for unanticipated reasons.
>>
>> We shall see...
>>
>> Nick
>>
>>
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Eric Blair]

Any reason the API is serving up an HTML page instead of the normal  
error XML?


--Eric

On May 8, 2009, at 1:50 PM, Michael Bailey wrote:

Thanks for the clarification Doug. Stock up on some RedBull and best  
of luck for a smooth upgrade to the team!

Michael Bailey



Doug Williams wrote:


@Michael: Yes. The OAuth server makes use of the twitter.com web  
servers as well as the database backing store which is the focus of  
the maintenance.


Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 10:00 AM, Michael Bailey  
 wrote:

Which includes OAuth verification functionalities, yes?
Michael Bailey
Thought Leader and Serial Entrepreneur
"Cultivating the landscape of the online multimedia community"
Blog: http://www.mobasoft.com
New Service: http://mobatalk.com




Doug Williams wrote:


Hi all.

To clarify: the site will be down. twitter.com and  
search.twitter.com will fail to serve content while the back end  
changes are being made.


Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 7:31 AM, Nick Arnett  
 wrote:



On Thu, May 7, 2009 at 10:33 PM, Abraham Williams  
<4bra...@gmail.com> wrote:

That is usually what "site maintenance" means...

In my experience, a maintenance window means that the site could  
be down for any or all of that period.  The way Doug wrote it, I'd  
imagine that they expect the site will only be down for a short  
time, but they're reserving an hour in case it takes longer for  
unanticipated reasons.


We shall see...

Nick

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Doug Williams]

The issues tell the story:

http://code.google.com/p/twitter-api/issues/list?can=1&q=maintenance&colspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Component&cells=tiles

Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 2:46 PM, Eric Blair  wrote:

>
> Any reason the API is serving up an HTML page instead of the normal error
> XML?
>
> --Eric
>
>
> On May 8, 2009, at 1:50 PM, Michael Bailey wrote:
>
>  Thanks for the clarification Doug. Stock up on some RedBull and best of
>> luck for a smooth upgrade to the team!
>> Michael Bailey
>>
>>
>>
>> Doug Williams wrote:
>>
>>>
>>> @Michael: Yes. The OAuth server makes use of the twitter.com web servers
>>> as well as the database backing store which is the focus of the maintenance.
>>>
>>> Thanks,
>>> Doug
>>> --
>>>
>>> Doug Williams
>>> Twitter Platform Support
>>> http://twitter.com/dougw
>>>
>>>
>>>
>>>
>>> On Fri, May 8, 2009 at 10:00 AM, Michael Bailey 
>>> wrote:
>>> Which includes OAuth verification functionalities, yes?
>>> Michael Bailey
>>> Thought Leader and Serial Entrepreneur
>>> "Cultivating the landscape of the online multimedia community"
>>> Blog: http://www.mobasoft.com
>>> New Service: http://mobatalk.com
>>>
>>>
>>>
>>>
>>> Doug Williams wrote:
>>>

 Hi all.

 To clarify: the site will be down. twitter.com and search.twitter.comwill 
 fail to serve content while the back end changes are being made.

 Thanks,
 Doug
 --

 Doug Williams
 Twitter Platform Support
 http://twitter.com/dougw




 On Fri, May 8, 2009 at 7:31 AM, Nick Arnett 
 wrote:


 On Thu, May 7, 2009 at 10:33 PM, Abraham Williams <4bra...@gmail.com>
 wrote:
 That is usually what "site maintenance" means...

 In my experience, a maintenance window means that the site could be down
 for any or all of that period.  The way Doug wrote it, I'd imagine that 
 they
 expect the site will only be down for a short time, but they're reserving 
 an
 hour in case it takes longer for unanticipated reasons.

 We shall see...

 Nick


>>>
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Jesse Stay]

Am I the only one confused on what exactly this is affecting regarding the
API?
@Jesse

On Fri, May 8, 2009 at 4:08 PM, Doug Williams  wrote:

> The issues tell the story:
>
>
> http://code.google.com/p/twitter-api/issues/list?can=1&q=maintenance&colspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Component&cells=tiles
>
> Thanks,
> Doug
> --
>
> Doug Williams
> Twitter Platform Support
> http://twitter.com/dougw
>
>
>
>
> On Fri, May 8, 2009 at 2:46 PM, Eric Blair  wrote:
>
>>
>> Any reason the API is serving up an HTML page instead of the normal error
>> XML?
>>
>> --Eric
>>
>>
>> On May 8, 2009, at 1:50 PM, Michael Bailey wrote:
>>
>>  Thanks for the clarification Doug. Stock up on some RedBull and best of
>>> luck for a smooth upgrade to the team!
>>> Michael Bailey
>>>
>>>
>>>
>>> Doug Williams wrote:
>>>

 @Michael: Yes. The OAuth server makes use of the twitter.com web
 servers as well as the database backing store which is the focus of the
 maintenance.

 Thanks,
 Doug
 --

 Doug Williams
 Twitter Platform Support
 http://twitter.com/dougw




 On Fri, May 8, 2009 at 10:00 AM, Michael Bailey 
 wrote:
 Which includes OAuth verification functionalities, yes?
 Michael Bailey
 Thought Leader and Serial Entrepreneur
 "Cultivating the landscape of the online multimedia community"
 Blog: http://www.mobasoft.com
 New Service: http://mobatalk.com




 Doug Williams wrote:

>
> Hi all.
>
> To clarify: the site will be down. twitter.com and search.twitter.comwill 
> fail to serve content while the back end changes are being made.
>
> Thanks,
> Doug
> --
>
> Doug Williams
> Twitter Platform Support
> http://twitter.com/dougw
>
>
>
>
> On Fri, May 8, 2009 at 7:31 AM, Nick Arnett 
> wrote:
>
>
> On Thu, May 7, 2009 at 10:33 PM, Abraham Williams <4bra...@gmail.com>
> wrote:
> That is usually what "site maintenance" means...
>
> In my experience, a maintenance window means that the site could be
> down for any or all of that period.  The way Doug wrote it, I'd imagine 
> that
> they expect the site will only be down for a short time, but they're
> reserving an hour in case it takes longer for unanticipated reasons.
>
> We shall see...
>
> Nick
>
>

>>
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Michael Bailey]

FYI,

Well, if Twitter is supposed to be back online now, it isn't here.
Good luck

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Michael Bailey]

And now it is - congrats! ;^)

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Michael Bailey]

No, and I spoke too soon, the site was up, but now it's not I guess 
there be a few whales.

Jesse Stay wrote:
> Am I the only one confused on what exactly this is affecting regarding 
> the API?
>
> @Jesse
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Eric Blair]

Yeah, I filed 300 a while ago and it was marked as fixed, so I was  
surprised when I saw HTML.


--Eric

On May 8, 2009, at 6:08 PM, Doug Williams wrote:


The issues tell the story:

http://code.google.com/p/twitter-api/issues/list?can=1&q=maintenance&colspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Component&cells=tiles

Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 2:46 PM, Eric Blair   
wrote:


Any reason the API is serving up an HTML page instead of the normal  
error XML?


--Eric


On May 8, 2009, at 1:50 PM, Michael Bailey wrote:

Thanks for the clarification Doug. Stock up on some RedBull and best  
of luck for a smooth upgrade to the team!

Michael Bailey



Doug Williams wrote:

@Michael: Yes. The OAuth server makes use of the twitter.com web  
servers as well as the database backing store which is the focus of  
the maintenance.


Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 10:00 AM, Michael Bailey  
 wrote:

Which includes OAuth verification functionalities, yes?
Michael Bailey
Thought Leader and Serial Entrepreneur
"Cultivating the landscape of the online multimedia community"
Blog: http://www.mobasoft.com
New Service: http://mobatalk.com




Doug Williams wrote:

Hi all.

To clarify: the site will be down. twitter.com and  
search.twitter.com will fail to serve content while the back end  
changes are being made.


Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 7:31 AM, Nick Arnett   
wrote:



On Thu, May 7, 2009 at 10:33 PM, Abraham Williams  
<4bra...@gmail.com> wrote:

That is usually what "site maintenance" means...

In my experience, a maintenance window means that the site could be  
down for any or all of that period.  The way Doug wrote it, I'd  
imagine that they expect the site will only be down for a short  
time, but they're reserving an hour in case it takes longer for  
unanticipated reasons.


We shall see...

Nick

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Doug Williams]

Eric,
I was right next to Alex when he made the fix for Issue 300 and I remember
seeing it deployed. I'll check into it.


Thanks,
Doug


Doug Williams | Platform Support | Twitter, Inc.

539 Bryant St. Suite 402, San Francisco, CA 94107 http://twitter.com/dougw



On Fri, May 8, 2009 at 3:38 PM, Eric Blair  wrote:

>
> Yeah, I filed 300 a while ago and it was marked as fixed, so I was
> surprised when I saw HTML.
>
> --Eric
>
>
> On May 8, 2009, at 6:08 PM, Doug Williams wrote:
>
>  The issues tell the story:
>>
>>
>> http://code.google.com/p/twitter-api/issues/list?can=1&q=maintenance&colspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Component&cells=tiles
>>
>> Thanks,
>> Doug
>> --
>>
>> Doug Williams
>> Twitter Platform Support
>> http://twitter.com/dougw
>>
>>
>>
>>
>> On Fri, May 8, 2009 at 2:46 PM, Eric Blair 
>> wrote:
>>
>> Any reason the API is serving up an HTML page instead of the normal error
>> XML?
>>
>> --Eric
>>
>>
>> On May 8, 2009, at 1:50 PM, Michael Bailey wrote:
>>
>> Thanks for the clarification Doug. Stock up on some RedBull and best of
>> luck for a smooth upgrade to the team!
>> Michael Bailey
>>
>>
>>
>> Doug Williams wrote:
>>
>> @Michael: Yes. The OAuth server makes use of the twitter.com web servers
>> as well as the database backing store which is the focus of the maintenance.
>>
>> Thanks,
>> Doug
>> --
>>
>> Doug Williams
>> Twitter Platform Support
>> http://twitter.com/dougw
>>
>>
>>
>>
>> On Fri, May 8, 2009 at 10:00 AM, Michael Bailey 
>> wrote:
>> Which includes OAuth verification functionalities, yes?
>> Michael Bailey
>> Thought Leader and Serial Entrepreneur
>> "Cultivating the landscape of the online multimedia community"
>> Blog: http://www.mobasoft.com
>> New Service: http://mobatalk.com
>>
>>
>>
>>
>> Doug Williams wrote:
>>
>> Hi all.
>>
>> To clarify: the site will be down. twitter.com and search.twitter.comwill 
>> fail to serve content while the back end changes are being made.
>>
>> Thanks,
>> Doug
>> --
>>
>> Doug Williams
>> Twitter Platform Support
>> http://twitter.com/dougw
>>
>>
>>
>>
>> On Fri, May 8, 2009 at 7:31 AM, Nick Arnett 
>> wrote:
>>
>>
>> On Thu, May 7, 2009 at 10:33 PM, Abraham Williams <4bra...@gmail.com>
>> wrote:
>> That is usually what "site maintenance" means...
>>
>> In my experience, a maintenance window means that the site could be down
>> for any or all of that period.  The way Doug wrote it, I'd imagine that they
>> expect the site will only be down for a short time, but they're reserving an
>> hour in case it takes longer for unanticipated reasons.
>>
>> We shall see...
>>
>> Nick
>>
>>
>>
>>
>>
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Eric Blair]

Another maintenance-related question - are things completely back up  
and running? I'm asking because I'm seeing inconsistent behavior.


In my servers logs, I'm seeing a number of timeouts when talking to  
Twitter. Also, when I try to run curl commands from my servers to  
Twitter, they are _extremely_ slow, sometimes timing out. Some of my  
servers are systems that regularly communicate with Twitter while  
others rarely do so.


However, when I run the same curl command from my desktop, it runs in  
a snap.


I don't recall seeing anything like this in the past, where there  
would be such a consistent difference between the two environments.


--Eric

On May 8, 2009, at 6:41 PM, Doug Williams wrote:


Eric,
I was right next to Alex when he made the fix for Issue 300 and I  
remember seeing it deployed. I'll check into it.



Thanks,
Doug


Doug Williams | Platform Support | Twitter, Inc.

539 Bryant St. Suite 402, San Francisco, CA 94107 http://twitter.com/dougw



On Fri, May 8, 2009 at 3:38 PM, Eric Blair   
wrote:


Yeah, I filed 300 a while ago and it was marked as fixed, so I was  
surprised when I saw HTML.


--Eric


On May 8, 2009, at 6:08 PM, Doug Williams wrote:

The issues tell the story:

http://code.google.com/p/twitter-api/issues/list?can=1&q=maintenance&colspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Component&cells=tiles

Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 2:46 PM, Eric Blair   
wrote:


Any reason the API is serving up an HTML page instead of the normal  
error XML?


--Eric


On May 8, 2009, at 1:50 PM, Michael Bailey wrote:

Thanks for the clarification Doug. Stock up on some RedBull and best  
of luck for a smooth upgrade to the team!

Michael Bailey



Doug Williams wrote:

@Michael: Yes. The OAuth server makes use of the twitter.com web  
servers as well as the database backing store which is the focus of  
the maintenance.


Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 10:00 AM, Michael Bailey  
 wrote:

Which includes OAuth verification functionalities, yes?
Michael Bailey
Thought Leader and Serial Entrepreneur
"Cultivating the landscape of the online multimedia community"
Blog: http://www.mobasoft.com
New Service: http://mobatalk.com




Doug Williams wrote:

Hi all.

To clarify: the site will be down. twitter.com and  
search.twitter.com will fail to serve content while the back end  
changes are being made.


Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw




On Fri, May 8, 2009 at 7:31 AM, Nick Arnett   
wrote:



On Thu, May 7, 2009 at 10:33 PM, Abraham Williams  
<4bra...@gmail.com> wrote:

That is usually what "site maintenance" means...

In my experience, a maintenance window means that the site could be  
down for any or all of that period.  The way Doug wrote it, I'd  
imagine that they expect the site will only be down for a short  
time, but they're reserving an hour in case it takes longer for  
unanticipated reasons.


We shall see...

Nick

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Yu-Shan Fung]

I'm seeing that as well. Guessing is the caches slowly warming up?

On Fri, May 8, 2009 at 4:11 PM, Eric Blair  wrote:

>
> Another maintenance-related question - are things completely back up and
> running? I'm asking because I'm seeing inconsistent behavior.
>
> In my servers logs, I'm seeing a number of timeouts when talking to
> Twitter. Also, when I try to run curl commands from my servers to Twitter,
> they are _extremely_ slow, sometimes timing out. Some of my servers are
> systems that regularly communicate with Twitter while others rarely do so.
>
> However, when I run the same curl command from my desktop, it runs in a
> snap.
>
> I don't recall seeing anything like this in the past, where there would be
> such a consistent difference between the two environments.
>
> --Eric
>
>
> On May 8, 2009, at 6:41 PM, Doug Williams wrote:
>
>  Eric,
>> I was right next to Alex when he made the fix for Issue 300 and I remember
>> seeing it deployed. I'll check into it.
>>
>>
>> Thanks,
>> Doug
>>
>> 
>> Doug Williams | Platform Support | Twitter, Inc.
>>
>> 539 Bryant St. Suite 402, San Francisco, CA 94107
>> http://twitter.com/dougw
>>
>>
>>
>> On Fri, May 8, 2009 at 3:38 PM, Eric Blair 
>> wrote:
>>
>> Yeah, I filed 300 a while ago and it was marked as fixed, so I was
>> surprised when I saw HTML.
>>
>> --Eric
>>
>>
>> On May 8, 2009, at 6:08 PM, Doug Williams wrote:
>>
>> The issues tell the story:
>>
>>
>> http://code.google.com/p/twitter-api/issues/list?can=1&q=maintenance&colspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Component&cells=tiles
>>
>> Thanks,
>> Doug
>> --
>>
>> Doug Williams
>> Twitter Platform Support
>> http://twitter.com/dougw
>>
>>
>>
>>
>> On Fri, May 8, 2009 at 2:46 PM, Eric Blair 
>> wrote:
>>
>> Any reason the API is serving up an HTML page instead of the normal error
>> XML?
>>
>> --Eric
>>
>>
>> On May 8, 2009, at 1:50 PM, Michael Bailey wrote:
>>
>> Thanks for the clarification Doug. Stock up on some RedBull and best of
>> luck for a smooth upgrade to the team!
>> Michael Bailey
>>
>>
>>
>> Doug Williams wrote:
>>
>> @Michael: Yes. The OAuth server makes use of the twitter.com web servers
>> as well as the database backing store which is the focus of the maintenance.
>>
>> Thanks,
>> Doug
>> --
>>
>> Doug Williams
>> Twitter Platform Support
>> http://twitter.com/dougw
>>
>>
>>
>>
>> On Fri, May 8, 2009 at 10:00 AM, Michael Bailey 
>> wrote:
>> Which includes OAuth verification functionalities, yes?
>> Michael Bailey
>> Thought Leader and Serial Entrepreneur
>> "Cultivating the landscape of the online multimedia community"
>> Blog: http://www.mobasoft.com
>> New Service: http://mobatalk.com
>>
>>
>>
>>
>> Doug Williams wrote:
>>
>> Hi all.
>>
>> To clarify: the site will be down. twitter.com and search.twitter.comwill 
>> fail to serve content while the back end changes are being made.
>>
>> Thanks,
>> Doug
>> --
>>
>> Doug Williams
>> Twitter Platform Support
>> http://twitter.com/dougw
>>
>>
>>
>>
>> On Fri, May 8, 2009 at 7:31 AM, Nick Arnett 
>> wrote:
>>
>>
>> On Thu, May 7, 2009 at 10:33 PM, Abraham Williams <4bra...@gmail.com>
>> wrote:
>> That is usually what "site maintenance" means...
>>
>> In my experience, a maintenance window means that the site could be down
>> for any or all of that period.  The way Doug wrote it, I'd imagine that they
>> expect the site will only be down for a short time, but they're reserving an
>> hour in case it takes longer for unanticipated reasons.
>>
>> We shall see...
>>
>> Nick
>>
>>
>>
>>
>>
>>
>>
>


-- 
“When nothing seems to help, I go look at a stonecutter hammering away at
his rock perhaps a hundred times without as much as a crack showing in it.
Yet at the hundred and first blow it will split in two, and I know it was
not that blow that did it, but all that had gone before.” — Jacob Riis

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Nick Arnett]

On Fri, May 8, 2009 at 4:11 PM, Eric Blair  wrote:

>
> Another maintenance-related question - are things completely back up and
> running? I'm asking because I'm seeing inconsistent behavior.
>
> In my servers logs, I'm seeing a number of timeouts when talking to
> Twitter. Also, when I try to run curl commands from my servers to Twitter,
> they are _extremely_ slow, sometimes timing out. Some of my servers are
> systems that regularly communicate with Twitter while others rarely do so.


Same here.

Nick

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Arnaud]

I don't think the API is back in service yet, Eric.
At least on my side, I can't connect to twitter servers.

Arnaud.

On 9 mai, 01:11, Eric Blair  wrote:
> Another maintenance-related question - are things completely back up  
> and running? I'm asking because I'm seeing inconsistent behavior.
>
> In my servers logs, I'm seeing a number of timeouts when talking to  
> Twitter. Also, when I try to run curl commands from my servers to  
> Twitter, they are _extremely_ slow, sometimes timing out. Some of my  
> servers are systems that regularly communicate with Twitter while  
> others rarely do so.
>
> However, when I run the same curl command from my desktop, it runs in  
> a snap.
>
> I don't recall seeing anything like this in the past, where there  
> would be such a consistent difference between the two environments.
>
> --Eric
>
> On May 8, 2009, at 6:41 PM, Doug Williams wrote:
>
> > Eric,
> > I was right next to Alex when he made the fix for Issue 300 and I  
> > remember seeing it deployed. I'll check into it.
>
> > Thanks,
> > Doug
>
> > 
> > Doug Williams | Platform Support | Twitter, Inc.
>
> > 539 Bryant St. Suite 402, San Francisco, CA 94107http://twitter.com/dougw
>
> > On Fri, May 8, 2009 at 3:38 PM, Eric Blair   
> > wrote:
>
> > Yeah, I filed 300 a while ago and it was marked as fixed, so I was  
> > surprised when I saw HTML.
>
> > --Eric
>
> > On May 8, 2009, at 6:08 PM, Doug Williams wrote:
>
> > The issues tell the story:
>
> >http://code.google.com/p/twitter-api/issues/list?can=1&q=maintenance&;...
>
> > Thanks,
> > Doug
> > --
>
> > Doug Williams
> > Twitter Platform Support
> >http://twitter.com/dougw
>
> > On Fri, May 8, 2009 at 2:46 PM, Eric Blair   
> > wrote:
>
> > Any reason the API is serving up an HTML page instead of the normal  
> > error XML?
>
> > --Eric
>
> > On May 8, 2009, at 1:50 PM, Michael Bailey wrote:
>
> > Thanks for the clarification Doug. Stock up on some RedBull and best  
> > of luck for a smooth upgrade to the team!
> > Michael Bailey
>
> > Doug Williams wrote:
>
> > @Michael: Yes. The OAuth server makes use of the twitter.com web  
> > servers as well as the database backing store which is the focus of  
> > the maintenance.
>
> > Thanks,
> > Doug
> > --
>
> > Doug Williams
> > Twitter Platform Support
> >http://twitter.com/dougw
>
> > On Fri, May 8, 2009 at 10:00 AM, Michael Bailey  
> >  wrote:
> > Which includes OAuth verification functionalities, yes?
> > Michael Bailey
> > Thought Leader and Serial Entrepreneur
> > "Cultivating the landscape of the online multimedia community"
> > Blog:http://www.mobasoft.com
> > New Service:http://mobatalk.com
>
> > Doug Williams wrote:
>
> > Hi all.
>
> > To clarify: the site will be down. twitter.com and  
> > search.twitter.com will fail to serve content while the back end  
> > changes are being made.
>
> > Thanks,
> > Doug
> > --
>
> > Doug Williams
> > Twitter Platform Support
> >http://twitter.com/dougw
>
> > On Fri, May 8, 2009 at 7:31 AM, Nick Arnett   
> > wrote:
>
> > On Thu, May 7, 2009 at 10:33 PM, Abraham Williams  
> > <4bra...@gmail.com> wrote:
> > That is usually what "site maintenance" means...
>
> > In my experience, a maintenance window means that the site could be  
> > down for any or all of that period.  The way Doug wrote it, I'd  
> > imagine that they expect the site will only be down for a short  
> > time, but they're reserving an hour in case it takes longer for  
> > unanticipated reasons.
>
> > We shall see...
>
> > Nick

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Doug Williams]

You guys are smart. Three things:

1) The cache is slow to warm up. Therefore there was some latency involved
with the restart of the service.

2) Some of you may have noticed problems about an hour after the restart.
Some important objects expire from cache after an hour. Since there was a
huge influx of objects expiring 1 hour after the restart presumably together
there were problems as the database began to be overworked. We're doing a
post-mortem now to determine how to better keep the cache hot even after
objects are cached nearly simultaneously after restarts like we experienced
today.

3) The downtime today was to increase capacity, which does not exactly
translate to performance.

Thanks,
Doug
--

Doug Williams
Twitter Platform Support
http://twitter.com/dougw



On Fri, May 8, 2009 at 5:28 PM, Nick Arnett  wrote:

>
>
> On Fri, May 8, 2009 at 4:11 PM, Eric Blair  wrote:
>
>>
>> Another maintenance-related question - are things completely back up and
>> running? I'm asking because I'm seeing inconsistent behavior.
>>
>> In my servers logs, I'm seeing a number of timeouts when talking to
>> Twitter. Also, when I try to run curl commands from my servers to Twitter,
>> they are _extremely_ slow, sometimes timing out. Some of my servers are
>> systems that regularly communicate with Twitter while others rarely do so.
>
>
> Same here.
>
> Nick
>

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Dossy Shiobara]

On 5/8/09 10:15 PM, Doug Williams wrote:

2) Some of you may have noticed problems about an hour after the
restart. Some important objects expire from cache after an hour. Since
there was a huge influx of objects expiring 1 hour after the restart
presumably together there were problems as the database began to be
overworked. We're doing a post-mortem now to determine how to better
keep the cache hot even after objects are cached nearly simultaneously
after restarts like we experienced today.


Ah, the stampeding herd cold start design problem ...

The simplest solution is to add some jitter - i.e., don't expire things 
exactly after 3600 seconds.  Set the expiration to be X +/- N, where X 
is your lifetime and N is the jitter which might be small like 10 or 
large like 300, computed per object usually at the time it's cached.


Over time, given a reasonable PRNG, you should have sufficient drift 
that cache expiration will remain random but on a cold start you won't 
have all your hot objects expiring at exactly the same time every X period.


--
Dossy Shiobara  | do...@panoptic.com | http://dossy.org/
Panoptic Computer Network   | http://panoptic.com/
  "He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST

[Chad Etzel]

On Sat, May 9, 2009 at 9:43 AM, Dossy Shiobara  wrote:
> Ah, the stampeding herd cold start design problem ...
>
> The simplest solution is to add some jitter - i.e., don't expire things
> exactly after 3600 seconds.  Set the expiration to be X +/- N, where X is
> your lifetime and N is the jitter which might be small like 10 or large like
> 300, computed per object usually at the time it's cached.
>
> Over time, given a reasonable PRNG, you should have sufficient drift that
> cache expiration will remain random but on a cold start you won't have all
> your hot objects expiring at exactly the same time every X period.

Ah, something we agree on!  Never underestimate the value of jitter.
I have used exactly this technique on many projects and it works
amazingly well.

Just to be pedantic:  The expiration should be "X +/- rand(n, N)"
Where "n" is probably 0, but is some lower bound to the jitter, and
less than "N" which is probably non-zero and is the upper bound to the
jitter.  As time goes to infinity, statistically the objects will
expire on average every X seconds.

You can also play with "startup jitter" where all the objects are not
created at time 0 when the switch is flipped, but randomly over a
period of time.  This may or may not apply depending on your
situation.

I'm sure you guys there have thought about this, but it can be useful
for a lot of things, so maybe other devs on the list will glean
something.

-Chad