[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
On 5/8/09 10:15 PM, Doug Williams wrote: 2) Some of you may have noticed problems about an hour after the restart. Some important objects expire from cache after an hour. Since there was a huge influx of objects expiring 1 hour after the restart presumably together there were problems as the database began to be overworked. We're doing a post-mortem now to determine how to better keep the cache hot even after objects are cached nearly simultaneously after restarts like we experienced today. Ah, the stampeding herd cold start design problem ... The simplest solution is to add some jitter - i.e., don't expire things exactly after 3600 seconds. Set the expiration to be X +/- N, where X is your lifetime and N is the jitter which might be small like 10 or large like 300, computed per object usually at the time it's cached. Over time, given a reasonable PRNG, you should have sufficient drift that cache expiration will remain random but on a cold start you won't have all your hot objects expiring at exactly the same time every X period. -- Dossy Shiobara | do...@panoptic.com | http://dossy.org/ Panoptic Computer Network | http://panoptic.com/ He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on. (p. 70)
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
On Sat, May 9, 2009 at 9:43 AM, Dossy Shiobara do...@panoptic.com wrote: Ah, the stampeding herd cold start design problem ... The simplest solution is to add some jitter - i.e., don't expire things exactly after 3600 seconds. Set the expiration to be X +/- N, where X is your lifetime and N is the jitter which might be small like 10 or large like 300, computed per object usually at the time it's cached. Over time, given a reasonable PRNG, you should have sufficient drift that cache expiration will remain random but on a cold start you won't have all your hot objects expiring at exactly the same time every X period. Ah, something we agree on! Never underestimate the value of jitter. I have used exactly this technique on many projects and it works amazingly well. Just to be pedantic: The expiration should be X +/- rand(n, N) Where n is probably 0, but is some lower bound to the jitter, and less than N which is probably non-zero and is the upper bound to the jitter. As time goes to infinity, statistically the objects will expire on average every X seconds. You can also play with startup jitter where all the objects are not created at time 0 when the switch is flipped, but randomly over a period of time. This may or may not apply depending on your situation. I'm sure you guys there have thought about this, but it can be useful for a lot of things, so maybe other devs on the list will glean something. -Chad
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.com wrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
Hi all. To clarify: the site will be down. twitter.com and search.twitter.com will fail to serve content while the back end changes are being made. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 7:31 AM, Nick Arnett nick.arn...@gmail.com wrote: On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.comwrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
Which includes OAuth verification functionalities, yes? Michael Bailey Thought Leader and Serial Entrepreneur Cultivating the landscape of the online multimedia community Blog: http://www.mobasoft.com New Service: http://mobatalk.com Doug Williams wrote: Hi all. To clarify: the site will be down. twitter.com http://twitter.com and search.twitter.com http://search.twitter.com will fail to serve content while the back end changes are being made. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 7:31 AM, Nick Arnett nick.arn...@gmail.com mailto:nick.arn...@gmail.com wrote: On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.com mailto:4bra...@gmail.com wrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
@Michael: Yes. The OAuth server makes use of the twitter.com web servers as well as the database backing store which is the focus of the maintenance. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 10:00 AM, Michael Bailey mbai...@mobasoft.comwrote: Which includes OAuth verification functionalities, yes? Michael Bailey Thought Leader and Serial Entrepreneur Cultivating the landscape of the online multimedia community Blog: http://www.mobasoft.com New Service: http://mobatalk.com Doug Williams wrote: Hi all. To clarify: the site will be down. twitter.com and search.twitter.com will fail to serve content while the back end changes are being made. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 7:31 AM, Nick Arnett nick.arn...@gmail.com wrote: On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.comwrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
Periscope down. Preparing to dive! On Fri, May 8, 2009 at 1:38 PM, Doug Williams d...@twitter.com wrote: @Michael: Yes. The OAuth server makes use of the twitter.com web servers as well as the database backing store which is the focus of the maintenance. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
Thanks for the clarification Doug. Stock up on some RedBull and best of luck for a smooth upgrade to the team! Michael Bailey Doug Williams wrote: @Michael: Yes. The OAuth server makes use of the twitter.com http://twitter.com web servers as well as the database backing store which is the focus of the maintenance. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 10:00 AM, Michael Bailey mbai...@mobasoft.com mailto:mbai...@mobasoft.com wrote: Which includes OAuth verification functionalities, yes? Michael Bailey Thought Leader and Serial Entrepreneur Cultivating the landscape of the online multimedia community Blog: http://www.mobasoft.com New Service: http://mobatalk.com Doug Williams wrote: Hi all. To clarify: the site will be down. twitter.com http://twitter.com and search.twitter.com http://search.twitter.com will fail to serve content while the back end changes are being made. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 7:31 AM, Nick Arnett nick.arn...@gmail.com mailto:nick.arn...@gmail.com wrote: On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.com mailto:4bra...@gmail.com wrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
FYI, Well, if Twitter is supposed to be back online now, it isn't here. Good luck
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
And now it is - congrats! ;^)
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
No, and I spoke too soon, the site was up, but now it's not I guess there be a few whales. Jesse Stay wrote: Am I the only one confused on what exactly this is affecting regarding the API? @Jesse
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
Yeah, I filed 300 a while ago and it was marked as fixed, so I was surprised when I saw HTML. --Eric On May 8, 2009, at 6:08 PM, Doug Williams wrote: The issues tell the story: http://code.google.com/p/twitter-api/issues/list?can=1q=maintenancecolspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Componentcells=tiles Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 2:46 PM, Eric Blair eric.s.bl...@gmail.com wrote: Any reason the API is serving up an HTML page instead of the normal error XML? --Eric On May 8, 2009, at 1:50 PM, Michael Bailey wrote: Thanks for the clarification Doug. Stock up on some RedBull and best of luck for a smooth upgrade to the team! Michael Bailey Doug Williams wrote: @Michael: Yes. The OAuth server makes use of the twitter.com web servers as well as the database backing store which is the focus of the maintenance. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 10:00 AM, Michael Bailey mbai...@mobasoft.com wrote: Which includes OAuth verification functionalities, yes? Michael Bailey Thought Leader and Serial Entrepreneur Cultivating the landscape of the online multimedia community Blog: http://www.mobasoft.com New Service: http://mobatalk.com Doug Williams wrote: Hi all. To clarify: the site will be down. twitter.com and search.twitter.com will fail to serve content while the back end changes are being made. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 7:31 AM, Nick Arnett nick.arn...@gmail.com wrote: On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.com wrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
Eric, I was right next to Alex when he made the fix for Issue 300 and I remember seeing it deployed. I'll check into it. Thanks, Doug Doug Williams | Platform Support | Twitter, Inc. 539 Bryant St. Suite 402, San Francisco, CA 94107 http://twitter.com/dougw On Fri, May 8, 2009 at 3:38 PM, Eric Blair eric.s.bl...@gmail.com wrote: Yeah, I filed 300 a while ago and it was marked as fixed, so I was surprised when I saw HTML. --Eric On May 8, 2009, at 6:08 PM, Doug Williams wrote: The issues tell the story: http://code.google.com/p/twitter-api/issues/list?can=1q=maintenancecolspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Componentcells=tiles Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 2:46 PM, Eric Blair eric.s.bl...@gmail.com wrote: Any reason the API is serving up an HTML page instead of the normal error XML? --Eric On May 8, 2009, at 1:50 PM, Michael Bailey wrote: Thanks for the clarification Doug. Stock up on some RedBull and best of luck for a smooth upgrade to the team! Michael Bailey Doug Williams wrote: @Michael: Yes. The OAuth server makes use of the twitter.com web servers as well as the database backing store which is the focus of the maintenance. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 10:00 AM, Michael Bailey mbai...@mobasoft.com wrote: Which includes OAuth verification functionalities, yes? Michael Bailey Thought Leader and Serial Entrepreneur Cultivating the landscape of the online multimedia community Blog: http://www.mobasoft.com New Service: http://mobatalk.com Doug Williams wrote: Hi all. To clarify: the site will be down. twitter.com and search.twitter.comwill fail to serve content while the back end changes are being made. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 7:31 AM, Nick Arnett nick.arn...@gmail.com wrote: On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.com wrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
Another maintenance-related question - are things completely back up and running? I'm asking because I'm seeing inconsistent behavior. In my servers logs, I'm seeing a number of timeouts when talking to Twitter. Also, when I try to run curl commands from my servers to Twitter, they are _extremely_ slow, sometimes timing out. Some of my servers are systems that regularly communicate with Twitter while others rarely do so. However, when I run the same curl command from my desktop, it runs in a snap. I don't recall seeing anything like this in the past, where there would be such a consistent difference between the two environments. --Eric On May 8, 2009, at 6:41 PM, Doug Williams wrote: Eric, I was right next to Alex when he made the fix for Issue 300 and I remember seeing it deployed. I'll check into it. Thanks, Doug Doug Williams | Platform Support | Twitter, Inc. 539 Bryant St. Suite 402, San Francisco, CA 94107 http://twitter.com/dougw On Fri, May 8, 2009 at 3:38 PM, Eric Blair eric.s.bl...@gmail.com wrote: Yeah, I filed 300 a while ago and it was marked as fixed, so I was surprised when I saw HTML. --Eric On May 8, 2009, at 6:08 PM, Doug Williams wrote: The issues tell the story: http://code.google.com/p/twitter-api/issues/list?can=1q=maintenancecolspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Componentcells=tiles Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 2:46 PM, Eric Blair eric.s.bl...@gmail.com wrote: Any reason the API is serving up an HTML page instead of the normal error XML? --Eric On May 8, 2009, at 1:50 PM, Michael Bailey wrote: Thanks for the clarification Doug. Stock up on some RedBull and best of luck for a smooth upgrade to the team! Michael Bailey Doug Williams wrote: @Michael: Yes. The OAuth server makes use of the twitter.com web servers as well as the database backing store which is the focus of the maintenance. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 10:00 AM, Michael Bailey mbai...@mobasoft.com wrote: Which includes OAuth verification functionalities, yes? Michael Bailey Thought Leader and Serial Entrepreneur Cultivating the landscape of the online multimedia community Blog: http://www.mobasoft.com New Service: http://mobatalk.com Doug Williams wrote: Hi all. To clarify: the site will be down. twitter.com and search.twitter.com will fail to serve content while the back end changes are being made. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 7:31 AM, Nick Arnett nick.arn...@gmail.com wrote: On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.com wrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
I'm seeing that as well. Guessing is the caches slowly warming up? On Fri, May 8, 2009 at 4:11 PM, Eric Blair eric.s.bl...@gmail.com wrote: Another maintenance-related question - are things completely back up and running? I'm asking because I'm seeing inconsistent behavior. In my servers logs, I'm seeing a number of timeouts when talking to Twitter. Also, when I try to run curl commands from my servers to Twitter, they are _extremely_ slow, sometimes timing out. Some of my servers are systems that regularly communicate with Twitter while others rarely do so. However, when I run the same curl command from my desktop, it runs in a snap. I don't recall seeing anything like this in the past, where there would be such a consistent difference between the two environments. --Eric On May 8, 2009, at 6:41 PM, Doug Williams wrote: Eric, I was right next to Alex when he made the fix for Issue 300 and I remember seeing it deployed. I'll check into it. Thanks, Doug Doug Williams | Platform Support | Twitter, Inc. 539 Bryant St. Suite 402, San Francisco, CA 94107 http://twitter.com/dougw On Fri, May 8, 2009 at 3:38 PM, Eric Blair eric.s.bl...@gmail.com wrote: Yeah, I filed 300 a while ago and it was marked as fixed, so I was surprised when I saw HTML. --Eric On May 8, 2009, at 6:08 PM, Doug Williams wrote: The issues tell the story: http://code.google.com/p/twitter-api/issues/list?can=1q=maintenancecolspec=ID+Stars+Type+Status+Priority+Owner+Summary+Opened+Modified+Componentcells=tiles Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 2:46 PM, Eric Blair eric.s.bl...@gmail.com wrote: Any reason the API is serving up an HTML page instead of the normal error XML? --Eric On May 8, 2009, at 1:50 PM, Michael Bailey wrote: Thanks for the clarification Doug. Stock up on some RedBull and best of luck for a smooth upgrade to the team! Michael Bailey Doug Williams wrote: @Michael: Yes. The OAuth server makes use of the twitter.com web servers as well as the database backing store which is the focus of the maintenance. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 10:00 AM, Michael Bailey mbai...@mobasoft.com wrote: Which includes OAuth verification functionalities, yes? Michael Bailey Thought Leader and Serial Entrepreneur Cultivating the landscape of the online multimedia community Blog: http://www.mobasoft.com New Service: http://mobatalk.com Doug Williams wrote: Hi all. To clarify: the site will be down. twitter.com and search.twitter.comwill fail to serve content while the back end changes are being made. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 7:31 AM, Nick Arnett nick.arn...@gmail.com wrote: On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.com wrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick -- “When nothing seems to help, I go look at a stonecutter hammering away at his rock perhaps a hundred times without as much as a crack showing in it. Yet at the hundred and first blow it will split in two, and I know it was not that blow that did it, but all that had gone before.” — Jacob Riis
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
On Fri, May 8, 2009 at 4:11 PM, Eric Blair eric.s.bl...@gmail.com wrote: Another maintenance-related question - are things completely back up and running? I'm asking because I'm seeing inconsistent behavior. In my servers logs, I'm seeing a number of timeouts when talking to Twitter. Also, when I try to run curl commands from my servers to Twitter, they are _extremely_ slow, sometimes timing out. Some of my servers are systems that regularly communicate with Twitter while others rarely do so. Same here. Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
I don't think the API is back in service yet, Eric. At least on my side, I can't connect to twitter servers. Arnaud. On 9 mai, 01:11, Eric Blair eric.s.bl...@gmail.com wrote: Another maintenance-related question - are things completely back up and running? I'm asking because I'm seeing inconsistent behavior. In my servers logs, I'm seeing a number of timeouts when talking to Twitter. Also, when I try to run curl commands from my servers to Twitter, they are _extremely_ slow, sometimes timing out. Some of my servers are systems that regularly communicate with Twitter while others rarely do so. However, when I run the same curl command from my desktop, it runs in a snap. I don't recall seeing anything like this in the past, where there would be such a consistent difference between the two environments. --Eric On May 8, 2009, at 6:41 PM, Doug Williams wrote: Eric, I was right next to Alex when he made the fix for Issue 300 and I remember seeing it deployed. I'll check into it. Thanks, Doug Doug Williams | Platform Support | Twitter, Inc. 539 Bryant St. Suite 402, San Francisco, CA 94107http://twitter.com/dougw On Fri, May 8, 2009 at 3:38 PM, Eric Blair eric.s.bl...@gmail.com wrote: Yeah, I filed 300 a while ago and it was marked as fixed, so I was surprised when I saw HTML. --Eric On May 8, 2009, at 6:08 PM, Doug Williams wrote: The issues tell the story: http://code.google.com/p/twitter-api/issues/list?can=1q=maintenance;... Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 2:46 PM, Eric Blair eric.s.bl...@gmail.com wrote: Any reason the API is serving up an HTML page instead of the normal error XML? --Eric On May 8, 2009, at 1:50 PM, Michael Bailey wrote: Thanks for the clarification Doug. Stock up on some RedBull and best of luck for a smooth upgrade to the team! Michael Bailey Doug Williams wrote: @Michael: Yes. The OAuth server makes use of the twitter.com web servers as well as the database backing store which is the focus of the maintenance. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 10:00 AM, Michael Bailey mbai...@mobasoft.com wrote: Which includes OAuth verification functionalities, yes? Michael Bailey Thought Leader and Serial Entrepreneur Cultivating the landscape of the online multimedia community Blog:http://www.mobasoft.com New Service:http://mobatalk.com Doug Williams wrote: Hi all. To clarify: the site will be down. twitter.com and search.twitter.com will fail to serve content while the back end changes are being made. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 7:31 AM, Nick Arnett nick.arn...@gmail.com wrote: On Thu, May 7, 2009 at 10:33 PM, Abraham Williams 4bra...@gmail.com wrote: That is usually what site maintenance means... In my experience, a maintenance window means that the site could be down for any or all of that period. The way Doug wrote it, I'd imagine that they expect the site will only be down for a short time, but they're reserving an hour in case it takes longer for unanticipated reasons. We shall see... Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
You guys are smart. Three things: 1) The cache is slow to warm up. Therefore there was some latency involved with the restart of the service. 2) Some of you may have noticed problems about an hour after the restart. Some important objects expire from cache after an hour. Since there was a huge influx of objects expiring 1 hour after the restart presumably together there were problems as the database began to be overworked. We're doing a post-mortem now to determine how to better keep the cache hot even after objects are cached nearly simultaneously after restarts like we experienced today. 3) The downtime today was to increase capacity, which does not exactly translate to performance. Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw On Fri, May 8, 2009 at 5:28 PM, Nick Arnett nick.arn...@gmail.com wrote: On Fri, May 8, 2009 at 4:11 PM, Eric Blair eric.s.bl...@gmail.com wrote: Another maintenance-related question - are things completely back up and running? I'm asking because I'm seeing inconsistent behavior. In my servers logs, I'm seeing a number of timeouts when talking to Twitter. Also, when I try to run curl commands from my servers to Twitter, they are _extremely_ slow, sometimes timing out. Some of my servers are systems that regularly communicate with Twitter while others rarely do so. Same here. Nick
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
Can you share what exactly this will affect? What symptoms will we see? Thanks, Jesse On Thu, May 7, 2009 at 4:21 PM, Doug Williams d...@twitter.com wrote: Prepare your apps! The status blog has the details [1]: We’re planning a significant database upgrade that will greatly increase system performance. To do this, we will require two planned maintenance windows: Friday, May 8th from 2p-3p Pacific and again on Monday, May 11th from Noon-1p Pacific. We’re taking these maintenance windows earlier in the day because of the extent of the changes involved. We want to make sure that we are able to respond quickly and efficiently to any issues that may arise from the work being performed. We will update [status.twitter.com] with any changes or additional information about this upgrade. http://status.twitter.com/post/104738920/planned-maintenance-tomorrow-monday Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw
[twitter-dev] Re: Planned site maintenance Friday, May 8th 2PM-3PM PST and Monday, May 11th Noon-1PM PST
Since the message doesn't make it clear (unless my sleep-deprived mind is missing it), will this mean downtime? Jesse Stay wrote: Can you share what exactly this will affect? What symptoms will we see? Thanks, Jesse On Thu, May 7, 2009 at 4:21 PM, Doug Williams d...@twitter.com mailto:d...@twitter.com wrote: Prepare your apps! The status blog has the details [1]: We’re planning a significant database upgrade that will greatly increase system performance. To do this, we will require two planned maintenance windows: Friday, May 8th from 2p-3p Pacific and again on Monday, May 11th from Noon-1p Pacific. We’re taking these maintenance windows earlier in the day because of the extent of the changes involved. We want to make sure that we are able to respond quickly and efficiently to any issues that may arise from the work being performed. We will update [status.twitter.com http://status.twitter.com] with any changes or additional information about this upgrade. http://status.twitter.com/post/104738920/planned-maintenance-tomorrow-monday Thanks, Doug -- Doug Williams Twitter Platform Support http://twitter.com/dougw
