On Tue, Jul 14, 2009 at 2:35 PM, Nicolas Sylvain <nsylv...@chromium.org>wrote:
>
>
> On Tue, Jul 14, 2009 at 2:25 PM, Evan Martin <e...@chromium.org> wrote:
>
>> On Tue, Jul 14, 2009 at 2:18 PM, Nicolas Sylvain<nsylv...@chromium.org>
>> wrote:
>> > The underlying problem with buildbot is the database format, which is
>> just
>> > hundred of
>> > thousand of files on the harddrive, with no "seek" capability, and the
>> fact
>> > that the
>> > webserver itself is single threaded.
>> > We currently have 63 slaves on our main waterfall. I think this is too
>> > many for what
>> > buildbot can really support. We would ideally need to split it.
>>
>> Can we get upstream buildbot devs involved in this discussion? It
>> seems they ought to be able to scale better than they have.
>
>
> I talked to them a little. They do plan some of that for their 1.0 release,
> but they
> said that it was not on the radar until then.
>
>
>>
>>
>> It seems to me a caching layer that only ever hit the "backend" every
>> ten seconds would allow it ten seconds to grind through its
>> computations, which should be more than sufficient, without any extra
>> splitting up required. That is, we should (a) fix the proxy and (b)
>> make every use the proxy.
>
>
> That makes a lot of sense. I agree that we should fix the proxy, and
> make more people use it. Some direct buildbot access would still be
> required internally to force a build and stuff like that.
>
>
>>
>> > Q3: What kind of auto-refresh do we need?
>> > We used to be at 60 secs for a long time, and I changed it a couple of
>> > weeks ago to 90 secs.
>> > No one complained, so I guess this is good. Should we go even higher
>> than
>> > that?
>>
>> I personally hate auto-refresh. We should make it opt-in since I
>> doubt most users need it and it adds load. I expect many people
>> (myself included) leave the buildbot page open in a background tab and
>> have it continually refreshing despite not looking at it.
>>
>> (My other common use case: the tree is red, I start scrolling down to
>> see what's gone wrong, and then the page refreshes out from under me
>> and I lose where I was looking.)
>
>
> Yep, a lot of people told me the same thing. Some other told me they would
> like
> a faster reload. Now i'm tempted to let the user control the reload and
> not give
> a default one.
>
>
>
>>
>> > - Get a better machine. It's already running on a dedicated dual quad
>> core
>> > nehalem server
>> > with 24gb of RAM and 15k rpm drives.
>>
>> This is absurdly powerful! It should have all the data necessary to
>> generate the page in RAM already, no need for even touching the disks
>> (?).
>
>
> Yeah, i'm not too sure how to debug this. When I strace the process I only
> see file reads, scrolling, like crazy, all the time.
>
That is pretty nuts. Is it calling fsync or something crazy? Since you
said strace, I'm assmuming linux. In that case, the buffer cache should be
saving you from disk accesses for most everything.
-Albert
--~--~---------~--~----~------------~-------~--~----~
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
-~----------~----~----~----~------~----~------~--~---
