Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[Radomir Dopieralski]

On Tue, Jul 10, 2012 at 8:57 AM, Radomir Dopieralski  wrote:
> On Tue, Jul 10, 2012 at 12:23 AM,   wrote:
 > Hopefully by migrating it to the new infrastructure.

 It appears it didn't help. Creating of a new page took 35seconds,
 which is twice the worst editing operation that I personally
 experienced with previous install.
>>>
>>> Did we look into the potential problem with mail notifications...?
>>
>> No, we didn't look into this at all.
>
> Actually I did, and I tried to find a way of deleting the unused user
> accounts to alleviate the problem, but it's not easy to determine
> which accounts are unused with the current moin.
>
> I will look at that patch, but I have my doubts about how much can be
> cached with the current architecture. Also, it will no longer be
> moinmoin as provided by the system package, unless I find a way to
> monkey-patch that code from the configuration (it can still break on
> updates then).

I finally reviewed the patch at
, modified it to a
form that can be applied from the configuration without actually
touching the files installed from the system package, and installed it
on wiki.python.org.

You can see that the patch is installed by a new item in the timings,
"getSubscribersCached". It will still take several seconds on saving a
page that wasn't edited yet, but on subsequent edits it should be much
faster.

Please let me know if you notice any problems or anomalies with the
edit notifications.
-- 
Radomir Dopieralski, http://sheep.art.pl
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[Radomir Dopieralski]

On Tue, Jul 10, 2012 at 12:23 AM,   wrote:
>>> > Hopefully by migrating it to the new infrastructure.
>>>
>>> It appears it didn't help. Creating of a new page took 35seconds,
>>> which is twice the worst editing operation that I personally
>>> experienced with previous install.
>>
>> Did we look into the potential problem with mail notifications...?
>
> No, we didn't look into this at all.

Actually I did, and I tried to find a way of deleting the unused user
accounts to alleviate the problem, but it's not easy to determine
which accounts are unused with the current moin.

I will look at that patch, but I have my doubts about how much can be
cached with the current architecture. Also, it will no longer be
moinmoin as provided by the system package, unless I find a way to
monkey-patch that code from the configuration (it can still break on
updates then).

-- 
Radomir Dopieralski, http://sheep.art.pl
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[anatoly techtonik]

On Tue, Jul 10, 2012 at 1:53 AM, anatoly techtonik  wrote:
> On Tue, Jul 10, 2012 at 1:28 AM,   wrote:
> total = 35.281s


 I can't reproduce this. I got

 total = 9.319s
>>>
>>>
>>> I don't understand what do you mean by "can't reproduce". Do you want
>>> to say that 10s is ok?
>>
>>
>> No, I want to say that 10s is significantly less than 30s; I was not
>> able to reproduce a run-time of 30s.
>
> Ok. So the performance question stays open.
>
 Is it possible to speed up wiki.python.org somehow?
>>>
>>> Hopefully by migrating it to the new infrastructure.
>>
>> It appears it didn't help.
>
> What's next?

Nevermind.

There is branch about email notifications in a separate thread.
--
anatoly t.
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[anatoly techtonik]

On Tue, Jul 10, 2012 at 1:28 AM,   wrote:
 total = 35.281s
>>>
>>>
>>> I can't reproduce this. I got
>>>
>>> total = 9.319s
>>
>>
>> I don't understand what do you mean by "can't reproduce". Do you want
>> to say that 10s is ok?
>
>
> No, I want to say that 10s is significantly less than 30s; I was not
> able to reproduce a run-time of 30s.

Ok. So the performance question stays open.

>>> Is it possible to speed up wiki.python.org somehow?
>>
>> Hopefully by migrating it to the new infrastructure.
>
> It appears it didn't help.

What's next?
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[martin]

total = 35.281s


I can't reproduce this. I got

total = 9.319s


I don't understand what do you mean by "can't reproduce". Do you want
to say that 10s is ok?


No, I want to say that 10s is significantly less than 30s; I was not
able to reproduce a run-time of 30s.

Regards,
Martin


___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[martin]

> Hopefully by migrating it to the new infrastructure.

It appears it didn't help. Creating of a new page took 35seconds,
which is twice the worst editing operation that I personally
experienced with previous install.


Did we look into the potential problem with mail notifications...?


No, we didn't look into this at all.

Regards,
Martin


___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[anatoly techtonik]

On Mon, Jul 9, 2012 at 9:04 PM, Ralf Hildebrandt
 wrote:
> * anatoly techtonik :
>> On Sat, Jun 16, 2012 at 1:27 PM, "Martin v. Löwis"  
>> wrote:
>> >> Is it possible to speed up wiki.python.org somehow?
>> >
>> > Hopefully by migrating it to the new infrastructure.
>>
>> It appears it didn't help. Creating of a new page took 35seconds,
>> which is twice the worst editing operation that I personally
>> experienced with previous install.
>>
>> Page.execute = 0.001s
>> antispam = 5.375s
>> getACL = 0.010s
>> init = 0.012s
>> load_multi_cfg = 0.000s
>> parsePageLinks = 0.021s
>> run = 35.269s
>> send_page = 0.393s
>> send_page_content = 0.027s
>> total = 35.281s
>
> Looks like a 30s DNS tiemout if you ask me.
>
> (Basically 5s for antispam and the 30s of basically nothing)

Disk I/O?
--
anatoly t.
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[anatoly techtonik]

On Mon, Jul 9, 2012 at 9:27 PM, "Martin v. Löwis"  wrote:
> On 09.07.2012 19:59, anatoly techtonik wrote:
>> On Sat, Jun 16, 2012 at 1:27 PM, "Martin v. Löwis"  
>> wrote:
 Is it possible to speed up wiki.python.org somehow?
>>>
>>> Hopefully by migrating it to the new infrastructure.
>>
>> It appears it didn't help. Creating of a new page took 35seconds,
>> which is twice the worst editing operation that I personally
>> experienced with previous install.
>>
>> Page.execute = 0.001s
>> antispam = 5.375s
>> getACL = 0.010s
>> init = 0.012s
>> load_multi_cfg = 0.000s
>> parsePageLinks = 0.021s
>> run = 35.269s
>> send_page = 0.393s
>> send_page_content = 0.027s
>> total = 35.281s
>
> I can't reproduce this. I got
>
> Page.execute = 0.000s
> antispam = 2.084s
> getACL = 0.001s
> init = 0.002s
> load_multi_cfg = 0.000s
> parsePageLinks = 0.004s
> run = 9.317s
> send_page = 0.041s
> send_page_content = 0.007s
> total = 9.319s

I don't understand what do you mean by "can't reproduce". Do you want
to say that 10s is ok? Just to remind that the letter that started
this thread mentioned that 10s is not accessible for any kind of edit.
I've tried plain editing once more - the result is still no good.

Page.execute = 0.000s
antispam = 6.726s
getACL = 0.002s
init = 0.002s
load_multi_cfg = 0.000s
parsePageLinks = 0.004s
run = 16.726s
send_page = 0.153s
send_page_content = 0.010s
total = 16.728s
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[Martin v. Löwis]

On 09.07.2012 19:59, anatoly techtonik wrote:
> On Sat, Jun 16, 2012 at 1:27 PM, "Martin v. Löwis"  wrote:
>>> Is it possible to speed up wiki.python.org somehow?
>>
>> Hopefully by migrating it to the new infrastructure.
> 
> It appears it didn't help. Creating of a new page took 35seconds,
> which is twice the worst editing operation that I personally
> experienced with previous install.
> 
> Page.execute = 0.001s
> antispam = 5.375s
> getACL = 0.010s
> init = 0.012s
> load_multi_cfg = 0.000s
> parsePageLinks = 0.021s
> run = 35.269s
> send_page = 0.393s
> send_page_content = 0.027s
> total = 35.281s

I can't reproduce this. I got

Page.execute = 0.000s
antispam = 2.084s
getACL = 0.001s
init = 0.002s
load_multi_cfg = 0.000s
parsePageLinks = 0.004s
run = 9.317s
send_page = 0.041s
send_page_content = 0.007s
total = 9.319s

Regards,
Martin
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[Ralf Hildebrandt]

* anatoly techtonik :
> On Sat, Jun 16, 2012 at 1:27 PM, "Martin v. Löwis"  wrote:
> >> Is it possible to speed up wiki.python.org somehow?
> >
> > Hopefully by migrating it to the new infrastructure.
> 
> It appears it didn't help. Creating of a new page took 35seconds,
> which is twice the worst editing operation that I personally
> experienced with previous install.
> 
> Page.execute = 0.001s
> antispam = 5.375s
> getACL = 0.010s
> init = 0.012s
> load_multi_cfg = 0.000s
> parsePageLinks = 0.021s
> run = 35.269s
> send_page = 0.393s
> send_page_content = 0.027s
> total = 35.281s

Looks like a 30s DNS tiemout if you ask me.

(Basically 5s for antispam and the 30s of basically nothing)

-- 
Ralf Hildebrandt   Charite Universitätsmedizin Berlin
ralf.hildebra...@charite.deCampus Benjamin Franklin
http://www.charite.de  Hindenburgdamm 30, 12203 Berlin
Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[anatoly techtonik]

On Sat, Jun 16, 2012 at 1:27 PM, "Martin v. Löwis"  wrote:
>> Is it possible to speed up wiki.python.org somehow?
>
> Hopefully by migrating it to the new infrastructure.

It appears it didn't help. Creating of a new page took 35seconds,
which is twice the worst editing operation that I personally
experienced with previous install.

Page.execute = 0.001s
antispam = 5.375s
getACL = 0.010s
init = 0.012s
load_multi_cfg = 0.000s
parsePageLinks = 0.021s
run = 35.269s
send_page = 0.393s
send_page_content = 0.027s
total = 35.281s
--
anatoly t.
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[anatoly techtonik]

On Sat, Jun 16, 2012 at 1:46 PM, Radomir Dopieralski  wrote:
> On Sat, Jun 16, 2012 at 12:31 PM, anatoly techtonik  
> wrote:
>> On Sat, Jun 16, 2012 at 1:27 PM, "Martin v. Löwis"  
>> wrote:
 Is it possible to speed up wiki.python.org somehow?
>>>
>>> Hopefully by migrating it to the new infrastructure.
>>
>> And still - that's the cause of the bottleneck?
>
> I enabled the timing information.
>
> I'm not 100% sure here, but on other large wikis the long saving times
> were usualy due to having to process a lot of regular expressions from
> all the user preferences in order to see whether the page title
> matches their mail notification patterns.

The results of write operations are very sad - 14 seconds.
Is it possible to postpone mail notifications or do them asyncronously?

Page.execute = 0.000s
antispam = 1.281s
getACL = 0.003s
init = 0.001s
load_multi_cfg = 0.000s
parsePageLinks = 0.010s
run = 13.566s
send_page = 0.050s
send_page_content = 0.006s
total = 13.567s

If we're going to stick these stats (they're worthy to have at hand)
then rendering it in one line that expanded if clicked for copy/paste
would make pages look as good as before.
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[Radomir Dopieralski]

On Sat, Jun 16, 2012 at 12:31 PM, anatoly techtonik  wrote:
> On Sat, Jun 16, 2012 at 1:27 PM, "Martin v. Löwis"  wrote:
>>> Is it possible to speed up wiki.python.org somehow?
>>
>> Hopefully by migrating it to the new infrastructure.
>
> And still - that's the cause of the bottleneck?

I enabled the timing information.

I'm not 100% sure here, but on other large wikis the long saving times
were usualy due to having to process a lot of regular expressions from
all the user preferences in order to see whether the page title
matches their mail notification patterns.

-- 
Radomir Dopieralski, http://sheep.art.pl
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[anatoly techtonik]

On Sat, Jun 16, 2012 at 1:27 PM, "Martin v. Löwis"  wrote:
>> Is it possible to speed up wiki.python.org somehow?
>
> Hopefully by migrating it to the new infrastructure.

And still - that's the cause of the bottleneck?
___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www

Re: [pydotorg-www] Wiki is too slow (Fwd: 1.2 beta release date/roadmap)

[Martin v. Löwis]

> Is it possible to speed up wiki.python.org somehow?

Hopefully by migrating it to the new infrastructure.

Regards,
Martin

___
pydotorg-www mailing list
pydotorg-www@python.org
http://mail.python.org/mailman/listinfo/pydotorg-www