Re: [RFC PATCH] REST: Add new setting for maximum API page size

2018-08-07 Thread Andrew Donnellan

On 07/08/18 17:20, Daniel Axtens wrote:

I wonder if we could let authenticated users do slower queries. Maybe
something for later. Let's split the difference at 250 then, I'd be
happy to merge that.


If you want to take this patch and s/500/250/g during merge then I'd be 
happy with that.


--
Andrew Donnellan  OzLabs, ADL Canberra
andrew.donnel...@au1.ibm.com  IBM Australia Limited

___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork


Re: [RFC PATCH] REST: Add new setting for maximum API page size

2018-08-07 Thread Daniel Axtens
 FWIW we now have this applied on patchwork.ozlabs.org and it appears to
 be working. Would like some more input as to what an appropriate default
 limit is.
>>> 
>>> My completely fact-free feeling/opinion is that if it takes more than
>>> ~1sec to run on Ozlabs, it's probably too high. Apart from that, I'm not
>>> fussed.
>>
>> OzLabs.org is not a fast machine. 1 second round trip time == 100 per 
>> page. 500 per page == ~2.3 seconds round trip.
>>
>> A single 500 item load is still under half the time of doing 5 * 100 
>> item loads...

I wonder if we could let authenticated users do slower queries. Maybe
something for later. Let's split the difference at 250 then, I'd be
happy to merge that.

> I bet MySQL would execute the query quicker :)

In general (and I have tested this) mysql 5.7 executes patchwork queries
slower than postgres 9.6 - this is on my laptop on an SSD with a couple
100ks of patches across 2 or 3 projects.

> Anyway, that sounds really quite slow and we should look at why on earth
> it's that slow - is there some kind of horrific hidden join or poor
> indexing or query plan or something?

Yes.

So the 1.0 -> 2.0 transition was not good for performance because it
turns out databases do not map well to object-oriented structures.* I
didn't think to check for performance at the time, so we're
progressively improving that by denormalising stuff. (In fact that was a
major driver of the 2.1 release!) There is still further to go.

Regards,
Daniel

* Yes, I probably should have realised this at the time. something
  something unpaid volunteers something something patch review something
  something.

>
> -- 
> Stewart Smith
> OPAL Architect, IBM.
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork