Tom, here's the results with github data (8 top level keys) only. Here's a sample object https://gist.github.com/igrigorik/2017462
All-Lenghts + Cache-Aware EXTERNAL Query 1: 516ms Query 2: 350ms The difference is small but I's definitely faster, which makes sense since cache line misses are probably slightly reduced. As in the previous runs, I ran the query a dozen times and took the average after excluding runs with a high deviation. compare to (copied from my previous email) HEAD (aka, all offsets) EXTERNAL Test query 1 runtime: 505ms Test query 2 runtime: 350ms All Lengths (Tom Lane patch) EXTERNAL Test query 1 runtime: 525ms Test query 2 runtime: 355ms -- Arthur Silva On Tue, Aug 26, 2014 at 7:11 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > I wrote: > > I wish it were cache-friendly too, per the upthread tangent about having > > to fetch keys from all over the place within a large JSON object. > > > ... and while I was typing that sentence, lightning struck. The existing > > arrangement of object subfields with keys and values interleaved is just > > plain dumb. We should rearrange that as all the keys in order, then all > > the values in the same order. Then the keys are naturally adjacent in > > memory and object-key searches become much more cache-friendly: you > > probably touch most of the key portion of the object, but none of the > > values portion, until you know exactly what part of the latter to fetch. > > This approach might complicate the lookup logic marginally but I bet not > > very much; and it will be a huge help if we ever want to do smart access > > to EXTERNAL (non-compressed) JSON values. > > > I will go prototype that just to see how much code rearrangement is > > required. > > This looks pretty good from a coding point of view. I have not had time > yet to see if it affects the speed of the benchmark cases we've been > trying. I suspect that it won't make much difference in them. I think > if we do decide to make an on-disk format change, we should seriously > consider including this change. > > The same concept could be applied to offset-based storage of course, > although I rather doubt that we'd make that combination of choices since > it would be giving up on-disk compatibility for benefits that are mostly > in the future. > > Attached are two patches: one is a "delta" against the last jsonb-lengths > patch I posted, and the other is a "merged" patch showing the total change > from HEAD, for ease of application. > > regards, tom lane > >
