Re: Avoid detoast overhead when possible

2023-12-05 Thread Andy Fan
Hi Nikita Malakhov writes: > > With your setup (table created with setup.sql): You need to "set jit to off" to turn on this feature, as I state in [1] [2]. [1] https://www.postgresql.org/message-id/87ttoyihgm.fsf%40163.com [2] https://www.postgresql.org/message-id/877cltvxgt.fsf%40163.com

Re: Avoid detoast overhead when possible

2023-12-05 Thread Nikita Malakhov
Hi, With your setup (table created with setup.sql): postgres@postgres=# explain analyze select big->'1', big->'2', big->'3', big->'5', big->'10' from b; QUERY PLAN

Re: Avoid detoast overhead when possible

2023-12-05 Thread zhihuifan1213
Nikita Malakhov writes: > Hi, > > Hmmm, I've checked this patch and can't see performance difference on a large > (2 key-value pairs) json, using toasted json column several times makes no > difference between current implementation on master (like queries mentioned > above). > > Maybe

Re: Avoid detoast overhead when possible

2023-12-05 Thread Nikita Malakhov
Hi, Hmmm, I've checked this patch and can't see performance difference on a large (2 key-value pairs) json, using toasted json column several times makes no difference between current implementation on master (like queries mentioned above). Maybe I'm doing something wrong? On Tue, Dec 5,

Re: Avoid detoast overhead when possible

2023-12-04 Thread zhihuifan1213
Hi, Matthias van de Meent writes: > SELECT toastable_col FROM t1 > WHERE f(t1.toastable_col) > ORDER BY nonindexed; Thanks for this example! it's true that the current design requires more memory to sort since toastable_col is detoasted at the scan stage and it is output to the sort node. It

Re: Avoid detoast overhead when possible

2023-12-04 Thread Matthias van de Meent
On Mon, 4 Dec 2023 at 14:23, wrote: > > > Hi, > > Matthias van de Meent writes: > > > On Mon, 4 Dec 2023 at 07:56, wrote: > > > ..It would also add overhead when > > we write results to disk, such as spilling merge sorts, hash join > > spills, or CTE materializations. > > > > Could you find a

Re: Avoid detoast overhead when possible

2023-12-04 Thread zhihuifan1213
Hi, Matthias van de Meent writes: > On Mon, 4 Dec 2023 at 07:56, wrote: > ..It would also add overhead when > we write results to disk, such as spilling merge sorts, hash join > spills, or CTE materializations. > > Could you find a way to reduce this memory and IO usage when the value > is

Re: Avoid detoast overhead when possible

2023-12-04 Thread Matthias van de Meent
On Mon, 4 Dec 2023 at 07:56, wrote: > 'SELECT f1(toast_col) FROM t;' will apply this code path, but nothing > gain and nothing lost. Applying this code path only when the toast > datum is accessed 1+ times needs some extra run-time effort. I don't > implement this so far, I'd like to see if I

Re: Avoid detoast overhead when possible

2023-12-04 Thread zhihuifan1213
Nikita Malakhov writes: Hi! > > There's a view from the other angle - detoast just attributes that are needed > (partial detoast), with optimized storage mechanics for JSONb. Very glad to know that, looking forward your design & patch! -- Best Regards Andy Fan

Re: Avoid detoast overhead when possible

2023-12-04 Thread Nikita Malakhov
Hi! There's a view from the other angle - detoast just attributes that are needed (partial detoast), with optimized storage mechanics for JSONb. I'm preparing a patch for it, so maybe the best results could be acquired by combining these two techniques. What do you think? -- Regards, Nikita

Avoid detoast overhead when possible

2023-12-03 Thread zhihuifan1213
Currently our code can do lazily detoast by design, for example: SELECT toast_col FROM t; SELECT toast_col FROM t ORDER BY b; SELECT toast_col FROM t join t2 using(c); it is only detoast at {type}_out function. The benefits includes: 1. The life time of detoast datum is pretty short which is