Re: remaining sql/json patches

2024-01-21 Thread John Naylor
On Mon, Nov 27, 2023 at 9:06 PM Alvaro Herrera wrote: > At this point one thing that IMO we cannot afford to do, is stop feature > progress work on the name of parser speed. I mean, parser speed is > important, and we need to be mindful that what we add is reasonable. > But at some point we'll

Re: Improve heapgetpage() performance, overhead from serializable

2024-01-21 Thread John Naylor
On Mon, Jul 17, 2023 at 9:58 PM Andres Freund wrote: > And 397->320ms > for something as core as this, is imo worth considering on its own. Hi Andres, this interesting work seems to have fallen off the radar -- are you still planning to move forward with this for v17?

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-21 Thread John Naylor
On Mon, Jan 22, 2024 at 10:28 AM Masahiko Sawada wrote: > > On further thought, as you pointed out before, "num_tids" should not > be in tidstore in terms of integration with tidbitmap.c, because > tidbitmap.c has "lossy pages". With lossy pages, "num_tids" is no > longer accurate and useful.

Re: Change GUC hashtable to use simplehash?

2024-01-21 Thread John Naylor
ed to explain usage, so I plan to hold on to this for later. Let me know what you think. [1] Examples of both in https://www.boost.org/doc/libs/1_84_0/boost/container_hash/detail/hash_mix.hpp From ad25c43c264c5857bf41cbf056ac7d4ab0995b40 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Sun, 21 Ja

Re: generate syscache info automatically

2024-01-21 Thread John Naylor
On Fri, Jan 19, 2024 at 9:03 PM Peter Eisentraut wrote: > > The DECLARE_* macros map to actual BKI commands ("declare toast", > "declare index"). So I wanted to use a different verb to distinguish > "generate code for this" from those BKI commands. That makes sense to me.

Re: Change GUC hashtable to use simplehash?

2024-01-19 Thread John Naylor
On Sat, Jan 20, 2024 at 7:13 AM Jeff Davis wrote: > > On Fri, 2024-01-19 at 13:38 -0800, Jeff Davis wrote: > > One post-commit question on 0aba255440: why do > > haszero64(pg_bswap64(chunk)) rather than just haszero64(chunk)? How > > does byteswapping affect whether a zero byte exists or not? > >

Re: Change GUC hashtable to use simplehash?

2024-01-19 Thread John Naylor
On Fri, Jan 19, 2024 at 11:54 PM Heikki Linnakangas wrote: > Thanks! I started to look at how to use this, and I have some questions. > I'd like to replace this murmurhash ussage in resowner.c with this: > > > /* > >* Most resource kinds store a pointer in 'value', and pointers are

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-19 Thread John Naylor
I wrote: > On Thu, Jan 18, 2024 at 8:31 AM Masahiko Sawada wrote: > > During trying this idea, I realized that there is a visibility problem > > in the radix tree template > > If it's broken even without the embedding I'll look into this (I don't > know if this configuration has ever been

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-19 Thread John Naylor
On Fri, Jan 19, 2024 at 2:26 PM Masahiko Sawada wrote: > > On Thu, Jan 18, 2024 at 1:30 PM John Naylor wrote: > > I'm not quite sure what the point of "num_items" is anymore, because > > it was really tied to the array in VacDeadItems. dead_items->num_items >

Re: Change GUC hashtable to use simplehash?

2024-01-18 Thread John Naylor
On Wed, Jan 17, 2024 at 9:54 PM Heikki Linnakangas wrote: > Maybe something like: > > " > Building blocks for creating fast inlineable hash functions. The > functions in this file are not guaranteed to be stable between versions, > and may differ by hardware platform. Hence they must not be used

Re: generate syscache info automatically

2024-01-18 Thread John Naylor
On Wed, Jan 17, 2024 at 7:46 PM Peter Eisentraut wrote: > > I updated the patch to use this style (but I swapped the first two > arguments from my example, so that the thing being created is named first). > > I also changed the names of the output files a bit to make them less > confusing. (I

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-17 Thread John Naylor
On Thu, Jan 18, 2024 at 8:31 AM Masahiko Sawada wrote: > It seems we cannot make this work nicely. IIUC VacDeadItems is > allocated in DSM and TidStore is embedded there. However, > dead_items->items.area is a local pointer to dsa_area. So we cannot > include dsa_area in neither TidStore nor

Re: Change GUC hashtable to use simplehash?

2024-01-16 Thread John Naylor
to have another look. After this first step is out of the way, we can look into using this more widely, including dynahash and the GUC hash. From cff2bfda6a3067936ef17162a2db2609185afb24 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Tue, 16 Jan 2024 16:32:48 +0700 Subject: [PATCH v14 2/2] Add

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-16 Thread John Naylor
I wrote: > > Hmm, I wonder if that's a side-effect of the "create" functions doing > > their own allocations and returning a pointer. Would it be less tricky > > if the structs were declared where we need them and passed to "init" > > functions? If this is a possibility, I thought I'd first send

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-16 Thread John Naylor
On Wed, Jan 17, 2024 at 8:39 AM Masahiko Sawada wrote: > > On Wed, Jan 17, 2024 at 9:20 AM John Naylor wrote: > > > > On Tue, Jan 16, 2024 at 1:18 PM Masahiko Sawada > > wrote: > > > Just changing "items" to be the local tidstore struct could make t

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-16 Thread John Naylor
On Tue, Jan 16, 2024 at 1:18 PM Masahiko Sawada wrote: > Just changing "items" to be the local tidstore struct could make the > code tricky a bit, since max_bytes and num_items are on the shared > memory while "items" is a local pointer to the shared tidstore. Thanks for trying it this way! I

Re: Tidy fill hstv array (src/backend/access/heap/pruneheap.c)

2024-01-14 Thread John Naylor
On Sat, Jan 13, 2024 at 9:36 PM Ranier Vilela wrote: > > Em ter., 9 de jan. de 2024 às 06:31, John Naylor > escreveu: >> This just moves an operation from one place to the other, while >> obliterating the explanatory comment, so I don't see an advantage. > > Well

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-14 Thread John Naylor
On Fri, Jan 12, 2024 at 3:49 PM Masahiko Sawada wrote: > > On Thu, Jan 11, 2024 at 9:28 AM Masahiko Sawada wrote: > > So I agree to remove both max_bytes and num_items from the control > > object.Also, as you mentioned, we can remove the tidstore control > > object itself. TidStoreGetHandle()

Re: Update docs for default value of fdw_tuple_cost

2024-01-10 Thread John Naylor
On Tue, Dec 26, 2023 at 11:27 PM Umair Shahid wrote: > > Commit cac169d686eddb277880a0d8a760ac3007b4846a updated the default value of > fdw_tuple_cost from 0.01 to 0.2. The attached patch updates the docs to > reflect this change. Pushed, thanks!

Re: generate syscache info automatically

2024-01-10 Thread John Naylor
On Thu, Nov 2, 2023 at 4:13 AM Peter Eisentraut wrote: > > Here is a rebased patch set, along with a summary of the questions I > have about these patches: This is an excellent summary of the issues, thanks. > v4-0001-Generate-syscache-info-from-catalog-files.patch > > What's a good syntax to

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-09 Thread John Naylor
On Wed, Jan 10, 2024 at 9:05 AM Masahiko Sawada wrote: > > I've done in 0010 patch in v51 patch set. Whereas RT_NODE_4 and > RT_NODE_16 structs declaration needs RT_FANOUT_4_HI and > RT_FANOUT_16_HI respectively, RT_FANOUT_16_LO and RT_FANOUT_48 need > RT_NODE_16 and RT_NODE_48 structs

Re: Add BF member koel-like indentation checks to SanityCheck CI

2024-01-09 Thread John Naylor
On Wed, Jan 10, 2024 at 2:20 AM Tristan Partin wrote: > > On Tue Jan 9, 2024 at 3:00 AM CST, John Naylor wrote: > > I don't indent during most of development, and don't intend to start. > > Could you expand on why you don't? I could understand as you're writing, > but I w

Re: add AVX2 support to simd.h

2024-01-09 Thread John Naylor
On Tue, Jan 9, 2024 at 11:20 PM Nathan Bossart wrote: > > On Tue, Jan 09, 2024 at 09:20:09AM +0700, John Naylor wrote: > > On Tue, Jan 9, 2024 at 12:37 AM Nathan Bossart > > wrote: > >> > >> > I suspect that there could be a regression lurking for some i

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-09 Thread John Naylor
On Tue, Jan 9, 2024 at 9:40 AM Masahiko Sawada wrote: > In addition, I've made some changes and cleanups: These look good to me, although I have not tried dumping a node in a while. > 0011 - simplify the radix tree iteration code. I hope it makes the > code clear and readable. Also I removed

Re: Tidy fill hstv array (src/backend/access/heap/pruneheap.c)

2024-01-09 Thread John Naylor
On Thu, Dec 28, 2023 at 7:58 PM Ranier Vilela wrote: > I think it would be more productive to initialize the entire array with -1, > and avoid flagging, one by one, the items that should be -1. This just moves an operation from one place to the other, while obliterating the explanatory comment,

Re: Add BF member koel-like indentation checks to SanityCheck CI

2024-01-09 Thread John Naylor
On Mon, Oct 30, 2023 at 2:21 PM Bharath Rupireddy wrote: > > Hi, > > How about adding code indent checks (like what BF member koel has) to > the SanityCheck CI task? This helps catch indentation problems way > before things are committed so that developers can find them out in > their respective

Re: add AVX2 support to simd.h

2024-01-08 Thread John Naylor
On Tue, Jan 9, 2024 at 12:37 AM Nathan Bossart wrote: > > > I suspect that there could be a regression lurking for some inputs > > that the benchmark doesn't look at: pg_lfind32() currently needs to be > > able to read 4 vector registers worth of elements before taking the > > fast path. There is

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-08 Thread John Naylor
On Wed, Jan 3, 2024 at 9:10 PM John Naylor wrote: > > On Tue, Jan 2, 2024 at 8:01 PM Masahiko Sawada wrote: > > > I agree that we expose RT_LOCK_* functions and have tidstore use them, > > but am not sure the if (TidStoreIsShared(ts) LWLockAcquire(..., ...)" > &g

Re: Change GUC hashtable to use simplehash?

2024-01-08 Thread John Naylor
On Mon, Jan 8, 2024 at 2:24 PM Junwang Zhao wrote: > > + * Portions Copyright (c) 2018-2023, PostgreSQL Global Development Group > > A kind reminder, it's already 2024 :) > > I'm also curious why the 2018, is there any convention for that? The convention I followed was "blind copy-paste", but

Re: add AVX2 support to simd.h

2024-01-08 Thread John Naylor
On Sat, Jan 6, 2024 at 12:04 AM Nathan Bossart wrote: > I've been thinking about the configuration option approach. ISTM that > would be the most feasible strategy, at least for v17. A couple things > come to mind: > > * This option would simply map to existing compiler flags. We already have

Re: add AVX2 support to simd.h

2024-01-07 Thread John Naylor
On Thu, Nov 30, 2023 at 12:15 AM Nathan Bossart wrote: > Using the same benchmark as we did for the SSE2 linear searches in > XidInMVCCSnapshot() (commit 37a6e5d) [1] [2], I see the following: I've been antagonistic towards the patch itself, but it'd be more productive if I paid some nuanced

Re: Change GUC hashtable to use simplehash?

2024-01-07 Thread John Naylor
On Sat, Jan 6, 2024 at 9:01 AM jian he wrote: > > latency average = 147.782 ms > select * from bench_cstring_hash_unaligned(10); > latency average = 101.179 ms > select * from bench_cstring_hash_aligned(10); > latency average = 101.219 ms Thanks for testing again! This looks closer to my

Re: Change GUC hashtable to use simplehash?

2024-01-05 Thread John Naylor
On Fri, Jan 5, 2024 at 6:58 PM jian he wrote: > -Dcassert=true \ > -Dbuildtype=debug \ These probably don't matter much for this test, but these should be off for any performance testing. > -DWRITE_READ_PARSE_PLAN_TREES > -DCOPY_PARSE_PLAN_TREES

Re: Change GUC hashtable to use simplehash?

2024-01-05 Thread John Naylor
(not for commit anyway) From cf64f9a0603837dd89efdf1aa455395906e75ded Mon Sep 17 00:00:00 2001 From: John Naylor Date: Fri, 5 Jan 2024 17:21:53 +0700 Subject: [PATCH v13 5/6] WIP: a safer way to accumulate a single struct member into the hash state --- src/backend/catalog/namespace.c | 4

Re: add AVX2 support to simd.h

2024-01-04 Thread John Naylor
On Wed, Jan 3, 2024 at 10:29 PM Nathan Bossart wrote: > If the requirement is that normal builds use AVX2, then I fear we will be > waiting a long time. IIUC the current proposals (building multiple > binaries or adding a configuration option that maps to compiler flags) > would still be opt-in,

Re: add AVX2 support to simd.h

2024-01-03 Thread John Naylor
On Tue, Jan 2, 2024 at 11:11 PM Nathan Bossart wrote: > > Perhaps I was too optimistic about adding support for newer instructions... > > I'm tempted to propose that we move forward with this patch as-is after > adding a buildfarm machine that compiles with -mavx2 or -march=x86-64-v3. That means

Re: Change GUC hashtable to use simplehash?

2024-01-03 Thread John Naylor
On Tue, Jan 2, 2024 at 6:56 AM jian he wrote: > > My local computer is slow. but here is the test results: > > select * from bench_cstring_hash_aligned(10);7318.893 ms > select * from bench_cstring_hash_unaligned(10);10383.173 ms > select * from bench_pgstat_hash(10);

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-03 Thread John Naylor
On Tue, Jan 2, 2024 at 8:01 PM Masahiko Sawada wrote: > I agree that we expose RT_LOCK_* functions and have tidstore use them, > but am not sure the if (TidStoreIsShared(ts) LWLockAcquire(..., ...)" > calls part. I think that even if we expose them, we will still need to > do something like "if

Re: warn if GUC set to an invalid shared library

2024-01-01 Thread John Naylor
On Thu, Dec 28, 2023 at 12:27 PM Shubham Khanna wrote: > > I was reviewing the Patch and came across a minor issue that the Patch > does not apply on the current Head. Please provide the updated version > of the patch. For your information, the commitfest manager has the ability to send private

Re: add AVX2 support to simd.h

2024-01-01 Thread John Naylor
On Thu, Nov 30, 2023 at 12:15 AM Nathan Bossart wrote: > I don't intend for this patch to be > seriously considered until we have better support for detecting/compiling > AVX2 instructions and a buildfarm machine that uses them. That's completely understandable, yet I'm confused why there is a

Re: Add support for __attribute__((returns_nonnull))

2023-12-31 Thread John Naylor
On Thu, Dec 28, 2023 at 1:20 AM Tristan Partin wrote: > I recently wound up in a situation where I was checking for NULL return > values of a function that couldn't ever return NULL because the > inability to allocate memory was always elog(ERROR)ed (aborted). It sounds like you have a ready

Re: Should "CRC" be in uppercase?

2023-12-26 Thread John Naylor
On Mon, Dec 25, 2023 at 12:51 PM Kyotaro Horiguchi wrote: > > A new function check_control_file() in pg_combinebackup.c has the > following message. > > > pg_fatal("%s: crc is incorrect", controlpath); > > I think "crc" should be in all uppercase in general and a brief >

Re: A typo in a messsage?

2023-12-26 Thread John Naylor
On Fri, Dec 22, 2023 at 1:50 PM Kyotaro Horiguchi wrote: > > I found the following message introduced by a recent commit. > > > errdetail("The first unsummarized LSN is this range is %X/%X.", > > Shouldn't the "is" following "LSN" be "in"? Pushed.

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-26 Thread John Naylor
On Tue, Dec 26, 2023 at 12:43 PM Masahiko Sawada wrote: > > On Thu, Dec 21, 2023 at 4:41 PM John Naylor wrote: > > +TidStoreSetBlockOffsets(TidStore *ts, BlockNumber blkno, OffsetNumber > > *offsets, > > + int num_offsets) > > +{ > > + char buf[MaxBlockta

Re: Change GUC hashtable to use simplehash?

2023-12-26 Thread John Naylor
On Wed, Dec 20, 2023 at 1:48 PM John Naylor wrote: > > On Wed, Dec 20, 2023 at 3:23 AM Jeff Davis wrote: > > > > The reason I looked here is that the inner while statement (to find the > > chunk size) looked out of place and possibly slow, and there's a > > bi

Re: A typo in a messsage?

2023-12-22 Thread John Naylor
On Fri, Dec 22, 2023 at 1:50 PM Kyotaro Horiguchi wrote: > > I found the following message introduced by a recent commit. > > > errdetail("The first unsummarized LSN is this range is %X/%X.", > > Shouldn't the "is" following "LSN" be "in"? I think you're right, will push.

Re: date_trunc function in interval version

2023-12-22 Thread John Naylor
On Sat, Dec 23, 2023 at 5:26 AM Przemysław Sztoch wrote: > > In my opinion date_trunc is very good name. > Truncated data is timestamp type, not interval. > First parameter has same meaning in original date_trunc and in my new version. > New version provides only more granularity. I haven't

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-22 Thread John Naylor
On Thu, Dec 21, 2023 at 6:27 PM Andres Freund wrote: > > Could either of you summarize what the design changes you've made in the last > months are and why you've done them? Unfortunately this thread is very long, > and the comments in the file just say "FIXME" in places that apparently are >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-20 Thread John Naylor
On Thu, Dec 21, 2023 at 8:33 AM Masahiko Sawada wrote: > > I found the following comment and wanted to discuss: > > // this might be better as "iterate over nodes", plus a callback to > RT_DUMP_NODE, > // which should really only concern itself with single nodes > RT_SCOPE void >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-20 Thread John Naylor
On Wed, Dec 20, 2023 at 6:36 PM Masahiko Sawada wrote: > > I've updated the new patch set that incorporated comments I got so > far. 0007, 0008, and 0012 patches are updates from the v45 patch set. > In addition to the review comments, I made some changes in tidstore to > make it independent from

Re: Change GUC hashtable to use simplehash?

2023-12-19 Thread John Naylor
On Wed, Dec 20, 2023 at 3:23 AM Jeff Davis wrote: > > On Tue, 2023-12-19 at 16:23 +0700, John Naylor wrote: > > That wasn't the next place I thought to look (that would be the > > strcmp > > call), but something like this could be worthwhile. > > The reason I looke

Re: Change GUC hashtable to use simplehash?

2023-12-19 Thread John Naylor
On Tue, Dec 19, 2023 at 2:32 PM Jeff Davis wrote: > > On Mon, 2023-12-18 at 13:39 +0700, John Naylor wrote: > > For now just two: > > v10-0002 is Jeff's change to the search path cache, but with the > > chunked interface that I found to be faster. > > Did you cons

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-18 Thread John Naylor
On Tue, Dec 19, 2023 at 12:37 PM Masahiko Sawada wrote: > > On Mon, Dec 18, 2023 at 3:41 PM John Naylor wrote: > > Let's do it in just one place. In TidStoreCreate(), do > > > > /* clamp max_bytes to at least the size of the empty tree with > > allocated blocks, so

Re: Simplify newNode()

2023-12-18 Thread John Naylor
On Fri, Dec 15, 2023 at 5:44 AM Tom Lane wrote: > > I did check that the v1 patch successfully inlines newNode() and > reduces it to just a MemoryContextAllocZeroAligned call, so it's > correct that modern compilers do that better than whatever I tested > in 2008. But I wonder what is happening

Re: meson: Stop using deprecated way getting path of files

2023-12-17 Thread John Naylor
On Tue, Dec 5, 2023 at 3:27 AM Tristan Partin wrote: > > On Mon Dec 4, 2023 at 2:10 PM CST, Tom Lane wrote: > > Not sure what you were using, but are you aware that SQL access to the > > buildfarm database is available to project members? My own stock > > approach to checking on this sort of

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-17 Thread John Naylor
On Fri, Dec 15, 2023 at 3:15 PM Masahiko Sawada wrote: > > On Fri, Dec 15, 2023 at 10:30 AM John Naylor wrote: > > parallel_vacuum_init(Relation rel, Relation *indrels, int nindexes, > > - int nrequested_workers, int max_items, > > - int elevel, BufferAccessStrat

Re: Change GUC hashtable to use simplehash?

2023-12-17 Thread John Naylor
ng buried in an earlier version: use in pgstat. Looks nicer, but not yet tested. From aecbc51780ada4634855727c50e3b85a8f7f Mon Sep 17 00:00:00 2001 From: John Naylor Date: Sat, 9 Dec 2023 16:24:56 +0700 Subject: [PATCH v10 3/3] Use fasthash32 for pgstat_hash_hash_key Currently this calls the 32-b

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-14 Thread John Naylor
On Thu, Dec 14, 2023 at 7:22 AM Masahiko Sawada wrote: > In v45, 0001 - 0006 are from earlier versions but I've merged previous > updates. So the radix tree now has RT_SET() and RT_FIND() but not > RT_GET() and RT_SEARCH(). 0007 and 0008 are the updates from previous > versions that incorporated

Re: Change GUC hashtable to use simplehash?

2023-12-14 Thread John Naylor
I wrote: > > * v8 with chunked interface: > latency average = 555.688 ms > > This starts to improve things for me. > > * v8 with chunked, and return lower 32 bits of full 64-bit hash: > latency average = 556.324 ms > > This is within the noise level. There doesn't seem to be much downside > of

Re: Change GUC hashtable to use simplehash?

2023-12-11 Thread John Naylor
On Sun, Dec 10, 2023 at 2:18 AM Jeff Davis wrote: > > On Sat, 2023-12-09 at 18:52 +0700, John Naylor wrote: > > > I tested using the new hash function APIs for my search path cache, > > > and > > > there's a significant speedup for cases not benefiting from &g

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-11 Thread John Naylor
On Mon, Dec 11, 2023 at 1:18 PM Masahiko Sawada wrote: > I've attached the updated patch set. From the previous patch set, I've > merged patches 0007 to 0010. The other changes such as adding RT_GET() > still are unmerged for now, for discussion. Probably we can make them > as follow-up patches

Re: Change GUC hashtable to use simplehash?

2023-12-10 Thread John Naylor
I wrote: > On Sun, Dec 10, 2023 at 2:18 AM Jeff Davis wrote: > > > > On Sat, 2023-12-09 at 18:52 +0700, John Naylor wrote: > > > > I tested using the new hash function APIs for my search path cache, > > > > and > > > > there's

Re: Change GUC hashtable to use simplehash?

2023-12-09 Thread John Naylor
On Sun, Dec 10, 2023 at 2:18 AM Jeff Davis wrote: > > On Sat, 2023-12-09 at 18:52 +0700, John Naylor wrote: > > > I tested using the new hash function APIs for my search path cache, > > > and > > > there's a significant speedup for cases not benefiting from &g

Re: Change GUC hashtable to use simplehash?

2023-12-09 Thread John Naylor
On Sat, Dec 9, 2023 at 3:32 AM Jeff Davis wrote: > > On Wed, 2023-11-29 at 20:31 +0700, John Naylor wrote: > > Attached is a rough start with Andres's earlier ideas, to get > > something concrete out there. > > The implementation of string hash in 0004 forgot t

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-08 Thread John Naylor
On Fri, Dec 8, 2023 at 3:06 PM Masahiko Sawada wrote: > > BTW Given that the actual value size can be calculated only by the > caller, how does the tree know if the value is embedded or not? It's > probably related to how to store combined pointer/value slots. Right, this is future work. At

Re: micro-optimizing json.c

2023-12-07 Thread John Naylor
On Fri, Dec 8, 2023 at 10:32 AM Nathan Bossart wrote: > > On Fri, Dec 08, 2023 at 04:11:52PM +1300, David Rowley wrote: > > + seplen = use_line_feeds ? sizeof(",\n ") - 1 : sizeof(",") - 1; > > > > Most modern compilers will be fine with just: > > > > seplen = strlen(sep); > > > > I had to go

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-07 Thread John Naylor
On Fri, Dec 8, 2023 at 8:57 AM Masahiko Sawada wrote: > It's still unclear to me why the value doesn't need to contain the size. > > If I understand you correctly, in RT_GET(), the tree allocs a new > memory and updates the slot where the value is embedded with the > pointer to the allocated

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-06 Thread John Naylor
On Mon, Nov 27, 2023 at 1:45 PM Masahiko Sawada wrote: > > On Sat, Oct 28, 2023 at 5:56 PM John Naylor wrote: > bool > RT_SET(RT_RADIX_TREE *tree, uint64 key, RT_VALUE_TYPE *value_p); > or for variable-length value support, > RT_SET(RT_RADIX_TREE *tree, uint64 key, RT_V

Re: Change GUC hashtable to use simplehash?

2023-12-06 Thread John Naylor
On Wed, Dec 6, 2023 at 11:48 PM Jeff Davis wrote: > > On Wed, 2023-12-06 at 07:39 +0700, John Naylor wrote: > > "git grep cstring_hash" found nothing, so not sure what you're > > asking. > > Sorry, I meant string_hash(). Your v5-0002 changes the way hashing &g

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-05 Thread John Naylor
On Wed, Dec 6, 2023 at 4:34 AM Masahiko Sawada wrote: > > On Mon, Dec 4, 2023 at 5:21 PM John Naylor wrote: > > > Given variable-length value support, RT_GET() would have to do > > > repalloc() if the existing value size is not big enough for the new > > > value

Re: Change GUC hashtable to use simplehash?

2023-12-05 Thread John Naylor
On Tue, Dec 5, 2023 at 1:57 AM Jeff Davis wrote: > > On Mon, 2023-12-04 at 12:12 +0700, John Naylor wrote: > There's already a patch to use simplehash, and the API is a bit > cleaner, and there's a minor performance improvement. It seems fairly > non-controversial -- should

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-12-05 Thread John Naylor
On Tue, Dec 5, 2023 at 7:55 AM Jeff Davis wrote: > > On Tue, 2023-11-21 at 09:24 -0500, Robert Haas wrote: > > As to the second, could we somehow come > > up with an API for guc.c where you can ask for an opaque handle that > > will later allow you to do a fast-SET of a GUC? > > Yes, attached.

Re: Move bki file pre-processing from initdb - part 1 - initdb->genbki.pl

2023-12-04 Thread John Naylor
On Mon, Dec 4, 2023 at 5:03 PM Krishnakumar R wrote: > > Hi, > > As per discussion in [1] splitting the patch. Part1 moves replacement > logic in initdb of NAMEDATALEN, FLOAT8PASSBYVAL, SIZEOF_VOID_P, > ALIGNOF_POINTER to compile time via genbki.pl. Hi Krishnakumar, Note this comment in

Commitfest 2023-11 is now closed

2023-12-04 Thread John Naylor
with very many patches, but after moving November over, there are plenty that have no reviewer of record. -- John Naylor

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-12-04 Thread John Naylor
On Mon, Dec 4, 2023 at 3:12 PM Pavel Borisov wrote: > I think this is complete and could be closed. Done.

Re: Add pg_basetype() function to obtain a DOMAIN base type

2023-12-04 Thread John Naylor
On Thu, Sep 28, 2023 at 12:22 AM Alexander Korotkov wrote: > The one thing triggering my perfectionism is that the patch does two > syscache lookups instead of one. For an admin function used interactively, I'm not sure why that matters? Or do you see another use case?

Re: [PoC] Improve dead tuple storage for lazy vacuum

2023-12-04 Thread John Naylor
On Mon, Nov 27, 2023 at 1:45 PM Masahiko Sawada wrote: > Since the variable-length values support is a big deal and would be > related to API design I'd like to discuss the API design first. Thanks for the fine summary of the issues here. [Swapping this back in my head] > RT_VALUE_TYPE >

Re: XID formatting and SLRU refactorings (was: Add 64-bit XIDs into PostgreSQL 15)

2023-12-03 Thread John Naylor
On Thu, Nov 30, 2023 at 4:37 PM Alexander Korotkov wrote: > > On Thu, Nov 30, 2023 at 10:29 AM Pavel Borisov wrote: > > Agree. The fix is attached. > > What an oversight. > Thank you, pushed! With that, is there any more work pending, or can we close the CF entry?

Re: Change GUC hashtable to use simplehash?

2023-12-03 Thread John Naylor
On Mon, Dec 4, 2023 at 4:16 AM Jeff Davis wrote: > I'm trying to follow the distinctions you're making between dynahash > and simplehash -- are you saying it's easier to do incremental hashing > with dynahash, and if so, why? That's a good thing to clear up. This thread has taken simplehash as a

Re: Change GUC hashtable to use simplehash?

2023-12-02 Thread John Naylor
On Wed, Nov 29, 2023 at 8:31 PM John Naylor wrote: > > Attached is a rough start with Andres's earlier ideas, to get > something concrete out there. While looking at the assembly out of curiosity, I found a couple bugs in the split API that I've fixed locally. I think the pat

Commitfest 2023-11 is almost over

2023-12-01 Thread John Naylor
remembered discussion on a "bulk move" button, but if it's there, I can't find it... -- John Naylor

Re: User functions for building SCRAM secrets

2023-12-01 Thread John Naylor
On Wed, Mar 22, 2023 at 9:06 PM Jonathan S. Katz wrote: > > Maybe I'm not conveying the problem this is solving -- I'm happy to go > one more round trying to make it clearer -- but if this is not clear, > it'd be good to at develop an alternative approach to this before > withdrawing the patch.

Re: [PGDOCS] Inconsistent linkends to "monitoring" views.

2023-11-30 Thread John Naylor
On Wed, Nov 8, 2023 at 2:02 PM John Naylor wrote: > My 2 cents: Comment typos are visible to readers, so more annoying > when seen in isolation, and less likely to have surroundings that > could change in back branches. Consistency would preferred all else > being equal, but then a

Re: [dynahash] do not refill the hashkey after hash_search

2023-11-30 Thread John Naylor
On Thu, Sep 14, 2023 at 3:28 PM Junwang Zhao wrote: > > Add a v2 with some change to fix warnings about unused-parameter. > > I will add this to Commit Fest. Pushed v2 after removing asserts, as well as the unnecessary cast that I complained about earlier. Some advice: I was added as a reviewer

Re: Change GUC hashtable to use simplehash?

2023-11-29 Thread John Naylor
On Wed, Nov 29, 2023 at 9:59 PM Heikki Linnakangas wrote: > > I didn't understand what you meant by the above. Did you wack around > fast-hash, or who did? I turned it into an init/accum/final style (shouldn't affect the result), and took out the input length from the calculation (will affect

Re: Change GUC hashtable to use simplehash?

2023-11-29 Thread John Naylor
for dynahash (getting rid of strlen) and guc hash. [1] https://code.google.com/archive/p/fast-hash/ [2] https://github.com/jonmaiga/mx3 From c2b799dd2418fb68fcfc6ccf006a50f74c9072fe Mon Sep 17 00:00:00 2001 From: John Naylor Date: Wed, 29 Nov 2023 16:28:58 +0700 Subject: [PATCH v5 4/4] Add bytewise

Re: autovectorize page checksum code included elsewhere

2023-11-28 Thread John Naylor
On Tue, Nov 28, 2023 at 7:51 AM Andres Freund wrote: > > Hi, > > On 2023-11-25 14:09:14 +0700, John Naylor wrote: > > * Note: I have seen the threads with the idea of compiling multiple > > entire binaries, and switching at postmaster start. I think it's a > >

Re: remaining sql/json patches

2023-11-27 Thread John Naylor
On Mon, Nov 27, 2023 at 8:57 PM Andrew Dunstan wrote: > Interesting. But inferring a speed effect from such changes is > difficult. I don't have a good idea about measuring parser speed, but a > tool to do that would be useful. Amit has made a start on such > measurements, but it's only a start.

Re: autovectorize page checksum code included elsewhere

2023-11-24 Thread John Naylor
On Thu, Nov 23, 2023 at 1:49 AM Nathan Bossart wrote: > > On Wed, Nov 22, 2023 at 02:54:13PM +0200, Ants Aasma wrote: > > For reference, executing the page checksum 10M times on a AMD 3900X CPU: > > > > clang-14 -O2 4.292s (17.8 GiB/s) > > clang-14 -O2 -msse4.12.859s (26.7

Re: autovectorize page checksum code included elsewhere

2023-11-24 Thread John Naylor
On Thu, Nov 23, 2023 at 11:51 PM Nathan Bossart wrote: > > On Thu, Nov 23, 2023 at 05:50:48PM +0700, John Naylor wrote: > > On Thu, Nov 23, 2023 at 1:49 AM Nathan Bossart > > wrote: > >> One half-formed idea I have is to introduce some sort of ./configure flag >

Re: Question about the Implementation of vector32_is_highbit_set on ARM

2023-11-23 Thread John Naylor
On Thu, Nov 23, 2023 at 4:29 PM Xiang Gao wrote: > > Thank you for your detailed explanation. > Can I do some testing and submit this patch? Please do, thanks.

Re: autovectorize page checksum code included elsewhere

2023-11-23 Thread John Naylor
On Thu, Nov 23, 2023 at 1:49 AM Nathan Bossart wrote: > > On Wed, Nov 22, 2023 at 02:54:13PM +0200, Ants Aasma wrote: > > On Wed, 22 Nov 2023 at 11:44, John Naylor wrote: > >> Poking in those files a bit, I also see references to building with > >> SSE 4.1. Maybe t

Re: Change GUC hashtable to use simplehash?

2023-11-23 Thread John Naylor
On Thu, Nov 23, 2023 at 5:34 AM Andres Freund wrote: > It's worth noting that the limited range of the input values means that > there's a lot of bias toward some bits being set ('a' to 'z' all start with > 0b011). We can take advantage of the limited range with a single additional instruction:

Re: Change GUC hashtable to use simplehash?

2023-11-22 Thread John Naylor
ollision behavior by collecting the bytes on a uint64 and calling our new murmur64 before returning the lower half, but that's speculative. From 1a516bb341afb72680470897d75c1d23f75fb37e Mon Sep 17 00:00:00 2001 From: John Naylor Date: Wed, 22 Nov 2023 17:28:41 +0700 Subject: [PATCH v4 1/3] Add finalize

Re: Output affected rows in EXPLAIN

2023-11-22 Thread John Naylor
On Wed, Nov 15, 2023 at 2:17 PM wrote: > > We can check the number of updated rows from execute plan, > I think there is no need to return the command tag > when EXPLAIN(ANALYZE) is executed by default. Given that several people have voiced an opinion against this patch, I'm marking it rejected.

Re: autovectorize page checksum code included elsewhere

2023-11-22 Thread John Naylor
On Tue, Nov 7, 2023 at 9:47 AM Nathan Bossart wrote: > > Presently, we ask compilers to autovectorize checksum.c and numeric.c. The > page checksum code actually lives in checksum_impl.h, and checksum.c just > includes it. But checksum_impl.h is also used in pg_upgrade/file.c and >

Commitfest 2023-11 update 2

2023-11-22 Thread John Naylor
. Withdrawn: 13. Rejected: 1. Total: 347. The pace seems to have picked up a bit, based on number of commits. -- John Naylor

Re: pipe_read_line for reading arbitrary strings

2023-11-22 Thread John Naylor
On Mon, Sep 25, 2023 at 2:55 PM Daniel Gustafsson wrote: > > Fixed, along with commit message wordsmithing in the attached. Unless > objected > to I'll go ahead with this version. +1

Re: Evaluate arguments of correlated SubPlans in the referencing ExprState

2023-11-22 Thread John Naylor
On Tue, Oct 10, 2023 at 10:00 AM Andres Freund wrote: > > Hi, > > On 2023-10-01 14:53:23 -0400, Tom Lane wrote: > > Peter Eisentraut writes: > > > Is this patch still being worked on? > > > > I thought Andres simply hadn't gotten back to it yet. > > It still seems like a worthwhile improvement.

Re: Unlinking Parallel Hash Join inner batch files sooner

2023-11-22 Thread John Naylor
On Wed, Sep 27, 2023 at 11:42 PM Heikki Linnakangas wrote: > > Looks good to me too at a quick glance. There's this one "XXX free" > comment though: > > > for (int i = 1; i < old_nbatch; ++i) > > { > > ParallelHashJoinBatch *shared = > >

<    1   2   3   4   5   6   7   8   9   10   >