reviously proposed creating a type of memory context that's
> intended for
> places where we never expect to allocate much which allocates from
> either a
> superior memory context or just from the system allocator and tracks
> memory
> via linked lists.
Why not just use ALLOCSET_SMALL_SIZES?
Regards,
Jeff Davis
me already existing provider and
> then
> do for example some pattern matching on the name of the new
> collation.
> Really ugly but works.
To add a catalog table for the locale providers, the main challenge is
around the database default collation and, relatedly, initdb. Do you
have some ideas around that?
Regards,
Jeff Davis
r, or lock the
same tuple.
If PD_ALL_VISIBLE is set it's even simpler.
Am I missing some major hazards?
Regards,
Jeff Davis
On Fri, 2024-03-08 at 00:20 -0800, Jeff Davis wrote:
> Implemented in v11, attached.
Rebased, v12 attached.
Regards,
Jeff Davis
From 5c2a8f5cb865becd70b08379d9fc72946be9a32a Mon Sep 17 00:00:00 2001
From: Jeff Davis
Date: Tue, 2 Jan 2024 13:42:48 -0800
Subject: [PATCH v12] CRE
dge cases
(or due to a bug), and it would be good to be able to see that.
I agree with others that we should look at changing the initial size or
type of the contexts, but that should be a separate commit.
Regards,
Jeff Davis
On Wed, 2024-10-30 at 10:28 +1300, David Rowley wrote:
> On Wed, 30 Oct 2024 at 09:29, Jeff Davis wrote:
> > Attached is a patch that reorganizes CacheMemoryContext to be
> > mostly a
> > parent context, with child contexts such as CatCacheContext,
> > RelCacheCon
,
Jeff Davis
From 80febd60550d1710186a111d95659d0f80cece4e Mon Sep 17 00:00:00 2001
From: Jeff Davis
Date: Tue, 29 Oct 2024 10:32:38 -0700
Subject: [PATCH v1] Reorganize cache memory contexts.
Create new memory contexts such as CatCacheContext, RelCacheContext,
etc. as children of
ame,
> 'inherited', false
> ) FROM generate_series(1, 2);
> ERROR: XX000: tuple already updated by self
Thank you for the report!
Attached a patch to add calls to CommandCounterIncrement().
Regards,
Jeff Davis
From ccff3df8b2d6f0c139a39e5aef8721b4480bdbd3 Mon Se
7;/'c'). The v6 patches brings us close to
> > this point, but I'm not sure if we want to go this far in v18.
>
> Probably necessary but I hate all the DDL commands the way to SQL
> standard is written forces us to add.
There is some precedent for a DDL-like thing without new grammar:
pg_replication_origin_create(). I don't have a strong opinion on
whether to do that or not.
>
Regards,
Jeff Davis
her we should only have one set of
functions.
Regards,
Jeff Davis
y functional change in my patch is to create the partitioned
table with relpages=-1 to be more consistent with the value after it's
analyzed.
Regards,
Jeff Davis
On Wed, 2024-10-23 at 04:47 +0200, Laurenz Albe wrote:
> On Tue, 2024-10-22 at 10:41 -0700, Jeff Davis wrote:
> > I attached a patch that creates partitioned tables with relpages=-
> > 1,
> > and updates the docs.
>
> Does this need any changes for pg_upgrade?
pg_u
ction.
> The attached small patch fixes the documentation.
Thank you again, fixed.
Regards,
Jeff Davis
> I've taken most of Jeff's work, reincorporated it into roughly the
> same patch structure as before, and am posting it now.
I committed 0001-0004 with significant revision.
Regards,
Jeff Davis
s() as well.
Regards,
Jeff Davis
From 34241c8a9e18c0e945dd38a2858676d52facc898 Mon Sep 17 00:00:00 2001
From: Jeff Davis
Date: Tue, 22 Oct 2024 10:04:11 -0700
Subject: [PATCH v1] Be more consistent about relpages for partitioned tables.
Use -1 for relpages, and document it.
Discussion:
ave any storage, but
relpages are always 0.
And lastly, it's undocumented: if -1 is allowable, it should be in the
public docs for pg_class.
I don't see any obvious reason that -1 is better than 0, or any code
that checks for it, so I'm inclined to just use zero instead.
Thoughts?
Regards,
Jeff Davis
Ez1gLOkWSh_Vd9LQh-JM4i%3DMu7PVT9ffc77TmH0Zh3TzA%40mail.gmail.com
Is a return value what you had in mind? Or some other function that can
help find missing stats later, or something else entirely?
Regards,
Jeff Davis
standing
ErrorData that hasn't been processed at all; while the latter is for an
ERROR specifically, for which errstart() and errfinish() have already
been called.
We also might consider adding some asserts.
Regards,
Jeff Davis
From 2fc18f71afc89d95eb38fe2d2606b5ff07a8b8f9 Mon Sep 17 00:00
because even partitioned indexes have
relpages=0. Furthermore, the parameter is of type BlockNumber, so
InvalidBlockNumber would make more sense.
Not the cleanest code, but if the value exists, we need to be able to
import it.
Regards,
Jeff Davis
e(), which uses an
array of booleans).
We are going to replace a maximum of 3 attributes, so the arrays have a
maximum size of 3. Predeclaring the arrays to be 3 elements is just
fine even if we only use the first 1-2 elements -- it avoids a needless
heap allocation/free.
Regards,
Jeff Davis
ntend to commit
soon.
Regards,
Jeff Davis
From c65eb2fda1d7c9a29846d61bdb0358a0e73e2226 Mon Sep 17 00:00:00 2001
From: Jeff Davis
Date: Wed, 9 Oct 2024 22:28:15 -0700
Subject: [PATCH v17] Fix missed case for builtin collation provider.
A missed check for the builtin collation provider cou
tching code due to the indirection.
What would be a good test for that? Just running it over long strings?
Regards,
Jeff Davis
On Fri, 2024-09-27 at 15:04 -0500, Nathan Bossart wrote:
> On Fri, Sep 27, 2024 at 09:22:48AM -0700, Jeff Davis wrote:
> > I suggest that we add the wording to the
> > query portion of the doc, near
> > "security-
> > restricted operation".
>
> How
.
The command is run effectively in two parts: the CREATE part and the
REFRESH part. The former just uses the session search path, while the
latter uses the safe search path.
I suggest that we add the wording to the
query portion of the doc, near "security-
restricted operation".
Regards,
Jeff Davis
On Wed, 2024-08-14 at 16:30 -0700, Jeff Davis wrote:
> On Thu, 2024-08-08 at 12:24 -0700, Jeff Davis wrote:
> > The collation cache, which maps collation oids to pg_locale_t
> > objects,
> > has a few longstanding issues:
>
> Here's a patch set v2.
Updated and
Regards,
Jeff Davis
Committed v2-0001.
On Tue, 2024-09-03 at 22:04 -0700, Jeff Davis wrote:
> * This patch may change the handling of collation oid 0, and I'm not
> sure whether that was intentional or not. lc_collate_is_c(0) returned
> false, whereas pg_newlocale_from_collation(0)->collate_is_c r
On Wed, 2024-08-28 at 18:43 +0200, Andreas Karlsson wrote:
> On 8/15/24 12:55 AM, Jeff Davis wrote:
> > This overlaps a bit with what Peter already proposed here:
> >
> > https://www.postgresql.org/message-id/4f562d84-87f4-44dc-8946-01d6c437936f%40eisentraut.org
> >
formance goes, I'm only looking at branch in add_path()
that calls compare_pathkeys(). Do you have some example queries which
would be a worst case for that path?
In general if you can post some details about how you are measuring,
that would be helpful.
Regards,
Jeff Davis
to
hold onto multiple paths for longer, similar to pathkeys, which might
offer some benefits or simplifications.
Regards,
Jeff Davis
[1]
https://www.postgresql.org/message-id/CA+TgmoZQyVxnRU--4g2bJonJ8RyJqNi2CHpy-=nwwbtnpaj...@mail.gmail.com
nd it still requires a solution for #4).
Regards,
Jeff Davis
[1]
https://www.postgresql.org/docs/devel/trigger-datachanges.html
On Wed, 2024-08-28 at 16:35 -0400, Robert Haas wrote:
> On Wed, Aug 28, 2024 at 4:29 PM Jeff Davis wrote:
> > Preserving a path for the right amount of time seems like the
> > primary
> > challenge for most of the use cases you raised (removing paths is
> > easier tha
where there's enough context to
know what's happening. There could be many such hooks, but I suspect
only a handful of important ones.
This idea allows the extension author to preserve the right paths long
enough to use set_rel_pathlist_hook/set_join_pathlist_hook, which can
editorialize on costs or do its own pruning.
Regards,
Jeff Davis
useful and seems relatively easy -- A JOIN B or B
JOIN A (though there's some nuance about when you try to make that
decision).
The latter requires controlling an explosion of possibilities, and
would be an entirely different kind of hook.
Regards,
Jeff Davis
On Mon, 2024-08-26 at 14:18 -0700, Jeff Davis wrote:
> 0001 implementation issues:
>
> * We need default implementations for AMs that don't implement the
> new
> APIs, so that the AM will still function even if it only defines the
> single-tuple APIs. If we need
using the
callback to copy tuples into the caller's context.
In 0003, why do you need the global insert_modify_buffer_flush_context?
0004 is the only place that calls table_modify_buffer_flush(). Is that
really necessary, or is automatic flushing enough?
Regards,
Jeff Davis
ad-of-row triggers, and volatile
functions in the query. We could also just consider RETURNING another
restriction, which could be lifted later by implementing the logic in
the callback (as described above) without an API change.
Regards,
Jeff Davis
ator
over the buffered tuples to the caller. The caller can then use the
iterator to insert into indexes, return a tuple to the executor, etc.,
and then release the iterator when done (freeing the buffer). That
control flow is less convenient for most callers, though, so perhaps
that should be optional?
Regards,
Jeff Davis
yway (which also materializes it; see
tts_virtual_copyslot()) at heapam.c:2710?
* After correcting the memory issues, can you get updated performance
numbers for COPY?
Regards,
Jeff Davis
be a table of methods,
which means we can add an extension hook to provide a different method
table. That still requires more work, I'm just mentioning it here for
context.
Regards,
Jeff Davis
From 6f0c0a9e05039cd295c6c090b3d98d381244b35c Mon Sep 17 00:00:00 2001
From: Jeff Davis
Date
On Thu, 2024-08-15 at 01:57 -0700, Jeff Davis wrote:
> On Sun, 2024-08-04 at 01:09 -0400, Corey Huinker wrote:
> >
> > > I believe 0001 and 0002 are in good shape API-wise, and I can
> > > start
> > > getting those committed. I will try to clean up the code
he report. Fixed and backported to version 17, where the
function was introduced.
Regards,
Jeff Davis
ound this would disappear, but I think somebody building from
> source
> can figure that out more easily than the subtle issue that pkg-config
> is
> not installed.
That looks good to me. Does anyone have a different opinion? If not,
I'll go ahead and commit (but not backport) this change.
Regards,
Jeff Davis
sample code that implements late locking for a FDW? I'm
not quite clear on how it's supposed to work.
Regards,
Jeff Davis
On Thu, 2024-08-08 at 12:24 -0700, Jeff Davis wrote:
> The collation cache, which maps collation oids to pg_locale_t
> objects,
> has a few longstanding issues:
Here's a patch set v2.
I changed it so that the pg_locale_t itself a resource kind, rather
than having separate locale_t
global locale), and also query the "" categories and make a copy of
> them in case anyone wants them later, and then never call setlocale()
> again.
+1.
Regards,
Jeff Davis
field makes it seem like there is more to it so I suggest that we
> should
> just remove it.
+1. When I added that, there was also a NULL check, but that was
removed so we might as well just read the field.
> 0006-Slightly-refactor-varstr_sortsupport-to-improve-read.patch
>
> Small refactor to make a hard to read function a bit easier to read.
Looks good to me.
Regards,
Jeff Davis
ary code it could be the
> singleton mechanism I showed in CF#5166.
+1 to this approach. It makes things more consistent across platforms
and avoids surprising dependencies on the global setting.
We'll have to be careful that each call site is either OK with C, or
that it gets changed to an _l() variant. We also have to be careful
about extensions.
Regards,
Jeff Davis
gt; somewhere to feed to _l functions...
If we're going to do that, why not just have ascii-only variants of our
own? pg_ascii_isspace(...) is at least as readable as isspace_l(...,
LC_C_LOCALE).
Regards,
Jeff Davis
place where we expect lc_collate_is_c() to work
without catalog access at all: libpq/hba.c uses regexes with
C_COLLATION_OID.
But I don't think that's a major problem -- we can just move the
hardcoded test into pg_newlocale_from_collation() and return a
predefined struct with collate_is_
On Mon, 2024-08-12 at 09:00 +0200, Peter Eisentraut wrote:
> > I assume it would now be okay to take out "locale &&" here?
>
> Correct, that check is no longer necessary.
Thank you, fixed.
Regards,
Jeff Davis
.@neon.tech
Sounds good, sorry I missed that.
Can you please rebase and we can discuss in that thread?
Regards,
Jeff Davis
rn different results for things like
isspace(), but it sounds plausible -- toupper() can return different
results for 'i' in tr_TR.
Also, what about the values outside 128-255, which are still valid
input to isspace()?
Regards,
Jeff Davis
e here, or just add
that to the release notes for 18?
Regards,
Jeff Davis
s, because "locale" is not even an
argument of the MATCH_LOWER(t) or GETCHAR(t) macros, it's taken
implicitly from the outer scope.
I don't think your patches cause this confusion, but is there a way you
can clarify some of this along the way?
Regards,
Jeff Davis
ted branches, but the fixes are
somewhat invasive so I'm not inclined to backport them unless someone
thinks the problems are serious enough.
Regards,
Jeff Davis
From 7a007d3573c6bda67729ab45bd58931bfcdaf702 Mon Sep 17 00:00:00 2001
From: Jeff Davis
Date: Tue, 6 Aug 2024 12:44:14 -0
o each of those call sites,
but I'm not sure that's a huge advantage over just using uselocale().
Regards,
Jeff Davis
iswspace(), but behavior can be
different at least in theory.
So I guess we're stuck with setlocale()/uselocale() for a while, unless
we're able to move most of those call sites over to an ascii-only
variant.
Regards,
Jeff Davis
le(), which uses the result of setlocale().
Thoughts?
Regards,
Jeff Davis
On Tue, 2024-07-30 at 12:13 -0700, Jeff Davis wrote:
> I found a couple issues with the later patches:
>
> * There was still some confusion about the default collation vs.
> datcollate/datctype for callers of wchar2char() and char2wchar()
> (those
> functions only work for lib
REFRESH, but not CONCURRENTLY.
4. Do #3 but also make it work for REFRESH ... CONCURRENTLY and provide
new information that's not available by only explaining the query.
And also figure out if any of this should be back-patched.
Regards,
Jeff Davis
not easy to make that all
work with REFRESH ... CONCURRENTLY, but perhaps it could work with
CREATE MATERIALIZED VIEW and REFRESH (without CONCURRENTLY).
Regards,
Jeff Davis
;s
> design this as an independent hook, separate from rmgrs.
That's a good way to look at it, agreed.
Regards,
Jeff Davis
row)
I think I like this, as well, except for the return value, which seems
like too much information and a bit over-engineered. Can we simplify it
to what's actually going to be used by pg_upgrade and other tools?
> Attached is v25.
I believe 0001 and 0002 are in good shape API-wise, and I can start
getting those committed. I will try to clean up the code in the
process.
Regards,
Jeff Davis
On Fri, 2024-08-02 at 00:13 +0500, Kirill Reshke wrote:
> On Thu, 1 Aug 2024 at 23:27, Jeff Davis wrote:
> >
> > EXPLAIN ANALYZE CREATE MATERIALIZED VIEW doesn't go through
> > ExecCreateTableAs(), but does use CreateIntoRelDestReceiver().
>
> EXPLAIN ANALYZ
eAs() but does go
through CreateIntoRelDestReceiver().
See:
https://postgr.es/m/20444c382e6cb5e21e93c94d679d0198b0dba4dd.ca...@j-davis.com
Should we refactor a bit and try to make EXPLAIN use the same code
paths?
Regards,
Jeff Davis
, too?
Comments?
Regards,
Jeff Davis
that inherently
maintains a connection between a prefix and a range? Does it remain
true even when strange rules are added to a collation?
Regards,
Jeff Davis
On Tue, 2024-07-30 at 23:01 +0300, Heikki Linnakangas wrote:
> +1. A comment in the pg_strnxfrm() function itself would be good too.
Committed, thank you. Backported to 16 (original email said 17, but
that was a mistake).
Regards,
Jeff Davis
of pg_strnxfrm() are not allowing for that possibility.
Patch attached, which should be backported to 17.
Regards,
Jeff Davis
From 9a9680455cab0cf747ef65cf11cf2cb06d5f500e Mon Sep 17 00:00:00 2001
From: Jeff Davis
Date: Tue, 30 Jul 2024 12:16:45 -0700
Subject: [PATCH] Relax check for ret
etlocale(). I
changed this to lookup the collation and then use pg_strxfrm(). That
should improve histogram selectivity estimates because it uses the
correct provider, rather than relying on setlocale(), right?
New series attached.
Regards,
Jeff Davis
From 5b903c82f34f5da9cab58ecd0a268345
is
> ready for committer.
Committed, thank you.
> And then we can discuss after committing if an additional cache of
> the
> last locale is worth it or not.
Yeah, I'm holding off on that until refactoring in the area settles,
and we'll see if it's still worth it.
Regards,
Jeff Davis
On Thu, 2024-07-25 at 13:29 -0700, Jeff Davis wrote:
> it may be a good idea to version collation and ctype
> separately. The ctype version is, more or less, the Unicode version,
> and we know what that is for the builtin provider as well as ICU.
Attached a rough patch for the pu
nds of
functions between releases. Even if the signatures remain the same, the
parse structures may change, which creates similar incompatibilities.
So let's just get rid of the 'params' argument from both functions.
Regards,
Jeff Davis
On Thu, 2024-06-20 at 17:07 +0700, John Naylor wrote:
> On Sat, Jun 15, 2024 at 6:46 AM Jeff Davis wrote:
> > Attached is a patch to use simplehash.h instead, which speeds
> > things up
> > enough to make them fairly close (from around 15% slower to around
> > 8%).
&
gt; Also is there any reaosn you do not squash th 4th and the 6th patch?
Done. I had to rearrange the patch ordering a bit because prior to the
cache refactoring patch, it's unsafe to call
pg_newlocale_from_collation() without checking lc_collate_is_c() or
lc_
; > {
> >
> > The patch sequencing might be a bit tricky here. Maybe it's ok if
> > patch 0004 stays as is in this respect if 0006 were to fix it back.
Addressed in v3-0006.
> > * v2-0005-Avoid-setlocale-in-lc_collate_is_c-and-lc_ctype_i.patch
> >
> &g
pg_database will be locale-
related?
Regards,
Jeff Davis
stics and control, the control parameters will be:
I don't like the idea of mixing statistics and control parameters in
the same list.
I do like the idea of returning a set, but I think it should be the
positive set (effectively a representation of what is now in the
pg_stats view) and any ignored settings would be output as WARNINGs.
Regards,
Jeff Davis
On Wed, 2024-07-24 at 14:47 -0400, Robert Haas wrote:
> On Wed, Jul 24, 2024 at 1:45 PM Jeff Davis wrote:
> > There's a qualitative difference between a collation update which
> > can
> > break your PKs and FKs, and a ctype update which definitely will
> > not.
>
logy doesn't quite capture this distinction. I don't mean to
over-emphasize this point, but I do think we need to keep some
perspective here.
But I agree with your general point that we shouldn't dismiss the
problem just because it's minor. We should expect the problem to
surface at some point and be reasonably prepared.
Regards,
Jeff Davis
re you
rebuild/fix objects to use the new collation, and when that's done then
you change the default so that queries use version 2. How does all that
work?
Regards,
Jeff Davis
;s much more tractable to review your expression indexes and look for
problems (not ideal, but better). Also, as Peter points out, CTYPE
changes are typically more narrow, so there's a good chance that
there's no problem at all.
Regards,
Jeff Davis
rité documented[1] cases where the libc C.UTF-8 locale changed
the *sort* behavior, thereby affecting primary keys.
Regards,
Jeff Davis
[1]
https://www.postgresql.org/message-id/8a3dc06f-9b9d-4ed7-9a12-2070d8b0165f%40manitou-mail.org
code are intolerable,
and only for PG_C_UTF8?
Regards,
Jeff Davis
tes, so primary keys will never be affected. The risks
we are talking about are for expression indexes, e.g. on LOWER(). Even
if you do have such expression indexes, the types of changes Unicode
makes to casing and character properties are typically much more mild.
Regards,
Jeff Davis
ffers code point order collation combined with
Unicode ctype semantics.
With PG17, between ICU and the builtin provider, there's little
remaining reason to use libc (aside from legacy).
Regards,
Jeff Davis
x27;t be imported from
an old version into a new version because it's either gone or the
meaning has changed too much. But that argument doesn't apply to a
bogus call, where the name/value pairs get misaligned or something.
Regards,
Jeff Davis
obody has commented yet.)
Regards,
Jeff Davis
rings and EXECUTE them.
Though perhaps not impossible if we use some kind of runtime detection.
We could have some kind of global context that tracks, at runtime, when
an expression is executing for the purposes of an index. If a function
depends on a versioned collation, then mark the index or add a version
somewhere.
Regards,
Jeff Davis
's mostly a
theoretical problem because, at least in my experience, I can't recall
ever seeing an index that would be affected.
Regards,
Jeff Davis
lways completely replaced, but the way you
can call pg_set_attribute_stats() doesn't imply that -- calling
pg_set_attribute_stats(..., most_common_vals => ..., most_common_freqs
=> ...) looks like it would just replace the most_common_vals+freqs and
leave histogram_bounds as it was, but it actually clears
histogram_bounds, right? Should we make that work or should we document
better that it doesn't?
Regards,
Jeff Davis
In other words, it would be reviewed like any other
change.
Ideally, some new developments would make it less worrisome, and
Unicode updates could become more routine. I have some ideas, which I
can propose in separate threads. But for now, I don't see a reason to
rush Unicode updates.
Regards,
Jeff Davis
e
more that you say so in the policy thread here:
https://www.postgresql.org/message-id/d75d2d0d1d2bd45b2c332c47e3e0a67f0640b49c.camel%40j-davis.com
which would get broader visibility and I believe provide you with
stronger assurances that *everyone* will be careful with Unicode
updates.
Regards,
version 18
like normal, because there's no actual problem now, I see no reason
your objections would be taken less seriously later.
Regards,
Jeff Davis
[1]
https://www.postgresql.org/message-id/d75d2d0d1d2bd45b2c332c47e3e0a67f0640b49c.camel%40j-davis.com
above is an accurate characterization. There's plenty of opportunity
for deliberation and compromise in version 18, and my mind is still
open to pretty much everything, up to and including freezing Unicode
updates if necessary[3].
Regards,
Jeff Davis
[1]
https://www.postgresql.org/m
x27;t engage in the version 18 policy
discussion.
> Maybe someone will change
> something in v18 so it's not like that, but don't count on it.
That's backwards. If nothing happens in v18, then there will be no
breaking Unicode change. It takes an active step by a
bd45b2c332c47e3e0a67f0640b49c.camel%40j-davis.com
which seems like a more direct (and more complete) path to a resolution
of your concerns. I speak only for myself, but I assure you that I have
an open mind in that discussion, and that I have no intention force a
Unicode update past objections.
b2c332c47e3e0a67f0640b49c.ca...@j-davis.com
Regards,
Jeff Davis
o in a complete way, and hard to do with much
accuracy. I don't oppose it though -- if someone finds a way to provide
enough information to be useful, then that's fine with me.
Regards,
Jeff Davis
1 - 100 of 1104 matches
Mail list logo
