On Thu, Jun 23, 2022 at 9:06 PM Andres Freund wrote:
> It looks like there's quite a bit of low hanging fruits to optimize...
Yeah, if escapes and control characters are rare, adding an SSE2 fast
path would give a boost to json_lex_string: check 16 bytes at a time
for those chars (plus the ending
Hi,
Thanks for working on this.
>> According to subj you can try to create many tables (induced by the case
> >> of partitioned table) with long prefix - see 6727v.sql for
> reproduction.
> >> But now it's impossible because of logic of the makeUniqueTypeName()
> >> routine.
> >> You get the
Just a note/reminder that "seawasp" has been unhappy for some days now
because of yet another change in the unstable API provided by LLVM:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=seawasp&dt=2022-06-23%2023%3A18%3A17
llvmjit.c:1115:50: error: use of undeclared identifier 'L
When parsing JSON strings need to be converted from the JSON string
format to a c-style string. A simple copy of the buffer does not suffice
because of the various escape sequences that that JSON supports. Because
of this our JSON parser wrote characters into the c-style string buffer
one at a time
On Fri, Jun 24, 2022 at 8:07 AM houzj.f...@fujitsu.com
wrote:
>
> >
> > +# wait for initial table synchronization to finish
> > +$node_subscriber->poll_query_until('postgres', $synced_query)
> >
> > We could probably slightly change the comment to say: "wait for table sync
> > to
> > finish". Nor
On Fri, Jun 24, 2022 at 2:09 AM Robert Haas wrote:
>
> On Thu, Jun 23, 2022 at 2:13 AM Amit Kapila wrote:
> > The patch looks good to me as well. I will push this patch in HEAD (as
> > per option (a)) tomorrow unless I see any more suggestions/comments.
>
> The example seems to demonstrate the po
On Fri, Jun 24, 2022 at 8:10 AM Masahiko Sawada wrote:
>
> On Thu, Jun 23, 2022 at 7:00 PM Amit Kapila wrote:
> >
> > On Wed, Jun 22, 2022 at 11:09 AM Masahiko Sawada
> > wrote:
> > >
> > > I've attached a WIP patch for adding regression tests for DDL deparse.
> > > The patch can be applied on
On Thu, 23 Jun 2022 at 18:45, Nathan Bossart wrote:
>
> Thanks for chiming in.
>
> On Thu, Jun 23, 2022 at 04:19:45PM +0100, Simon Riggs wrote:
> > I don't understand why you need this patch at all.
> >
> > Since you have automation, you can use that layer to automatically
> > restart all standbys
On Thu, 23 Jun 2022 at 18:15, Nathan Bossart wrote:
> I'm grateful for the discussion in this thread so far, but I'm not seeing a
> clear path forward.
+1 to add the new auxiliary process.
--
Simon Riggshttp://www.EnterpriseDB.com/
Hi everyone!
Sorry for bumping old thread.
> On 25 May 2021, at 21:14, Justin Pryzby wrote:
>
> Such aggregate functions should be dropped before upgrade/restore and then
> re-created afterwards using the "anycompatible" functions. The affected
> functions are: array_append, array_prepend, arr
On 22.06.22 15:45, Tom Lane wrote:
Peter Eisentraut writes:
macOS has traditionally used extension .dylib for shared libraries (used
at build time) and .so for dynamically loaded modules (used by
dlopen()). This complicates the build system a bit. Also, Meson uses
.dylib for both, so it would
Hi hackers,
Many thanks for the review!
Here is a patch updated according to all the recent feedback, except
for two suggestions:
> This adds support for "ADD COLUMN SET STORAGE" but it is not described
> in the doc. COMPRESSION is not described, too. Shouldn't we add the
> both this time? Or
On 6/24/22 00:54, Tom Lane wrote:
Does such code exist? I don't see any other calls in Debian code search,
and I find it hard to believe that anyone would think such a thing is
maintainable.
Such a thing does exist within PGLogical and BDR, yes.
Thanks for your concern about maintainability.
On Thu, Jun 23, 2022 at 6:43 PM Tom Lane wrote:
> Nonetheless, the presence of GETSTRUCT calls should be a good guide
> to where we need to do something.
Indubitably.
--
Robert Haas
EDB: http://www.enterprisedb.com
On Thu, Jun 23, 2022 at 11:11 PM John Naylor
wrote:
> Hmm, I must have misunderstood this aspect. In my mind I was thinking
> that if a varlen attribute were at the end, these functions would make
> it easier to access them quickly. But from this and the follow-on
> responses, these would be used
Hi,
I'm trying to have a setup where there is a primary, standby and
pg_receivewal (which acts as a server that maintains the entire WAL).
Quorum is any one of standby and pg_receivewal. In case of primary crash,
when I promote standby (timeline switch from 5 to 6) and restart
pg_receivewal to con
Hi hackers,
> Here is a patch updated according to all the recent feedback, except
> for two suggestions:
In v4 I forgot to list possible arguments for STORAGE in
alter_table.sgml, similarly as it is done for other subcommands. Here
is a corrected patch.
- SET STORAGE
+ SET STORAGE { PLAIN | EXT
> On 24 Jun 2022, at 16:09, Andrey Borodin wrote:
>
> Would it be backpatchable to add this check to pg_upgrade?
Just to be clear of what exactly I propose I drafted a patch. PFA.
I've tested it with PG13 and
CREATE AGGREGATE public.array_accum(anyelement) (
SFUNC = array_append,
STY
On Fri, Jun 24, 2022 at 04:09:46PM +0500, Andrey Borodin wrote:
> Hi everyone!
>
> Sorry for bumping old thread.
Please find this newer thread+patch here ;)
https://www.postgresql.org/message-id/20220614230949.gx29...@telsasoft.com
> On 25 May 2021, at 21:14, Justin Pryzby wrote:
> >
> > Such
Justin Pryzby writes:
> I realized that my latest patch would break upgrades from old servers, which
> do
> not have array_position/s nor width_bucket, so ::reprocedure would fail.
> Maybe
> Andrey's way is better (checking proname rather than its OID).
proname is dangerous, because there's no
Robert Haas writes:
> I don't know whether we can or should move all the "name" columns to
> the end of the catalog. It would be user-visible and probably not
> user-desirable,
I'm a strong -1 on changing that if we're not absolutely forced to.
> but it would save something in terms of tuple
> d
Peter Eisentraut writes:
> On 22.06.22 15:45, Tom Lane wrote:
>> Doesn't this amount to a fundamental ABI break for extensions?
>> Yesterday they had to ship foo.so, today they have to ship foo.dylib.
> Extensions generally only load the module files using the extension-free
> base name. And if
On 2022-06-23 Th 21:45, Andres Freund wrote:
> Hi,
>
> On 2022-06-23 11:38:29 +0300, Aleksander Alekseev wrote:
>>> I know little about parquet - can it support FROM STDIN efficiently?
>> Parquet is a compressed binary format with data grouped by columns
>> [1]. I wouldn't assume that this is a p
Hi,
I have a made a small deductive database on top of PostgreSQL for
educational/research purposes. In this setting, due to certain
VIEW-constructions, queries often end up being self-joins on primary keys, e.g.:
SELECT t1.id, t2.val
FROM t AS t1 JOIN t AS t2 USING (id);
where t(id) is a
> It's a decent amount of work to define one though... It's clearly not
> acceptable to just dump out the internal representation, as already discussed
> in this thread.
I totally agree that it should be a well-defined format that doesn't
leak stuff like endianness and alignment of the underlying
Hi,
I noticed that in many places we check or assert the OIDs of our
built-in AMs. E.g. in planning, we will only do row compares in
indexes that have BTREE_AM_OID, and we can only sort tuples for
btamhandler -based index ams if their oid is the BTREE_AM_OID. That
seems like an artificial limitati
> PS. I noticed this when checking the sortsupport code for some active
> patches, and after playing around a bit while trying to run PG
> extensions that are not system-registered. The attached test case
> fails, where I'd expect it to succeed, or at least I expected it not
> to fail at "This AM's
On 2022-06-23 Th 21:51, Andres Freund wrote:
> Hi,
>
> On 2022-06-23 16:38:12 +0900, Michael Paquier wrote:
>> On Tue, Jun 21, 2022 at 05:41:07PM -0400, Andrew Dunstan wrote:
>>> On 2022-06-21 Tu 17:25, Andres Freund wrote:
On 2022-06-21 17:11:33 -0400, Andrew Dunstan wrote:
> I and a cou
Matthias van de Meent writes:
> Although it makes sense to ensure that we don't accidentally call such
> functions from the 'wrong location', it does mean that a user cannot
> manually install the preinstalled access methods and get a working
> index AM, because the internal code is checking the O
On Fri, Jun 24, 2022 at 7:08 AM Dilip Kumar wrote:
> I have changed that. PFA, the updated patch.
Apart from one minor nitpick (see below) I don't see a problem with
this in isolation. It seems like a pretty clean renaming. So I think
we need to move onto the question of how clean the rest of the
Hi Andrew,
> > IMO decent COPY FROM / TO STDIN support is crucial, because otherwise you
> > can't do COPY from/to a client. Which would make the feature unusable for
> > anybody not superuser, including just about all users of hosted PG.
> >
>
> +1
>
> Note that Parquet puts the metadata at the e
>Thus, I still don't see what have happened at Imseih's hand, but I can
>cause PANIC with a bit tricky steps, which I don't think valid. This
>is what I wanted to know the exact steps to cause the PANIC.
>The attached 1 is the PoC of the TAP test (it uses system()..), and
>the
Hi,
On 2022-06-24 14:33:00 +0700, John Naylor wrote:
> On Thu, Jun 23, 2022 at 9:06 PM Andres Freund wrote:
> > It looks like there's quite a bit of low hanging fruits to optimize...
>
> Yeah, if escapes and control characters are rare, adding an SSE2 fast
> path would give a boost to json_lex_s
Hi, hackers!
We started discussion about overheads and how to calculate it correctly.
Julien Rouhaud wrote:
Can you give a bit more details on your bench scenario? I see
contradictory
results, where the patched version with more code is sometimes way
faster,
sometimes way slower. If you're
On 24/6/2022 18:58, Leif Harald Karlsen wrote:
I have a made a small deductive database on top of PostgreSQL for
educational/research purposes. In this setting, due to certain
VIEW-constructions, queries often end up being self-joins on primary
keys, e.g.:
SELECT t1.id, t2.val
FROM t AS t1 JOI
> On 24 Jun 2022, at 18:30, Justin Pryzby wrote:
>
> On Fri, Jun 24, 2022 at 04:09:46PM +0500, Andrey Borodin wrote:
>> Hi everyone!
>>
>> Sorry for bumping old thread.
>
> Please find this newer thread+patch here ;)
> https://www.postgresql.org/message-id/20220614230949.gx29...@telsasoft.com
> On 23 Jun 2022, at 04:58, Justin Pryzby wrote:
>
> On Fri, Jun 17, 2022 at 10:14:13AM -0400, Tom Lane wrote:
>> Robert Haas writes:
>>> On Thu, Jun 16, 2022 at 10:01 PM Justin Pryzby wrote:
To me, oid>=16384 seems more hard-wired than namespace!='pg_catalog'.
>>
>>> Extensions can be
On Thu, Jun 23, 2022 at 10:33 AM Jacob Champion wrote:
> - I think NOT is a important case in practice, which is effectively a
> negative OR ("anything but this/these")
Both NOT (via ! negation) and "none" are implemented in v4.
Examples:
# The server must use SCRAM.
require_auth=scram-sha-256
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation:not tested
Hello
I tested this patch in a setup where the standby is in the mid
On Mon, Jun 6, 2022 at 7:41 PM Stephen Frost wrote:
> Thankfully, at least from my reading, the spec isn't all that
> complicated on this particular point. The spec talks about "role
> authorization descriptor"s and those are "created with role name,
> grantee, and grantor" and then further says
On Fri, Jun 24, 2022 at 11:43:18PM +0500, Andrey Borodin wrote:
> > On 23 Jun 2022, at 04:58, Justin Pryzby wrote:
> >
> > On Fri, Jun 17, 2022 at 10:14:13AM -0400, Tom Lane wrote:
> >> Robert Haas writes:
> >>> On Thu, Jun 16, 2022 at 10:01 PM Justin Pryzby
> >>> wrote:
> To me, oid>=163
On Fri, Jun 24, 2022 at 1:19 PM Robert Haas wrote:
> On Mon, Jun 6, 2022 at 7:41 PM Stephen Frost wrote:
> >
> > In terms of how that's then used, yeah, it's during REVOKE because a
> > REVOKE is only able to 'find' role authorization descriptors which match
> > the triple of role revoked, grant
On Fri, Jun 24, 2022 at 4:30 PM David G. Johnston
wrote:
>> Upthread, I proposed that "drop role baz" should fail here
>
> I concur with this.
>
> I think that the grantor owns the grant, and that REASSIGNED OWNED should be
> able to move those grants to someone else.
>
> By extension, DROP OWNED
Hi,
Looking at the patch,
+ if (copyable_characters_length)
+ {
+ /* flush copyable characters */
+ appendBinaryStringInfo(
+ lex->strval,
+ s - copyable_characters_length,
+
On Fri, Jun 24, 2022 at 8:35 PM Fabien COELHO wrote:
> Just a note/reminder that "seawasp" has been unhappy for some days now
> because of yet another change in the unstable API provided by LLVM:
Hi Fabien,
Yeah, I've started on the changes needed for opaque pointers (that's
the change that's be
Commit 64919aaab made pull_up_simple_subquery set rte->subquery = NULL
after doing the deed, so that we don't waste cycles copying a
now-useless subquery tree around. I discovered today while
working on another patch that if you invoke query_tree_mutator
or range_table_mutator on the whole Query a
> + if (copyable_characters_length)
> + {
> + /* flush copyable characters */
> + appendBinaryStringInfo(
> + lex->strval,
> + s - copyable_characters_length,
> +
Hi Andrey,
Thank you for the quick answer, and for the pointer to the patch! This looks
like just the thing I need!
On a more general note: What would, in general, be the best way to implement
such optimizations? Is there a good way to do this as an extension, or is a
patch the preferred way
Hi Pgsql-Hackers
As part of ongoing work on PostgreSQL security hardening we have
added a capability to disable all file system access (COPY TO/FROM
[PROGRAM] , pg_*file*() functions, lo_*() functions
accessing files, etc) in a way that can not be re-enabled without
already having access to the f
On Thu, Jun 23, 2022 at 06:06:48PM -0700, Andres Freund wrote:
> > You are correct that we wouldn’t need to rely on the pd_flag bit to
> > determine page type for any access to a page where we come top down
> > following the hierarchy. However, for the purpose of debugging “from the
> > bottom up”
Hi,
On 2022-06-24 22:19:33 +, Shawn Debnath wrote:
> On Thu, Jun 23, 2022 at 06:06:48PM -0700, Andres Freund wrote:
>
> > > You are correct that we wouldn’t need to rely on the pd_flag bit to
> > > determine page type for any access to a page where we come top down
> > > following the hierarch
On Fri, Jun 24, 2022 at 3:08 PM Hannu Krosing wrote:
>
> 1) would it be enough to just disable WRITING to the filesystem (COPY
> ... TO ..., COPY TO ... PROGRAM ...) or are some reading functions
> also potentially exploitable or at least making attackers life easier
> ?
>
I would protect read p
On Sat, Jun 25, 2022 at 12:08:13AM +0200, Hannu Krosing wrote:
> As part of ongoing work on PostgreSQL security hardening we have
> added a capability to disable all file system access (COPY TO/FROM
> [PROGRAM] , pg_*file*() functions, lo_*() functions
> accessing files, etc) in a way that can not
Hi,
On 2022-06-25 00:08:13 +0200, Hannu Krosing wrote:
> Currently the file system access is controlled via being a SUPREUSER
> or having the pg_read_server_files, pg_write_server_files and
> pg_execute_server_program roles. The problem with this approach is
> that it will not stop an attacker who
On Sat, Jun 25, 2022 at 1:13 AM Andres Freund wrote:
>
> Hi,
>
> On 2022-06-25 00:08:13 +0200, Hannu Krosing wrote:
> > Currently the file system access is controlled via being a SUPREUSER
> > or having the pg_read_server_files, pg_write_server_files and
> > pg_execute_server_program roles. The pr
On Sat, Jun 25, 2022 at 1:23 AM Hannu Krosing wrote:
> My impression was that this was largely fixed via disabling the old
> direct file calling convention, but then again I did not pay much
> attention at that time :)
I meant of course direct FUNCTION calling convention (Version 0
Calling Conve
On Fri, Jun 24, 2022 at 4:13 PM Andres Freund wrote:
> Hi,
>
> On 2022-06-25 00:08:13 +0200, Hannu Krosing wrote:
> > Currently the file system access is controlled via being a SUPREUSER
> > or having the pg_read_server_files, pg_write_server_files and
> > pg_execute_server_program roles. The pro
On Fri, Jun 24, 2022 at 4:13 PM Andres Freund wrote:
> On 2022-06-25 00:08:13 +0200, Hannu Krosing wrote:
> > 3) should this be back-patched (we can provide batches for all
> > supported PgSQL versions)
>
> Err, what?
Translation: Backpatching these changes to any stable versions will
not be acc
My understanding was that unless activated by admin these changes
would change nothing.
And they would be (borderline :) ) security fixes
And the versioning policy link actually does not say anything about
not adding features to older versions (I know this is the policy, just
pointing out the inf
On Friday, June 24, 2022, Gurjeet Singh wrote:
> On Fri, Jun 24, 2022 at 4:13 PM Andres Freund wrote:
> > On 2022-06-25 00:08:13 +0200, Hannu Krosing wrote:
>
> > > 3) should this be back-patched (we can provide batches for all
> > > supported PgSQL versions)
> >
> > Err, what?
>
> Translation:
The old versions should definitely not have it turned on by default. I
probably was not as clear as I thought in bringing out that point..
For upcoming next ones the distributors may want to turn it on for
some more security-conscious ("enterprize") distributions.
-- Hannu
On Sat, Jun 25, 2022 a
Hi,
On 2022-06-24 08:47:09 +, Jelte Fennema wrote:
> To test performance of this change I used COPY BINARY from a JSONB table
> into another, containing fairly JSONB values of ~15kB.
This will have a lot of other costs included (DML is expensive). I'd suggest
storing the json in a text column
(fixed your top-posting)
On Fri, Jun 24, 2022 at 4:59 PM Hannu Krosing wrote:
> On Sat, Jun 25, 2022 at 1:46 AM Gurjeet Singh wrote:
> >
> > On Fri, Jun 24, 2022 at 4:13 PM Andres Freund wrote:
> > > On 2022-06-25 00:08:13 +0200, Hannu Krosing wrote:
> >
> > > > 3) should this be back-patched (
Resending with a problematic email removed from CC...
On Mon, Apr 04, 2022 at 03:21:54PM +, Jelte Fennema wrote:
> 2. Added some extra sleeps to the cancellation test, to remove random
> failures on FreeBSD.
Apparently there's still an occasional issue.
https://cirrus-ci.com/task/66133099851
Hi,
On 2022-06-24 17:18:10 -0700, Andres Freund wrote:
> On 2022-06-24 08:47:09 +, Jelte Fennema wrote:
> > To test performance of this change I used COPY BINARY from a JSONB table
> > into another, containing fairly JSONB values of ~15kB.
>
> This will have a lot of other costs included (DML
Hi,
On 2022-06-25 01:23:36 +0200, Hannu Krosing wrote:
> Are you claiming that one can manipulate PostgreSQL to do any file
> writes directly by manipulating pg_proc to call the functions "in a
> wrong way" ?
Yes.
> My impression was that this was largely fixed via disabling the old
> direct f
Hi,
On 2022-06-24 10:59:25 -0400, Robert Haas wrote:
> A preliminary refactoring that was discussed in the past and was
> originally in 0001 was to move the fields included in BufferTag via
> RelFileNode/Locator directly into the struct. I think maybe it doesn't
> make sense to include that in 000
On Fri, 24 Jun 2022 at 22:44, Tom Lane wrote:
>
> Commit 64919aaab made pull_up_simple_subquery set rte->subquery = NULL
> after doing the deed, so that we don't waste cycles copying a
> now-useless subquery tree around. I discovered today while
> working on another patch that if you invoke query
Dean Rasheed writes:
> Not directly related to that change ... I think it would be easier to
> follow if the CHECKFLATCOPY() was replaced with a separate Assert()
> and FLATCOPY() (I had to go and remind myself what CHECKFLATCOPY()
> did).
> Doing that would allow CHECKFLATCOPY() to be deleted, si
On 24/6/2022 23:43, Leif Harald Karlsen wrote:
Thank you for the quick answer, and for the pointer to the patch! This
looks like just the thing I need!
On a more general note: What would, in general, be the best way to
implement such optimizations? Is there a good way to do this as an
extensio
Hello Thomas,
llvmjit.c:1233:81: error: too few arguments to function call, expected 3,
have 2
ref_gen =
LLVMOrcCreateCustomCAPIDefinitionGenerator(llvm_resolve_symbols, NULL);
Ah yes, I hadn't seen that one yet. That function grew a "Dispose"
argument, which we can just pass NU
71 matches
Mail list logo
