Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-28 Thread Joe Conway
On 3/28/22 15:56, Robert Haas wrote: On Mon, Mar 21, 2022 at 4:15 PM Joe Conway wrote: Robert -- any opinion on this? If I am not mistaken it is code that you are actively working on. Woops, I only just saw this. I don't mind if you want to change the calls to is_member_of_role

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-28 Thread Joe Conway
On 3/21/22 16:15, Joe Conway wrote: On 3/20/22 12:38, Stephen Frost wrote: Greetings, On Sun, Mar 20, 2022 at 18:31 Joshua Brindle mailto:joshua.brin...@crunchydata.com>> wrote: On Sun, Mar 20, 2022 at 12:27 PM Joe Conway mailto:m...@joeconway.com>> wrote: > &g

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-21 Thread Joe Conway
On 3/20/22 12:38, Stephen Frost wrote: Greetings, On Sun, Mar 20, 2022 at 18:31 Joshua Brindle mailto:joshua.brin...@crunchydata.com>> wrote: On Sun, Mar 20, 2022 at 12:27 PM Joe Conway mailto:m...@joeconway.com>> wrote: > > On 3/3/22 11:26, Jo

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-20 Thread Joe Conway
On 3/20/22 12:31, Joshua Brindle wrote: On Sun, Mar 20, 2022 at 12:27 PM Joe Conway wrote: On 3/3/22 11:26, Joshua Brindle wrote: > On Thu, Feb 10, 2022 at 2:37 PM Joe Conway wrote: >> >> On 2/10/22 14:28, Nathan Bossart wrote: >> > On Wed, Feb 09, 2022 at 04:39:11PM

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-03-20 Thread Joe Conway
On 3/3/22 11:26, Joshua Brindle wrote: On Thu, Feb 10, 2022 at 2:37 PM Joe Conway wrote: On 2/10/22 14:28, Nathan Bossart wrote: > On Wed, Feb 09, 2022 at 04:39:11PM -0500, Joe Conway wrote: >> On 2/9/22 13:13, Nathan Bossart wrote: >>> I do wonder if users find the d

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Joe Conway
On 2/23/22 09:52, Aleksander Alekseev wrote: > What about scanning for "PG_MODULE_MAGIC"? An extension can be written without using C at all. BTW some extensions [1] are written in Rust these days. Sure, but scanning for PG_MODULE_MAGIC may well pick up repos that would otherwise have been

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-23 Thread Joe Conway
On 2/23/22 09:33, Euler Taveira wrote: On Wed, Feb 23, 2022, at 6:00 AM, Joel Jacobson wrote: On Fri, Feb 11, 2022, at 04:46, Noah Misch wrote: > How did you make the list?  (I'd imagine doing it by searching for > repositories containing evidence like \bpgxs\b matches.) Searching Github for

Re: Time to drop plpython2?

2022-02-18 Thread Joe Conway
On 2/18/22 15:53, Andres Freund wrote: the next run succeeded, with 'PYTHON' => 'python3' in build env. But presumably this just was because you installed the python3-devel package? Ok, I guess I got confused when it failed due to the missing devel package, because I removed the PYTHON =>

Re: Time to drop plpython2?

2022-02-18 Thread Joe Conway
On 2/18/22 15:25, Andres Freund wrote: On 2022-02-18 14:46:39 -0500, Joe Conway wrote: $ ll /usr/bin/python lrwxrwxrwx. 1 root root 7 Mar 13 2021 /usr/bin/python -> python2 8<--- Yea, that all looks fine. What's the problem if you don't specify the PYTHON=python3?

Re: Time to drop plpython2?

2022-02-18 Thread Joe Conway
On 2/18/22 14:37, Andres Freund wrote: Hi, On 2022-02-18 14:19:49 -0500, Joe Conway wrote: On 2/17/22 13:08, Andres Freund wrote: > On 2022-02-16 23:14:46 -0800, Andres Freund wrote: > > > Done. Curious how red the BF will turn out to be. Let's hope it's not > > > too b

Re: Time to drop plpython2?

2022-02-18 Thread Joe Conway
On 2/17/22 13:08, Andres Freund wrote: On 2022-02-16 23:14:46 -0800, Andres Freund wrote: > Done. Curious how red the BF will turn out to be. Let's hope it's not > too bad. - rhinoceros Joe replied that he is afk, looking into it tomorrow. I installed python3 packages (initially

Re: pgsql: Move scanint8() to numutils.c

2022-02-15 Thread Joe Conway
On 2/15/22 13:47, Robert Haas wrote: On Tue, Feb 15, 2022 at 10:39 AM Joe Conway wrote: (moving to hackers) I guess shame on me for not noticing the thread, but I don't see any discussion about the potential for breakage to external projects. scanint8() is exported, and this change breaks

Re: pgsql: Move scanint8() to numutils.c

2022-02-15 Thread Joe Conway
On 2/14/22 16:18, Peter Eisentraut wrote: Move scanint8() to numutils.c Move scanint8() to numutils.c and rename to pg_strtoint64(). We already have a "16" and "32" version of that, and the code inside the functions was aligned, so this move makes all three versions consistent. The API is

Re: Observability in Postgres

2022-02-15 Thread Joe Conway
On 2/15/22 07:30, Dave Page wrote: On Mon, 14 Feb 2022 at 20:16, Greg Stark > wrote: So I've been dealing a lot with building and maintaining dashboards for (fleets of) Postgres servers. And it's a pain. I have a few strongly held ideas about where the pain

Re: List of all* PostgreSQL EXTENSIONs in the world

2022-02-10 Thread Joe Conway
On 2/10/22 15:35, Robert Haas wrote: On Thu, Feb 10, 2022 at 3:19 PM Joel Jacobson wrote: I've compiled a list of all* PostgreSQL EXTENSIONs in the world: https://gist.github.com/joelonsql/e5aa27f8cc9bd22b8999b7de8aee9d47 *) It's not all, but 1041, compared to the 338 found on PGXN. Maybe

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-10 Thread Joe Conway
On 2/10/22 14:28, Nathan Bossart wrote: On Wed, Feb 09, 2022 at 04:39:11PM -0500, Joe Conway wrote: On 2/9/22 13:13, Nathan Bossart wrote: I do wonder if users find the differences between predefined roles and role attributes confusing. INHERIT doesn't govern role attributes

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-09 Thread Joe Conway
On 2/9/22 13:13, Nathan Bossart wrote: On Tue, Feb 08, 2022 at 10:54:50PM -0500, Robert Haas wrote: On Tue, Feb 8, 2022 at 7:38 PM Joe Conway wrote: If we were to start all over again with this feature my vote would be to do things differently than we have done. I would not have called them

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-08 Thread Joe Conway
On 2/8/22 10:07, Robert Haas wrote: On Tue, Feb 8, 2022 at 10:00 AM Joshua Brindle wrote: 4 predefined roles currently use has_privs_of_role in master. Further, pg_monitor, as an SQL-only predefined role, also behaves consistently with the INHERIT rules that other roles do. In order for

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-08 Thread Joe Conway
On 2/7/22 12:09, Robert Haas wrote: On Mon, Feb 7, 2022 at 11:13 AM Joe Conway wrote: It is confusing and IMHO dangerous that the predefined roles currently work differently than regular roles eith respect to privilege inheritance. I feel like that's kind of a conclusory statement

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-07 Thread Joe Conway
On 2/7/22 10:35, Robert Haas wrote: On Sun, Feb 6, 2022 at 12:24 PM Tom Lane wrote: Joe Conway writes: > I'd like to pick this patch up and see it through to commit/push. > Presumably that will include back-patching to all supported pg versions. > Before I go through the effort to b

Re: [PATCH v2] use has_privs_for_role for predefined roles

2022-02-06 Thread Joe Conway
On 1/4/22 16:51, Joshua Brindle wrote: On Tue, Jan 4, 2022 at 3:56 PM Tom Lane wrote: "Bossart, Nathan" writes: > On 11/12/21, 12:34 PM, "Joshua Brindle" wrote: >> All of these and also adminpack.sgml updated. I think that is all of >> them but docs broken across lines and irregular

Re: [RFC] speed up count(*)

2021-10-21 Thread Joe Conway
On 10/21/21 4:23 PM, Robert Haas wrote: On Thu, Oct 21, 2021 at 4:19 PM Joe Conway wrote: That is a grossly overstated position. When I have looked, it is often not that terribly far off. And for many use cases that I have heard of at least, quite adequate. I don't think it's grossly

Re: [RFC] speed up count(*)

2021-10-21 Thread Joe Conway
On 10/21/21 4:06 PM, Robert Haas wrote: On Thu, Oct 21, 2021 at 9:09 AM Joe Conway wrote: I think you are exactly correct. People seem to understand that with a predicate it is harder, but they expect select count(*) from foo; to be nearly instantaneous, and they don't really need

Re: [RFC] speed up count(*)

2021-10-21 Thread Joe Conway
On 10/20/21 2:33 PM, John Naylor wrote: On Wed, Oct 20, 2021 at 2:23 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > > Couldn't we simply inspect the visibility map, use the index data only > for fully visible/summarized ranges, and inspect the heap for the > remaining

Re: Returning to Postgres community work

2021-08-31 Thread Joe Conway
On 8/31/21 1:53 AM, Gurjeet Singh wrote: I'm very happy to announce that I now work for Supabase [1]. They have hired me so that I can participate in, and contribute to the Postgres community. Welcome back! :-) Joe -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread Joe Conway
On 8/30/21 8:22 PM, Tom Lane wrote: Joe Conway writes: It would be interesting to step through a few times to see if it is really stuck in that loop. Yeah, this single data point is not enough justification to blame dynahash.c (which is *extremely* battle-tested code, you'll recall). I'm

Re: Pg stuck at 100% cpu, for multiple days

2021-08-30 Thread Joe Conway
On 8/30/21 3:34 PM, Justin Pryzby wrote: On Mon, Aug 30, 2021 at 09:09:20PM +0200, Laurenz Albe wrote: On Mon, 2021-08-30 at 17:18 +0200, hubert depesz lubaczewski wrote: > The thing is - I can't close it with pg_terminate_backend(), and I'd > rather not kill -9, as it will, I think, close all

Re: Cosmic ray hits integerset

2021-07-07 Thread Joe Conway
On 7/7/21 2:53 AM, Jakub Wartak wrote: Hi, Asking out of pure technical curiosity about "the rhinoceros" - what kind of animal is it ? Physical box or VM? How one could get dmidecode(1) / dmesg(1) / mcelog (1) from what's out there (e.g. does it run ECC or not ?) Rhinoceros is just a VM on

Re: Support for CREATE MODULE?

2021-06-02 Thread Joe Conway
On 6/2/21 10:43 AM, Jim Mlodgenski wrote: On Wed, Jun 2, 2021 at 9:58 AM Tom Lane wrote: I wonder whether it'd be better to consider modules as a kind of extension, or at least things with the same sort of ownership relations as extensions have. That would solve the problem of associating

Re: Freenode woes

2021-05-20 Thread Joe Conway
On 5/19/21 4:27 PM, Robert Treat wrote: On Wed, May 19, 2021 at 10:19 AM Christoph Berg wrote: Fwiw, if the PostgreSQL projects is considering moving the #postgresql IRC channel(s) elsewhere given [1,2], I'm a member of OFTC.net's network operations committee and would be happy to help. [1]

Re: View invoker privileges

2021-05-14 Thread Joe Conway
On 5/14/21 4:11 AM, Noah Misch wrote: On Wed, Apr 14, 2021 at 10:25:08AM +0300, Ivan Ivanov wrote: In Postgres we can create view with view owner privileges only. What’s the reason that there is no option to create view with invoker privileges? Is there any technical or security subtleties

Re: PG 14 release notes, first draft

2021-05-11 Thread Joe Conway
On 5/11/21 1:30 PM, Bruce Momjian wrote: On Tue, May 11, 2021 at 12:31:01PM -0400, Joe Conway wrote: On 5/11/21 11:37 AM, Bruce Momjian wrote: > On Tue, May 11, 2021 at 11:26:48AM -0400, Joe Conway wrote: > > On 5/11/21 11:11 AM, Bruce Momjian wrote: > > > > Previously exis

Re: PG 14 release notes, first draft

2021-05-11 Thread Joe Conway
On 5/11/21 11:37 AM, Bruce Momjian wrote: On Tue, May 11, 2021 at 11:26:48AM -0400, Joe Conway wrote: On 5/11/21 11:11 AM, Bruce Momjian wrote: > > Previously existence of such columns were ignored when caller had table > > level privileges. > > I can't reproduce the NULL

Re: PG 14 release notes, first draft

2021-05-11 Thread Joe Conway
On 5/11/21 11:11 AM, Bruce Momjian wrote: On Tue, May 11, 2021 at 06:57:19AM -0400, Joe Conway wrote: On 5/10/21 9:56 PM, Bruce Momjian wrote: > On Mon, May 10, 2021 at 07:50:14AM -0400, Joe Conway wrote: > > On 5/10/21 2:03 AM, Bruce Momjian wrote: > > > I have committ

Re: PG 14 release notes, first draft

2021-05-11 Thread Joe Conway
On 5/10/21 9:56 PM, Bruce Momjian wrote: On Mon, May 10, 2021 at 07:50:14AM -0400, Joe Conway wrote: On 5/10/21 2:03 AM, Bruce Momjian wrote: > I have committed the first draft of the PG 14 release notes. You can > see the most current build of them here: > > https://momjian.u

Re: PG 14 release notes, first draft

2021-05-10 Thread Joe Conway
On 5/10/21 2:03 AM, Bruce Momjian wrote: I have committed the first draft of the PG 14 release notes. You can see the most current build of them here: https://momjian.us/pgsql_docs/release-14.html I need clarification on many items, and the document still needs its items properly

Re: PG in container w/ pid namespace is init, process exits cause restart

2021-05-04 Thread Joe Conway
(Joe Conway if memory serves) reported years ago. Guilty as charged ;-) Joe -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure Enterprises Consulting, Training, & Open Source Development

Re: function for testing that causes the backend to terminate

2021-04-29 Thread Joe Conway
On 4/29/21 6:56 AM, Dave Cramer wrote: For testing unusual situations I'd like to be able to cause a backend to terminate due to something like a segfault. Do we currently have this in testing ? If you can run SQL as a superuser from that backend, try: COPY (SELECT pg_backend_pid()) TO

Re: PL/R regression on windows, but not linux with master.

2021-04-11 Thread Joe Conway
On 4/11/21 12:51 PM, Dave Cramer wrote: On Sun, 11 Apr 2021 at 12:43, Tom Lane <mailto:t...@sss.pgh.pa.us>> wrote: I wrote: > Joe Conway mailto:m...@joeconway.com>> writes: >> Would an equivalent "PGWARNING" be something we are open t

Re: PL/R regression on windows, but not linux with master.

2021-04-11 Thread Joe Conway
On 4/11/21 10:13 AM, Tom Lane wrote: Andrew Dunstan writes: Well, plr.h does this: #define WARNING 19 #define ERROR 20 The coding pattern in plr.h looks quite breakable. Meh -- that code has gone 18+ years before breaking. Indeed. elog.h already provides a "PGERROR"

Re: documentation fix for SET ROLE

2021-04-02 Thread Joe Conway
On 4/2/21 10:21 AM, Laurenz Albe wrote: On Mon, 2021-03-15 at 17:09 +, Bossart, Nathan wrote: On 3/15/21, 7:06 AM, "Laurenz Albe" wrote: > On Fri, 2021-03-12 at 21:41 +, Bossart, Nathan wrote: > > On 3/12/21, 11:14 AM, "Joe Conway" wrote: > > &

Re: policies with security definer option for allowing inline optimization

2021-04-02 Thread Joe Conway
On 4/2/21 10:23 AM, Stephen Frost wrote: Greetings, * Joe Conway (m...@joeconway.com) wrote: On 4/2/21 9:57 AM, Isaac Morland wrote: >Views already run security definer, allowing them to be used for some of >the same information-hiding purposes as RLS. But I just found something &g

Re: policies with security definer option for allowing inline optimization

2021-04-02 Thread Joe Conway
On 4/2/21 9:57 AM, Isaac Morland wrote: Views already run security definer, allowing them to be used for some of the same information-hiding purposes as RLS. But I just found something strange: current_user/_role returns the user's role, not the view owner's role: postgres=# set role to t1;

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-03-31 Thread Joe Conway
On 3/30/21 8:17 PM, Joe Conway wrote: On 3/30/21 6:22 PM, Tom Lane wrote: Joe Conway writes: Heh, I missed the forest for the trees it seems. That version undid the changes fixing what Ian was originally complaining about. Duh, right. It would be a good idea for there to be a code comment

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-03-30 Thread Joe Conway
On 3/30/21 6:22 PM, Tom Lane wrote: Joe Conway writes: Heh, I missed the forest for the trees it seems. That version undid the changes fixing what Ian was originally complaining about. Duh, right. It would be a good idea for there to be a code comment explaining this, because it's *far

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-03-30 Thread Joe Conway
On 3/30/21 3:37 PM, Joe Conway wrote: On 3/21/21 12:27 PM, Tom Lane wrote: I think we may have to adjust the acl.c APIs, or maybe better provide new entry points, so that we can have variants of pg_xxx_aclcheck that won't throw a hard error upon not finding the row. We cheesily tried to avoid

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-03-30 Thread Joe Conway
On 3/21/21 12:27 PM, Tom Lane wrote: I think we may have to adjust the acl.c APIs, or maybe better provide new entry points, so that we can have variants of pg_xxx_aclcheck that won't throw a hard error upon not finding the row. We cheesily tried to avoid adjusting those APIs to support the

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-03-21 Thread Joe Conway
On 3/16/21 2:45 PM, Joe Conway wrote: Ian, or anyone else, any comments/complaints on my changes? If not I will commit and push that version sooner rather than later. Any thoughts on back-patching this? On one hand, in my view it is clearly a bug. On the other hand, no one has complained

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-03-16 Thread Joe Conway
On 3/16/21 1:42 AM, Chengxi Sun wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested I tested the patch and it

Re: documentation fix for SET ROLE

2021-03-12 Thread Joe Conway
On 3/12/21 1:16 PM, Bossart, Nathan wrote: On 3/12/21, 6:35 AM, "Laurenz Albe" wrote: On Fri, 2021-03-12 at 10:16 +0100, I wrote: After sleeping on it, I have come to think that it is excessive to write so much documentation for a feature that is that unimportant. It takes some effort to

Re: Procedures versus the "fastpath" API

2021-03-09 Thread Joe Conway
On 3/9/21 2:15 PM, Tom Lane wrote: > So the question on the table is what to do about this. As far as > window functions go, it seems clear that fastpath.c should just reject > any attempt to call a window function that way (or an aggregate for > that matter; aggregates fail already, but with

Re: [PATCH] pg_permissions

2021-03-08 Thread Joe Conway
On 3/6/21 2:03 PM, Joel Jacobson wrote: > ...but to answer the question... > >    - What permissions are there for a specific role in the database? > > you need to manually query all relevant pg_catalog or > information_schema.*_privileges views, > which is a O(n) mental effort, while the first

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-03-07 Thread Joe Conway
On 3/7/21 2:35 PM, Zhihong Yu wrote: > Joe: > I don't seem to find attachment. > > Maybe attach again ? Oops -- I did forget that, didn't I. This time patch is attached :-) Joe -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure Enterprises Consulting, Training, & Open

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-03-07 Thread Joe Conway
On 3/3/21 9:43 AM, Joe Conway wrote: > On 3/3/21 8:50 AM, David Steele wrote: >> On 1/29/21 4:56 AM, Joe Conway wrote: >>> On 1/29/21 12:13 AM, Ian Lawrence Barwick wrote: >>>> 2021年1月28日(木) 17:18 Peter Eisentraut: >>>> I'm not convinced the

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-03-03 Thread Joe Conway
On 3/3/21 8:50 AM, David Steele wrote: > On 1/29/21 4:56 AM, Joe Conway wrote: >> On 1/29/21 12:13 AM, Ian Lawrence Barwick wrote: >>> 2021年1月28日(木) 17:18 Peter Eisentraut: >>> I'm not convinced the current behavior is wrong.  Is there some >>>

Re: documentation fix for SET ROLE

2021-02-17 Thread Joe Conway
On 2/17/21 2:12 PM, David G. Johnston wrote: > On Wednesday, February 17, 2021, Bossart, Nathan > wrote: > > >     postgres=# ALTER ROLE test1 SET ROLE test2; >     ALTER ROLE > > > I would not have expected this to work - “role” isn’t a >

Re: "has_column_privilege()" issue with attnums and non-existent columns

2021-01-29 Thread Joe Conway
On 1/29/21 12:13 AM, Ian Lawrence Barwick wrote: > 2021年1月28日(木) 17:18 Peter Eisentraut: > I'm not convinced the current behavior is wrong.  Is there some > practical use case that is affected by this behavior? > >   > I was poking around at the function with a view to using it for

Re: Should we document IS [NOT] OF?

2020-11-19 Thread Joe Conway
On 11/19/20 12:08 PM, Tom Lane wrote: > Bruce Momjian writes: >> On Thu, Nov 19, 2020 at 11:15:33AM -0500, Joe Conway wrote: >>> On 11/19/20 11:06 AM, Tom Lane wrote: >>>> Let's just rip it out and be done. If anyone is ever >>>> motivated to

Re: Should we document IS [NOT] OF?

2020-11-19 Thread Joe Conway
On 11/19/20 11:06 AM, Tom Lane wrote: > Let's just rip it out and be done. If anyone is ever > motivated to make it work per spec, they can resurrect > whatever seems useful from the git history. +1 Joe -- Crunchy Data - http://crunchydata.com PostgreSQL Support for Secure Enterprises

Re: Should we document IS [NOT] OF?

2020-11-19 Thread Joe Conway
On 11/19/20 2:03 AM, Tom Lane wrote: > "David G. Johnston" writes: >> Is there a feature code? I skimmed the standard and non-standard tables in >> our appendix and couldn’t find this in either. > > a19d9d3c4 seems to have thought it was S151. Here is a link to previous list discussions:

Re: security_context_t marked as deprecated in libselinux 3.1

2020-08-13 Thread Joe Conway
On 8/13/20 1:22 AM, Michael Paquier wrote: > On Wed, Aug 12, 2020 at 10:50:21PM -0400, Tom Lane wrote: >> Ummm ... aren't you going to get some cast-away-const warnings now? >> Or are all of the called functions declared as taking "const char *" >> not just "char *"? > > Let me see.. The

Re: pg_read_file() with virtual files returns empty string

2020-07-04 Thread Joe Conway
On 7/4/20 1:10 PM, Joe Conway wrote: > On 7/4/20 12:52 PM, Tom Lane wrote: >> Justin Pryzby writes: >>> But I noticed that cfbot is now populating with failures like: >> >>> genfile.c: In function ‘read_binary_file’: >>> genfile.c:192:5: error:

Re: pg_read_file() with virtual files returns empty string

2020-07-04 Thread Joe Conway
On 7/4/20 12:52 PM, Tom Lane wrote: > Justin Pryzby writes: >> But I noticed that cfbot is now populating with failures like: > >> genfile.c: In function ‘read_binary_file’: >> genfile.c:192:5: error: ignoring return value of ‘fread’, declared with >> attribute warn_unused_result

Re: pg_read_file() with virtual files returns empty string

2020-07-04 Thread Joe Conway
On 7/2/20 6:29 PM, Tom Lane wrote: > Joe Conway writes: >> On 7/2/20 5:37 PM, Tom Lane wrote: >>> I still can't get excited about contorting the code to remove that >>> issue. > >> It doesn't seem much worse than the oom test that was there before -- see

Re: pg_read_file() with virtual files returns empty string

2020-07-02 Thread Joe Conway
On 7/2/20 5:37 PM, Tom Lane wrote: > Joe Conway writes: >> In fact, in principle there is no reason we can't get to max - 4 with this >> code >> except that when the filesize is exactly 1073741819, we need to try to read >> one >> more byte to find the EO

Re: pg_read_file() with virtual files returns empty string

2020-07-02 Thread Joe Conway
On 7/2/20 4:27 PM, Tom Lane wrote: > Joe Conway writes: >> When I saw originally MaxAllocSize - 5 fail I skipped to something smaller by >> 4096 and it worked. But here I see that the actual max size is MaxAllocSize >> - 6. > > Huh, I wonder why it's not max - 5.

Re: pg_read_file() with virtual files returns empty string

2020-07-02 Thread Joe Conway
On 7/2/20 3:36 PM, Tom Lane wrote: > Joe Conway writes: >> On 7/1/20 6:22 PM, Tom Lane wrote: >>> Hm, I was expecting that the last successful iteration of >>> enlargeStringInfo would increase the buffer size to MaxAllocSize, >>> so that we'd really only be l

Re: pg_read_file() with virtual files returns empty string

2020-07-02 Thread Joe Conway
On 7/1/20 6:22 PM, Tom Lane wrote: > Joe Conway writes: >> The only downside is that the max filesize is reduced to (MaxAllocSize - >> MIN_READ_SIZE - 1) compared to MaxAllocSize with the old method. > > Hm, I was expecting that the last successful iteration of > enlargeS

Re: pg_read_file() with virtual files returns empty string

2020-07-01 Thread Joe Conway
On 7/1/20 5:17 PM, Joe Conway wrote: > On 7/1/20 4:12 PM, Tom Lane wrote: >> Joe Conway writes: >>> I did some performance testing of the worst case/largest possible file and >>> found >>> that skipping the stat and bulk read does cause a significant regr

Re: pg_read_file() with virtual files returns empty string

2020-07-01 Thread Joe Conway
On 7/1/20 4:12 PM, Tom Lane wrote: > Joe Conway writes: >> I did some performance testing of the worst case/largest possible file and >> found >> that skipping the stat and bulk read does cause a significant regression. > > Yeah, I was wondering a littl

Re: pg_read_file() with virtual files returns empty string

2020-06-30 Thread Joe Conway
On 6/28/20 6:00 PM, Tom Lane wrote: > Joe Conway writes: >> All good stuff -- I believe the attached checks all the boxes. > > Looks okay to me, except I think you want > > ! if (bytes_to_read > 0) > > to be > > ! if (bytes_to_read >= 0) Yep --

Re: pg_read_file() with virtual files returns empty string

2020-06-28 Thread Joe Conway
On 6/27/20 3:43 PM, Tom Lane wrote: > Joe Conway writes: >> The attached patch fixes this for me. I think it ought to be backpatched >> through >> pg11. > >> Comments? > > 1. Doesn't seem to be accounting for the possibility of an error in fread(). >

pg_read_file() with virtual files returns empty string

2020-06-27 Thread Joe Conway
Since pg11 pg_read_file() and friends can be used with absolute paths as long as the user is superuser or explicitly granted the role pg_read_server_files. I noticed that when trying to read a virtual file, e.g.: SELECT pg_read_file('/proc/self/status'); the returned result is a zero length

Re: language cleanups in code and docs

2020-06-16 Thread Joe Conway
On 6/16/20 3:26 AM, Magnus Hagander wrote: > On Tue, Jun 16, 2020 at 2:23 AM Andres Freund wrote: > postmaster is just a symlink, which we very well could just leave in > place... I was really just thinking of the code level stuff. And I think > there's some clarity reasons to rename

Re: Building PostgreSQL extensions on Windows

2020-06-12 Thread Joe Conway
On 6/11/20 6:42 PM, David Rowley wrote: > I've heard from a few people that building PostgreSQL extensions on > Windows is a bit of a pain. I've heard from these people that their > solution was to temporarily add their extension as a contrib module > and have the extension building code take care

Re: Recording test runtimes with the buildfarm

2020-06-11 Thread Joe Conway
On 6/10/20 6:00 PM, Andres Freund wrote: > On June 10, 2020 2:13:51 PM PDT, David Rowley wrote: >>On Thu, 11 Jun 2020 at 02:13, Tom Lane wrote: >>> I have in the past scraped the latter results and tried to make sense >>of >>> them. They are *mighty* noisy, even when considering just one animal

Re: repeat() function, CHECK_FOR_INTERRUPTS(), and unlikely()

2020-06-05 Thread Joe Conway
On 6/4/20 5:20 PM, Alvaro Herrera wrote: > On 2020-May-28, Joe Conway wrote: > >> I backpatched and pushed the changes to the repeat() function. Any other >> opinions regarding backpatch of the unlikely() addition to >> CHECK_FOR_INTERRUPTS()? > > We don't use

Re: repeat() function, CHECK_FOR_INTERRUPTS(), and unlikely()

2020-06-04 Thread Joe Conway
On 5/28/20 1:23 PM, Joe Conway wrote: > On 5/27/20 3:29 AM, Michael Paquier wrote: >>> I think that each of those tests should have a separate unlikely() marker, >>> since the whole point here is that we don't expect either of those tests >>> to y

Re: repeat() function, CHECK_FOR_INTERRUPTS(), and unlikely()

2020-05-28 Thread Joe Conway
On 5/27/20 3:29 AM, Michael Paquier wrote: >> I think that each of those tests should have a separate unlikely() marker, >> since the whole point here is that we don't expect either of those tests >> to yield true in the huge majority of CHECK_FOR_INTERRUPTS executions. > > +1. I am not sure

Re: repeat() function, CHECK_FOR_INTERRUPTS(), and unlikely()

2020-05-25 Thread Joe Conway
On 5/25/20 9:52 AM, Tom Lane wrote: > Joe Conway writes: >>> Comments or objections? > >> Seeing none ... I intend to backpatch and push these two patches in the next >> day >> or so. > > There was some question as to what (if anything) to do with the Wi

Re: repeat() function, CHECK_FOR_INTERRUPTS(), and unlikely()

2020-05-25 Thread Joe Conway
On 5/12/20 8:06 AM, Joe Conway wrote: > I was doing some memory testing under fractional CPU allocations and it became > painfully obvious that the repeat() function needs CHECK_FOR_INTERRUPTS(). > > I exchanged a few emails offlist with Tom about it, and (at the risk of > putting

repeat() function, CHECK_FOR_INTERRUPTS(), and unlikely()

2020-05-12 Thread Joe Conway
I was doing some memory testing under fractional CPU allocations and it became painfully obvious that the repeat() function needs CHECK_FOR_INTERRUPTS(). I exchanged a few emails offlist with Tom about it, and (at the risk of putting words in his mouth) he agreed and felt it was a candidate for

Re: RFC: seccomp-bpf support

2020-01-07 Thread Joe Conway
On 1/6/20 8:37 PM, Tomas Vondra wrote: > Hi, > > This patch is currently in "needs review" state, but the last message is > from August 29, and my understanding is that there have been a couple of > objections / disagreements about the architecture, difficulties with > producing the set of

Re: string literal continuations in C

2019-12-24 Thread Joe Conway
On 12/23/19 2:51 PM, Alvaro Herrera wrote: > Per a recent thread, these patches remove string literals split with > \-escaped newlines. The first is for the message "materialize mode > required, but it is not allowed in this context" where it's more > prevalent, and we keep perpetuating it; the

Re: add a MAC check for TRUNCATE

2019-11-23 Thread Joe Conway
On 11/22/19 3:07 AM, Michael Paquier wrote: > On Wed, Nov 20, 2019 at 02:30:12PM -0500, Joe Conway wrote: >> I tested this successfully on Rhinoceros, both with and without >> "db_table: { truncate }" loaded in the policy. Updated patches attached >>

Re: add a MAC check for TRUNCATE

2019-11-20 Thread Joe Conway
On 11/20/19 2:30 PM, Joe Conway wrote: > On 11/8/19 9:16 AM, Joe Conway wrote: >> On 11/8/19 9:02 AM, Yuli Khodorkovskiy wrote: >>> On Thu, Nov 7, 2019 at 7:46 PM Michael Paquier wrote: >>>> >>>> On Mon, Sep 30, 2019 at 11:38:05AM -0300, Alvaro Herrer

Re: add a MAC check for TRUNCATE

2019-11-20 Thread Joe Conway
On 11/8/19 9:16 AM, Joe Conway wrote: > On 11/8/19 9:02 AM, Yuli Khodorkovskiy wrote: >> On Thu, Nov 7, 2019 at 7:46 PM Michael Paquier wrote: >>> >>> On Mon, Sep 30, 2019 at 11:38:05AM -0300, Alvaro Herrera wrote: >>> > On 2019-Sep-30, Joe Conway wrote:

Re: add a MAC check for TRUNCATE

2019-11-08 Thread Joe Conway
On 11/8/19 9:02 AM, Yuli Khodorkovskiy wrote: > On Thu, Nov 7, 2019 at 7:46 PM Michael Paquier wrote: >> >> On Mon, Sep 30, 2019 at 11:38:05AM -0300, Alvaro Herrera wrote: >> > On 2019-Sep-30, Joe Conway wrote: >> > >> > > I am not sure I will get to

Re: add a MAC check for TRUNCATE

2019-09-30 Thread Joe Conway
On 9/25/19 4:47 PM, Joe Conway wrote: > On 9/25/19 3:56 PM, Alvaro Herrera wrote: >> Hello >> >> On 2019-Sep-09, Yuli Khodorkovskiy wrote: >> >>> I have included an updated version of the sepgql patch. The >>> Truncate-Hook patch is unchanged from

Re: add a MAC check for TRUNCATE

2019-09-25 Thread Joe Conway
On 9/25/19 3:56 PM, Alvaro Herrera wrote: > Hello > > On 2019-Sep-09, Yuli Khodorkovskiy wrote: > >> I have included an updated version of the sepgql patch. The >> Truncate-Hook patch is unchanged from the last version. > > This patch no longer applies. Can you please rebase? > > Joe, do you

Re: add a MAC check for TRUNCATE

2019-09-06 Thread Joe Conway
On 9/6/19 8:07 PM, Tom Lane wrote: > Joe Conway writes: >> On 9/6/19 2:18 PM, Tom Lane wrote: >>> sepgsql hasn't worked on RHEL6 in a long time, if ever; it requires >>> a newer version of libselinux than what ships in RHEL6. So I'm not >>> concerned about th

Re: add a MAC check for TRUNCATE

2019-09-06 Thread Joe Conway
On 9/6/19 2:13 PM, Yuli Khodorkovskiy wrote: > As Joe Conway pointed out to me out of band, the build animal for RHEL > 7 has handle_unknown set to `0`. Are there any other concerns with > this approach? You mean deny_unknown I believe. "Allow unknown object class / permissions.

Re: add a MAC check for TRUNCATE

2019-09-06 Thread Joe Conway
On 9/6/19 2:18 PM, Tom Lane wrote: > Yuli Khodorkovskiy writes: >> On Fri, Sep 6, 2019 at 11:57 AM Tom Lane wrote: >>> Well, the larger question, independent of the regression tests, is >>> will the new policy work at all on older SELinux? If not, that >>> doesn't seem very acceptable. > >>

Re: add a MAC check for TRUNCATE

2019-09-06 Thread Joe Conway
On 9/6/19 11:26 AM, Yuli Khodorkovskiy wrote: > On Fri, Sep 6, 2019 at 10:40 AM Stephen Frost wrote: >> There are actual reasons why the 'DELETE' privilege is *not* the same as >> 'TRUNCATE' in PostgreSQL and I'm really not convinced that we should >> just be tossing that distinction out the

Re: RFC: seccomp-bpf support

2019-08-29 Thread Joe Conway
On 8/29/19 10:00 AM, Tom Lane wrote: > Joe Conway writes: >> Clearly Joshua and I disagree, but understand that the consensus is not >> on our side. It is our assessment that PostgreSQL will be subject to >> seccomp willingly or not (e.g., via docker, systemd, etc.) and t

Re: RFC: seccomp-bpf support

2019-08-29 Thread Joe Conway
On 8/28/19 4:07 PM, Peter Eisentraut wrote: > On 2019-08-28 21:38, Joshua Brindle wrote: >> I think we need to reign in the thread somewhat. The feature allows >> end users to define some sandboxing within PG. Nothing is being forced >> on anyone > > Features come with a maintenance cost. If we

Re: RFC: seccomp-bpf support

2019-08-28 Thread Joe Conway
On 8/28/19 12:47 PM, David Fetter wrote: > On Wed, Aug 28, 2019 at 11:13:27AM -0400, Joe Conway wrote: >> SECCOMP ("SECure COMPuting with filters") is a Linux kernel syscall >> filtering mechanism which allows reduction of the kernel attack surface >> by preven

Re: RFC: seccomp-bpf support

2019-08-28 Thread Joe Conway
On 8/28/19 1:03 PM, Peter Eisentraut wrote: > On 2019-08-28 17:13, Joe Conway wrote: >> * systemd does not implement seccomp filters by default. Packagers may >> decide to do so, but there is no guarantee. Adding them post install >> potentially requires cooperation by g

RFC: seccomp-bpf support

2019-08-28 Thread Joe Conway
SECCOMP ("SECure COMPuting with filters") is a Linux kernel syscall filtering mechanism which allows reduction of the kernel attack surface by preventing (or at least audit logging) normally unused syscalls. Quoting from this link: https://www.kernel.org/doc/Documentation/prctl/seccomp_filter.txt

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-26 Thread Joe Conway
On 8/26/19 2:53 AM, Masahiko Sawada wrote: > I guess that this depends on the number of encryption keys we use. If > we have encryption keys per tablespace or database the number of keys > would be at most several dozen or several hundred. It's enough to have > them in flat-file format on the disk

<    1   2   3   4   5   >