Re: pg_ls_tmpdir()

2018-09-26 Thread Laurenz Albe
Bossart, Nathan wrote: > Attached is a patch to add a pg_ls_tmpdir() function that lists the > contents of a specified tablespace's pgsql_tmp directory. This is > very similar to the existing pg_ls_logdir() and pg_ls_waldir() > functions. > > By providing more visibility

Re: pg_ls_tmpdir()

2018-10-01 Thread Andres Freund
Hi, On 2018-09-26 22:36:03 +0200, Laurenz Albe wrote: > 2. There already are functions "pg_ls_logdir" and "pg_ls_waldir", >and I can imagine new requests, e.g. pg_ls_datafiles() to list the >data files in a database directory. > >It may make sense to have a generic function like > >

Re: pg_ls_tmpdir()

2018-10-02 Thread Christoph Berg
Re: Bossart, Nathan 2018-10-01 <69fd7e51-2b13-41fd-9438-17395c73f...@amazon.com> > > 1. Do we really need two functions, one without input argument > >to list the default tablespace? > >I think that anybody who uses with such a function whould > >be able to feed the OID of "pg_default".

Re: pg_ls_tmpdir()

2018-10-02 Thread Laurenz Albe
Christoph Berg wrote: > Re: Bossart, Nathan 2018-10-01 > <69fd7e51-2b13-41fd-9438-17395c73f...@amazon.com> > > > 1. Do we really need two functions, one without input argument > > > to list the default tablespace? > > > I think that anybody who uses with such a function whould > > > be

Re: pg_ls_tmpdir()

2018-10-02 Thread Andrew Dunstan
On 10/02/2018 08:00 AM, Laurenz Albe wrote: Christoph Berg wrote: Re: Bossart, Nathan 2018-10-01 <69fd7e51-2b13-41fd-9438-17395c73f...@amazon.com> 1. Do we really need two functions, one without input argument to list the default tablespace? I think that anybody who uses with such

Re: pg_ls_tmpdir()

2018-10-02 Thread Bossart, Nathan
during bootstrap. >> At least I failed in the attempt. > > > See the bottom of src/backend/catalog/system_views.sql starting around > line 1010. AFAICT the cleanest way to do this in system_views.sql is to hard-code the pg_default tablespace OID in the DEFAULT expression. So, it might be best to use the two function approach if we want pg_ls_tmpdir() to default to the pg_default tablespace. Nathan

Re: pg_ls_tmpdir()

2018-10-02 Thread Tom Lane
tablespace OID in the DEFAULT expression. So, it might > be best to use the two function approach if we want pg_ls_tmpdir() to > default to the pg_default tablespace. That would be pretty bletcherous, so +1 for just making two C functions. regards, tom lane

Re: pg_ls_tmpdir()

2018-10-04 Thread Laurenz Albe
Bossart, Nathan wrote: > >> AFAICT the cleanest way to do this in system_views.sql is to hard-code > >> the pg_default tablespace OID in the DEFAULT expression. So, it might > >> be best to use the two function approach if we want pg_ls_tmpdir() to > >>

Re: pg_ls_tmpdir()

2018-10-04 Thread Michael Paquier
On Thu, Oct 04, 2018 at 02:23:34PM +0200, Laurenz Albe wrote: > Bossart, Nathan wrote: >> Alright, here's an updated patch. > > Looks, good; marking as "ready for committer". Like Tom, I think it is less dirty to use the two-function approach. Committed this way with a catalog version bump. -- Mi

Re: pg_ls_tmpdir()

2018-10-04 Thread Bossart, Nathan
On 10/4/18, 7:31 PM, "Michael Paquier" wrote: > Committed this way with a catalog version bump. Thanks! Nathan

pg_ls_tmpdir to show shared filesets

2019-12-12 Thread Justin Pryzby
On Thu, Dec 12, 2019, Justin Pryzby wrote in 20191212180506.gr2...@telsasoft.com: > Actually, I tried using pg_ls_tmpdir(), but it unconditionally masks > non-regular files and thus shared filesets. Maybe that's worth discussion on > a > new thread ? > > src/back

Re: pg_ls_tmpdir to show shared filesets

2019-12-14 Thread Justin Pryzby
On Thu, Dec 12, 2019 at 11:39:31PM -0600, Justin Pryzby wrote: > I suggested that pg_ls_tmpdir should show shared filesets like > > 169347 5492 -rw-r- 1 postgres postgres 5619712 Dec 7 01:35 > > /var/lib/pgsql/12/data/base/pgsql_tmp/pgsql_tmp11025.0.sharedfileset/0.0 ..

Re: [PATCH v1] pg_ls_tmpdir to show directories

2020-03-03 Thread David Steele
- .foo | 4096 | 2020-01-16 08:57:04-05 | t In my mind, pg_ls_tmpdir() is for showing tmpfiles, not just a shortcut to SELECT pg_ls_dir((SELECT 'base/pgsql_tmp'p)); -- or, for all tablespaces: WITH x AS (SELECT format('/PG_%s_%s', split_part(current_setting('

Re: [PATCH v1] pg_ls_tmpdir to show directories

2020-03-03 Thread Justin Pryzby
base/pgsql_tmp'p)p, pg_ls_dir(p)name, pg_stat_file(p||'/'||name)s; > > name | size | modification | isdir > >--+--++--- > > .foo | 4096 | 2020-01-16 08:57:04-05 | t > > > >In my mind, pg_ls_tmpdir() is for showing tmpfiles,

Re: [PATCH v1] pg_ls_tmpdir to show directories

2020-03-03 Thread Alvaro Herrera
On 2020-Mar-03, Justin Pryzby wrote: > But I don't think it makes sense to go through more implementation/review > cycles without some agreement from a larger group regarding the > desired/intended interface. Should there be a column for "parent dir" ? Or a > column for "is_dir" ? Should dirs b

Re: [PATCH v1] pg_ls_tmpdir to show directories

2020-03-03 Thread Bossart, Nathan
On 3/3/20, 12:24 PM, "Alvaro Herrera" wrote: > IMO: is_dir should be there (and subdirs should be listed), but > parent_dir should not appear. Also, the "path" should show the complete > pathname, including containing dirs, starting from whatever the "root" > is for the operation. > > So for the

Re: [PATCH v1] pg_ls_tmpdir to show directories

2019-12-27 Thread Justin Pryzby
Re-added -hackers. Thanks for reviewing. On Fri, Dec 27, 2019 at 05:22:47PM +0100, Fabien COELHO wrote: > The implementation simply extends an existing functions with a boolean to > allow for sub-directories. However, the function does not seem to show > subdir contents recursively. Should it be

Re: [PATCH v1] pg_ls_tmpdir to show directories

2019-12-27 Thread Fabien COELHO
Re-added -hackers. Indeed, I left it out by accident. I tried to bounce the original mail but it did not work. Thanks for reviewing. On Fri, Dec 27, 2019 at 05:22:47PM +0100, Fabien COELHO wrote: The implementation simply extends an existing functions with a boolean to allow for sub-dir

Re: [PATCH v1] pg_ls_tmpdir to show directories

2019-12-27 Thread Justin Pryzby
On Fri, Dec 27, 2019 at 06:50:24PM +0100, Fabien COELHO wrote: > >On Fri, Dec 27, 2019 at 05:22:47PM +0100, Fabien COELHO wrote: > >>The implementation simply extends an existing functions with a boolean to > >>allow for sub-directories. However, the function does not seem to show > >>subdir conten

Re: [PATCH v1] pg_ls_tmpdir to show directories

2019-12-27 Thread Fabien COELHO
Hello Justin, Why not simply showing the files underneath their directories? /path/to/tmp/file1 /path/to/tmp/subdir1/file2 In which case probably showing the directory itself is not useful, and the is_dir column could be dropped? The names are expected to look like this: $ sudo find /v

Re: [PATCH v1] pg_ls_tmpdir to show directories

2019-12-28 Thread Justin Pryzby
quot;, dir))); + errmsg("could not stat file \"%s\": %m", path))); /* Ignore anything but regular files */ if (!S_ISREG(attrib.st_mode)) -- 2.7.4 >From fff91aec87f635755527e91aebb7554fa6385fec Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sat, 14 Dec 2019 16

Re: [PATCH v1] pg_ls_tmpdir to show directories

2019-12-28 Thread Fabien COELHO
Hello Justin, Ok, so this suggests recursing into subdirs, which requires to make a separate function of the inner loop. Yea, it suggests that; but, SRF_RETURN_NEXT doesn't make that very easy. It'd need to accept the fcinfo argument, and pg_ls_dir_files would call it once for every tuple to

Re: [PATCH v1] pg_ls_tmpdir to show directories

2019-12-28 Thread Justin Pryzby
n Pryzby Date: Fri, 27 Dec 2019 23:34:14 -0600 Subject: [PATCH v4 1/2] BUG: in errmsg Note there's two changes here. Should backpatch to v12, where pg_ls_tmpdir was added. --- src/backend/utils/adt/genfile.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/backend/utils

Re: [PATCH v1] pg_ls_tmpdir to show directories

2020-01-15 Thread Fabien COELHO
Hello Justin, About this v4. It applies cleanly. I'm trying to think about how to get rid of the strange structure and hacks, and the arbitrary looking size 2 array. Also the recursion is one step, but I'm not sure why, ISTM it could/should go on always? Maybe the recursive implementati

Re: [PATCH v1] pg_ls_tmpdir to show directories

2020-01-15 Thread Justin Pryzby
On Wed, Jan 15, 2020 at 11:21:36AM +0100, Fabien COELHO wrote: > I'm trying to think about how to get rid of the strange structure and hacks, > and the arbitrary looking size 2 array. > > Also the recursion is one step, but I'm not sure why, ISTM it could/should > go on always? Because tmpfiles o

Re: [PATCH v1] pg_ls_tmpdir to show directories

2020-01-16 Thread Fabien COELHO
Hello Justin, I'm trying to think about how to get rid of the strange structure and hacks, and the arbitrary looking size 2 array. Also the recursion is one step, but I'm not sure why, ISTM it could/should go on always? Because tmpfiles only go one level deep. I'm not sure it is a general

Re: [PATCH v1] pg_ls_tmpdir to show directories

2020-01-16 Thread Justin Pryzby
an do this: postgres=# SELECT name, s.size, s.modification, s.isdir FROM (SELECT 'base/pgsql_tmp'p)p, pg_ls_dir(p)name, pg_stat_file(p||'/'||name)s; name | size | modification | isdir --+--++--- .foo | 4096 | 2020-01-16 08:57:04-

Re: pg_ls_tmpdir(); AND, Function for listing archive_status directory

2018-10-01 Thread Justin Pryzby
On Wed, Sep 26, 2018 at 10:36:03PM +0200, Laurenz Albe wrote: > Bossart, Nathan wrote: > > Attached is a patch to add a pg_ls_tmpdir() function that lists the > > contents of a specified tablespace's pgsql_tmp directory. This is > > very similar to the existing pg_ls

Re: pg_ls_tmpdir to show directories and shared filesets

2020-03-05 Thread Justin Pryzby
urn type. Or if they should show hidden or special files, which doesn't require breaking the return. But until then I am to leave the behavior alone. -- Justin >From a5b9a03445d1c768662cafebd8ab3bd7a62890aa Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Fri, 27 Dec 2019 23:34:14 -0600 Subject: [PATCH v6 1/4] BUG

Re: pg_ls_tmpdir to show directories and shared filesets

2020-03-06 Thread Justin Pryzby
em - that was my original complaint. I propose to *also* change at least tmpdir and logdir to show dirs, but don't decend. I left waldir alone for now. Since v12 ls_tmpdir and since v10 logdir and waldir exclude dirs, I think we should backpatch documentation to say so. ISTM pg_ls_

Re: pg_ls_tmpdir to show directories and shared filesets

2020-03-07 Thread Fabien COELHO
Hello Justin, Some feedback about the v7 patch set. About v7.1, seems ok. About v7.2 & v7.3 seems ok, altought the two could be merged. About v7.4: The documentation sentences could probably be improved "for for", "used ... used". Maybe: For the temporary directory for tablespace, ...

Re: pg_ls_tmpdir to show directories and shared filesets

2020-03-07 Thread Justin Pryzby
ercize pg_ls_dir_files at least once.. My previous version had a bug with ignore_missing with pg_ls_tmpdir, which would've been caught by a test like: SELECT FROM pg_ls_tmpdir() WHERE name='Does not exist'; -- Never true, so the function runs to completion but returns zero rows. The 000

Re: pg_ls_tmpdir to show directories and shared filesets

2020-03-07 Thread Fabien COELHO
It seems that lists are used as FIFO structures by appending, fetching & deleting last, all of which are O(n). ISTM it would be better to use the head of the list by inserting, getting and deleting first, which are O(1). I think you're referring to linked lists, but pglists are now arrays, O

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-16 Thread Fabien COELHO
Hello Justin, psql> SELECT * FROM pg_ls_dir_recurse('.'); ERROR: could not stat file "./base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-16 Thread Justin Pryzby
on, 9 Mar 2020 22:40:24 -0500 Subject: [PATCH v12 04/11] Add tests exercizing pg_ls_tmpdir.. ..and its backing function pg_ls_dir_files --- src/test/regress/expected/misc_functions.out | 7 +++ src/test/regress/input/tablespace.source | 5 + src/test/regress/output/tablespace.source

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-16 Thread Alvaro Herrera
I pushed 0001 and 0003 (as a single commit). archive_statusdir didn't get here until 12, so your commit message was mistaken. Also, pg10 is slightly different so it didn't apply there, so I left it alone. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-16 Thread Justin Pryzby
0:00 2001 From: Justin Pryzby Date: Mon, 9 Mar 2020 22:40:24 -0500 Subject: [PATCH v13 2/8] Add tests exercising pg_ls_tmpdir.. ..and its backing function pg_ls_dir_files --- src/test/regress/expected/misc_functions.out | 7 +++ src/test/regress/input/tablespace.source | 5 + src/

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-17 Thread Fabien COELHO
About v13, seens as one patch: Function "pg_ls_dir_metadata" documentation suggests a variable number of arguments with brackets, but parameters are really mandatory. postgres=# SELECT pg_ls_dir_metadata('.'); ERROR: function pg_ls_dir_metadata(unknown) does not exist LINE 1: SELECT pg_ls

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-17 Thread Justin Pryzby
ame); - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_METADATA) + { + values[1] = Int64GetDatum((int64) lattrib.st_size); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(lattrib

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-17 Thread Tom Lane
Justin Pryzby writes: > It seems like the only way to make variable number of arguments is is with > multiple entries in pg_proc.dat, one for each "number of" arguments. Is that > right ? Another way to do it is to have one entry, put the full set of arguments into the initial pg_proc.dat data,

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-31 Thread Justin Pryzby
mp; LS_DIR_METADATA) + { + values[1] = Int64GetDatum((int64) attrib.st_size); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_ISDIR) + { +values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); +#ifdef WIN32 +/* Links should have isdir=false */

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-04-12 Thread Fabien COELHO
Hello Justin, About v15, seen as one patch. Patches serie applies cleanly, compiles, "make check" ok. Documentation: - indent documentation text around 80 cols, as done around? - indent SQL example for readability and capitalize keywords (pg_ls_dir_metadata) - "For each file in a direct

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-11-23 Thread Tom Lane
Justin Pryzby writes: >> This patch has been waiting for input from a committer on the approach I've >> taken with the patches since March 10, so I'm planning to set to "Ready" - at >> least ready for senior review. I took a quick look through this. This is just MHO, of course: * I don't think

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-11-23 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Justin Pryzby writes: > >> This patch has been waiting for input from a committer on the approach I've > >> taken with the patches since March 10, so I'm planning to set to "Ready" - > >> at > >> least ready for senior review. > > I took a qui

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-11-23 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> I took a quick look through this. This is just MHO, of course: >> >> * I don't think it's okay to change the existing signatures of >> pg_ls_logdir() et al. > I disagree that we need to stress over this- we pretty routinely chang

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-11-24 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> I took a quick look through this. This is just MHO, of course: > >> > >> * I don't think it's okay to change the existing signatures of > >> pg_ls_logdir() et al. > > > I d

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-11-29 Thread Justin Pryzby
ntinue; + } values[0] = CStringGetTextDatum(de->d_name); - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_METADATA) + { + values[1] = Int64GetDatum((int64)

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-12-04 Thread Tom Lane
Justin Pryzby writes: [ v24-0001-Document-historic-behavior-of-links-to-directori.patch ] The cfbot is unhappy with one of the test cases you added: 6245@@ -259,9 +259,11 @@ 6246 select path, filename, type from pg_ls_dir_metadata('.', true, false, true) where path!~'[0-9]|pg_internal.init|glob

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-12-09 Thread Justin Pryzby
f (flags & LS_DIR_METADATA) + { + values[1] = Int64GetDatum((int64) attrib.st_size); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_ISDIR) + { +values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); +#ifdef WIN32 + /* Links should have is

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-12-23 Thread Justin Pryzby
ir as a convenience > column. I agree it'll now be a bit duplicative but that seems alright. Maybe we should do what Tom said, and leave pg_ls_* unchanged, but also mark them as deprecated in favour of: | pg_ls_dir_metadata(dir), dir={'pg_wal/archive_status', 'log', 

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-12-23 Thread Stephen Frost
Greetings, * Justin Pryzby (pry...@telsasoft.com) wrote: > On Mon, Nov 23, 2020 at 04:14:18PM -0500, Tom Lane wrote: > > * I don't think it's okay to change the existing signatures of > > pg_ls_logdir() et al. Even if you can make an argument that it's > > not too harmful to add more output colum

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-03-15 Thread David Steele
On 12/23/20 2:27 PM, Stephen Frost wrote: * Justin Pryzby (pry...@telsasoft.com) wrote: On Mon, Nov 23, 2020 at 04:14:18PM -0500, Tom Lane wrote: * I don't think it's okay to change the existing signatures of pg_ls_logdir() et al. Even if you can make an argument that it's not too harmful to a

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-04-06 Thread Justin Pryzby
tach the re-arranged patches. Now I also renumbered OIDs following best practice. The first handful of patches address the original issue, and I think could be "ready": $ git log --oneline origin..pg-ls-dir-new |tac ... Document historic behavior of links to directories.. ... Add tests on pg

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-04-08 Thread Justin Pryzby
dir=false */ +if (pgwin32_is_junction(path)) + values[3] = BoolGetDatum(false); +#endif + } + } + memset(nulls, 0, sizeof(nulls)); tuplestore_putvalues(tupstore, tupdesc, values, nulls); @@ -665,14 +683,14 @@ pg_ls_dir_files(FunctionCallInfo fcinfo, const char *

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-07-14 Thread Justin Pryzby
values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_ISDIR) + { +values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); +#ifdef WIN32 +/* Links should have isdir=false */ +if (pgwin32_is_junction(path)) + values[3] = BoolGetDatum(false

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-07-18 Thread Justin Pryzby
64GetDatum((int64) attrib.st_size); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_ISDIR) + { +values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); +#ifdef WIN32 +/* Links should have isdir=false */ +if (pgwin32_is_junction(path))

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-10-28 Thread Justin Pryzby
ef WIN32 + /* Links should have isdir=false */ +if (pgwin32_is_junction(path)) + values[3] = BoolGetDatum(false); +#endif + } + } + memset(nulls, 0, sizeof(nulls)); tuplestore_putvalues(tupstore, tupdesc, values, nulls); @@ -662,14 +680,14 @@ pg_ls_dir_files(FunctionCa

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-11-05 Thread Justin Pryzby
ze); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_ISDIR) + { +values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); +#ifdef WIN32 +/* Links should have isdir=false */ +if (pgwin32_is_junction(path)) + values[3] = BoolGetDatu

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-09-08 Thread Justin Pryzby
On Sat, Jul 18, 2020 at 03:15:32PM -0500, Justin Pryzby wrote: > Still waiting for feedback from a committer. This patch has been waiting for input from a committer on the approach I've taken with the patches since March 10, so I'm planning to set to "Ready" - at least ready for senior review. --

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-10 Thread Fabien COELHO
Hello Justin, I hope to look at it over the week-end. -- Fabien Coelho - CRI, MINES ParisTech

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-12 Thread Fabien COELHO
dded which check that the result columns are configured as required, including error cases. "make check" is ok. OK. # part 04 Add a new "isdir" column to "pg_ls_tmpdir" output. This is a small behavioral change. I'm ok with that, however I must say that I'

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-12 Thread Michael Paquier
On Wed, Mar 09, 2022 at 10:50:45AM -0600, Justin Pryzby wrote: > Rebased over 9e9858389 (Michael may want to look at the tuplestore part?). Are you referring to the contents of 0003 here that changes the semantics of pg_ls_dir_files() regarding its setup call? -- Michael signature.asc Descriptio

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-12 Thread Justin Pryzby
On Sun, Mar 13, 2022 at 09:45:35AM +0900, Michael Paquier wrote: > On Wed, Mar 09, 2022 at 10:50:45AM -0600, Justin Pryzby wrote: > > Rebased over 9e9858389 (Michael may want to look at the tuplestore part?). > > Are you referring to the contents of 0003 here that changes the > semantics of pg_ls_

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-13 Thread Michael Paquier
rom pg_ls_logicalsnapdir() limit 0; +select * from pg_ls_replslotdir('') limit 0; +select * from pg_ls_tmpdir() limit 0; +select * from pg_ls_waldir() limit 0; +select * from pg_stat_file('.') limit 0; The rest of the patch set should be stable AFAIK, there are various steps wh

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-14 Thread Michael Paquier
On Mon, Mar 14, 2022 at 01:53:54PM +0900, Michael Paquier wrote: > +select * from pg_ls_logicalmapdir() limit 0; > +select * from pg_ls_logicalsnapdir() limit 0; > +select * from pg_ls_replslotdir('') limit 0; > +select * from pg_ls_tmpdir() limit 0; > +select * fr

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-14 Thread Justin Pryzby
n it as a recursive CTE, so I rewrote it in C. 129225.1606166...@sss.pgh.pa.us > Hmm. I am not convinced that we need a new set of SQL functions as > presented in 0003 (addition of meta-data in pg_ls_dir()), and > extensions of 0004 (do the same but for pg_ls_tmpdir) and 0005 (same > fo

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-14 Thread Michael Paquier
On Mon, Mar 14, 2022 at 09:37:25PM -0500, Justin Pryzby wrote: > One could argue that most of the pg_ls_* functions aren't needed (including > 1922d7c6e), since the same things are possible with pg_ls_dir() and > pg_stat_file(). > |1922d7c6e Add SQL functions to monitor the directory contents of re

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-21 Thread Andres Freund
Hi, On 2022-03-09 10:50:45 -0600, Justin Pryzby wrote: > Rebased over 9e9858389 (Michael may want to look at the tuplestore part?). Doesn't apply cleanly anymore: http://cfbot.cputube.org/patch_37_2377.log Marked as waiting-on-author. Greetings, Andres Freund

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-22 Thread Michael Paquier
On Mon, Mar 21, 2022 at 06:28:28PM -0700, Andres Freund wrote: > Doesn't apply cleanly anymore: http://cfbot.cputube.org/patch_37_2377.log > > Marked as waiting-on-author. FWIW, per my review the bit of the patch set that I found the most relevant is the addition of a note in the docs of pg_stat_

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-26 Thread Michael Paquier
On Wed, Mar 23, 2022 at 03:17:35PM +0900, Michael Paquier wrote: > FWIW, per my review the bit of the patch set that I found the most > relevant is the addition of a note in the docs of pg_stat_file() about > the case where "filename" is a link, because the code internally uses > stat(). The func

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-28 Thread Justin Pryzby
On Sat, Mar 26, 2022 at 08:23:54PM +0900, Michael Paquier wrote: > On Wed, Mar 23, 2022 at 03:17:35PM +0900, Michael Paquier wrote: > > FWIW, per my review the bit of the patch set that I found the most > > relevant is the addition of a note in the docs of pg_stat_file() about > > the case where "f

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-31 Thread Justin Pryzby
> 20200503024215.gj28...@telsasoft.com > 20191227195918.gf12...@telsasoft.com > 20200116003924.gj26...@telsasoft.com > 20200908195126.gb18...@telsasoft.com Michael said he's not enthusiastic about the patch. But I haven't heard a suggestion about how else to address the iss

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-04-03 Thread Greg Stark
The cfbot is failing testing this patch. It seems... unlikely given the nature of the patch modifying an admin function that doesn't even modify the database that it should be breaking a streaming test. Perhaps the streaming test is using this function in the testing scaffolding? [03:19:29.564] #

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-10-27 Thread Justin Pryzby
On Fri, Dec 13, 2019 at 03:03:47PM +1300, Thomas Munro wrote: > > Actually, I tried using pg_ls_tmpdir(), but it unconditionally masks > > non-regular files and thus shared filesets. Maybe that's worth > > discussion on a new thread ? > > > &

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2024-01-21 Thread Peter Smith
2024-01 Commitfest. Hi, this patch was marked in CF as "Needs Review", but there has been no activity on this thread for 14+ months. Since there seems not much interest, I have changed the status to "Returned with Feedback" [1]. Feel free to propose a stronger use case for the patch and add an en

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-09 Thread Justin Pryzby
rom pg_ls_logicalmapdir() limit 0; + name | size | modification +--+--+-- +(0 rows) + +select * from pg_ls_logicalsnapdir() limit 0; + name | size | modification +--+--+-- +(0 rows) + +select * from pg_ls_replslotdir('') limit 0; + name | size |

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-11-22 Thread Bossart, Nathan
discussion on this point upthread, so perhaps there is a good reason to make a new function. In any case, I like the idea of having pg_ls_dir() use pg_ls_dir_files() internally like the rest of the pg_ls_* functions. 0004: This one changes pg_ls_tmpdir to show directories as well. I think this is

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-11-23 Thread Justin Pryzby
this, but I figure I'd at least > share my thoughts. Thanks for looking. The patches are separate since the early patches are the most necessary, least disputable parts, to allow the possibility of (say) chaging pg_ls_tmpdir() without changing other functions, since pg_ls_tmpdir was w

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-12-23 Thread Fabien COELHO
Hello Justin, It seems that the v31 patch does not apply anymore: postgresql> git apply ~/v31-0001-Document-historic-behavior-of-links-to-directori.patch error: patch failed: doc/src/sgml/func.sgml:27410 error: doc/src/sgml/func.sgml: patch does not apply -- Fabien.

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-12-23 Thread Justin Pryzby
ues[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_ISDIR) + { +values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); +#ifdef WIN32 +/* Links should have isdir=false */ +if (pgwin32_is_junction(path)) + values[3] = BoolGetDatum(false); +#endi

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-01-02 Thread Fabien COELHO
. ? ... : ...; #endif New tests are added which check that the result columns are configured as required, including error cases. "make check" is ok. OK. # part 04 Add a new "isdir" column to "pg_ls_tmpdir" output. This is a small behavioral change. I'm o

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-01-02 Thread Fabien COELHO
Here is my review about v32: I forgot to tell that doc generation for the cumulated changes also works. -- Fabien.

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-01-17 Thread Julien Rouhaud
Hi, On Sun, Jan 02, 2022 at 02:55:04PM +0100, Fabien COELHO wrote: > > > Here is my review about v32: > > I forgot to tell that doc generation for the cumulated changes also works. Unfortunately the patchset doesn't apply anymore: http://cfbot.cputube.org/patch_36_2377.log === Applying patches

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-01-25 Thread Justin Pryzby
p;& b){}" >> I'm going to leave it up to a committer. > The ifdef WIN32 (which probably detects windows 64 bits…) overwrites > values[3]. ISTM > it could be reordered so that there is no overwrite, and simpler single > assignements. > > #ifndef WIN32 >

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2021-07-02 Thread Justin Pryzby
issue, and I think could be > "ready": > > $ git log --oneline origin..pg-ls-dir-new |tac > ... Document historic behavior of links to directories.. > ... Add tests on pg_ls_dir before changing it > ... Add pg_ls_dir_metadata to list a dir with file metadata.. > ...

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-06-23 Thread Justin Pryzby
;p)p, pg_ls_dir(p)name, pg_stat_file(p||'/'||name)s; > name | size | modification | isdir > --+--++--- > .foo | 4096 | 2020-01-16 08:57:04-05 | t > > In my mind, pg_ls_tmpdir() is for showing tmpfiles, not just a shortcut to

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-07 Thread Justin Pryzby
:00:00 2001 From: Justin Pryzby Date: Fri, 27 Dec 2019 23:34:14 -0600 Subject: [PATCH v8 1/6] BUG: in errmsg Note there's two changes here. Should backpatch to v12, where pg_ls_tmpdir was added. --- src/backend/utils/adt/genfile.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-08 Thread Fabien COELHO
Hello Justin, Patch series applies cleanly. The last status compiles and passes "make check". A few more comments: * v8.[123] ok. * v8.4 Avoid using the type name as a field name? "enum dir_action dir_action;" -> "enum dir_action action", or maybe rename "dir_action" enum "dir_action_t".

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-10 Thread Justin Pryzby
dir to show metadata columns including is_dir, but not decend. It's pretty unfortunate if a function called pg_ls_tmpdir hides shared filesets, so maybe it really is best to change that (it's new in v12). I'm interested to in feedback on the alternative approach, as attached. The f

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-13 Thread Justin Pryzby
guments to pg_ls_* functions (including pg_ls_tmpdir but not pg_ls_dir). - maybe add pg_ls_dir_recurse, which satisfies the original need; - retire pg_ls_dir (does this work with tuplestore?) - profit The alternative seems to be to go back to Alvaro's earlier proposal: - not only add &q

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-15 Thread Fabien COELHO
Hello Justin, Some feedback on v10: All patches apply cleanly, one on top of the previous. I really wish there would be less than 9 patches… * v10.1 doc change: ok * v10.2 doc change: ok, not sure why it is not merged with previous * v10.3 test add: could be merge with both previous Tests

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-15 Thread Justin Pryzby
n't* exist until someone creates tmpfiles there. As it mentions: +-- This tests the missing_ok parameter, which causes pg_ls_tmpdir to succeed even if the tmpdir doesn't exist yet > * v10.4 at least, some code! > I'm not sure of the "FLAG_" prefix which seems too

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-16 Thread Fabien COELHO
About v11, ISTM that the recursive function should check for symbolic links and possibly avoid them: sh> cd data/base sh> ln -s .. foo psql> SELECT * FROM pg_ls_dir_recurse('.'); ERROR: could not stat file "./base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/foo/base/

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-16 Thread Justin Pryzby
On Mon, Mar 16, 2020 at 04:20:21PM +0100, Fabien COELHO wrote: > > About v11, ISTM that the recursive function should check for symbolic links > and possibly avoid them: > > sh> cd data/base > sh> ln -s .. foo > > psql> SELECT * FROM pg_ls_dir_recurse('.'); > ERROR: could not stat file > "

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-05-02 Thread Justin Pryzby
etadata of an arbitrary dir; | - add isdir arguments to pg_ls_* functions (including pg_ls_tmpdir but not |pg_ls_dir). | - maybe add pg_ls_dir_recurse, which satisfies the original need; | - retire pg_ls_dir (does this work with tuplestore?) | - profit -- Justin >From 6175cecd312296bbb9099834

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-05-07 Thread Justin Pryzby
= Int64GetDatum((int64) attrib.st_size); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_ISDIR) + { +values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); +#ifdef WIN32 +/* Links should have isdir=false */ + if (pgwin32_is_junction(pa

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-05-25 Thread Justin Pryzby
A) + { + values[1] = Int64GetDatum((int64) attrib.st_size); + values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_ISDIR) + { +values[3] = BoolGetDatum(S_ISDIR(attrib.st_mode)); +#ifdef WIN32 +/* Links should have isdir=false */ + if (pgwin

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-06-07 Thread Fabien COELHO
Hello Justin, Rebased onto 7b48f1b490978a8abca61e9a9380f8de2a56f266 and renumbered OIDs. Some feedback about v18, seen as one patch. Patch applies cleanly & compiles. "make check" is okay. pg_stat_file() and pg_stat_dir_files() now return a char type, as well as the function which call th

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-06-21 Thread Justin Pryzby
etTextDatum(de->d_name); - values[1] = Int64GetDatum((int64) attrib.st_size); - values[2] = TimestampTzGetDatum(time_t_to_timestamptz(attrib.st_mtime)); + if (flags & LS_DIR_METADATA) + { + values[1] = Int64GetDatum((int64) attrib.st_size); + values[2] = TimestampTzGetDatum(time_t_to_ti