[PATCH] initdb: do not exit after warn_on_mount_point

2022-09-10 Thread andrey . arapov
Hello,

please find my first patch for PostgreSQL is attached.
Kind regards,
Andrey Arapov


0001-initdb-do-not-exit-after-warn_on_mount_point.patch
Description: Binary data


Re: [PATCH] initdb: do not exit after warn_on_mount_point

2022-09-11 Thread andrey . arapov
Hi Tom,

I've updated the patch by adding the explanation behind and more comments. 
(please see the attachment)

Have slightly improved the logic so that it does not report an error
"directory \"%s\" exists but is not empty"
when it is only supposed to warn the user about the mountpoint, without exiting.

To me, my patch looks like a typo fix where exit(1) should not be called on the 
warn_on_mount_point(),
but only warn and continue as more people are mounting the device at 
`/var/lib/postgresql/data` (PGDATA) in the containerized world (K8s 
deployments, especially now in the Akash Network I am working for) for making 
sure their data persist.

As a workaround, we either have to `rmdir /var/lib/postgresql/data/lost+found` 
before running `docker-entrypoint.sh postgres` which in turn calls the 
`initdb`, or, alternatively we have to pass 
`PGDATA=/var/lib/postgresql/data/` while mounting persistent storage 
over `/var/lib/postgresql/data` path so that it won't exit on the very first 
run.
To me, this in itself is an odd behavior, which led me to finding this typo 
(from my point of view) to which I've made this patch.


Please let me know if it makes sense or requires more information / explanation.


Kind regards,
Andrey Arapov


September 10, 2022 5:10 PM, "Tom Lane"  wrote:

> andrey.ara...@nixaid.com writes:
> 
>> please find my first patch for PostgreSQL is attached.
> 
> You haven't explained why you think this would be a good
> change, or even a safe one.
> 
> regards, tom lane


0001-initdb-do-not-exit-when-PGDATA-PGDATA-pg_wal-is-a-mo.patch
Description: Binary data


Re: [PATCH] initdb: do not exit after warn_on_mount_point

2022-09-12 Thread andrey . arapov
September 11, 2022 12:18 PM, "Julien Rouhaud"  wrote:

> Hi,
> 
> On Sat, Sep 10, 2022 at 07:14:59PM +, andrey.ara...@nixaid.com wrote:
> 
>> Have slightly improved the logic so that it does not report an error
>> "directory \"%s\" exists but is not empty"
>> when it is only supposed to warn the user about the mountpoint, without
>> exiting.
>> 
>> To me, my patch looks like a typo fix where exit(1) should not be called on
>> the warn_on_mount_point(), but only warn and continue as more people are
>> mounting the device at `/var/lib/postgresql/data` (PGDATA) in the
>> containerized world (K8s deployments, especially now in the Akash Network I
>> am working for) for making sure their data persist.
> 
> This definitely isn't a typo, as it's really strongly discouraged to use a
> mount point for the data directory. You can refer to this thread [1] for more
> explanations.
> 
> [1]
> https://www.postgresql.org/message-id/flat/CAKoxK+6H40imynM5P31bf0DnpN-5f5zeROjcaj6BKVAjxdEA9w@mail.
> mail.com


I've read the "why not using a mountpoint as PGDATA?" thread [1] as well as 
Bugzilla "postgresql-setup fails when /var/lib/pgsql/data is mount point" thead 
[2] but could not find any good reason why not to mount the PGDATA directly,
except probably for the NFS mount point, but who does that anyway?
And using NFS for PostgreSQL PGDATA is a way for finding problems starting with 
poor performance ending up with corrupted DB. lol

The only point that hooked my attention was the pg_upgrade, but then I've tried 
pg_upgrade'ing postgresql:13 to postgresql:14, everything went without issues. 
I wrote a step-by-step doc for that purpose [3].

That leaves me unconvinced on why would `initdb` quit when detecting PGDATA 
being a mountpoint.

Everyone using containerized postgresql image cannot use /var/lib/postgresql as 
the mountpoint but has to use /var/lib/postgresql/data instead due to this 
issue [4] due to [5].
Hence, everyone using containerized version of postgresql with the device (say 
Ceph's RBD) mounted over /var/lib/postgresql/data directory has to either 
specify:

- PGDATA=/var/lib/postgresql/data/

OR

make sure to remove $PGDATA/lost+found directory.

Both of these hacks are only for the initdb to fail detect the mountpoint 
which, on its own, is supposed to be only a warning which is seen from the 
function name warn_on_mount_point() and its messages [6] look to be of only 
advisory nature to me.

I would understand initdb to exit if it detects the mountpoint being is NFS, 
otherwise, there is no reason for not letting one use the mountpoint over 
PGDATA as neither it breaks the pg_upgrade functionality as shown in this 
message.


[1] 
https://www.postgresql.org/message-id/flat/cakoxk+6h40imynm5p31bf0dnpn-5f5zerojcaj6bkvajxde...@mail.gmail.com
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1247477
[3] https://gist.github.com/andy108369/aa3bf1707054542f2fa944f6d39aef64
[4] https://github.com/docker-library/postgres/issues/404#issuecomment-773755801
[5] 
https://github.com/docker-library/postgres/blob/3c20b7bdb915ecb1648fb468ab53080c58bb1716/Dockerfile-debian.template#L184
[6] 
https://github.com/postgres/postgres/blob/7fed801135bae14d63b11ee4a10f6083767046d8/src/bin/initdb/initdb.c#L2616-L2626


Kind regards,
Andrey Arapov




Re: [PATCH] initdb: do not exit after warn_on_mount_point

2022-09-12 Thread andrey . arapov
filesystems without 
`lost+found` directory.


[1] https://bugzilla.redhat.com/show_bug.cgi?id=1247477#c1
[2] 
https://github.com/docker-library/postgres/blob/74e51d102aede/docker-entrypoint.sh#L317
[3] 
https://github.com/postgres/postgres/blob/REL_14_5/contrib/start-scripts/linux#L94
[4] 
https://github.com/postgres/postgres/commit/17f15239325a88581bb4f9cf91d38005f1f52d69


Kind regards,
Andrey Arapov