Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-12 Thread Thomas Munro
On Thu, Sep 12, 2024 at 8:54 PM Pecsök Ján wrote: > In link you provided there is mention, that in PostgreSQL 16 data is not being > compressed for PostgreSQL 16 server. Does it mean, that PosgreSQL 16 use much > more space while computing queries? > If that is the case, it can be our problem, be

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-11 Thread Thomas Munro
I don't understand what ENOSPC has to do with the file descriptor limits, but this person reported: # touch test touch: cannot touch ‘test’: No space left on device https://serverfault.com/questions/746032/rsync-and-scp-failing-with-no-space-left-on-xfs-device ... with plenty of free space, and

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-11 Thread Thomas Munro
On Thu, Sep 12, 2024 at 12:39 AM Alvaro Herrera wrote: >> On 2024-Sep-11, Pecsök Ján wrote: > > In our case: > > Kernel: Linux version 4.18.0-513.18.1.el8_9.ppc64le > > (mockbu...@ppc-hv-13.build.eng.rdu2.redhat.com) (gcc version 8.5.0 20210514 > > (Red Hat 8.5.0-20) (GCC)) #1 SMP Thu Feb 1 02:5

Re: Error:could not extend file " with FileFallocate(): No space left on device

2024-09-11 Thread Thomas Munro
On Wed, Sep 11, 2024 at 9:56 PM Alvaro Herrera wrote: > On 2024-Sep-10, Pecsök Ján wrote: > > After upgrade of Posgres from version 13.5 to 16.2 we experience following > > error: > > could not extend file "pg_tblspc/16401/PG_16_202307071/17820/3968302971" > > with FileFallocate(): No space left

Re: Windows installation problem at post-install step

2024-08-07 Thread Thomas Munro
Thanks. The log didn't offer any more clues, and my colleague David R has Windows and knows how to work its debugger so we sat down together and chased this down (thanks David!). 1. It is indeed calling abort(), but it's not a PANIC or Assert() in PostgreSQL, it's an assertion inside Windows' ow

Re: Windows installation problem at post-install step

2024-08-06 Thread Thomas Munro
On Tue, Aug 6, 2024 at 11:44 PM Peter J. Holzer wrote: > I assume that "1254" here is the code page. > But you specified --encoding=UTF-8 above, so your default locale uses a > different encoding than the template databases. I would expect that to > cause problems if the template databases contain

Re: Windows installation problem at post-install step

2024-08-06 Thread Thomas Munro
On Tue, Aug 6, 2024 at 10:38 PM Sandeep Thakkar wrote: > On Tue, Aug 6, 2024 at 4:06 PM Sandeep Thakkar > wrote: [v15] >>> XXX debug raw: setup_locale_encoding = "Turkish_Türkiye.1254" >>> XXX debug hex: setup_locale_encoding = { 54 75 72 6b 69 73 68 5f 54 fc 72 >>> 6b 69 79 65 2e 31 32 35

Re: Windows installation problem at post-install step

2024-08-05 Thread Thomas Munro
less confusing than looking at script output via email (I don't even know how many onion layers of transcoding are involved...) From b97fe5a55e50a447d41a439412922ffe3f7e168b Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Tue, 6 Aug 2024 16:06:29 +1200 Subject: [PATCH 1/3] xxx debug --- src/

Re: Windows installation problem at post-install step

2024-07-22 Thread Thomas Munro
On Mon, Jul 22, 2024 at 11:51 PM Sandeep Thakkar wrote: > EDB's windows installer gets the locales on the system using the > https://github.com/EnterpriseDB/edb-installers/blob/REL-16/server/scripts/windows/getlocales/getlocales.cpp > and then substitute some patterns > (https://github.com/Ente

Re: Windows installation problem at post-install step

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 11:58 AM Ertan Küçükoglu wrote: > Thomas Munro , 21 Tem 2024 Paz, 23:27 tarihinde şunu > yazdı: >> 2. Some existing database clusters which had been installed with the >> name "Turkish_Turkey.1254" became unstartable when the OS upgra

Re: Windows installation problem at post-install step

2024-07-21 Thread Thomas Munro
On Mon, Jul 22, 2024 at 7:29 AM Adrian Klaver wrote: > On 7/21/24 12:00, Ertan Küçükoglu wrote: > > My main purpose was and still is to reach EDB people using the forum and > > let them know about the problem. > > I believe it is something to be fixed for future installations. I would > > like to

Re: libpq v17 PQsocketPoll timeout is not granular enough

2024-06-10 Thread Thomas Munro
On Tue, Jun 11, 2024 at 2:36 AM Dominique Devienne wrote: > Hi. I've noticed [that libpq API in v17 beta1][1], and wanted to use > it to replace an existing Boost.ASIO-based async polling of the > connection's socket, waiting for notifications. The use case being > using PostgreSQL LISTEN/NOTIFY f

Re: Preallocation changes in Postgresql 16

2024-04-25 Thread Thomas Munro
On Fri, Apr 26, 2024 at 4:37 AM Riku Iki wrote: > I am wondering if there were preallocation related changes in PG16, and if it > is possible to disable preallocation in PostgreSQL 16? I have no opinion on the btrfs details, but I was wondering if someone might show up with a system that doesn't

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-04-03 Thread Thomas Munro
On Sat, Mar 23, 2024 at 3:01 AM Nick Renders wrote: > We now have a second machine with this issue: it is an Intel Mac mini running > macOS Sonoma (14.4) and PostgreSQL 16.2. > This one only has a single Data directory, so there are no multiple instances > running. BTW if you're running databas

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-04-03 Thread Thomas Munro
On Thu, Apr 4, 2024 at 3:11 AM Nick Renders wrote: > In the macOS Finder, when you show the Info (command+i) for an external drive > (or any partition that is not the boot drive), there is a checkbox "Ignore > ownership on this volume" in the Permissions section. I think it is by > default "on"

Re: could not open file "global/pg_filenode.map": Operation not permitted

2024-03-28 Thread Thomas Munro
On Fri, Mar 29, 2024 at 4:47 AM Nick Renders wrote: > Looking at the 2 machines that are having this issue (and the others that > don't), I think it is somehow related to the following setup: > - macOS Sonoma (14.4 and 14.4.1) > - data directory on an external drive > > That external drive (a Pro

Re: How do you optimize the disk IO when you cannot assume a file will start at a boundary then?

2024-02-22 Thread Thomas Munro
working on bringing to PostgreSQL in useful form, we had to do this: commit faeedbcefd40bfdf314e048c425b6d9208896d90 Author: Thomas Munro Date: Sat Apr 8 10:38:09 2023 +1200 Introduce PG_IO_ALIGN_SIZE and align all I/O buffers. ... to avoid EINVAL errors, falling back to buffered mode or pa

Re: Query crash with 15.5 on debian bookworm/armv8

2023-12-26 Thread Thomas Munro
On Wed, Dec 27, 2023 at 5:17 AM Clemens Eisserer wrote: > > FWIW, since this crash is inside LLVM you could presumably dodge the bug > > by setting "jit" to off. > > Thanks, this indeed solved the crash. > Just to make sure this crash doesn't have anything to do with my > setup/config (I'd changed

Re: How to generate random bigint

2023-12-20 Thread Thomas Munro
On Thu, Dec 21, 2023 at 7:21 PM Tom Lane wrote: > Phillip Diffley writes: > > Postgres's random() function generates a random double. That can be > > converted to a random int for smaller integers, but a double can't > > represent all of the values in a bigint. Is there a recommended way to > > g

Re: psql crash with custom build on RedHat 7

2023-12-19 Thread Thomas Munro
On Wed, Dec 20, 2023 at 4:41 AM Dominique Devienne wrote: > On Tue, Dec 19, 2023 at 2:02 PM Thomas Munro wrote: >> On Wed, Dec 20, 2023 at 1:39 AM Dominique Devienne >> wrote: >> > Program received signal SIGSEGV, Segmentation fault. >> > 0x004232b8 in

Re: psql crash with custom build on RedHat 7

2023-12-19 Thread Thomas Munro
On Wed, Dec 20, 2023 at 1:39 AM Dominique Devienne wrote: > Program received signal SIGSEGV, Segmentation fault. > 0x004232b8 in slash_yylex () I think this might have to do with flex changing. Does it help if you "make maintainer-clean"?

Re: fsync data directory after DB crash

2023-07-20 Thread Thomas Munro
On Wed, Jul 19, 2023 at 2:09 PM Pandora wrote: > Yes, I saw the usage of syncfs in PG14, but it is recommended to use it on > Linux 5.8 or higher. If my OS version is lower than 5.8, can I still enable > it? Nothing stops you from enabling it, it's fairly ancient and should work. It just doesn

Re: fsync data directory after DB crash

2023-07-18 Thread Thomas Munro
On Wed, Jul 19, 2023 at 12:41 PM Michael Paquier wrote: > On Tue, Jul 18, 2023 at 04:50:25PM +0800, Pandora wrote: > > I found that starting from version 9.5, PostgreSQL will do fsync on > > the entire data directory after DB crash. Here's a question: if I > > have FPW = on, why is this step still

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-08 Thread Thomas Munro
On Tue, May 9, 2023 at 10:04 AM Tom Lane wrote: > Michael Paquier writes: > > One thing I was wondering about to improve the odds of the hits is to > > be more aggressive with the number of relations created at once, so as > > we are much more aggressive with the number of pages extended in > > p

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Thomas Munro
On Mon, May 8, 2023 at 2:24 PM Michael Paquier wrote: > I can reproduce the same backtrace here. That's just my usual laptop > with ext4, so this would be a Postgres bug. First, here are the four > things running in parallel so as I can get a failure in loading a > critical index when connecting

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Thomas Munro
On Mon, May 8, 2023 at 4:10 AM Evgeny Morozov wrote: > On 6/05/2023 11:13 pm, Thomas Munro wrote: > > Would you like to try requesting FILE_COPY for a while and see if it > > eventually happens like that too? > Sure, we can try that. Maybe you could do some one way and some

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Thomas Munro
On Sun, May 7, 2023 at 1:21 PM Tom Lane wrote: > Thomas Munro writes: > > Did you previously run this same workload on versions < 15 and never > > see any problem? 15 gained a new feature CREATE DATABASE ... > > STRATEGY=WAL_LOG, which is also the default. I wond

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Thomas Munro
On Sun, May 7, 2023 at 10:23 AM Jeffrey Walton wrote: > This may be related... I seem to recall the GNUlib folks talking about > a cp bug on sparse files. It looks like it may be fixed in coreutils > release 9.2 (2023-03-20): > https://github.com/coreutils/coreutils/blob/master/NEWS#L233 > > If I

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Thomas Munro
On Sun, May 7, 2023 at 12:29 AM Evgeny Morozov wrote: > On 6/05/2023 12:34 pm, Thomas Munro wrote: > > So it does indeed look like something unknown has replaced 32KB of > > data with 32KB of zeroes underneath us. Are there more non-empty > > files that are all-zeroes? Some

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Thomas Munro
On Sat, May 6, 2023 at 9:58 PM Evgeny Morozov wrote: > Right - I should have realised that! base/1414389/2662 is indeed all > nulls, 32KB of them. I included the file anyway in > https://objective.realityexists.net/temp/pgstuff2.zip OK so it's not just page 0, you have 32KB or 4 pages of all zero

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-05 Thread Thomas Munro
On Fri, May 5, 2023 at 7:50 PM Evgeny Morozov wrote: > The OID of the bad DB ('test_behavior_638186279733138190') is 1414389 and > I've uploaded base/1414389/pg_filenode.map and also base/5/2662 (in case > that's helpful) as https://objective.realityexists.net/temp/pgstuff1.zip Thanks. That pg

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Thomas Munro
On Fri, May 5, 2023 at 11:15 AM Thomas Munro wrote: > What does select > pg_relation_filepath('pg_class_oid_index') show in the corrupted > database, base/5/2662 or something else? Oh, you can't get that far, but perhaps you could share the pg_filenode.map file? Or

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Thomas Munro
On Fri, May 5, 2023 at 11:15 AM Thomas Munro wrote: > Now *that* is a piece of > logic that changed in PostgreSQL 15. It changed from sector-based > atomicity assumptions to a directory entry swizzling trick, in commit > d8cd0c6c95c0120168df93aae095df4e0682a08a. Hmm. I spoke to

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Thomas Munro
On Fri, May 5, 2023 at 6:11 AM Evgeny Morozov wrote: > Meanwhile, what do I do with the existing server, though? Just try to > drop the problematic DBs again manually? That earlier link to a FreeBSD thread is surely about bleeding edge new ZFS stuff that was briefly broken then fixed, being disco

Re: src/test/examples/testlibpq2.c where the HAVE_SYS_SELECT_H is defined.

2023-03-17 Thread Thomas Munro
On Fri, Mar 17, 2023 at 7:48 PM jian he wrote: > Hi, > playing around with $[0] testlibpq2.c example. I wondered where > HAVE_SYS_SELECT_H is defined? > > I searched on the internet, founded that people also asked the same question > in $[1]. > > In my machine, I do have . > system version: Ubun

Re: 13.x, stream replication and locale(?) issues

2023-03-03 Thread Thomas Munro
On Wed, Mar 1, 2023 at 10:30 AM Thomas Munro wrote: > On Wed, Mar 1, 2023 at 12:09 AM Eugene M. Zheganin wrote: > > 3) how do I fix it ? Should I take locale sources for ru_RU.utf8 on Linux > > and compile it on FreeBSD - will it help ? > > Out of curiosity (I'm n

Re: Interval in hours but not in days Leap second not taken into account

2023-02-28 Thread Thomas Munro
On Mon, Feb 27, 2023 at 8:26 PM PALAYRET Jacques wrote: > # PostgreSQL does not take into account the additional second (leap second) > in some calendar days ; eg. 2016, 31 dec. : > SELECT to_timestamp('20170102 10:11:12','mmdd hh24:mi:ss') - > to_timestamp('20161230 00:00:00','mmdd hh24

Re: 13.x, stream replication and locale(?) issues

2023-02-28 Thread Thomas Munro
On Wed, Mar 1, 2023 at 12:09 AM Eugene M. Zheganin wrote: > 3) how do I fix it ? Should I take locale sources for ru_RU.utf8 on Linux and > compile it on FreeBSD - will it help ? Out of curiosity (I'm not saying it's a good idea!), do you know if FreeBSD's localedef can compile glibc's collatio

Re: Getting PSQL in Windows to support TAB/Autocomplete via modified readline...

2022-11-22 Thread Thomas Munro
On Tue, Nov 22, 2022 at 4:25 AM Kirk Wolak wrote: > In researching this problem, it appears that the decision was made like > 17yrs ago, when windows did not have a realistic "terminal" type interface. > Assuming we target Windows 8.1 or higher, I believe this goes away. FWIW PostgreSQL 16 w

Re: PANIC: could not flush dirty data: Cannot allocate memory

2022-11-15 Thread Thomas Munro
On Wed, Nov 16, 2022 at 1:24 AM wrote: > Filesystem is ext4. VM technology is mixed: VMware, KVM and XEN PV. > Kernel is 5.15.0-52-generic. > > We have not seen this with Ubutnu 18.04 and 20.04 (although we might not > have noticed it). > > I guess upgrading to postgresql 13/14/15 does not help as

Re: PANIC: could not flush dirty data: Cannot allocate memory

2022-11-14 Thread Thomas Munro
On Tue, Nov 15, 2022 at 10:54 AM Christoph Moench-Tegeder wrote: > ## klaus.mailingli...@pernau.at (klaus.mailingli...@pernau.at): > > On several servers we see the error message: PANIC: could not flush > > dirty data: Cannot allocate memory > Of these three places, there's an sync_file_range(),

Re: Segmentation Fault PG 14

2022-11-07 Thread Thomas Munro
On Tue, Nov 8, 2022 at 11:45 AM Willian Colognesi wrote: > root@ip-10-x-x-x:/home/ubuntu# pg_config --configure > ... --with-extra-version= (Ubuntu 14.5-2.pgdg20.04+2)' ... > ... '--with-llvm' 'LLVM_CONFIG=/usr/bin/llvm-config-10' ... > There is no llvm installed on ubuntu server, postgresql was

Re: Strange collation names ("hu_HU.UTF-8")

2022-08-02 Thread Thomas Munro
On Wed, Aug 3, 2022 at 1:43 AM Tom Lane wrote: > I believe most if not all variants of Unix are > permissive about the spelling of the encoding part. I've only seen glibc doing that downcase-and-strip-hyphens thing to the codeset part of a locale name when looking for locale definition files. Ot

Re: AIX and EAGAIN on open()

2022-07-03 Thread Thomas Munro
On Mon, Jun 20, 2022 at 9:53 PM Christoph Berg wrote: > IBM's reply to the issue back in December 2020 was this: > > The man page / infocenter document is not intended as an exhaustive > list of all possible error codes returned and their circumstances. > "Resource temporarily unavailable" m

Re: Order of rows in statement triggers NEW/OLD tables

2022-05-31 Thread Thomas Munro
On Fri, May 6, 2022 at 6:20 PM hubert depesz lubaczewski wrote: > when defining statement triggers on update I can use: > > REFERENCING OLD TABLE AS xxx NEW TABLE as YYY > > these "pseudo" tables contain rows that were before and after. > > Is the order guaranteed? > > Can I assume that "first" ro

Re: Improve configurability for IO related behavoir

2022-05-28 Thread Thomas Munro
On Sun, May 29, 2022 at 4:29 AM 浩辰 何 wrote: > Furthermore, the results above are also related to IO API supported by OS. > MySQL support synchronized IO and Linux libaio. It seems > that PostgreSQL only supports synchronized IO, so shall we support more IO > engines? like io_uring which is very

Re: Pg14 possible index corruption after reindex concurrently

2022-05-24 Thread Thomas Munro
On Wed, May 25, 2022 at 6:17 AM Aleš Zelený wrote: > SELECT format('REINDEX SCHEMA CONCURRENTLY %I;', n.nspname) This may be related to bug #17485, discussed at: https://www.postgresql.org/message-id/flat/17485-396609c6925b982d%40postgresql.org

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Thomas Munro
On Sun, Apr 24, 2022 at 8:00 AM Peter wrote: > More than a million files in a single directory, this is > inacceptable. You didn't show EXPLAIN (ANALYZE) but if [Parallel] Hash is making insane numbers of temporary files then something is not working as intended... and I can take a guess at what

Re: frequency of positive and negative numbers, sizes of numbers and frequency of alteration of polarity

2022-02-17 Thread Thomas Munro
On Fri, Feb 18, 2022 at 10:42 AM Shaozhong SHI wrote: > Given 2 or more such columns, is there any measure that can be calculated to > tell which one alternates more than others? Well, you could report non-flips as NULL and flips as magnitude, and then wrap that query in another query to compute

Re: frequency of positive and negative numbers, sizes of numbers and frequency of alteration of polarity

2022-02-17 Thread Thomas Munro
On Fri, Feb 18, 2022 at 9:11 AM Shaozhong SHI wrote: > How to calculate frequency of positive and negative numbers and define and > calculate frequency of alteration of polarity? > > Surely, we can use frequency of alteration of polarity and level of change > (e.g., size of positive and negative

Re: Compile 14.1 in EL5.8

2022-02-09 Thread Thomas Munro
On Thu, Feb 10, 2022 at 2:23 PM Tom Lane wrote: > Thomas Munro writes: > > ... I wondered about also removing the leftover comment > > "We assume that any system that has Linux epoll() also has Linux > > signalfd()" which was my attempt to explain that there wasn&#

Re: Compile 14.1 in EL5.8

2022-02-09 Thread Thomas Munro
On Thu, Feb 10, 2022 at 3:38 AM Tom Lane wrote: > Gabriela Serventi writes: > > Hi! I'm trying to compile release 14.1 in a very old Linux Server (Red Hat > > Enterprise Linux Server release 5.8). I can run configure successfully, but > > when I try to run make, I get the following error: > > lat

Re: sort order for UTF-8 char column with Japanese UTF-8

2022-02-03 Thread Thomas Munro
On Fri, Feb 4, 2022 at 8:11 AM Matthias Apitz wrote: > On my FreeBSD laptop the same file sorts as > > guru@c720-r368166:~ $ LANG=de_DE.UTF-8 sort swd > A > ゲアハルト・A・リッター > ゲルハルト・A・リッター > チャールズ・A・ビアード > A010STRUKTUR > A010STRUKTUR > A010STRUKTUR > A0150SUPRALEITER Wow, so it's one thing to have a

Re: could not accept SSL connection: Success

2022-01-19 Thread Thomas Munro
On Thu, Jan 20, 2022 at 12:06 AM Carla Iriberri wrote: > On Wed, Jan 19, 2022 at 5:42 AM Michael Paquier wrote: >> "On an unexpected EOF, versions before OpenSSL 3.0 returned >> SSL_ERROR_SYSCALL, nothing was added to the error stack, and errno was >> 0. Since OpenSSL 3.0 the returned error is SS

Re: create database hangs forever on WSL - autovacuum deadlock?

2022-01-09 Thread Thomas Munro
On Sun, Jan 9, 2022 at 2:15 PM Alicja Kucharczyk wrote: > sob., 8 sty 2022 o 22:40 Thomas Munro napisał(a): >> On Sat, Jan 8, 2022 at 8:11 PM Alicja Kucharczyk >> wrote: >> > this is postgres running on windows wsl: PostgreSQL 14.1 (Ubuntu >> > 14.1-1.pg

Re: create database hangs forever on WSL - autovacuum deadlock?

2022-01-08 Thread Thomas Munro
On Sat, Jan 8, 2022 at 8:11 PM Alicja Kucharczyk wrote: > this is postgres running on windows wsl: PostgreSQL 14.1 (Ubuntu > 14.1-1.pgdg18.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu > 7.5.0-3ubuntu1~18.04) 7.5.0, 64-bit Is this WSL1 (some kind of Linux system call emulator running o

Re: Need to know more about pg_test_fsync utility

2021-12-12 Thread Thomas Munro
On Mon, Dec 13, 2021 at 3:04 PM PGSQL DBA wrote: > As you mentioned in question-8, "I'd investigate whether data is being cached > unexpectedly, perhaps indicating that committed transactions be lost in a > system crash event." So, I would like to know that if we configure the disk > for the WA

Re: Need to know more about pg_test_fsync utility

2021-12-09 Thread Thomas Munro
On Fri, Dec 10, 2021 at 3:20 PM PGSQL DBA wrote: > 1) How to interpret the output of pg_test_fsync? The main interesting area is probably the top section that compares the different wal_sync_method settings. For example, it's useful to verify the claim that fdatasync() is faster than fsync() (be

Re: Wrong sorting on docker image

2021-10-16 Thread Thomas Munro
On Sun, Oct 17, 2021 at 4:42 AM Tom Lane wrote: > Speaking of ICU, if you are using an ICU-enabled Postgres build, > maybe you could find an ICU collation that acts the way you want. > This wouldn't be a perfect solution, because we don't yet have > the ability to set an ICU collation as a databas

Re: To JIT (or not!) in Ubuntu packages

2021-09-14 Thread Thomas Munro
On Wed, Sep 15, 2021 at 3:30 PM Ben Chobot wrote: > So I've installed > http://apt.postgresql.org/pub/repos/apt/pool/14/p/postgresql-14/postgresql-14_14~beta3-1.pgdg18.04+1_arm64.deb, > after which I see: Ahhh, so you're on 18.04, an old LTS. I remember now, there was this issue in LLVM 3.9 on

Re: To JIT (or not!) in Ubuntu packages

2021-09-14 Thread Thomas Munro
On Tue, Sep 14, 2021 at 10:11 AM Ben Chobot wrote: > We've noticed that the Ubuntu postgresql-12 package has --with-llvm > enabled on x86_64, but not on aarch64. Does anybody know if this was > intentional, or just an oversight? > > For what it's worth, it seems the beta postgresql-14 package for

Re: spiexceptions.UndefinedFile: could not open file "base/16482/681830": No such file or directory

2021-09-08 Thread Thomas Munro
On Thu, Sep 9, 2021 at 9:19 AM Celia McInnis wrote: > Note that the file does exist:! (How do I know if it is looking under the > correct directory? Other times I have done similar temporary table creations > with no problems!): PostgreSQL internally uses relative paths. It's probably not a ve

Re: Is there something similar like flashback query from Oracle planned for PostgreSQL

2021-06-23 Thread Thomas Munro
On Thu, Jun 24, 2021 at 6:54 AM Dirk Krautschick wrote: > Is there something planned to get a behaviour like Oracle's flashback query > based on the old values > before deleted by vacuum? > > So a feature to recreate old versions of rows if still there? > > Or are there any related extensions or

Re: order by

2021-06-10 Thread Thomas Munro
On Thu, Jun 10, 2021 at 8:40 PM Luca Ferrari wrote: > On Thu, Jun 10, 2021 at 10:15 AM Vijaykumar Jain > > when you provide a table in query in the order by clause, it is > > ordered by cols of that table in that order. > > Clever, thanks! > I also realized that this "table to tuples" expansion wo

Re: CREATE/REFRESH MATERIALIZED VIEW planner difference?

2021-06-01 Thread Thomas Munro
On Wed, Jun 2, 2021 at 7:15 AM Vijaykumar Jain wrote: > i only get workers to create mv, but refresh mv plan does not use workers for > the same conf params. Yeah, this changed in v14: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=9e7ccd9ef64d05e87ceb1985d459bef9031205c0

Re: Chain Hashing

2021-05-06 Thread Thomas Munro
On Thu, May 6, 2021 at 9:48 PM Jian He wrote: > The following part is about the Chain Hashing. >> >> Maintain a linked list of buckets for each slot in the hash table. >> Resolve collisions by placing all elements with the same hash key into the >> same bucket. >> → To determine whether an elem

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-04 Thread Thomas Munro
On Wed, May 5, 2021 at 2:12 PM Thomas Munro wrote: > It might be interesting to know how that 40ms time scales as you add > more workers. ... Another thought: I'd also try tests like that in large databases (ie large virtual memory) vs small ones, and with and without huge/locked m

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-04 Thread Thomas Munro
On Wed, May 5, 2021 at 3:50 AM Hans Buschmann wrote: > (BTW: Is this cost multiplied by the real count of workers choosen > (max_parallel_workers_per_gather) or only a value independent of the number > of workers?. This would matter in windows-high-parallel scenarios) It's not multiplied: http

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-04 Thread Thomas Munro
On Tue, May 4, 2021 at 7:40 PM Hans Buschmann wrote: > The problem seems that this (probably inherent) performance disadvantage of > windows is not reflected in the cost model. https://www.postgresql.org/docs/13/runtime-config-query.html#GUC-PARALLEL-SETUP-COST is for that. It might be interest

Re: Huge performance penalty with parallel queries in Windows x64 v. Linux x64

2021-05-03 Thread Thomas Munro
On Tue, May 4, 2021 at 4:05 AM Hans Buschmann wrote: > The main difference is the time shown for the Gather Merge step (65 ms vs. 7 > ms) No Windows here, but could it be super slow at launching workers? How does a trivial parallel query compare, something like? SET force_parallel_mode = on; E

Re: -1/0 virtualtransaction

2021-04-30 Thread Thomas Munro
On Wed, Apr 28, 2021 at 2:25 AM Mike Beachy wrote: > On Tue, Apr 27, 2021 at 2:56 AM Laurenz Albe wrote: >> >> Not sure, but do you see prepared transactions in "pg_prepared_xacts"? > > No, the -1 in the virtualtransaction > (https://www.postgresql.org/docs/11/view-pg-locks.html) for pg_prepared

Re: Postgres crashes at memcopy() after upgrade to PG 13.

2021-03-15 Thread Thomas Munro
On Mon, Mar 15, 2021 at 1:29 PM Avinash Kumar wrote: > Is this expected when replication is happening between PostgreSQL databases > hosted on different OS versions like Ubuntu 16 and Ubuntu 20 ? Or, do we > think this is some sort of corruption ? Is this index on a text datatype, and using a c

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-11 Thread Thomas Munro
Hi Andrus, On Thu, Mar 11, 2021 at 2:21 AM Andrus wrote: > Windows Resource manger shows that wal files are used by large number of > postgres processes: > > postgres.exe22656FileC:\Program > Files\PostgreSQL\13\data\pg_wal\0001000A0075 > postgres.exe30788File

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-09 Thread Thomas Munro
On Tue, Mar 9, 2021 at 9:43 PM Andrus wrote: > > Any hints in Windows event viewer? Events occurring at the same time > showing up there. > > Looked into Administrative Events/Custom views and few others. There are no > messages about this. Windowsi perfomance monitor and Filemon show files >

Re: Log files polluted with permission denied error messages after every 10 seconds

2021-03-08 Thread Thomas Munro
On Sat, Mar 6, 2021 at 2:36 PM Michael Paquier wrote: > On Fri, Mar 05, 2021 at 07:36:37PM +0200, Andrus wrote: > > Then turned real-time protection off: > > > > Problem persists. New entry is written after every 10 seconds. > > On which files are those complaints? It seems to me that you may hav

Re: [LDAPS] Test connection user with ldaps server

2021-02-15 Thread Thomas Munro
On Tue, Feb 16, 2021 at 4:32 AM Laurenz Albe wrote: > What I would do is experiment with the "ldapsearch" executable from OpenLDAP > and see > if you can reproduce the problem from the command line. Also, maybe try doing this as the "postgres" user (or whatever user PostgreSQL runs as), just in

Re: How to post to this mailing list from a web based interface

2021-02-14 Thread Thomas Munro
On Fri, Jan 29, 2021 at 4:27 AM Alvaro Herrera wrote: > On 2021-Jan-28, Ravi Krishna wrote: > > I recollect there use to be a website from where one can reply from web. > > The community does not maintain such a service. > > There used to be a Gmane archive of this list that you could use to > pos

Re: Unable To Drop Tablespace

2021-02-04 Thread Thomas Munro
On Fri, Feb 5, 2021 at 12:43 PM Ian Lawrence Barwick wrote: > 2021年2月5日(金) 3:52 Pavan Pusuluri : >> We are trying to drop a table space on RDS Postgres . We have removed the >> objects etc, but it still won't drop. >> >> I have checked and there's no reference anywhere to this tablespace but it

Re: Unable to compile postgres 13.1 on Slackware current x64

2020-11-17 Thread Thomas Munro
On Tue, Nov 17, 2020 at 8:02 PM Condor wrote: > I try to compile postgres again with (cd src/backend/commands; sed > 's/TRUE/true/' collationcmds.c > collationcmds.c; ) and it's compiled > but get new error on linking: Doesn't that produce an empty file collationcmds.c? I think you want: sed 's/

Re: Unable to compile postgres 13.1 on Slackware current x64

2020-11-16 Thread Thomas Munro
On Mon, Nov 16, 2020 at 10:10 PM Laurenz Albe wrote: > On Mon, 2020-11-16 at 09:15 +0200, Condor wrote: > > collationcmds.c: In function ‘get_icu_language_tag’: > > collationcmds.c:467:51: error: ‘TRUE’ undeclared (first use in this > > function); did you mean ‘IS_TRUE’? > >467 | uloc_toLangu

Re: PostgreSQL on Windows' state

2020-09-23 Thread Thomas Munro
On Wed, Sep 23, 2020 at 10:53 PM Alessandro Dentella wrote: > Thanks Pavel, but psycopg2 (that I always use is just for Python). T > hey claim/complain that from c# there's no native solution. Maybe https://www.npgsql.org/?

Re: Check replication lag

2020-08-05 Thread Thomas Munro
On Thu, Aug 6, 2020 at 7:02 AM Sreejith P wrote: > IN SYNC mode of replication what would be the impact on Master DB server in > terms of over all performance ? The pg_stat_replication columns write_lag, flush_lag and replay_lag are designed tell you how long to expect commits to take for synch

Re: PG 9.5.5 cores on AIX 7.1

2020-07-19 Thread Thomas Munro
On Sun, Jul 19, 2020 at 11:01 PM Abraham, Danny wrote: > Segmentation fault in _alloc_initial_pthread at 0x9521474 > 0x9521474 (_alloc_initial_pthread+0x1d4) e803 ld > r0,0x0(r3) > (dbx) where > _alloc_initial_pthread(??) at 0x9521474 > __pth_init(??) at

Re: Same query taking less time in low configuration machine

2020-07-15 Thread Thomas Munro
On Tue, Jul 14, 2020 at 9:27 PM Vishwa Kalyankar wrote: > Function Scan on kseb_geometry_trace_with_barrier_partition > (cost=0.25..10.25 rows=1000 width=169) (actual time=11626.548..11626.568 > rows=254 loops=1) > Function Scan on kseb_geometry_trace_with_barrier_partition > (cost=0.25..1

Re: Definition of REPEATABLE READ

2020-06-21 Thread Thomas Munro
On Mon, Jun 22, 2020 at 12:25 AM Peter J. Holzer wrote: > I've read http://jepsen.io/analyses/postgresql-12.3 which reports a > problem in PostgreSQL's SERIALIZABLE isolation leven (which has since been > fixed) and also shows an example of a violation of what they consider to > be the correct def

Re: troubleshooting postgresql ldap authentication

2020-06-08 Thread Thomas Munro
On Tue, Jun 9, 2020 at 9:05 AM Chris Stephens wrote: > hostsslall all 0.0.0.0/0 ldap > ldapserver="ldaps://xxx" ldapbasedn="yyy" ldaptls=1 > does anyone know what might be causing "LDAP: Bad parameter to an ldap > routine" You probably want ldapurl="lda

Re: Shared memory error

2020-06-04 Thread Thomas Munro
On Fri, Jun 5, 2020 at 1:00 AM Sonam Sharma wrote: > The dynamic_shared_memory_type was set to POSIX . Because of this it was > using tmpfs /dev/shm. When the query was running I saw the file system was > filling. So I extended the file system and luckily the query worked for that > time Oh,

Re: Shared memory error

2020-06-04 Thread Thomas Munro
>> Do you see any other errors around this one, in the PostgreSQL logs? > No , only this is the error from db and jdbc end .. and queries are failing If you need a workaround right now you could always set max_parallel_workers_per_gather=0 so that it doesn't try to use parallel query. That could

Re: Shared memory error

2020-06-04 Thread Thomas Munro
On Thu, Jun 4, 2020 at 6:18 AM Sonam Sharma wrote: >>> 1) Postgres version : 11.2 FYI This is missing over a year's worth of bugfixes. That said, I don't know of anything fixed that has this symptom. >>> 4) Is this only with one query and if so what is it doing? : No , few >>> queries work, fe

Re: Can we get SQL Server-like cross database queries

2020-06-03 Thread Thomas Munro
On Thu, Jun 4, 2020 at 4:26 PM Ron wrote: > On 6/3/20 2:57 PM, Rob Sargent wrote: > >> On Jun 3, 2020, at 1:46 PM, Tom Lane wrote: > >> Guyren Howe writes: > >>> Is it practical to provide the SQL Server-like feature in Postgres? > >> No. > > That got me chuckling. > > I had just decided not to

Re: Help understanding SIReadLock growing without bound on completed transaction

2020-05-22 Thread Thomas Munro
On Fri, May 22, 2020 at 7:48 AM Mike Klaas wrote: > It's my understanding that these locks should be cleared when there are no > conflicting transactions. These locks had existed for > 1 week and we have > no transactions that last more than a few seconds (the oldest transaction in > pg_stat_a

Re: Help understanding SIReadLock growing without bound on completed transaction

2020-05-21 Thread Thomas Munro
On Fri, May 22, 2020 at 7:48 AM Mike Klaas wrote: > locktype: page > relation::regclass::text: _pkey > virtualtransaction: 36/296299968 > granted:t > pid:2263461 That's an unusually high looking pid. Is that expected, for example did you crank Linux's pid_max right up, or is this AIX, or somethi

Re: Ubuntu 20.04: apt.postgresql.org/pub/repos/apt focal-pgdg InRelease' doesn't support architecture 'i386'

2020-05-16 Thread Thomas Munro
On Sun, May 17, 2020 at 10:45 AM Hugh wrote: > While this doesn't appear to be a bug that causes problems of any kind, I do > have a question about its cause. > > The "error" listed in the Subject: line is basically what I'm seeing. The > entire message is below, particularly the 'N:' at the end

Re: Is there a significant difference in Memory settings between 9.5 and 12

2020-05-11 Thread Thomas Munro
On Tue, May 12, 2020 at 2:52 PM Tory M Blue wrote: > It took the change but didn't help. So 10GB of shared_buffers in 12 is still > a no go. I'm down to 5GB and it works, but this is the same hardware, the > same exact 9.5 configuration. So I'm missing something. WE have not had to > mess with

Re: 12.2: Howto check memory-leak in worker?

2020-05-04 Thread Thomas Munro
On Tue, May 5, 2020 at 10:13 AM Peter wrote: > BTW, I would greatly appreciate if we would reconsider the need for > the server to read the postmaster.pid file every few seconds (probably > needed for something, I don't know). > That makes it necessary to set atime=off to get a spindown, and I > u

Re: Transition tables for column-specific UPDATE triggers

2020-05-03 Thread Thomas Munro
On Wed, Oct 9, 2019 at 3:59 PM Guy Burgess wrote: > The manual says: > https://www.postgresql.org/docs/current/sql-createtrigger.html > > A column list cannot be specified when requesting transition relations. > > And (I think the same point): > > The standard allows transition tables to be used

Re: EINTR while resizing dsm segment.

2020-04-07 Thread Thomas Munro
On Tue, Apr 7, 2020 at 8:58 PM Nicola Contu wrote: > So that seems to be a bug, correct? > Just to confirm, I am not using NFS, it is directly on disk. > > Other than that, is there a particular option we can set in the postgres.conf > to mitigate the issue? Hi Nicola, Yeah, I think it's a bug.

Re: EINTR while resizing dsm segment.

2020-04-03 Thread Thomas Munro
On Thu, Apr 2, 2020 at 9:25 PM Kyotaro Horiguchi wrote: > I provided the subject, and added -hackers. > > > Hello, > > I am running postgres 11.5 and we were having issues with shared segments. > > So I increased the max_connection as suggested by you guys and reduced my > > work_mem to 600M. > >

Re: \COPY to accept non UTF-8 chars in CHAR columns

2020-03-27 Thread Thomas Munro
On Sat, Mar 28, 2020 at 4:46 AM Tom Lane wrote: > Matthias Apitz writes: > > In short, it there a way to let \COPY accept such broken ISO bytes, just > > complaining about, but not stopping the insert of the row? > > No. We don't particularly believe in the utility of invalid data. > > If you do

Re: LDAP with TLS is taking more time in Postgresql 11.5

2020-02-25 Thread Thomas Munro
On Wed, Feb 26, 2020 at 7:37 AM Adrian Klaver wrote: > On 2/25/20 10:23 AM, Mani Sankar wrote: > > Hi Adrian, > > > > Both the machines are in same network and both are pointing towards the > > same LDAP server > > I don't see any errors in the Postgres logs. > > You probably should take a look at

  1   2   3   >