Re: Move defaults toward ICU in 16?

2023-05-03 Thread Jonathan S. Katz
On 4/17/23 2:33 PM, Tom Lane wrote: Jeff Davis writes: Is now a reasonable time to check it in and see what breaks? It looks like there are quite a few buildfarm members that specify neither -- with-icu nor --without-icu. I see you just pinged buildfarm-members again, so I'd think it's

Re: Move defaults toward ICU in 16?

2023-04-17 Thread Tom Lane
Jeff Davis writes: > Is now a reasonable time to check it in and see what breaks? It looks > like there are quite a few buildfarm members that specify neither -- > with-icu nor --without-icu. I see you just pinged buildfarm-members again, so I'd think it's polite to give people 24 hours or so to

Re: Move defaults toward ICU in 16?

2023-04-17 Thread Jeff Davis
On Mon, 2023-04-17 at 08:23 -0500, Justin Pryzby wrote: > > I don't object to this patch.  I suggest waiting until next week to > > commit > > it and then see what happens.  It's easy to revert if it goes > > terribly. > > Is this still being considered for v16 ? Yes, unless someone raises a

Re: Move defaults toward ICU in 16?

2023-04-17 Thread Justin Pryzby
On Wed, Apr 05, 2023 at 09:33:25AM +0200, Peter Eisentraut wrote: > On 16.03.23 14:52, Peter Eisentraut wrote: > > On 09.03.23 20:14, Jeff Davis wrote: > > > > Let's come back to that after dealing with the other two. > > > > > > Leaving 0001 open for now. > > > > I suspect making a change like

Re: Move defaults toward ICU in 16?

2023-04-05 Thread Peter Eisentraut
On 16.03.23 14:52, Peter Eisentraut wrote: On 09.03.23 20:14, Jeff Davis wrote: Let's come back to that after dealing with the other two. Leaving 0001 open for now. I suspect making a change like this now would result in a bloodbath on the build farm that we could do without.  I suggest

Re: Move defaults toward ICU in 16?

2023-03-16 Thread Peter Eisentraut
On 09.03.23 20:14, Jeff Davis wrote: Let's come back to that after dealing with the other two. Leaving 0001 open for now. I suspect making a change like this now would result in a bloodbath on the build farm that we could do without. I suggest revisiting this after the commit fest ends.

Re: Move defaults toward ICU in 16?

2023-03-09 Thread Jeff Davis
On Thu, 2023-03-09 at 10:36 +0100, Peter Eisentraut wrote: > 0002 seems fine to me. Committed 0002 with some test improvements. > > Let's come back to that after dealing with the other two. Leaving 0001 open for now. 0003 committed after addressing your comments. -- Jeff Davis PostgreSQL

Re: Move defaults toward ICU in 16?

2023-03-09 Thread Peter Eisentraut
On 08.03.23 06:55, Jeff Davis wrote: On Fri, 2023-03-03 at 21:45 -0800, Jeff Davis wrote:    0002: update template0 in new cluster (as described above) I think 0002 is about ready and I plan to commit it soon unless someone has more comments. 0002 seems fine to me. I'm holding off on

Re: Move defaults toward ICU in 16?

2023-03-07 Thread Jeff Davis
On Fri, 2023-03-03 at 21:45 -0800, Jeff Davis wrote: > > >    0002: update template0 in new cluster (as described above) I think 0002 is about ready and I plan to commit it soon unless someone has more comments. I'm holding off on 0001 for now, because you objected. But I still think 0001 is a

Re: Move defaults toward ICU in 16?

2023-03-03 Thread Jeff Davis
On Thu, 2023-03-02 at 10:37 +0100, Peter Eisentraut wrote: > I would skip this.  There was a brief discussion about this at [0], > where I pointed out that if we are going to do something like that, > there would be other candidates among the optional dependencies to > promote, such as

Re: Move defaults toward ICU in 16?

2023-03-02 Thread Peter Eisentraut
On 25.02.23 00:54, Jeff Davis wrote: On Fri, 2023-02-17 at 15:07 -0800, Jeff Davis wrote: 2. Update the pg_database entry for template0. This has less potential for surprise in case people are actually using template0 for a template. New patches attached. 0001: default autoconf to build

Re: Move defaults toward ICU in 16?

2023-02-27 Thread Tom Lane
Jeff Davis writes: > On Fri, 2023-02-24 at 15:54 -0800, Jeff Davis wrote: >> 0001: default autoconf to build with ICU (meson already uses >> 'auto') > What's the best way to prepare for the impact of this on the buildfarm? > How should we migrate to using --without-icu for those animals not >

Re: Move defaults toward ICU in 16?

2023-02-27 Thread Jeff Davis
On Fri, 2023-02-24 at 15:54 -0800, Jeff Davis wrote: >   0001: default autoconf to build with ICU (meson already uses > 'auto') What's the best way to prepare for the impact of this on the buildfarm? How should we migrate to using --without-icu for those animals not currently specifying

Re: Move defaults toward ICU in 16?

2023-02-24 Thread Jeff Davis
On Fri, 2023-02-17 at 15:07 -0800, Jeff Davis wrote: > 2. Update the pg_database entry for template0. This has less > potential > for surprise in case people are actually using template0 for a > template. New patches attached. 0001: default autoconf to build with ICU (meson already uses

Re: Move defaults toward ICU in 16?

2023-02-20 Thread Jeff Davis
On Mon, 2023-02-20 at 15:55 +0100, Peter Eisentraut wrote: > I'm confused.  We are not going to try to change existing databases > to a > different collation provider during pg_upgrade, are we? No, certainly not. I interpreted Pavel's comments as a comparison of ICU and libc in general and not

Re: Move defaults toward ICU in 16?

2023-02-20 Thread Peter Eisentraut
On 17.02.23 21:43, Jeff Davis wrote: select row_number() over (order by nazev collate "cs-x-icu"), nazev from obce except select row_number() over (order by nazev collate "cs_CZ"), nazev from obce; returns a not empty set. So minimally for Czech collate, an index rebuild is necessary Yes,

Re: Move defaults toward ICU in 16?

2023-02-18 Thread Robert Haas
On Fri, Feb 17, 2023 at 10:32 PM Jeff Davis wrote: > Thinking about this more, it's not clear to me if this would be in > scope for pg_upgrade or not. If pg_upgrade is fixing up the new cluster > rather than checking for compatibility, why doesn't it just take over > and do the initdb for the new

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Pavel Stehule
pá 17. 2. 2023 v 21:43 odesílatel Jeff Davis napsal: > On Fri, 2023-02-17 at 18:27 +0100, Pavel Stehule wrote: > > Today I tested icu for Czech sorting. It is a little bit slower, but > > not too much, but it produces partially different results. > > Thank you for trying it. > > If it's a

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Jeff Davis
On Fri, 2023-02-17 at 12:50 -0800, Andres Freund wrote: > I think we just drop/recreate template1 and postgres during > pg_upgrade. Thank you, that makes much more sense now. I was confused because pg_upgrade loops through to check compatibility with all the databases, which makes zero sense if

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Andres Freund
Hi, On 2023-02-17 12:36:05 -0800, Jeff Davis wrote: > > > There's already a check that the new cluster is empty, so I think > > > it's > > > safe to hack the pg_database locale fields. > > > > I don't think we need to, we do issue the CREATE DATABASEs. So we > > just need to > > make sure that

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Jeff Davis
On Fri, 2023-02-17 at 18:27 +0100, Pavel Stehule wrote: > Today I tested icu for Czech sorting. It is a little bit slower, but > not too much, but it produces partially different results. Thank you for trying it. If it's a significant slowdown, can you please send more information? ICU version,

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Jeff Davis
On Fri, 2023-02-17 at 10:09 -0800, Andres Freund wrote: > -1. That's just going to cause pain one major version upgrade further > down the > line. Why would we want to incur that pain? OK, we can just always do the fixup as long as the old one is libc and the new one is ICU. I'm just trying to

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Justin Pryzby
On Fri, Feb 17, 2023 at 09:01:54AM -0800, Jeff Davis wrote: > On Fri, 2023-02-17 at 00:06 -0800, Jeff Davis wrote: > > On Tue, 2023-02-14 at 09:59 -0800, Andres Freund wrote: > > > I am saying that pg_upgrade should be able to deal with the > > > difference. The > > > details of how to implement

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Andres Freund
Hi, On 2023-02-17 10:00:41 -0800, Jeff Davis wrote: > I guess I'm fine hacking pg_upgrade, but I think I'd like to make it > conditional on this specific case: only perform the fixup if the old > cluster is 15 or earlier and using libc and the newer cluster is 16 or > later and using icu. -1.

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Jeff Davis
On Fri, 2023-02-17 at 09:05 -0800, Andres Freund wrote: > > Thinking about this more, it's not clear to me if this would be in > > scope for pg_upgrade or not. > > I don't think we should consider changing the default collation > provider > without making this more seamless, one way or another.

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Pavel Stehule
pá 17. 2. 2023 v 18:02 odesílatel Jeff Davis napsal: > On Fri, 2023-02-17 at 00:06 -0800, Jeff Davis wrote: > > On Tue, 2023-02-14 at 09:59 -0800, Andres Freund wrote: > > > I am saying that pg_upgrade should be able to deal with the > > > difference. The > > > details of how to implement that,

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Andres Freund
Hi, On 2023-02-17 09:01:54 -0800, Jeff Davis wrote: > On Fri, 2023-02-17 at 00:06 -0800, Jeff Davis wrote: > > On Tue, 2023-02-14 at 09:59 -0800, Andres Freund wrote: > > > I am saying that pg_upgrade should be able to deal with the > > > difference. The > > > details of how to implement that,

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Jeff Davis
On Fri, 2023-02-17 at 00:06 -0800, Jeff Davis wrote: > On Tue, 2023-02-14 at 09:59 -0800, Andres Freund wrote: > > I am saying that pg_upgrade should be able to deal with the > > difference. The > > details of how to implement that, don't matter that much. > > To clarify, you're saying that

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Laurenz Albe
On Fri, 2023-02-17 at 14:40 +0900, Michael Paquier wrote: > Separate question: what's the state of the Windows installers provided > by the community regarding libicu?  Is that embedded in the MSI? The EDB installer installs a quite old version of the ICU library for compatibility reasons, as far

Re: Move defaults toward ICU in 16?

2023-02-17 Thread Jeff Davis
On Tue, 2023-02-14 at 09:59 -0800, Andres Freund wrote: > I am saying that pg_upgrade should be able to deal with the > difference. The > details of how to implement that, don't matter that much. To clarify, you're saying that pg_upgrade should simply update pg_database to set the new databases'

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Andres Freund
Hi, On February 16, 2023 9:40:17 PM PST, Michael Paquier wrote: >On Fri, Feb 17, 2023 at 10:44:05AM +0530, Robert Haas wrote: >> Uh, I had the contrary impression from the discussion upthread, but it >> sounds like I might be misunderstanding the situation? > >IMO, it would be nice to be able

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Michael Paquier
On Fri, Feb 17, 2023 at 10:44:05AM +0530, Robert Haas wrote: > Uh, I had the contrary impression from the discussion upthread, but it > sounds like I might be misunderstanding the situation? IMO, it would be nice to be able to have the automatic detection of meson work in the CFbot to see how

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Andres Freund
Hi, On February 16, 2023 9:14:05 PM PST, Robert Haas wrote: >On Thu, Feb 16, 2023 at 9:45 PM Andres Freund wrote: >> On 2023-02-16 15:05:10 +0530, Robert Haas wrote: >> > The fact that we can't use ICU on Windows, though, weakens this >> > argument a lot. In my experience, we have a lot of

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Robert Haas
On Thu, Feb 16, 2023 at 9:45 PM Andres Freund wrote: > On 2023-02-16 15:05:10 +0530, Robert Haas wrote: > > The fact that we can't use ICU on Windows, though, weakens this > > argument a lot. In my experience, we have a lot of Windows users, and > > they're not any happier with the operating

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Andres Freund
Hi, On 2023-02-16 15:05:10 +0530, Robert Haas wrote: > The fact that we can't use ICU on Windows, though, weakens this > argument a lot. In my experience, we have a lot of Windows users, and > they're not any happier with the operating system collations than > Linux users. Possibly less so. Why

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Jonathan S. Katz
On 2/16/23 4:35 AM, Robert Haas wrote: On Thu, Feb 16, 2023 at 1:01 AM Jeff Davis wrote: It feels very wrong to me to explain that sort order is defined by the operating system on which Postgres happens to run. Saying that it's defined by ICU, which is part of the Unicode consotium, is much

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Laurenz Albe
On Thu, 2023-02-16 at 15:05 +0530, Robert Haas wrote: > On Thu, Feb 16, 2023 at 1:01 AM Jeff Davis wrote: > > It feels very wrong to me to explain that sort order is defined by the > > operating system on which Postgres happens to run. Saying that it's > > defined by ICU, which is part of the

Re: Move defaults toward ICU in 16?

2023-02-16 Thread Robert Haas
On Thu, Feb 16, 2023 at 1:01 AM Jeff Davis wrote: > It feels very wrong to me to explain that sort order is defined by the > operating system on which Postgres happens to run. Saying that it's > defined by ICU, which is part of the Unicode consotium, is much better. > It doesn't eliminate

Re: Move defaults toward ICU in 16?

2023-02-15 Thread Jeff Davis
On Tue, 2023-02-14 at 16:27 -0500, Jonathan S. Katz wrote: > Would it be any different than a vulnerability in OpenSSL et al? In principle, no, but sometimes the details matter. I'm just trying to add data to the discussion. > It seems that > in general, users would see performance gains

Re: Move defaults toward ICU in 16?

2023-02-14 Thread Jonathan S. Katz
On 2/13/23 8:11 PM, Jeff Davis wrote: On Thu, 2023-02-02 at 05:13 -0800, Jeff Davis wrote: As a project, do we want to nudge users toward ICU as the collation provider as the best practice going forward? One consideration here is security. Any vulnerability in ICU collation routines could

Re: Move defaults toward ICU in 16?

2023-02-14 Thread Andres Freund
Hi, On 2023-02-14 09:48:08 -0800, Jeff Davis wrote: > On Fri, 2023-02-10 at 18:00 -0800, Andres Freund wrote: > > But, shouldn't pg_upgrade be able to deal with this? As long as the > > databases > > are created with template0, we can create the collations at that > > point? > > Are you saying

Re: Move defaults toward ICU in 16?

2023-02-14 Thread Jeff Davis
On Fri, 2023-02-10 at 18:00 -0800, Andres Freund wrote: > Until something like my patch above is done more generally > applicable, I think > your patch should disable ICU on windows. Can't just fail to build. > > Perhaps we don't need to force ICU use to on with the meson build, > given that > it

Re: Move defaults toward ICU in 16?

2023-02-13 Thread Jeff Davis
On Thu, 2023-02-02 at 05:13 -0800, Jeff Davis wrote: > As a project, do we want to nudge users toward ICU as the collation > provider as the best practice going forward? One consideration here is security. Any vulnerability in ICU collation routines could easily become a vulnerability in

Re: Move defaults toward ICU in 16?

2023-02-10 Thread Andres Freund
Hi, On 2023-02-10 16:17:00 -0800, Jeff Davis wrote: > One CI test is failing: "Windows - Server 2019, VS 2019 - Meson & > ninja"; if I apply Andres patch ( > https://github.com/anarazel/postgres/commit/dde7c68 ), then it works. Until something like my patch above is done more generally

Re: Move defaults toward ICU in 16?

2023-02-10 Thread Jeff Davis
On Wed, 2023-02-08 at 18:22 -0800, Andres Freund wrote: > On 2023-02-08 12:16:46 -0800, Jeff Davis wrote: > > On Thu, 2023-02-02 at 18:10 -0500, Tom Lane wrote: > > > Yeah.  I would be resistant to making ICU a required dependency, > > > but it doesn't seem unreasonable to start moving towards it

Re: Move defaults toward ICU in 16?

2023-02-08 Thread Andres Freund
On 2023-02-08 12:16:46 -0800, Jeff Davis wrote: > On Thu, 2023-02-02 at 18:10 -0500, Tom Lane wrote: > > Yeah.  I would be resistant to making ICU a required dependency, > > but it doesn't seem unreasonable to start moving towards it being > > our default collation support. > > Patch attached.

Re: Move defaults toward ICU in 16?

2023-02-08 Thread Jeff Davis
On Thu, 2023-02-02 at 18:10 -0500, Tom Lane wrote: > Yeah.  I would be resistant to making ICU a required dependency, > but it doesn't seem unreasonable to start moving towards it being > our default collation support. Patch attached. To get the default locale, the patch initializes a UCollator

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Tom Lane
Thomas Munro writes: > It's still important to have libc support as an option, though, > because it's a totally reasonable thing to want sort order to agree > with the "sort" command on the same host, and you are willing to deal > with all the complexities that we're trying to escape. Yeah. I

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Peter Geoghegan
On Thu, Feb 2, 2023 at 2:15 PM Thomas Munro wrote: > Sure, there could be a clean-room implementation that replaces it in > some sense (just as there is a Java implementation) but it would very > likely be "the same" because the real thing we're buying here is the > set of algorithms and data

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Thomas Munro
On Fri, Feb 3, 2023 at 5:31 AM Jeff Davis wrote: > On Thu, 2023-02-02 at 08:44 -0500, Robert Haas wrote: > > On Thu, Feb 2, 2023 at 8:13 AM Jeff Davis wrote: > > > If we don't want to nudge users toward ICU, is it because we are > > > waiting for something, or is there a lack of consensus that

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Jeff Davis
On Thu, 2023-02-02 at 08:44 -0500, Robert Haas wrote: > On Thu, Feb 2, 2023 at 8:13 AM Jeff Davis wrote: > > If we don't want to nudge users toward ICU, is it because we are > > waiting for something, or is there a lack of consensus that ICU is > > actually better? > > Do you think it's better?

Re: Move defaults toward ICU in 16?

2023-02-02 Thread Robert Haas
On Thu, Feb 2, 2023 at 8:13 AM Jeff Davis wrote: > If we don't want to nudge users toward ICU, is it because we are > waiting for something, or is there a lack of consensus that ICU is > actually better? Do you think it's better? -- Robert Haas EDB: http://www.enterprisedb.com

Move defaults toward ICU in 16?

2023-02-02 Thread Jeff Davis
As a project, do we want to nudge users toward ICU as the collation provider as the best practice going forward? If so, is version 16 the right time to adjust defaults to favor ICU? * At build time, default to --with-icu (-Dicu=enabled); users who don't want ICU can specify --without-icu