On Thu, Nov 24, 2022 at 5:48 PM Thomas Munro <thomas.mu...@gmail.com> wrote:
> On Thu, Nov 24, 2022 at 3:07 PM Jeff Davis <pg...@j-davis.com> wrote:
> > I'd vote for 1 on the grounds that it's easier to document and
> > understand a single collation version, which comes straight from
> > ucol_getVersion(). This approach makes it a separate problem to find
> > the collation version among whatever libraries the admin can provide;
> > but adding some observability into the search should mitigate any
> > confusion.
>
> OK, it sounds like I should code that up next.
Here's the first iteration. The version rosetta stone functions look like this:
postgres=# select * from pg_icu_library_versions();
icu_version | unicode_version | cldr_version
-------------+-----------------+--------------
67.1 | 13.0 | 37.0
63.1 | 11.0 | 34.0
57.1 | 8.0 | 29.0
(3 rows)
postgres=# select * from pg_icu_collation_versions('zh');
icu_version | uca_version | collator_version
-------------+-------------+------------------
67.1 | 13.0 | 153.14.37
63.1 | 11.0 | 153.88.34
57.1 | 8.0 | 153.64.29
(3 rows)
It's no longer necessary to put anything in PG_TEST_EXTRA to run
"meson test irc/020_multiversion" usefully. It will find extra ICU
versions all by itself in your system library search path and SKIP if
it doesn't find a second major version. I have tried to cover the
main scenarios that I expect users to encounter in the update TAP
tests, with commentary that I hope will be helpful to assess the
usability of this thing.
Other changes:
* now using RTLD_LOCAL instead of RTLD_GLOBAL (I guess the latter
might cause trouble for someone using --disable-renaming, but I
haven't tested that and am not an expert on linker/loader arcana)
* fixed library names on Windows (based on reading the manual, but I
haven't tested that)
* fixed failure on non-ICU builds (the reason CI was failing in v7,
some misplaced #ifdefs)
* various cleanup
* I've attached a throwaway patch to install a second ICU version on
Debian/amd64 on CI, since otherwise the new test would SKIP on all
systems
This is just a first cut, but enough to try out and see if we like it,
what needs to be improved, what edge cases we haven't thought about
etc. Let me know what you think.
From 0d96bfbec02245ddce6c985250ff0f8d38e41df9 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.mu...@gmail.com>
Date: Wed, 8 Jun 2022 17:43:53 +1200
Subject: [PATCH v8 1/2] WIP: Multi-version ICU.
Add a layer of indirection when accessing ICU, so that multiple major
versions of the library can be used at once. Versions other than the
one that PostgreSQL was linked against are opened with dlopen(), but we
refuse to open version higher than the one were were compiled against.
The ABI might change in future releases so that wouldn't be safe.
Whenever creating a DATABASE or COLLATION object that uses ICU, we'll
use the "default" ICU library and record its ucol_getVersion() in the
catalog. That's usually the one we're linked against but another can be
selected with the setting default_icu_version_library.
Whenever opening an existing DATABASE or COLLATION object that uses ICU,
we'll see the recorded [dat]collversion and try to find the ICU library
that provides that version. If we can't, we'll fall back to using the
default ICU library with a warning that the user should either install
another ICU library version, or rebuild affected database objects and
REFRESH.
New GUCs:
icu_library_path
A place to find ICU libraries, if not the default system library
search path.
icu_library_versions
A comma-separated list of ICU major or major.minor versions to make
available to PostgreSQL, or * for every major version that can be
found (the default).
default_icu_library_version
The major or major.minor version to use for new objects and as a
fallback (with warnings) if the right version can't be found.
Reviewed-by: Peter Eisentraut <peter.eisentr...@enterprisedb.com>
Reviewed-by: Jeff Davis <pg...@j-davis.com>
Discussion: https://postgr.es/m/CA%2BhUKGL4VZRpP3CkjYQkv4RQ6pRYkPkSNgKSxFBwciECQ0mEuQ%40mail.gmail.com
---
src/backend/access/hash/hashfunc.c | 16 +-
src/backend/commands/collationcmds.c | 20 +
src/backend/utils/adt/formatting.c | 53 +-
src/backend/utils/adt/pg_locale.c | 748 +++++++++++++++++-
src/backend/utils/adt/varchar.c | 16 +-
src/backend/utils/adt/varlena.c | 56 +-
src/backend/utils/init/postinit.c | 34 +-
src/backend/utils/misc/guc_tables.c | 40 +-
src/backend/utils/misc/postgresql.conf.sample | 10 +
src/include/catalog/pg_proc.dat | 23 +
src/include/utils/pg_locale.h | 85 +-
src/test/icu/meson.build | 1 +
src/test/icu/t/020_multiversion.pl | 274 +++++++
src/tools/pgindent/typedefs.list | 4 +
14 files changed, 1275 insertions(+), 105 deletions(-)
create mode 100644 src/test/icu/t/020_multiversion.pl
diff --git a/src/backend/access/hash/hashfunc.c b/src/backend/access/hash/hashfunc.c
index b57ed946c4..0a61538efd 100644
--- a/src/backend/access/hash/hashfunc.c
+++ b/src/backend/access/hash/hashfunc.c
@@ -298,11 +298,11 @@ hashtext(PG_FUNCTION_ARGS)
ulen = icu_to_uchar(&uchar, VARDATA_ANY(key), VARSIZE_ANY_EXHDR(key));
- bsize = ucol_getSortKey(mylocale->info.icu.ucol,
- uchar, ulen, NULL, 0);
+ bsize = PG_ICU_LIB(mylocale)->getSortKey(PG_ICU_COL(mylocale),
+ uchar, ulen, NULL, 0);
buf = palloc(bsize);
- ucol_getSortKey(mylocale->info.icu.ucol,
- uchar, ulen, buf, bsize);
+ PG_ICU_LIB(mylocale)->getSortKey(PG_ICU_COL(mylocale),
+ uchar, ulen, buf, bsize);
result = hash_any(buf, bsize);
@@ -355,11 +355,11 @@ hashtextextended(PG_FUNCTION_ARGS)
ulen = icu_to_uchar(&uchar, VARDATA_ANY(key), VARSIZE_ANY_EXHDR(key));
- bsize = ucol_getSortKey(mylocale->info.icu.ucol,
- uchar, ulen, NULL, 0);
+ bsize = PG_ICU_LIB(mylocale)->getSortKey(PG_ICU_COL(mylocale),
+ uchar, ulen, NULL, 0);
buf = palloc(bsize);
- ucol_getSortKey(mylocale->info.icu.ucol,
- uchar, ulen, buf, bsize);
+ PG_ICU_LIB(mylocale)->getSortKey(PG_ICU_COL(mylocale),
+ uchar, ulen, buf, bsize);
result = hash_any_extended(buf, bsize, PG_GETARG_INT64(1));
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 81e54e0ce6..4fb0c77f38 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -853,6 +853,26 @@ pg_import_system_collations(PG_FUNCTION_ARGS)
CreateComments(collid, CollationRelationId, 0,
icucomment);
}
+
+ /* Also create an object pinned to an ICU major version. */
+ collid = CollationCreate(psprintf("%s-x-icu-%d", langtag, U_ICU_VERSION_MAJOR_NUM),
+ nspid, GetUserId(),
+ COLLPROVIDER_ICU, true, -1,
+ NULL, NULL,
+ psprintf("%d:%s", U_ICU_VERSION_MAJOR_NUM, iculocstr),
+ get_collation_actual_version(COLLPROVIDER_ICU, iculocstr),
+ true, true);
+ if (OidIsValid(collid))
+ {
+ ncreated++;
+
+ CommandCounterIncrement();
+
+ icucomment = get_icu_locale_comment(name);
+ if (icucomment)
+ CreateComments(collid, CollationRelationId, 0,
+ icucomment);
+ }
}
}
#endif /* USE_ICU */
diff --git a/src/backend/utils/adt/formatting.c b/src/backend/utils/adt/formatting.c
index 26f498b5df..0c3c7724d7 100644
--- a/src/backend/utils/adt/formatting.c
+++ b/src/backend/utils/adt/formatting.c
@@ -1599,6 +1599,11 @@ typedef int32_t (*ICU_Convert_Func) (UChar *dest, int32_t destCapacity,
const UChar *src, int32_t srcLength,
const char *locale,
UErrorCode *pErrorCode);
+typedef int32_t (*ICU_Convert_BI_Func) (UChar *dest, int32_t destCapacity,
+ const UChar *src, int32_t srcLength,
+ UBreakIterator *bi,
+ const char *locale,
+ UErrorCode *pErrorCode);
static int32_t
icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
@@ -1623,18 +1628,41 @@ icu_convert_case(ICU_Convert_Func func, pg_locale_t mylocale,
}
if (U_FAILURE(status))
ereport(ERROR,
- (errmsg("case conversion failed: %s", u_errorName(status))));
+ (errmsg("case conversion failed: %s",
+ PG_ICU_LIB(mylocale)->errorName(status))));
return len_dest;
}
+/*
+ * Like icu_convert_case, but func takes a break iterator (which we don't
+ * make use of).
+ */
static int32_t
-u_strToTitle_default_BI(UChar *dest, int32_t destCapacity,
- const UChar *src, int32_t srcLength,
- const char *locale,
- UErrorCode *pErrorCode)
+icu_convert_case_bi(ICU_Convert_BI_Func func, pg_locale_t mylocale,
+ UChar **buff_dest, UChar *buff_source, int32_t len_source)
{
- return u_strToTitle(dest, destCapacity, src, srcLength,
- NULL, locale, pErrorCode);
+ UErrorCode status;
+ int32_t len_dest;
+
+ len_dest = len_source; /* try first with same length */
+ *buff_dest = palloc(len_dest * sizeof(**buff_dest));
+ status = U_ZERO_ERROR;
+ len_dest = func(*buff_dest, len_dest, buff_source, len_source, NULL,
+ mylocale->info.icu.locale, &status);
+ if (status == U_BUFFER_OVERFLOW_ERROR)
+ {
+ /* try again with adjusted length */
+ pfree(*buff_dest);
+ *buff_dest = palloc(len_dest * sizeof(**buff_dest));
+ status = U_ZERO_ERROR;
+ len_dest = func(*buff_dest, len_dest, buff_source, len_source, NULL,
+ mylocale->info.icu.locale, &status);
+ }
+ if (U_FAILURE(status))
+ ereport(ERROR,
+ (errmsg("case conversion failed: %s",
+ PG_ICU_LIB(mylocale)->errorName(status))));
+ return len_dest;
}
#endif /* USE_ICU */
@@ -1702,7 +1730,8 @@ str_tolower(const char *buff, size_t nbytes, Oid collid)
UChar *buff_conv;
len_uchar = icu_to_uchar(&buff_uchar, buff, nbytes);
- len_conv = icu_convert_case(u_strToLower, mylocale,
+ len_conv = icu_convert_case(PG_ICU_LIB(mylocale)->strToLower,
+ mylocale,
&buff_conv, buff_uchar, len_uchar);
icu_from_uchar(&result, buff_conv, len_conv);
pfree(buff_uchar);
@@ -1824,7 +1853,8 @@ str_toupper(const char *buff, size_t nbytes, Oid collid)
UChar *buff_conv;
len_uchar = icu_to_uchar(&buff_uchar, buff, nbytes);
- len_conv = icu_convert_case(u_strToUpper, mylocale,
+ len_conv = icu_convert_case(PG_ICU_LIB(mylocale)->strToUpper,
+ mylocale,
&buff_conv, buff_uchar, len_uchar);
icu_from_uchar(&result, buff_conv, len_conv);
pfree(buff_uchar);
@@ -1947,8 +1977,9 @@ str_initcap(const char *buff, size_t nbytes, Oid collid)
UChar *buff_conv;
len_uchar = icu_to_uchar(&buff_uchar, buff, nbytes);
- len_conv = icu_convert_case(u_strToTitle_default_BI, mylocale,
- &buff_conv, buff_uchar, len_uchar);
+ len_conv = icu_convert_case_bi(PG_ICU_LIB(mylocale)->strToTitle,
+ mylocale,
+ &buff_conv, buff_uchar, len_uchar);
icu_from_uchar(&result, buff_conv, len_conv);
pfree(buff_uchar);
pfree(buff_conv);
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2b42d9ccd8..004100af66 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "funcapi.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/guc_hooks.h"
@@ -69,6 +71,8 @@
#ifdef USE_ICU
#include <unicode/ucnv.h>
+#include <unicode/ulocdata.h>
+#include <unicode/ustring.h>
#endif
#ifdef __GLIBC__
@@ -79,14 +83,36 @@
#include <shlwapi.h>
#endif
+#include <dlfcn.h>
+
#define MAX_L10N_DATA 80
+#ifdef USE_ICU
+
+/*
+ * We don't want to call into dlopen'd ICU libraries that are newer than the
+ * one we were compiled and linked against, just in case there is an
+ * incompatible API change.
+ */
+#define PG_MAX_ICU_MAJOR_VERSION U_ICU_VERSION_MAJOR_NUM
+
+/*
+ * The oldest ICU release we're likely to encounter, and that has all the
+ * funcitons required.
+ */
+#define PG_MIN_ICU_MAJOR_VERSION 50
+
+#endif
+
/* GUC settings */
char *locale_messages;
char *locale_monetary;
char *locale_numeric;
char *locale_time;
+char *icu_library_path;
+char *icu_library_versions;
+char *default_icu_library_version;
/*
* lc_time localization cache.
@@ -123,7 +149,9 @@ static char *IsoLocaleName(const char *);
#endif
#ifdef USE_ICU
-static void icu_set_collation_attributes(UCollator *collator, const char *loc);
+static void icu_set_collation_attributes(pg_icu_library *lib,
+ UCollator *collator,
+ const char *loc);
#endif
/*
@@ -1400,33 +1428,544 @@ lc_ctype_is_c(Oid collation)
struct pg_locale_struct default_locale;
-void
+#ifdef USE_ICU
+
+static pg_icu_library *icu_library_list;
+static pg_icu_library *default_icu_library;
+static bool icu_library_list_fully_loaded;
+
+static void *
+get_icu_function(void *handle, const char *function, int version)
+{
+ char function_with_version[80];
+ void *result;
+
+ /*
+ * Try to look it up using the symbols with major versions, but if that
+ * doesn't work, also try the unversioned name in case the library was
+ * configured with --disable-renaming.
+ */
+ snprintf(function_with_version, sizeof(function_with_version), "%s_%d",
+ function, version);
+ result = dlsym(handle, function_with_version);
+
+ return result ? result : dlsym(handle, function);
+}
+
+static void
+make_icu_library_name(char *output,
+ const char *name,
+ int major_version,
+ int minor_version)
+{
+ /*
+ * See
+ * https://unicode-org.github.io/icu/userguide/icu4c/packaging.html#icu-versions
+ * for conventions on library naming on POSIX and Windows systems. Apple
+ * isn't mentioned but varies in the usual way.
+ *
+ * Format 1 is expected to be a major version-only symlink pointing to a
+ * specific minor version (or on Windows it may be the actual library).
+ * Format 2 is expected to be an actual library.
+ */
+#ifdef WIN32
+#define ICU_LIBRARY_NAME_FORMAT1 "%s%sicu%s%d" DLSUFFIX
+#define ICU_LIBRARY_NAME_FORMAT2 "%s%sicu%s%d.%d" DLSUFFIX
+#elif defined(__darwin__)
+#define ICU_LIBRARY_NAME_FORMAT1 "%s%slibicu%s.%d" DLSUFFIX
+#define ICU_LIBRARY_NAME_FORMAT2 "%s%slibicu%s.%d.%d" DLSUFFIX
+#else
+#define ICU_LIBRARY_NAME_FORMAT1 "%s%slibicu%s" DLSUFFIX ".%d"
+#define ICU_LIBRARY_NAME_FORMAT2 "%s%slibicu%s" DLSUFFIX ".%d.%d"
+#endif
+
+#ifdef WIN32
+#define PATH_SEPARATOR "\\"
+#define ICU_I18N "in"
+#define ICU_UC "uc"
+#else
+#define PATH_SEPARATOR "/"
+#define ICU_I18N "i18n"
+#define ICU_UC "uc"
+#endif
+
+ if (minor_version < 0)
+ snprintf(output,
+ MAXPGPATH,
+ ICU_LIBRARY_NAME_FORMAT1,
+ icu_library_path,
+ icu_library_path[0] ? PATH_SEPARATOR : "",
+ name,
+ major_version);
+ else
+ snprintf(output,
+ MAXPGPATH,
+ ICU_LIBRARY_NAME_FORMAT2,
+ icu_library_path,
+ icu_library_path[0] ? PATH_SEPARATOR : "",
+ name,
+ major_version,
+ minor_version);
+}
+
+/*
+ * Given an ICU library major version and optionally minor version (or -1 for
+ * any), return the object we need to access all the symbols in the pair of
+ * libraries we need. Returns NULL if the library can't be found. Returns
+ * NULL and logs a warning if the library can be found but cannot be used for
+ * some reason.
+ */
+static pg_icu_library *
+load_icu_library(int major_version, int minor_version)
+{
+ UVersionInfo version_info;
+ pg_icu_library *lib;
+ void *libicui18n_handle = NULL;
+ void *libicuuc_handle = NULL;
+
+ /*
+ * We don't dare open libraries outside the range that we know has an API
+ * compatible with the headers we are compiling against.
+ */
+ if (major_version < PG_MIN_ICU_MAJOR_VERSION ||
+ major_version > PG_MAX_ICU_MAJOR_VERSION)
+ {
+ elog(WARNING,
+ "ICU version must be between %d and %d",
+ PG_MIN_ICU_MAJOR_VERSION,
+ PG_MAX_ICU_MAJOR_VERSION);
+ return NULL;
+ }
+
+ /*
+ * We were compiled against a certain version of ICU, though the minor
+ * version might have changed if the library was upgraded. Does it
+ * satisfy the request?
+ */
+ u_getVersion(version_info);
+ if (version_info[0] == major_version &&
+ (minor_version == -1 || version_info[1] == minor_version))
+ {
+ /*
+ * These assignments will fail to compile if an incompatible API
+ * change is made to some future version of ICU, at which point we
+ * might need to consider special treatment for different major
+ * version ranges, with intermediate trampoline functions.
+ */
+ lib = MemoryContextAllocZero(TopMemoryContext, sizeof(*lib));
+ lib->getICUVersion = u_getVersion;
+ lib->getUnicodeVersion = u_getUnicodeVersion;
+ lib->getCLDRVersion = ulocdata_getCLDRVersion;
+ lib->open = ucol_open;
+ lib->close = ucol_close;
+ lib->getCollatorVersion = ucol_getVersion;
+ lib->getUCAVersion = ucol_getUCAVersion;
+ lib->versionToString = u_versionToString;
+ lib->strcoll = ucol_strcoll;
+ lib->strcollUTF8 = ucol_strcollUTF8;
+ lib->getSortKey = ucol_getSortKey;
+ lib->nextSortKeyPart = ucol_nextSortKeyPart;
+ lib->setUTF8 = uiter_setUTF8;
+ lib->errorName = u_errorName;
+ lib->strToUpper = u_strToUpper;
+ lib->strToLower = u_strToLower;
+ lib->strToTitle = u_strToTitle;
+ lib->setAttribute = ucol_setAttribute;
+
+ /*
+ * Also assert the size of a couple of types used as output buffers,
+ * as a canary to tell us to add extra padding in the (unlikely) event
+ * that a later release makes these values smaller.
+ */
+ StaticAssertStmt(U_MAX_VERSION_STRING_LENGTH == 20,
+ "u_versionToString output buffer size changed incompatibly");
+ StaticAssertStmt(U_MAX_VERSION_LENGTH == 4,
+ "ucol_getVersion output buffer size changed incompatibly");
+ }
+ else
+ {
+ /* This is an older version, so we'll need to use dlopen(). */
+ char libicui18n_name[MAXPGPATH];
+ char libicuuc_name[MAXPGPATH];
+
+ /* Load the internationalization library. */
+ make_icu_library_name(libicui18n_name, ICU_I18N, major_version, minor_version);
+ libicui18n_handle = dlopen(libicui18n_name, RTLD_NOW | RTLD_LOCAL);
+ if (!libicui18n_handle)
+ return NULL;
+
+ /* Load the common library. */
+ make_icu_library_name(libicuuc_name, ICU_UC, major_version, minor_version);
+ libicuuc_handle = dlopen(libicuuc_name, RTLD_NOW | RTLD_LOCAL);
+ if (!libicui18n_handle)
+ {
+ elog(WARNING, "found library \"%s\" but not companion library \"%s\"",
+ libicui18n_name, libicuuc_name);
+ dlclose(libicui18n_handle);
+ return NULL;
+ }
+
+ /*
+ * We only allocate the pg_icu_library object after successfully
+ * opening the libraries to minimize the work done in the ENOENT case,
+ * when probing a range of versions. That means we might need to
+ * clean up on allocation failure.
+ */
+ lib = MemoryContextAllocExtended(TopMemoryContext, sizeof(*lib),
+ MCXT_ALLOC_NO_OOM);
+ if (!lib)
+ {
+ dlclose(libicui18n_handle);
+ dlclose(libicuuc_handle);
+ elog(ERROR, "out of memory");
+ }
+
+ /* Now try to find all the symbols we need. */
+ lib->getICUVersion = get_icu_function(libicui18n_handle,
+ "u_getVersion",
+ major_version);
+ lib->getUnicodeVersion = get_icu_function(libicui18n_handle,
+ "u_getUnicodeVersion",
+ major_version);
+ lib->getCLDRVersion = get_icu_function(libicui18n_handle,
+ "ulocdata_getCLDRVersion",
+ major_version);
+ lib->open = get_icu_function(libicui18n_handle,
+ "ucol_open",
+ major_version);
+ lib->close = get_icu_function(libicui18n_handle,
+ "ucol_close",
+ major_version);
+ lib->getCollatorVersion = get_icu_function(libicui18n_handle,
+ "ucol_getVersion",
+ major_version);
+ lib->getUCAVersion = get_icu_function(libicui18n_handle,
+ "ucol_getUCAVersion",
+ major_version);
+ lib->versionToString = get_icu_function(libicui18n_handle,
+ "u_versionToString",
+ major_version);
+ lib->strcoll = get_icu_function(libicui18n_handle,
+ "ucol_strcoll",
+ major_version);
+ lib->strcollUTF8 = get_icu_function(libicui18n_handle,
+ "ucol_strcollUTF8",
+ major_version);
+ lib->getSortKey = get_icu_function(libicui18n_handle,
+ "ucol_getSortKey",
+ major_version);
+ lib->nextSortKeyPart = get_icu_function(libicui18n_handle,
+ "ucol_nextSortKeyPart",
+ major_version);
+ lib->setUTF8 = get_icu_function(libicui18n_handle,
+ "uiter_setUTF8",
+ major_version);
+ lib->errorName = get_icu_function(libicui18n_handle,
+ "u_errorName",
+ major_version);
+ lib->strToUpper = get_icu_function(libicuuc_handle,
+ "u_strToUpper",
+ major_version);
+ lib->strToLower = get_icu_function(libicuuc_handle,
+ "u_strToLower",
+ major_version);
+ lib->strToTitle = get_icu_function(libicuuc_handle,
+ "u_strToTitle",
+ major_version);
+ lib->setAttribute = get_icu_function(libicui18n_handle,
+ "ucol_setAttribute",
+ major_version);
+
+ /* Did we find everything? */
+ if (!lib->getICUVersion ||
+ !lib->getUnicodeVersion ||
+ !lib->getCLDRVersion ||
+ !lib->open ||
+ !lib->close ||
+ !lib->getCollatorVersion ||
+ !lib->getUCAVersion ||
+ !lib->versionToString ||
+ !lib->strcoll ||
+ !lib->strcollUTF8 ||
+ !lib->getSortKey ||
+ !lib->nextSortKeyPart ||
+ !lib->setUTF8 ||
+ !lib->errorName ||
+ !lib->strToUpper ||
+ !lib->strToLower ||
+ !lib->strToTitle ||
+ !lib->setAttribute)
+ {
+ dlclose(libicui18n_handle);
+ dlclose(libicuuc_handle);
+ pfree(lib);
+ ereport(WARNING,
+ (errmsg("could not find all expected symbols in libraries \"%s\" and \"%s\"",
+ libicui18n_name, libicuuc_name)));
+ return NULL;
+ }
+ }
+
+ /* Is this major.minor already loaded? */
+ lib->getICUVersion(version_info);
+ lib->major_version = version_info[0];
+ lib->minor_version = version_info[1];
+ for (pg_icu_library *lib2 = icu_library_list; lib2; lib2 = lib2->next)
+ {
+ if (lib2->major_version == lib->major_version &&
+ lib2->minor_version == lib->minor_version)
+ {
+ if (libicui18n_handle)
+ dlclose(libicui18n_handle);
+ if (libicuuc_handle)
+ dlclose(libicuuc_handle);
+ pfree(lib);
+
+ /* Return the one we already had. */
+ return lib2;
+ }
+ }
+
+ /* Add to list of loaded libraries. */
+ lib->next = icu_library_list;
+ icu_library_list = lib;
+
+ return lib;
+}
+
+static pg_icu_library *
+get_icu_library_list(void)
+{
+ char *copy;
+ char *token;
+ char *saveptr;
+
+ if (icu_library_list_fully_loaded)
+ return icu_library_list;
+
+ copy = pstrdup(icu_library_versions);
+ token = strtok_r(copy, ",", &saveptr);
+ while (token)
+ {
+ int major_version;
+ int minor_version;
+
+ /* Ignore spaces between commas. */
+ while (*token == ' ')
+ ++token;
+
+ if (strcmp(token, "*") == 0)
+ {
+ /* Try to load every supportable major library version. */
+ for (int i = PG_MIN_ICU_MAJOR_VERSION; i <= PG_MAX_ICU_MAJOR_VERSION; ++i)
+ load_icu_library(i, -1);
+ }
+ else if (sscanf(token, "%d.%d", &major_version, &minor_version) == 2)
+ {
+ /* Try to load a version with an explicit minor version provided. */
+ if (!load_icu_library(major_version, minor_version))
+ ereport(WARNING,
+ (errmsg("could not open ICU library \"%s\"", token)));
+ }
+ else if (sscanf(token, "%d", &major_version) == 1)
+ {
+ /* Try to load a major version through symlinks. */
+ if (!load_icu_library(major_version, -1))
+ ereport(WARNING,
+ (errmsg("could not open ICU library \"%s\"", token)));
+ }
+ else
+ ereport(WARNING,
+ (errmsg("could not parse ICU library version \"%s\"", token)));
+
+ token = strtok_r(NULL, ",", &saveptr);
+ }
+ pfree(copy);
+
+ icu_library_list_fully_loaded = true;
+
+ return icu_library_list;
+}
+
+static pg_icu_library *
+get_default_icu_library(void)
+{
+ int major_version;
+ int minor_version;
+
+ if (default_icu_library)
+ return default_icu_library;
+
+ if (default_icu_library_version[0] == 0)
+ {
+ /* Use the linked version by default. */
+ default_icu_library = load_icu_library(PG_MAX_ICU_MAJOR_VERSION, -1);
+ Assert(default_icu_library);
+ }
+ else if (sscanf(default_icu_library_version, "%d.%d", &major_version, &minor_version) == 2)
+ {
+ /* Try to load a version with an explicit major.minor version. */
+ default_icu_library = load_icu_library(major_version, minor_version);
+ }
+ else if (sscanf(default_icu_library_version, "%d", &major_version) == 1)
+ {
+ /* Try to load a version using only major (usually a symlink on Unix). */
+ default_icu_library = load_icu_library(major_version, -1);
+ }
+ else
+ {
+ ereport(WARNING,
+ (errmsg("could not parse default_icu_library_version \"%s\"",
+ default_icu_library_version)));
+ }
+
+ if (!default_icu_library_version)
+ {
+ /*
+ * Fall back to the linked version with a warning if the above
+ * attempts failed.
+ */
+ default_icu_library = load_icu_library(PG_MAX_ICU_MAJOR_VERSION, -1);
+ Assert(default_icu_library);
+ ereport(WARNING,
+ (errmsg("could not load ICU library version \"%s\", so using linked version %d.%d instead",
+ default_icu_library_version,
+ default_icu_library->major_version,
+ default_icu_library->minor_version)));
+ Assert(default_icu_library);
+ }
+
+ return default_icu_library;
+}
+
+/*
+ * Try to open a collator with a specific version from a given library.
+ * Returns NULL on failure.
+ */
+static UCollator *
+get_icu_collator(pg_icu_library *lib,
+ const char *locale,
+ const char *collversion)
+{
+ UErrorCode status;
+ UCollator *collator;
+ UVersionInfo version_info;
+ char version_info_string[U_MAX_VERSION_STRING_LENGTH];
+
+ /* Can we even open it? */
+ status = U_ZERO_ERROR;
+ collator = lib->open(locale, &status);
+ if (!collator)
+ return NULL;
+
+ /*
+ * Does it have the requested version? We tolerate a null collversion
+ * argument only for bootrapping in initdb --locale-provider=icu, where we
+ * accept the first library we try.
+ */
+ if (collversion)
+ {
+ lib->getCollatorVersion(collator, version_info);
+ lib->versionToString(version_info, version_info_string);
+ if (strcmp(version_info_string, collversion) != 0)
+ {
+ lib->close(collator);
+ return NULL;
+ }
+ } else
+ Assert(!IsUnderPostmaster);
+
+ /* XXX this can raise an error and leak collator! */
+ if (lib->major_version < 54)
+ icu_set_collation_attributes(lib, collator, locale);
+
+ return collator;
+}
+
+#endif
+
+/*
+ * Returns true if a collator with u_getVersion() matching collversion could
+ * not be found in any available ICU library, so the default library was used
+ * instead.
+ */
+bool
make_icu_collator(const char *iculocstr,
+ const char *collversion,
struct pg_locale_struct *resultp)
{
+ bool using_default = false;
#ifdef USE_ICU
- UCollator *collator;
- UErrorCode status;
+ pg_icu_library *lib = NULL;
+ UCollator *collator = NULL;
- status = U_ZERO_ERROR;
- collator = ucol_open(iculocstr, &status);
- if (U_FAILURE(status))
- ereport(ERROR,
- (errmsg("could not open collator for locale \"%s\": %s",
- iculocstr, u_errorName(status))));
+ /*
+ * Try the default library first, which might avoid the need to dlopen()
+ * libraries in the common case that it's the version we're linked
+ * against.
+ */
+ lib = get_default_icu_library();
+ collator = get_icu_collator(lib, iculocstr, collversion);
- if (U_ICU_VERSION_MAJOR_NUM < 54)
- icu_set_collation_attributes(collator, iculocstr);
+ /*
+ * If that didn't succeed, try every available library.
+ */
+ if (!collator)
+ {
+ for (lib = get_icu_library_list(); lib; lib = lib->next)
+ {
+ collator = get_icu_collator(lib, iculocstr, collversion);
+ if (collator)
+ break;
+ }
+ }
+
+ /*
+ * If we didn't find a match, it's time to fall back to our default
+ * library. We'll also return true so the caller can generate a more
+ * specific warning about what to do.
+ */
+ if (!collator)
+ {
+ UVersionInfo version_info;
+ char version_info_string[U_MAX_VERSION_STRING_LENGTH];
+ UErrorCode status;
+
+ lib = get_default_icu_library();
+
+ status = U_ZERO_ERROR;
+ collator = lib->open(iculocstr, &status);
+ if (!collator)
+ ereport(ERROR,
+ (errmsg("could not open collator for locale \"%s\": %s",
+ iculocstr, lib->errorName(status))));
+
+ lib->getCollatorVersion(collator, version_info);
+ lib->versionToString(version_info, version_info_string);
+ ereport(WARNING,
+ (errmsg("could not find ICU collator for locale \"%s\" with "
+ "version %s, so using version %s from default "
+ "ICU library %d.%d instead",
+ iculocstr, collversion, version_info_string,
+ lib->major_version, lib->minor_version)));
+
+ using_default = true;
+ }
+
+ Assert(lib);
/* We will leak this string if the caller errors later :-( */
resultp->info.icu.locale = MemoryContextStrdup(TopMemoryContext, iculocstr);
resultp->info.icu.ucol = collator;
+ resultp->info.icu.lib = lib;
#else /* not USE_ICU */
/* could get here if a collation was created by a build with ICU */
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("ICU is not supported in this build")));
#endif /* not USE_ICU */
+
+ return using_default;
}
@@ -1504,6 +2043,7 @@ pg_newlocale_from_collation(Oid collid)
pg_locale_t resultp;
Datum datum;
bool isnull;
+ const char *collversion;
tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
if (!HeapTupleIsValid(tp))
@@ -1515,6 +2055,10 @@ pg_newlocale_from_collation(Oid collid)
result.provider = collform->collprovider;
result.deterministic = collform->collisdeterministic;
+ datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
+ &isnull);
+ collversion = isnull ? NULL : TextDatumGetCString(datum);
+
if (collform->collprovider == COLLPROVIDER_LIBC)
{
#ifdef HAVE_LOCALE_T
@@ -1584,23 +2128,37 @@ pg_newlocale_from_collation(Oid collid)
datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_colliculocale, &isnull);
Assert(!isnull);
iculocstr = TextDatumGetCString(datum);
- make_icu_collator(iculocstr, &result);
+ if (!collversion)
+ elog(ERROR, "ICU collation lacks version");
+ if (make_icu_collator(iculocstr, collversion, &result))
+ {
+ ereport(WARNING,
+ errmsg("collation \"%s\" version mismatch",
+ NameStr(collform->collname)),
+ errdetail("The collation in the database was created using "
+ "locale \"%s\" version %s, "
+ "but no ICU library with a matching collator is available",
+ iculocstr, collversion),
+ errhint("Install a version of ICU that provides locale \"%s\" "
+ "version %s, or rebuild all objects "
+ "affected by this collation and run "
+ "ALTER COLLATION %s REFRESH VERSION.",
+ iculocstr, collversion,
+ quote_qualified_identifier(get_namespace_name(collform->collnamespace),
+ NameStr(collform->collname))));
+ }
}
- datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collversion,
- &isnull);
- if (!isnull)
+ if (collform->collprovider == COLLPROVIDER_LIBC && collversion)
{
char *actual_versionstr;
- char *collversionstr;
+ char *locale;
- collversionstr = TextDatumGetCString(datum);
-
- datum = SysCacheGetAttr(COLLOID, tp, collform->collprovider == COLLPROVIDER_ICU ? Anum_pg_collation_colliculocale : Anum_pg_collation_collcollate, &isnull);
+ datum = SysCacheGetAttr(COLLOID, tp, Anum_pg_collation_collcollate, &isnull);
Assert(!isnull);
+ locale = TextDatumGetCString(datum);
- actual_versionstr = get_collation_actual_version(collform->collprovider,
- TextDatumGetCString(datum));
+ actual_versionstr = get_collation_actual_version(collform->collprovider, locale);
if (!actual_versionstr)
{
/*
@@ -1613,18 +2171,20 @@ pg_newlocale_from_collation(Oid collid)
NameStr(collform->collname))));
}
- if (strcmp(actual_versionstr, collversionstr) != 0)
+ if (strcmp(actual_versionstr, collversion) != 0)
+ {
ereport(WARNING,
(errmsg("collation \"%s\" has version mismatch",
NameStr(collform->collname)),
errdetail("The collation in the database was created using version %s, "
"but the operating system provides version %s.",
- collversionstr, actual_versionstr),
+ collversion, actual_versionstr),
errhint("Rebuild all objects affected by this collation and run "
"ALTER COLLATION %s REFRESH VERSION, "
"or build PostgreSQL with the right library version.",
quote_qualified_identifier(get_namespace_name(collform->collnamespace),
NameStr(collform->collname)))));
+ }
}
ReleaseSysCache(tp);
@@ -1651,21 +2211,27 @@ get_collation_actual_version(char collprovider, const char *collcollate)
#ifdef USE_ICU
if (collprovider == COLLPROVIDER_ICU)
{
+ pg_icu_library *lib;
UCollator *collator;
UErrorCode status;
UVersionInfo versioninfo;
char buf[U_MAX_VERSION_STRING_LENGTH];
+ /*
+ * Use the default library, but other versions might also be active
+ * and can be seen with pg_icu_collation_versions().
+ */
+ lib = get_default_icu_library();
status = U_ZERO_ERROR;
- collator = ucol_open(collcollate, &status);
+ collator = lib->open(collcollate, &status);
if (U_FAILURE(status))
ereport(ERROR,
(errmsg("could not open collator for locale \"%s\": %s",
- collcollate, u_errorName(status))));
- ucol_getVersion(collator, versioninfo);
- ucol_close(collator);
+ collcollate, lib->errorName(status))));
+ lib->getCollatorVersion(collator, versioninfo);
+ lib->close(collator);
- u_versionToString(versioninfo, buf);
+ lib->versionToString(versioninfo, buf);
collversion = pstrdup(buf);
}
else
@@ -1731,8 +2297,110 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+Datum
+pg_icu_library_versions(PG_FUNCTION_ARGS)
+{
+#ifdef USE_ICU
+#define PG_ICU_AVAILABLE_ICU_LIRBARIES_COLS 3
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Datum values[PG_ICU_AVAILABLE_ICU_LIRBARIES_COLS];
+ bool nulls[PG_ICU_AVAILABLE_ICU_LIRBARIES_COLS];
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ for (pg_icu_library *lib = get_icu_library_list(); lib; lib = lib->next)
+ {
+ UErrorCode status;
+ UVersionInfo version_info;
+ char version_string[U_MAX_VERSION_STRING_LENGTH];
+
+ lib->getICUVersion(version_info);
+ lib->versionToString(version_info, version_string);
+ values[0] = PointerGetDatum(cstring_to_text(version_string));
+ nulls[0] = false;
+
+ lib->getUnicodeVersion(version_info);
+ lib->versionToString(version_info, version_string);
+ values[1] = PointerGetDatum(cstring_to_text(version_string));
+ nulls[1] = false;
+
+ status = U_ZERO_ERROR;
+ lib->getCLDRVersion(version_info, &status);
+ if (U_SUCCESS(status))
+ {
+ lib->versionToString(version_info, version_string);
+ values[2] = PointerGetDatum(cstring_to_text(version_string));
+ nulls[2] = false;
+ }
+ else
+ {
+ nulls[2] = true;
+ }
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+ }
+#endif
+
+ return (Datum) 0;
+}
+
+Datum
+pg_icu_collation_versions(PG_FUNCTION_ARGS)
+{
+#ifdef USE_ICU
+#define PG_ICU_AVAILABLE_ICU_LIRBARIES_COLS 3
+ const char *locale = text_to_cstring(PG_GETARG_TEXT_PP(0));
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ Datum values[PG_ICU_AVAILABLE_ICU_LIRBARIES_COLS];
+ bool nulls[PG_ICU_AVAILABLE_ICU_LIRBARIES_COLS];
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ for (pg_icu_library *lib = get_icu_library_list(); lib; lib = lib->next)
+ {
+ UErrorCode status;
+ UCollator *collator;
+ UVersionInfo version_info;
+ char version_string[U_MAX_VERSION_STRING_LENGTH];
+
+ status = U_ZERO_ERROR;
+ collator = lib->open(locale, &status);
+ if (!collator)
+ {
+ if (U_FAILURE(status))
+ ereport(WARNING,
+ (errmsg("could not open collator for locale \"%s\" from ICU %d.%d: %s",
+ locale,
+ lib->major_version,
+ lib->minor_version,
+ lib->errorName(status))));
+ continue;
+ }
+
+ lib->getICUVersion(version_info);
+ lib->versionToString(version_info, version_string);
+ values[0] = PointerGetDatum(cstring_to_text(version_string));
+ nulls[0] = false;
+
+ lib->getUCAVersion(collator, version_info);
+ lib->versionToString(version_info, version_string);
+ values[1] = PointerGetDatum(cstring_to_text(version_string));
+ nulls[1] = false;
+
+ lib->getCollatorVersion(collator, version_info);
+ lib->versionToString(version_info, version_string);
+ values[2] = PointerGetDatum(cstring_to_text(version_string));
+ nulls[2] = false;
+
+ tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
+ }
+#endif
+
+ return (Datum) 0;
+}
#ifdef USE_ICU
+
/*
* Converter object for converting between ICU's UChar strings and C strings
* in database encoding. Since the database encoding doesn't change, we only
@@ -1855,9 +2523,10 @@ icu_from_uchar(char **result, const UChar *buff_uchar, int32_t len_uchar)
* ucol_open(), so this is only necessary for emulating this behavior on older
* versions.
*/
-pg_attribute_unused()
static void
-icu_set_collation_attributes(UCollator *collator, const char *loc)
+icu_set_collation_attributes(pg_icu_library *lib,
+ UCollator *collator,
+ const char *loc)
{
char *str = asc_tolower(loc, strlen(loc));
@@ -1886,6 +2555,8 @@ icu_set_collation_attributes(UCollator *collator, const char *loc)
/*
* See attribute name and value lists in ICU i18n/coll.cpp
+ *
+ * XXX Are these enumerator values stable across releases?
*/
if (strcmp(name, "colstrength") == 0)
uattr = UCOL_STRENGTH;
@@ -1931,7 +2602,7 @@ icu_set_collation_attributes(UCollator *collator, const char *loc)
status = U_ILLEGAL_ARGUMENT_ERROR;
if (status == U_ZERO_ERROR)
- ucol_setAttribute(collator, uattr, uvalue, &status);
+ lib->setAttribute(collator, uattr, uvalue, &status);
/*
* Pretend the error came from ucol_open(), for consistent error
@@ -1940,7 +2611,7 @@ icu_set_collation_attributes(UCollator *collator, const char *loc)
if (U_FAILURE(status))
ereport(ERROR,
(errmsg("could not open collator for locale \"%s\": %s",
- loc, u_errorName(status))));
+ loc, lib->errorName(status))));
}
}
}
@@ -1954,19 +2625,18 @@ void
check_icu_locale(const char *icu_locale)
{
#ifdef USE_ICU
+ pg_icu_library *lib;
UCollator *collator;
UErrorCode status;
+ lib = get_default_icu_library();
status = U_ZERO_ERROR;
- collator = ucol_open(icu_locale, &status);
+ collator = lib->open(icu_locale, &status);
if (U_FAILURE(status))
ereport(ERROR,
(errmsg("could not open collator for locale \"%s\": %s",
- icu_locale, u_errorName(status))));
-
- if (U_ICU_VERSION_MAJOR_NUM < 54)
- icu_set_collation_attributes(collator, icu_locale);
- ucol_close(collator);
+ icu_locale, lib->errorName(status))));
+ lib->close(collator);
#else
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
diff --git a/src/backend/utils/adt/varchar.c b/src/backend/utils/adt/varchar.c
index 68e2e6f7a7..e0c86870e0 100644
--- a/src/backend/utils/adt/varchar.c
+++ b/src/backend/utils/adt/varchar.c
@@ -1026,11 +1026,11 @@ hashbpchar(PG_FUNCTION_ARGS)
ulen = icu_to_uchar(&uchar, keydata, keylen);
- bsize = ucol_getSortKey(mylocale->info.icu.ucol,
- uchar, ulen, NULL, 0);
+ bsize = PG_ICU_LIB(mylocale)->getSortKey(PG_ICU_COL(mylocale),
+ uchar, ulen, NULL, 0);
buf = palloc(bsize);
- ucol_getSortKey(mylocale->info.icu.ucol,
- uchar, ulen, buf, bsize);
+ PG_ICU_LIB(mylocale)->getSortKey(PG_ICU_COL(mylocale),
+ uchar, ulen, buf, bsize);
result = hash_any(buf, bsize);
@@ -1087,11 +1087,11 @@ hashbpcharextended(PG_FUNCTION_ARGS)
ulen = icu_to_uchar(&uchar, VARDATA_ANY(key), VARSIZE_ANY_EXHDR(key));
- bsize = ucol_getSortKey(mylocale->info.icu.ucol,
- uchar, ulen, NULL, 0);
+ bsize = PG_ICU_LIB(mylocale)->getSortKey(PG_ICU_COL(mylocale),
+ uchar, ulen, NULL, 0);
buf = palloc(bsize);
- ucol_getSortKey(mylocale->info.icu.ucol,
- uchar, ulen, buf, bsize);
+ PG_ICU_LIB(mylocale)->getSortKey(PG_ICU_COL(mylocale),
+ uchar, ulen, buf, bsize);
result = hash_any_extended(buf, bsize, PG_GETARG_INT64(1));
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index c5e7ee7ca2..cf891a5654 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -1667,13 +1667,14 @@ varstr_cmp(const char *arg1, int len1, const char *arg2, int len2, Oid collid)
UErrorCode status;
status = U_ZERO_ERROR;
- result = ucol_strcollUTF8(mylocale->info.icu.ucol,
- arg1, len1,
- arg2, len2,
- &status);
+ result = PG_ICU_LIB(mylocale)->strcollUTF8(PG_ICU_COL(mylocale),
+ arg1, len1,
+ arg2, len2,
+ &status);
if (U_FAILURE(status))
ereport(ERROR,
- (errmsg("collation failed: %s", u_errorName(status))));
+ (errmsg("collation failed: %s",
+ PG_ICU_LIB(mylocale)->errorName(status))));
}
else
#endif
@@ -1686,9 +1687,9 @@ varstr_cmp(const char *arg1, int len1, const char *arg2, int len2, Oid collid)
ulen1 = icu_to_uchar(&uchar1, arg1, len1);
ulen2 = icu_to_uchar(&uchar2, arg2, len2);
- result = ucol_strcoll(mylocale->info.icu.ucol,
- uchar1, ulen1,
- uchar2, ulen2);
+ result = PG_ICU_LIB(mylocale)->strcoll(PG_ICU_COL(mylocale),
+ uchar1, ulen1,
+ uchar2, ulen2);
pfree(uchar1);
pfree(uchar2);
@@ -2388,13 +2389,14 @@ varstrfastcmp_locale(char *a1p, int len1, char *a2p, int len2, SortSupport ssup)
UErrorCode status;
status = U_ZERO_ERROR;
- result = ucol_strcollUTF8(sss->locale->info.icu.ucol,
- a1p, len1,
- a2p, len2,
- &status);
+ result = PG_ICU_LIB(sss->locale)->strcollUTF8(PG_ICU_COL(sss->locale),
+ a1p, len1,
+ a2p, len2,
+ &status);
if (U_FAILURE(status))
ereport(ERROR,
- (errmsg("collation failed: %s", u_errorName(status))));
+ (errmsg("collation failed: %s",
+ PG_ICU_LIB(sss->locale)->errorName(status))));
}
else
#endif
@@ -2407,9 +2409,9 @@ varstrfastcmp_locale(char *a1p, int len1, char *a2p, int len2, SortSupport ssup)
ulen1 = icu_to_uchar(&uchar1, a1p, len1);
ulen2 = icu_to_uchar(&uchar2, a2p, len2);
- result = ucol_strcoll(sss->locale->info.icu.ucol,
- uchar1, ulen1,
- uchar2, ulen2);
+ result = PG_ICU_LIB(sss->locale)->strcoll(PG_ICU_COL(sss->locale),
+ uchar1, ulen1,
+ uchar2, ulen2);
pfree(uchar1);
pfree(uchar2);
@@ -2569,24 +2571,24 @@ varstr_abbrev_convert(Datum original, SortSupport ssup)
uint32_t state[2];
UErrorCode status;
- uiter_setUTF8(&iter, sss->buf1, len);
+ PG_ICU_LIB(sss->locale)->setUTF8(&iter, sss->buf1, len);
state[0] = state[1] = 0; /* won't need that again */
status = U_ZERO_ERROR;
- bsize = ucol_nextSortKeyPart(sss->locale->info.icu.ucol,
- &iter,
- state,
- (uint8_t *) sss->buf2,
- Min(sizeof(Datum), sss->buflen2),
- &status);
+ bsize = PG_ICU_LIB(sss->locale)->nextSortKeyPart(PG_ICU_COL(sss->locale),
+ &iter,
+ state,
+ (uint8_t *) sss->buf2,
+ Min(sizeof(Datum), sss->buflen2),
+ &status);
if (U_FAILURE(status))
ereport(ERROR,
(errmsg("sort key generation failed: %s",
- u_errorName(status))));
+ PG_ICU_LIB(sss->locale)->errorName(status))));
}
else
- bsize = ucol_getSortKey(sss->locale->info.icu.ucol,
- uchar, ulen,
- (uint8_t *) sss->buf2, sss->buflen2);
+ bsize = PG_ICU_LIB(sss->locale)->getSortKey(PG_ICU_COL(sss->locale),
+ uchar, ulen,
+ (uint8_t *) sss->buf2, sss->buflen2);
}
else
#endif
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index a990c833c5..236ec6d682 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -317,6 +317,7 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
char *collate;
char *ctype;
char *iculocale;
+ char *collversion;
/* Fetch our pg_database row normally, via syscache */
tup = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
@@ -404,6 +405,9 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_datctype, &isnull);
Assert(!isnull);
ctype = TextDatumGetCString(datum);
+ datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_datcollversion,
+ &isnull);
+ collversion = isnull ? NULL : TextDatumGetCString(datum);
if (pg_perm_setlocale(LC_COLLATE, collate) == NULL)
ereport(FATAL,
@@ -424,7 +428,20 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_daticulocale, &isnull);
Assert(!isnull);
iculocale = TextDatumGetCString(datum);
- make_icu_collator(iculocale, &default_locale);
+ if (make_icu_collator(iculocale, collversion, &default_locale))
+ {
+ ereport(WARNING,
+ errmsg("database \"%s\" has a collation version mismatch",
+ name),
+ errdetail("The database was created using ICU locale \"%s\" version %s, "
+ "but no ICU library with a matching collator is available",
+ iculocale, collversion),
+ errhint("Install a version of ICU that provides locale \"%s\" "
+ "version %s, or rebuild all objects "
+ "in this database that use the default collation and run "
+ "ALTER DATABASE %s REFRESH COLLATION VERSION.",
+ iculocale, collversion, name));
+ }
}
else
iculocale = NULL;
@@ -443,32 +460,29 @@ CheckMyDatabase(const char *name, bool am_superuser, bool override_allow_connect
* pg_newlocale_from_collation(). Note that here we warn instead of error
* in any case, so that we don't prevent connecting.
*/
- datum = SysCacheGetAttr(DATABASEOID, tup, Anum_pg_database_datcollversion,
- &isnull);
- if (!isnull)
+ if (dbform->datlocprovider == COLLPROVIDER_LIBC && collversion)
{
char *actual_versionstr;
- char *collversionstr;
-
- collversionstr = TextDatumGetCString(datum);
- actual_versionstr = get_collation_actual_version(dbform->datlocprovider, dbform->datlocprovider == COLLPROVIDER_ICU ? iculocale : collate);
+ actual_versionstr = get_collation_actual_version(dbform->datlocprovider, collate);
if (!actual_versionstr)
/* should not happen */
elog(WARNING,
"database \"%s\" has no actual collation version, but a version was recorded",
name);
- else if (strcmp(actual_versionstr, collversionstr) != 0)
+ else if (strcmp(actual_versionstr, collversion) != 0)
+ {
ereport(WARNING,
(errmsg("database \"%s\" has a collation version mismatch",
name),
errdetail("The database was created using collation version %s, "
"but the operating system provides version %s.",
- collversionstr, actual_versionstr),
+ collversion, actual_versionstr),
errhint("Rebuild all objects in this database that use the default collation and run "
"ALTER DATABASE %s REFRESH COLLATION VERSION, "
"or build PostgreSQL with the right library version.",
quote_identifier(name))));
+ }
}
/* Make the locale settings visible as GUC variables, too */
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 349dd6a537..5fd08f7693 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -3941,7 +3941,45 @@ struct config_string ConfigureNamesString[] =
},
{
- {"krb_server_keyfile", PGC_SIGHUP, CONN_AUTH_AUTH,
+ {"icu_library_path", PGC_SUSET, COMPAT_OPTIONS_PREVIOUS,
+ gettext_noop("Sets the path for dynamically loadable ICU libraries."),
+ gettext_noop("If versions of ICU other than the one that "
+ "PostgreSQL is linked against are needed, they will "
+ "be opened from this directory. If empty, the "
+ "system linker search path will be used."),
+ GUC_SUPERUSER_ONLY
+ },
+ &icu_library_path,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"icu_library_versions", PGC_SUSET, COMPAT_OPTIONS_PREVIOUS,
+ gettext_noop("Sets the available ICU library versions."),
+ gettext_noop("A comma-separated list of major or major.minor ICU versions "
+ "that will be searched for referenced collation versions. Use * "
+ "for all possible versions."),
+ GUC_SUPERUSER_ONLY
+ },
+ &icu_library_versions,
+ "*",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"default_icu_library_version", PGC_SIGHUP, COMPAT_OPTIONS_PREVIOUS,
+ gettext_noop("Sets the ICU library version used to create new collations and databases."),
+ gettext_noop("A major or major.minor ICU version, or empty string for the linked version."),
+ GUC_SUPERUSER_ONLY
+ },
+ &default_icu_library_version,
+ "",
+ NULL, NULL, NULL
+ },
+
+ {
+ {"krb_server_keyfile", PGC_POSTMASTER, CONN_AUTH_AUTH,
gettext_noop("Sets the location of the Kerberos server key file."),
NULL,
GUC_SUPERUSER_ONLY
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 868d21c351..93a0ad6406 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -727,6 +727,16 @@
#lc_numeric = 'C' # locale for number formatting
#lc_time = 'C' # locale for time formatting
+#icu_library_path = '' # path for dynamically loaded ICU
+ # libraries
+#icu_library_versions = '*' # comma-separated list of ICU major
+ # or major.minor versions to make
+ # available, or * for all major
+ # versions that can be found
+#default_icu_library_version = '' # version of ICU to use for new
+ # databases and collations, defaults
+ # to the latest version
+
# default configuration for text search
#default_text_search_config = 'pg_catalog.simple'
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f9301b2627..607968b340 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11733,6 +11733,29 @@
proname => 'pg_database_collation_actual_version', procost => '100',
provolatile => 'v', prorettype => 'text', proargtypes => 'oid',
prosrc => 'pg_database_collation_actual_version' },
+{ oid => '8888', descr => 'get available ICU library versions',
+ proname => 'pg_icu_library_versions', prorettype => 'record',
+ procost => '10',
+ prorows => '2',
+ proretset => 't',
+ provolatile => 'v',
+ proargtypes => '',
+ proallargtypes => '{text,text,text}',
+ proargmodes => '{o,o,o}',
+ proargnames => '{icu_version,unicode_version,cldr_version}',
+ prosrc => 'pg_icu_library_versions' },
+{ oid => '8889', descr => 'get available ICU collation versions',
+ proname => 'pg_icu_collation_versions', prorettype => 'record',
+ procost => '10',
+ prorows => '2',
+ proretset => 't',
+ provolatile => 'v',
+ proargtypes => 'text',
+ proallargtypes => '{text,text,text,text}',
+ proargmodes => '{i,o,o,o}',
+ proargnames => '{locale,icu_version,uca_version,collator_version}',
+ prosrc => 'pg_icu_collation_versions' },
+
# system management/monitoring related functions
{ oid => '3353', descr => 'list files in the log directory',
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index a875942123..554b335df9 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -17,6 +17,7 @@
#endif
#ifdef USE_ICU
#include <unicode/ucol.h>
+#include <unicode/ubrk.h>
#endif
#ifdef USE_ICU
@@ -40,6 +41,9 @@ extern PGDLLIMPORT char *locale_messages;
extern PGDLLIMPORT char *locale_monetary;
extern PGDLLIMPORT char *locale_numeric;
extern PGDLLIMPORT char *locale_time;
+extern PGDLLIMPORT char *icu_library_path;
+extern PGDLLIMPORT char *icu_library_versions;
+extern PGDLLIMPORT char *default_icu_library_version;
/* lc_time localization cache */
extern PGDLLIMPORT char *localized_abbrev_days[];
@@ -63,6 +67,78 @@ extern struct lconv *PGLC_localeconv(void);
extern void cache_locale_time(void);
+#ifdef USE_ICU
+
+/*
+ * An ICU library version that we're either linked against or have loaded at
+ * runtime.
+ */
+typedef struct pg_icu_library
+{
+ int major_version;
+ int minor_version;
+ void (*getICUVersion) (UVersionInfo info);
+ void (*getUnicodeVersion) (UVersionInfo into);
+ void (*getCLDRVersion) (UVersionInfo info, UErrorCode *status);
+ UCollator *(*open) (const char *loc, UErrorCode *status);
+ void (*close) (UCollator *coll);
+ void (*getCollatorVersion) (const UCollator *coll, UVersionInfo info);
+ void (*getUCAVersion) (const UCollator *coll, UVersionInfo info);
+ void (*versionToString) (const UVersionInfo versionArray,
+ char *versionString);
+ UCollationResult (*strcoll) (const UCollator *coll,
+ const UChar *source,
+ int32_t sourceLength,
+ const UChar *target,
+ int32_t targetLength);
+ UCollationResult (*strcollUTF8) (const UCollator *coll,
+ const char *source,
+ int32_t sourceLength,
+ const char *target,
+ int32_t targetLength,
+ UErrorCode *status);
+ int32_t (*getSortKey) (const UCollator *coll,
+ const UChar *source,
+ int32_t sourceLength,
+ uint8_t *result,
+ int32_t resultLength);
+ int32_t (*nextSortKeyPart) (const UCollator *coll,
+ UCharIterator *iter,
+ uint32_t state[2],
+ uint8_t *dest,
+ int32_t count,
+ UErrorCode *status);
+ void (*setUTF8) (UCharIterator *iter,
+ const char *s,
+ int32_t length);
+ const char *(*errorName) (UErrorCode code);
+ int32_t (*strToUpper) (UChar *dest,
+ int32_t destCapacity,
+ const UChar *src,
+ int32_t srcLength,
+ const char *locale,
+ UErrorCode *pErrorCode);
+ int32_t (*strToLower) (UChar *dest,
+ int32_t destCapacity,
+ const UChar *src,
+ int32_t srcLength,
+ const char *locale,
+ UErrorCode *pErrorCode);
+ int32_t (*strToTitle) (UChar *dest,
+ int32_t destCapacity,
+ const UChar *src,
+ int32_t srcLength,
+ UBreakIterator *titleIter,
+ const char *locale,
+ UErrorCode *pErrorCode);
+ void (*setAttribute) (UCollator *coll,
+ UColAttribute attr,
+ UColAttributeValue value,
+ UErrorCode *status);
+ struct pg_icu_library *next;
+} pg_icu_library;
+
+#endif
/*
* We define our own wrapper around locale_t so we can keep the same
@@ -84,17 +160,24 @@ struct pg_locale_struct
{
const char *locale;
UCollator *ucol;
+ pg_icu_library *lib;
} icu;
#endif
int dummy; /* in case we have neither LOCALE_T nor ICU */
} info;
};
+#ifdef USE_ICU
+#define PG_ICU_LIB(x) ((x)->info.icu.lib)
+#define PG_ICU_COL(x) ((x)->info.icu.ucol)
+#endif
+
typedef struct pg_locale_struct *pg_locale_t;
extern PGDLLIMPORT struct pg_locale_struct default_locale;
-extern void make_icu_collator(const char *iculocstr,
+extern bool make_icu_collator(const char *iculocstr,
+ const char *collversion,
struct pg_locale_struct *resultp);
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
diff --git a/src/test/icu/meson.build b/src/test/icu/meson.build
index 5a4f53f37f..ac2672190e 100644
--- a/src/test/icu/meson.build
+++ b/src/test/icu/meson.build
@@ -5,6 +5,7 @@ tests += {
'tap': {
'tests': [
't/010_database.pl',
+ 't/020_multiversion.pl',
],
'env': {'with_icu': icu.found() ? 'yes' : 'no'},
},
diff --git a/src/test/icu/t/020_multiversion.pl b/src/test/icu/t/020_multiversion.pl
new file mode 100644
index 0000000000..c04df4c65d
--- /dev/null
+++ b/src/test/icu/t/020_multiversion.pl
@@ -0,0 +1,274 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+#
+# If one or more extra ICU versions is installed in the standard system library
+# search path, this test will detect them and run.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+if ($ENV{with_icu} ne 'yes')
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+my $node1 = PostgreSQL::Test::Cluster->new('node1');
+$node1->init;
+$node1->start;
+
+# Check which ICU versions are installed.
+my $highest_version = $node1->safe_psql('postgres', 'select max(icu_version::decimal) from pg_icu_library_versions()');
+my $lowest_version = $node1->safe_psql('postgres', 'select min(icu_version::decimal) from pg_icu_library_versions()');
+my $highest_major_version = int($highest_version);
+my $lowest_major_version = int($lowest_version);
+
+if ($highest_major_version == $lowest_major_version)
+{
+ $node1->stop;
+ plan skip_all => 'no extra ICU library versions found';
+}
+
+sub set_default_icu_library_version
+{
+ my $icu_version = shift;
+ $node1->safe_psql('postgres', "alter system set default_icu_library_version = '$icu_version'; select pg_reload_conf()");
+}
+
+sub set_icu_library_versions
+{
+ my $icu_versions = shift;
+ $node1->safe_psql('postgres', "alter system set icu_library_versions = '$icu_versions'");
+ $node1->restart;
+}
+
+my $ret;
+my $stderr;
+
+# === DATABASE objects ===
+
+# ===== Scenario 1: user creates database with all default settings
+
+$node1->safe_psql('postgres', "create database db2 locale_provider = icu template = template0 icu_locale = 'en'");
+
+# No warning when logging into this database.
+$ret = $node1->psql('db2', "select", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+# ===== Scenario 2: user wants to use an old library
+
+# Create a database using the older library by changing the default. This
+# might be done for compatibility with some other system, but it also simulates
+# a database that was created with all default settings when the binary was
+# linked against the older version.
+set_default_icu_library_version($lowest_major_version);
+$node1->safe_psql('postgres', "create database db3 locale_provider = icu template = template0 icu_locale = 'en'");
+
+isnt($node1->safe_psql('postgres', "select datcollversion from pg_database where datname = 'db2'"),
+ $node1->safe_psql('postgres', "select datcollversion from pg_database where datname = 'db3'"),
+ 'db2 and db3 should have different datcollversion');
+
+# ===== Scenario 3: user has the old library avaliable, is happy to keep using it
+
+# Unset the default ICU library version (meaning use the linked version for
+# newly created databases). No warning, because we can still find that older
+# version via dlopen(). User can happily go on using that old version in this
+# database for the rest of time.
+set_default_icu_library_version("");
+$ret = $node1->psql('db3', "select", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+# ===== Scenario 4: user doesn't have the old library, installs after warnings
+
+# Hide the old library version. This simulates a system that doesn't have that
+# version installed yet, by making it unavailable. We get a warning.
+set_icu_library_versions("$highest_major_version");
+$ret = $node1->psql('db3', "select", stderr => \$stderr);
+is($ret, 0, "success");
+like($stderr, qr/WARNING: database "db3" has a collation version mismatch/, "warning for incorrect datcollversion");
+like($stderr, qr/HINT: Install a version of ICU that provides/, "warning suggests installing another ICU version");
+
+# Make the old version available again, this time explicitly (whereas before it
+# worked becuase the default is * which would find it automatically).
+set_icu_library_versions("$lowest_major_version,$highest_major_version");
+$ret = $node1->psql('db3', "select", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+# It also works if you use a major.minor version explicitly.
+set_icu_library_versions("$lowest_version,$highest_major_version");
+$ret = $node1->psql('db3', "select", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+# Or *, the default value that we started with.
+set_icu_library_versions("*");
+$ret = $node1->psql('db3', "select", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+# ===== Scenario 5: user doesn't have the old library, rebuilds after warnings
+
+# Hide the old library version again, and we get the warning again.
+set_icu_library_versions("$highest_major_version");
+$ret = $node1->psql('db3', "select", stderr => \$stderr);
+is($ret, 0, "success");
+like($stderr, qr/WARNING: database "db3" has a collation version mismatch/, "warning for incorrect datcollversion");
+like($stderr, qr/HINT: Install a version of ICU that provides/, "warning suggests installing another ICU version");
+
+# If we don't want to install a new library, we have the option of clobbering
+# the version. It's the administrator's job to rebuild any database objects
+# that depend on the collation (most interestingly indexes) before doing so.
+# In this scenario, the REFRESH command can be run before or *after* rebuilding
+# indexes, because either way we're already using the default ICU library (due
+# to failure to find the named version).
+$ret = $node1->psql('postgres', "alter database db3 refresh collation version", stderr => \$stderr);
+is($ret, 0, "success");
+like($stderr, qr/NOTICE: changing version/, "version changes");
+
+# Now no warning.
+$ret = $node1->psql('db3', "select", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning after refresh");
+
+# ===== Scenario 6: user has the old library, but eventually decides to rebuild/upgrade
+
+# Make a new database with the old version active
+set_default_icu_library_version($lowest_major_version);
+$node1->safe_psql('postgres', "create database db4 locale_provider = icu template = template0 icu_locale = 'en'");
+
+# No warning, it just load the old version.
+$ret = $node1->psql('db4', "select", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning after refresh");
+my $old_datcollversion = $node1->safe_psql('postgres', "select datcollversion from pg_database where datname = 'db4'");
+
+# The user would now like to upgrade to the new library. Presumably people
+# will want to do this eventually to avoid running very old unmaintained copies
+# of ICU. Unlike scenario 3, here it's actually a requirement to REFRESH
+# *before* doing all the rebuilds of indexes etc, which may be a little
+# confusing (not shown here). REFRESH is necessary to change datcollversion,
+# which is required to make us start opening the newer library.
+#
+# XXX Currently you also need to reconnect all sessions too, because the
+# default locale is cached and now out of date.
+set_default_icu_library_version("");
+$ret = $node1->psql('postgres', "alter database db4 refresh collation version", stderr => \$stderr);
+my $new_datcollversion = $node1->safe_psql('postgres', "select datcollversion from pg_database where datname = 'db4'");
+
+isnt($old_datcollversion, $new_datcollversion, "datcollversion changed");
+
+
+# === COLLATION objects ===
+
+# The same scenarios, this time with COLLATIONs.
+
+# ===== Scenario 1: user creates database with all default settings
+
+set_default_icu_library_version("");
+set_icu_library_versions("*");
+$node1->safe_psql('postgres', "create collation c1 (provider = icu, locale = 'en')");
+
+# No warning when using it.
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c1", stderr => \$stderr);
+is($ret, 0, "can use collation");
+unlike($stderr, qr/WARNING/, "no warning for default");
+
+# ===== Scenario 2: user wants to use an old library
+
+# Simulates a collation in a database that migrated from an older binary, or a
+# collation set up explicitly to match some other system.
+set_default_icu_library_version($lowest_major_version);
+$node1->safe_psql('postgres', "create collation c2 (provider = icu, locale = 'en')");
+
+isnt($node1->safe_psql('postgres', "select collversion from pg_collation where collname = 'c1'"),
+ $node1->safe_psql('postgres', "select collversion from pg_collation where collname = 'c2'"),
+ 'c1 and c2 should have different collversion');
+
+# ===== Scenario 3: user has the old library avaliable, is happy to keep using it
+
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c2", stderr => \$stderr);
+is($ret, 0, "can use collation");
+unlike($stderr, qr/WARNING/, "no warning when using old library collation");
+
+# ===== Scenario 4: user doesn't have the old library, installs after warnings
+
+# Hide the old library version.
+set_default_icu_library_version("");
+set_icu_library_versions("$highest_major_version");
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c2", stderr => \$stderr);
+is($ret, 0, "success");
+like($stderr, qr/WARNING: collation "c2" version mismatch/, "warning for incorrect collversion");
+like($stderr, qr/HINT: Install a version of ICU that provides/, "warning suggests installing another ICU version");
+
+# Make the old version available again.
+set_icu_library_versions("$lowest_major_version,$highest_major_version");
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c2", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+# It also works if you use a major.minor version explicitly.
+set_icu_library_versions("$lowest_version,$highest_major_version");
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c2", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+# Or *, the default value that we started with.
+set_icu_library_versions("*");
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c2", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+# ===== Scenario 5: user doesn't have the old library, rebuilds after warnings
+
+# Hide the old library version again.
+set_icu_library_versions("$highest_major_version");
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c2", stderr => \$stderr);
+is($ret, 0, "success");
+like($stderr, qr/WARNING: collation "c2" version mismatch/, "warning for incorrect collversion");
+like($stderr, qr/HINT: Install a version of ICU that provides/, "warning suggests installing another ICU version");
+
+# Rebuild things, and refresh.
+$ret = $node1->psql('postgres', "alter collation c2 refresh version", stderr => \$stderr);
+is($ret, 0, "success");
+like($stderr, qr/NOTICE: changing version/, "version changes");
+
+# Now no warning.
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c2", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+# ===== Scenario 6: user has the old library, but eventually decides to rebuild/upgrade
+
+set_default_icu_library_version($lowest_major_version);
+$node1->safe_psql('postgres', "create collation c3 (provider = icu, locale = 'en')");
+
+# No warning.
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c3", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+my $old_collversion = $node1->safe_psql('postgres', "select collversion from pg_collation where collname = 'c3'");
+
+# Rebuild things, and refresh. As with database scenario 6, we need to refresh
+# *before* rebuilding dependent objects (not shown here).
+set_default_icu_library_version("");
+$ret = $node1->psql('postgres', "alter collation c3 refresh version", stderr => \$stderr);
+is($ret, 0, "success");
+like($stderr, qr/NOTICE: changing version/, "version changes");
+
+# No warning.
+$ret = $node1->psql('postgres', "select 'x' < 'y' collate c3", stderr => \$stderr);
+is($ret, 0, "success");
+unlike($stderr, qr/WARNING/, "no warning");
+
+my $new_collversion = $node1->safe_psql('postgres', "select collversion from pg_collation where collname = 'c3'");
+
+isnt($old_collversion, $new_collversion, "collversion changed");
+
+$node1->stop;
+
+done_testing();
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 2f5802195d..50d9558cf4 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1101,6 +1101,7 @@ HeapTupleTableSlot
HistControl
HotStandbyState
I32
+ICU_Convert_BI_Func
ICU_Convert_Func
ID
INFIX
@@ -2852,10 +2853,12 @@ TypeName
U
U32
U8
+UBreakIterator
UChar
UCharIterator
UColAttribute
UColAttributeValue
+UCollationResult
UCollator
UConverter
UErrorCode
@@ -3482,6 +3485,7 @@ pg_funcptr_t
pg_gssinfo
pg_hmac_ctx
pg_hmac_errno
+pg_icu_library
pg_int64
pg_local_to_utf_combined
pg_locale_t
--
2.38.1
From aa9ac4f827f83abd4efe7f773efa2e2f45ad7640 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.mu...@gmail.com>
Date: Sat, 26 Nov 2022 14:39:18 +1300
Subject: [PATCH v8 2/2] ci: XXX install ICU63 on debian
This is not a good way to add the package, just doing this temporarily
as a demonstration via cfbot.
---
.cirrus.yml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/.cirrus.yml b/.cirrus.yml
index f31923333e..8c1fb63cad 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -296,6 +296,10 @@ task:
setup_additional_packages_script: |
#apt-get update
#DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ # this is debian 11 (bullseye) but we can install ICU 63 from debian 10 (buster)
+ curl -O http://ftp.debian.org/debian/pool/main/i/icu/libicu63_63.1-6+deb10u3_amd64.deb
+ dpkg -i libicu63_63.1-6+deb10u3_amd64.deb
+
matrix:
- name: Linux - Debian Bullseye - Autoconf
--
2.38.1
