On Fri, Oct 11, 2024 at 12:36:53PM +0900, Yugo NAGATA wrote:
> On Fri, 11 Oct 2024 12:16:50 +0900 (JST)
> Tatsuo Ishii <[email protected]> wrote:
>
> > > We can check non-ASCII letters SGML/XML files by preparing "allowlist"
> > > that contains lines which are allowed to have non-ascii characters,
> > > although this list will need to be maintained when lines in it are
> > > modified.
> > > I've attached a patch to add a simple Perl script to do this.
> >
> > I doubt it really works. For example, nbsp can be used formatting
> > (that's the purpose of the character in the first place). Whenever a
> > developer decides to or not to use nbsp, "allowlist" needs to be
> > maintained. It's too annoying.
>
> I suppose non-ascii characters including nbsp are basically disallowed,
> so the allowlist will not increase unless there is some special reason.
>
> However, it is true that there might be a cost for maintaining the list
> more or less, so if people don't think it is worth adding this check,
> I will withdraw this proposal.l.
I did some more research and we able to clarify our behavior in
release.sgml:
We can only use Latin1 characters, not all UTF8 characters,
because rendering engines must support the referenced characters,
and they currently only support Latin1. In the SGML files we
encode non-ASCII Latin1 characters as HTML entities, e.g.,
Álvaro Herrera. Oddly, it is possible to add Latin1
characters as UTF8, but we we currently prevent this via the
Makefile "check-non-ascii" check.
We used to use UTF8 characters in SGML files, but only UTF8 characters
that had Latin1 equivalents, and I think the toolchain would convert
UTF8 to Latin1 for us.
What I ended up doing was to change the UTF8 encoded characters to HTML
entities, and then modify the Makefile to check for any non-ASCII
characters. This will catch and any other UTF8 characters.
I also added a dummy 'pdf' target that is the same as the postgres.pdf
dummy target; we already had an "html" target, so I thought a "pdf" one
made sense.
Patch attached. I plan to apply this in a few days to master.
--
Bruce Momjian <[email protected]> https://momjian.us
EDB https://enterprisedb.com
When a patient asks the doctor, "Am I going to die?", he means
"Am I going to die soon?"
diff --git a/doc/src/sgml/Makefile b/doc/src/sgml/Makefile
index 65ed32cd0ab..87d21783e52 100644
--- a/doc/src/sgml/Makefile
+++ b/doc/src/sgml/Makefile
@@ -143,7 +143,7 @@ postgres.txt: postgres.html
## Print
##
-postgres.pdf:
+postgres.pdf pdf:
$(error Invalid target; use postgres-A4.pdf or postgres-US.pdf as targets)
XSLTPROC_FO_FLAGS += --stringparam img.src.path '$(srcdir)/'
@@ -194,7 +194,7 @@ MAKEINFO = makeinfo
##
# Quick syntax check without style processing
-check: postgres.sgml $(ALLSGML) check-tabs check-nbsp
+check: postgres.sgml $(ALLSGML) check-tabs check-non-ascii
$(XMLLINT) $(XMLINCLUDE) --noout --valid $<
@@ -262,10 +262,9 @@ check-tabs:
# Non-breaking spaces are harmless, but it is best to avoid them in SGML files.
# Use perl command because non-GNU grep or sed could not have hex escape sequence.
-check-nbsp:
- @ ( $(PERL) -ne '/\xC2\xA0/ and print("$$ARGV:$$_"),$$n++; END {exit($$n>0)}' \
- $(wildcard $(srcdir)/*.sgml $(srcdir)/ref/*.sgml $(srcdir)/*.xsl) ) || \
- (echo "Non-breaking spaces appear in SGML/XML files" 1>&2; exit 1)
+check-non-ascii:
+ @( ! grep -P '[^\x00-\x7f]' $(wildcard $(srcdir)/*.sgml $(srcdir)/ref/*.sgml $(srcdir)/*.xsl) ) || \
+ (echo "Non-ASCII characters appear in SGML/XML files; use HTML entities for Latin1 characters" 1>&2; exit 1)
##
## Clean
diff --git a/doc/src/sgml/charset.sgml b/doc/src/sgml/charset.sgml
index 1ef5322b912..f5e115e8d6e 100644
--- a/doc/src/sgml/charset.sgml
+++ b/doc/src/sgml/charset.sgml
@@ -1225,7 +1225,7 @@ CREATE COLLATION ignore_accents (provider = icu, locale = 'und-u-ks-level1-kc-tr
<programlisting>
-- ignore differences in accents and case
CREATE COLLATION ignore_accent_case (provider = icu, deterministic = false, locale = 'und-u-ks-level1');
-SELECT 'Å' = 'A' COLLATE ignore_accent_case; -- true
+SELECT 'Å' = 'A' COLLATE ignore_accent_case; -- true
SELECT 'z' = 'Z' COLLATE ignore_accent_case; -- true
-- upper case letters sort before lower case.
@@ -1282,7 +1282,7 @@ SELECT 'w;x*y-z' = 'wxyz' COLLATE num_ignore_punct; -- true
<entry><literal>'ab' = U&'a\2063b'</literal></entry>
<entry><literal>'x-y' = 'x_y'</literal></entry>
<entry><literal>'g' = 'G'</literal></entry>
- <entry><literal>'n' = 'ñ'</literal></entry>
+ <entry><literal>'n' = 'ñ'</literal></entry>
<entry><literal>'y' = 'z'</literal></entry>
</row>
</thead>
@@ -1346,7 +1346,7 @@ SELECT 'w;x*y-z' = 'wxyz' COLLATE num_ignore_punct; -- true
<para>
At every level, even with full normalization off, basic normalization is
- performed. For example, <literal>'á'</literal> may be composed of the
+ performed. For example, <literal>'á'</literal> may be composed of the
code points <literal>U&'\0061\0301'</literal> or the single code
point <literal>U&'\00E1'</literal>, and those sequences will be
considered equal even at the <literal>identic</literal> level. To treat
@@ -1430,8 +1430,8 @@ SELECT 'x-y' = 'x_y' COLLATE level4; -- false
<entry><literal>false</literal></entry>
<entry>
Backwards comparison for the level 2 differences. For example,
- locale <literal>und-u-kb</literal> sorts <literal>'àe'</literal>
- before <literal>'aé'</literal>.
+ locale <literal>und-u-kb</literal> sorts <literal>'àe'</literal>
+ before <literal>'aé'</literal>.
</entry>
</row>
diff --git a/doc/src/sgml/images/genetic-algorithm.svg b/doc/src/sgml/images/genetic-algorithm.svg
index fb9fdd1ba78..2ce5f1b2712 100644
--- a/doc/src/sgml/images/genetic-algorithm.svg
+++ b/doc/src/sgml/images/genetic-algorithm.svg
@@ -72,7 +72,7 @@
<title>a4->end</title>
<path fill="none" stroke="#000000" d="M259,-312.5834C259,-312.5834 259,-54.659 259,-54.659"/>
<polygon fill="#000000" stroke="#000000" points="262.5001,-54.659 259,-44.659 255.5001,-54.6591 262.5001,-54.659"/>
-<text text-anchor="middle" x="246" y="-186.6212" font-family="sans-serif" font-size="10.00" fill="#000000">true </text>
+<text text-anchor="middle" x="246" y="-186.6212" font-family="sans-serif" font-size="10.00" fill="#000000">true</text>
</g>
<!-- a5 -->
<g id="node7" class="node">
@@ -85,7 +85,7 @@
<title>a4->a5</title>
<path fill="none" stroke="#000000" d="M144,-298.269C144,-298.269 144,-286.5248 144,-286.5248"/>
<polygon fill="#000000" stroke="#000000" points="147.5001,-286.5248 144,-276.5248 140.5001,-286.5249 147.5001,-286.5248"/>
-<text text-anchor="middle" x="127" y="-284.3969" font-family="sans-serif" font-size="10.00" fill="#000000">false </text>
+<text text-anchor="middle" x="127" y="-284.3969" font-family="sans-serif" font-size="10.00" fill="#000000">false</text>
</g>
<!-- a6 -->
<g id="node8" class="node">
diff --git a/doc/src/sgml/release.sgml b/doc/src/sgml/release.sgml
index 8433690dead..65c86f54c0e 100644
--- a/doc/src/sgml/release.sgml
+++ b/doc/src/sgml/release.sgml
@@ -26,13 +26,15 @@ non-ASCII characters find using grep -P '[\x80-\xFF]' or
http://www.zipcon.net/~swhite/docs/computers/browsers/entities_page.html
https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references
- We cannot use UTF8 because rendering engines have to
- support the referenced characters.
-
- Do not use numeric _UTF_ numeric character escapes (&#nnn;),
- we can only use Latin1.
-
- Example: Alvaro Herrera is Álvaro Herrera
+ We can only use Latin1 characters, not all UTF8 characters,
+ because rendering engines must support the referenced characters,
+ and they currently only support Latin1. In the SGML files we
+ encode non-ASCII Latin1 characters as HTML entities, e.g.,
+ Álvaro Herrera. Oddly, it is possible to add Latin1
+ characters as UTF8, but we we currently prevent this via the
+ Makefile "check-non-ascii" check.
+
+ Do not use numeric _UTF_ numeric character escapes (&#nnn;).
wrap long lines
diff --git a/doc/src/sgml/stylesheet-man.xsl b/doc/src/sgml/stylesheet-man.xsl
index fcb485c2931..2e2564da683 100644
--- a/doc/src/sgml/stylesheet-man.xsl
+++ b/doc/src/sgml/stylesheet-man.xsl
@@ -213,12 +213,12 @@
<!-- Slight rephrasing to indicate that missing sections are found
in the documentation. -->
<l:context name="xref-number-and-title">
- <l:template name="chapter" text="Chapter %n, %t, in the documentation"/>
- <l:template name="sect1" text="Section %n, “%t”, in the documentation"/>
- <l:template name="sect2" text="Section %n, “%t”, in the documentation"/>
- <l:template name="sect3" text="Section %n, “%t”, in the documentation"/>
- <l:template name="sect4" text="Section %n, “%t”, in the documentation"/>
- <l:template name="sect5" text="Section %n, “%t”, in the documentation"/>
+ <l:template name="chapter" text="Chapter %n, "%t", in the documentation"/>
+ <l:template name="sect1" text="Section %n, "%t", in the documentation"/>
+ <l:template name="sect2" text="Section %n, "%t", in the documentation"/>
+ <l:template name="sect3" text="Section %n, "%t", in the documentation"/>
+ <l:template name="sect4" text="Section %n, "%t", in the documentation"/>
+ <l:template name="sect5" text="Section %n, "%t", in the documentation"/>
</l:context>
</l:l10n>
</l:i18n>
