Re: make update-po@master stops at pg_upgrade
On 2022-Aug-08, Tom Lane wrote: > Alvaro Herrera writes: > > Now that I did the translation chores again after a few years I am > > reminded of a point about this argument: in reality, few people ever > > run this recipe manually (I know I never do), because it's easier to > > grab the already-merged files from the NLS website. It all happens > > mechanically and there's nobody leaving random junnk files. > > Hmm, so where does the NLS website get its data? Well, the NLS website does invoke the recipe. Just not manually. > I'd be all for flushing the recipe altogether if no one uses it. > However, the existence of this thread suggests otherwise. I just meant it's not normally run manually. But if you do run it manually, and you translate a file that has a few extra messages because of the hypothetical junk source file, then you'll upload a catalog with those extra messages; these extra messages will be dropped the next time your file is merged through the NLS website. Maybe you'll do some extra work (translating useless messages) but there'll be no harm. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
Re: make update-po@master stops at pg_upgrade
Alvaro Herrera writes: > On 2022-Jul-28, Tom Lane wrote: >> Thanks for testing it. I think the only remaining concern is >> Peter's objection that $(wildcard) might pick up random junk files >> that end in ".c". That's true, but the backend's nls.mk also >> picks up everything matching "*.c" (over the whole backend tree, >> not just one directory!), > Now that I did the translation chores again after a few years I am > reminded of a point about this argument: in reality, few people ever > run this recipe manually (I know I never do), because it's easier to > grab the already-merged files from the NLS website. It all happens > mechanically and there's nobody leaving random junnk files. Hmm, so where does the NLS website get its data? I'd be all for flushing the recipe altogether if no one uses it. However, the existence of this thread suggests otherwise. regards, tom lane
Re: make update-po@master stops at pg_upgrade
On 2022-Jul-28, Tom Lane wrote: > Thanks for testing it. I think the only remaining concern is > Peter's objection that $(wildcard) might pick up random junk files > that end in ".c". That's true, but the backend's nls.mk also > picks up everything matching "*.c" (over the whole backend tree, > not just one directory!), Now that I did the translation chores again after a few years I am reminded of a point about this argument: in reality, few people ever run this recipe manually (I know I never do), because it's easier to grab the already-merged files from the NLS website. It all happens mechanically and there's nobody leaving random junnk files. -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ "El que vive para el futuro es un iluso, y el que vive para el pasado, un imbécil" (Luis Adler, "Los tripulantes de la noche")
Re: make update-po@master stops at pg_upgrade
Alvaro Herrera writes: > In short, +1 to this patch. Thanks for testing it. I think the only remaining concern is Peter's objection that $(wildcard) might pick up random junk files that end in ".c". That's true, but the backend's nls.mk also picks up everything matching "*.c" (over the whole backend tree, not just one directory!), and I don't recall people complaining about that. So I think the reduction in maintenance burden justifies the risk. What do others think? regards, tom lane
Re: make update-po@master stops at pg_upgrade
On 2022-Jul-13, Tom Lane wrote: > Actually, we can get rid of those easily enough anyway with $(sort). > Here's a draft that might solve these problems. The idea is to use > $(wildcard) for files in the srcdir, and manually enumerate only > built files. I checked the files in src/bin/scripts and they look OK; there are no missing messages now. I also checked the es.po.new files with and without patch; they look good, nothing is lost. Files that weren't previously being processed: src/interfaces/libpq/fe-print.c src/interfaces/ecpg/preproc/parser.c Incidentally, this patch is pretty similar to what Kyotaro-san sent when opening the thread, with the addition of the $(notdir) call. In short, +1 to this patch. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "El sentido de las cosas no viene de las cosas, sino de las inteligencias que las aplican a sus problemas diarios en busca del progreso." (Ernesto Hernández-Novich)
Re: make update-po@master stops at pg_upgrade
Peter Eisentraut writes: > On 13.07.22 20:19, Tom Lane wrote: >> Hmm. We could list built files explicitly, perhaps, and still be >> a good step ahead on the maintenance burden. Does xgettext get >> upset if the same input file is mentioned twice, ie would we have >> to filter sql_help.c out of the wildcard result? > It seems it would be ok with that. Actually, we can get rid of those easily enough anyway with $(sort). Here's a draft that might solve these problems. The idea is to use $(wildcard) for files in the srcdir, and manually enumerate only built files. regards, tom lane diff --git a/src/bin/initdb/nls.mk b/src/bin/initdb/nls.mk index 19c9136849..54d6d034b5 100644 --- a/src/bin/initdb/nls.mk +++ b/src/bin/initdb/nls.mk @@ -1,5 +1,7 @@ # src/bin/initdb/nls.mk CATALOG_NAME = initdb -GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) findtimezone.c initdb.c ../../common/exec.c ../../common/fe_memutils.c ../../common/file_utils.c ../../common/pgfnames.c ../../common/restricted_token.c ../../common/rmtree.c ../../common/username.c ../../common/wait_error.c ../../port/dirmod.c +GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) \ + $(notdir $(wildcard $(srcdir)/*.c)) \ + ../../common/exec.c ../../common/fe_memutils.c ../../common/file_utils.c ../../common/pgfnames.c ../../common/restricted_token.c ../../common/rmtree.c ../../common/username.c ../../common/wait_error.c ../../port/dirmod.c GETTEXT_TRIGGERS = $(FRONTEND_COMMON_GETTEXT_TRIGGERS) simple_prompt GETTEXT_FLAGS= $(FRONTEND_COMMON_GETTEXT_FLAGS) diff --git a/src/bin/pg_amcheck/nls.mk b/src/bin/pg_amcheck/nls.mk index 5e6171952c..1f70ed4c70 100644 --- a/src/bin/pg_amcheck/nls.mk +++ b/src/bin/pg_amcheck/nls.mk @@ -1,7 +1,7 @@ # src/bin/pg_amcheck/nls.mk CATALOG_NAME = pg_amcheck GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) \ - pg_amcheck.c \ + $(notdir $(wildcard $(srcdir)/*.c)) \ ../../fe_utils/cancel.c \ ../../fe_utils/connect_utils.c \ ../../fe_utils/option_utils.c \ diff --git a/src/bin/pg_archivecleanup/nls.mk b/src/bin/pg_archivecleanup/nls.mk index 801cf1c51e..8b4b64de8f 100644 --- a/src/bin/pg_archivecleanup/nls.mk +++ b/src/bin/pg_archivecleanup/nls.mk @@ -1,5 +1,6 @@ # src/bin/pg_archivecleanup/nls.mk CATALOG_NAME = pg_archivecleanup -GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) pg_archivecleanup.c +GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) \ + $(notdir $(wildcard $(srcdir)/*.c)) GETTEXT_TRIGGERS = $(FRONTEND_COMMON_GETTEXT_TRIGGERS) GETTEXT_FLAGS= $(FRONTEND_COMMON_GETTEXT_FLAGS) diff --git a/src/bin/pg_basebackup/nls.mk b/src/bin/pg_basebackup/nls.mk index 8d28e249de..04af63ac3c 100644 --- a/src/bin/pg_basebackup/nls.mk +++ b/src/bin/pg_basebackup/nls.mk @@ -1,18 +1,7 @@ # src/bin/pg_basebackup/nls.mk CATALOG_NAME = pg_basebackup GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) \ - bbstreamer_file.c \ - bbstreamer_gzip.c \ - bbstreamer_inject.c \ - bbstreamer_lz4.c \ - bbstreamer_tar.c \ - bbstreamer_zstd.c \ - pg_basebackup.c \ - pg_receivewal.c \ - pg_recvlogical.c \ - receivelog.c \ - streamutil.c \ - walmethods.c \ + $(notdir $(wildcard $(srcdir)/*.c)) \ ../../common/compression.c \ ../../common/fe_memutils.c \ ../../common/file_utils.c \ diff --git a/src/bin/pg_checksums/nls.mk b/src/bin/pg_checksums/nls.mk index f7cd2a5ee9..0b08f46f4d 100644 --- a/src/bin/pg_checksums/nls.mk +++ b/src/bin/pg_checksums/nls.mk @@ -1,7 +1,7 @@ # src/bin/pg_checksums/nls.mk CATALOG_NAME = pg_checksums GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) \ - pg_checksums.c \ + $(notdir $(wildcard $(srcdir)/*.c)) \ ../../fe_utils/option_utils.c GETTEXT_TRIGGERS = $(FRONTEND_COMMON_GETTEXT_TRIGGERS) GETTEXT_FLAGS= $(FRONTEND_COMMON_GETTEXT_FLAGS) diff --git a/src/bin/pg_config/nls.mk b/src/bin/pg_config/nls.mk index d86c28c404..8642be47e4 100644 --- a/src/bin/pg_config/nls.mk +++ b/src/bin/pg_config/nls.mk @@ -1,3 +1,4 @@ # src/bin/pg_config/nls.mk CATALOG_NAME = pg_config -GETTEXT_FILES= pg_config.c ../../common/config_info.c ../../common/exec.c +GETTEXT_FILES= $(notdir $(wildcard $(srcdir)/*.c)) \ + ../../common/config_info.c ../../common/exec.c diff --git a/src/bin/pg_controldata/nls.mk b/src/bin/pg_controldata/nls.mk index ab34205b96..805085a56b 100644 --- a/src/bin/pg_controldata/nls.mk +++ b/src/bin/pg_controldata/nls.mk @@ -1,5 +1,6 @@ # src/bin/pg_controldata/nls.mk CATALOG_NAME = pg_controldata -GETTEXT_FILES
Re: make update-po@master stops at pg_upgrade
On 13.07.22 20:19, Tom Lane wrote: Hmm. We could list built files explicitly, perhaps, and still be a good step ahead on the maintenance burden. Does xgettext get upset if the same input file is mentioned twice, ie would we have to filter sql_help.c out of the wildcard result? It seems it would be ok with that.
Re: make update-po@master stops at pg_upgrade
Peter Eisentraut writes: > Note that we have this in nls-global.mk which tries to avoid having the > vpath details sneak into the output: > po/$(CATALOG_NAME).pot: $(GETTEXT_FILES) $(MAKEFILE_LIST) > # Change to srcdir explicitly, don't rely on $^. That way we get > # consistent #: file references in the po files. > ... > $(XGETTEXT) -D $(srcdir) -D . -n $(addprefix -k, $(GETTEXT_TRIGGERS)) > $(addprefix --flag=, $(GETTEXT_FLAGS)) $(GETTEXT_FILES) Hmm ... so how does that work with built files in a VPATH build today? Anyway, I'll go revert the patch for now, since it's clearly got at least a couple problems that need sorting. regards, tom lane
Re: make update-po@master stops at pg_upgrade
On 13.07.22 19:41, Tom Lane wrote: Alvaro Herrera writes: Hmm, I got this failure: /usr/bin/xgettext: no se especificó el fichero de entrada Hmm ... are you doing this in a VPATH setup? Does it help if you make the entry be GETTEXT_FILES= $(wildcard $(srcdir)/*.c) I'd supposed we didn't need to be careful about that, because I saw uses of $(wildcard) without it ... but I now see other uses with. Note that we have this in nls-global.mk which tries to avoid having the vpath details sneak into the output: po/$(CATALOG_NAME).pot: $(GETTEXT_FILES) $(MAKEFILE_LIST) # Change to srcdir explicitly, don't rely on $^. That way we get # consistent #: file references in the po files. ... $(XGETTEXT) -D $(srcdir) -D . -n $(addprefix -k, $(GETTEXT_TRIGGERS)) $(addprefix --flag=, $(GETTEXT_FLAGS)) $(GETTEXT_FILES)
Re: make update-po@master stops at pg_upgrade
Peter Eisentraut writes: > In some cases, the listed files are build output from another rule, for > example sql_help.c. By using a wildcard, you just take whatever files > happen to be there, not observing proper make dependencies. Hmm. We could list built files explicitly, perhaps, and still be a good step ahead on the maintenance burden. Does xgettext get upset if the same input file is mentioned twice, ie would we have to filter sql_help.c out of the wildcard result? regards, tom lane
Re: make update-po@master stops at pg_upgrade
On 13.07.22 20:09, Peter Eisentraut wrote: In any case, someone should check that this creates identical output before and after. A quick check shows differences in pg_rewind.pot psql.pot ecpg.pot libpq.pot plpgsql.pot
Re: make update-po@master stops at pg_upgrade
On 13.07.22 18:07, Tom Lane wrote: Still, wildcarding the local *.c references seems like a clear step forward. I'll go push that part. In some cases, the listed files are build output from another rule, for example sql_help.c. By using a wildcard, you just take whatever files happen to be there, not observing proper make dependencies. (Conversely, you could also include files that are not part of the build, maybe leftover from something aborted.) So I'm not sure this is such a clear win. In any case, someone should check that this creates identical output before and after.
Re: make update-po@master stops at pg_upgrade
Alvaro Herrera writes: > Hmm, I got this failure: > /usr/bin/xgettext: no se especificó el fichero de entrada Hmm ... are you doing this in a VPATH setup? Does it help if you make the entry be GETTEXT_FILES= $(wildcard $(srcdir)/*.c) I'd supposed we didn't need to be careful about that, because I saw uses of $(wildcard) without it ... but I now see other uses with. regards, tom lane
Re: make update-po@master stops at pg_upgrade
On 2022-Jul-13, Tom Lane wrote: > I had to recreate the patch almost from scratch, because 88dad06b4 > touched adjacent lines in most of these files, scaring patch(1) > away from applying the changes. That being the case, I decided > to use $(wildcard *.c) everywhere, including the places where there's > currently just one *.c file. It seems to work for me, but please check. Hmm, I got this failure: make[4]: se entra en el directorio '/home/alvherre/Code/pgsql-build/master/src/interfaces/ecpg/ecpglib' /usr/bin/xgettext -ctranslator --copyright-holder='PostgreSQL Global Development Group' --msgid-bugs-address=pgsql-b...@lists.postgresql.org --no-wrap --sort-by-file --package-name='ecpglib (PostgreSQL)' --package-version='16' -D /pgsql/source/master/src/interfaces/ecpg/ecpglib -D . -n -kecpg_gettext -k_ --flag=ecpg_gettext:1:pass-c-format --flag=_:1:pass-c-format /usr/bin/xgettext: no se especificó el fichero de entrada Pruebe '/usr/bin/xgettext --help' para más información. make[4]: *** [/pgsql/source/master/src/nls-global.mk:108: po/ecpglib.pot] Error 1 The error from xgettext, of course, is /usr/bin/xgettext: no input file given -- Álvaro HerreraBreisgau, Deutschland — https://www.EnterpriseDB.com/ "Puedes vivir sólo una vez, pero si lo haces bien, una vez es suficiente"
Re: make update-po@master stops at pg_upgrade
I wrote: > Kyotaro Horiguchi writes: >> Since backend does that way, I think we can do that the same way >> also for the tools. Attached second does that except for tools that >> have only one *.c. The patch doesn't make a difference in the result >> of make update-po. > Still, wildcarding the local *.c references seems like a clear step > forward. I'll go push that part. I had to recreate the patch almost from scratch, because 88dad06b4 touched adjacent lines in most of these files, scaring patch(1) away from applying the changes. That being the case, I decided to use $(wildcard *.c) everywhere, including the places where there's currently just one *.c file. It seems to work for me, but please check. regards, tom lane
Re: make update-po@master stops at pg_upgrade
Kyotaro Horiguchi writes: > I find it annoying that make update-po stops at pg_upgrade on master. > The cause is that a file is renamed from relfilenode.c to > relfilenumber.c so just fixing the name works. (attached first). Ooops. > I wonder if we can use $(wildcard *.c) instead of explicitly > enumerating every *.c file in the directory then maintaining the > list. +1. We've repeatedly forgotten to update the nls.mk files when adding/removing files; they're just not on most hackers' radar. Anything we can do to make that more automatic seems like a win. > Since backend does that way, I think we can do that the same way > also for the tools. Attached second does that except for tools that > have only one *.c. The patch doesn't make a difference in the result > of make update-po. I wonder if there's anything we can do about the manual cross-references to src/common. We could wildcard that, but then every FE program would have to contain translations for all messages in src/common/, even for modules it doesn't use. Still, wildcarding the local *.c references seems like a clear step forward. I'll go push that part. regards, tom lane
make update-po@master stops at pg_upgrade
I'm not sure if it fits -hackers, but seems better than -translators. I find it annoying that make update-po stops at pg_upgrade on master. The cause is that a file is renamed from relfilenode.c to relfilenumber.c so just fixing the name works. (attached first). On the other hand, basically, every nls.mk contain all *.c files in the tool's directory plus some in other directories with some exceptions: a) nls.mk of pg_dump and psql contain some *.h files but they don't contain a translatable string. b) nls.mk of ecpglib is missing some files that contain translatable strings. (adding theses files doesn't change the result of make update-po, but I'll send another mail for this issue.) c) nls.mk of pg_waldump, psql, libpq and preproc excludes some *.c files which don't contain a translatable string. At least for (a) above, removing them from nls.mk is rather good. For (b), adding them is also good. For (c), I think few extra files doesn't make difference. I wonder if we can use $(wildcard *.c) instead of explicitly enumerating every *.c file in the directory then maintaining the list. Since backend does that way, I think we can do that the same way also for the tools. Attached second does that except for tools that have only one *.c. The patch doesn't make a difference in the result of make update-po. Either will do for me but at least I'd like (a) applied. regards. -- Kyotaro Horiguchi NTT Open Source Software Center diff --git a/src/bin/pg_upgrade/nls.mk b/src/bin/pg_upgrade/nls.mk index bfaacfbb7c..2488b8c406 100644 --- a/src/bin/pg_upgrade/nls.mk +++ b/src/bin/pg_upgrade/nls.mk @@ -2,7 +2,7 @@ CATALOG_NAME = pg_upgrade AVAIL_LANGUAGES = cs de es fr ja ko ru sv tr uk zh_CN GETTEXT_FILES= check.c controldata.c dump.c exec.c file.c function.c \ - info.c option.c parallel.c pg_upgrade.c relfilenode.c \ + info.c option.c parallel.c pg_upgrade.c relfilenumber.c \ server.c tablespace.c util.c version.c GETTEXT_TRIGGERS = pg_fatal pg_log:2 prep_status prep_status_progress report_status:2 GETTEXT_FLAGS= \ diff --git a/src/bin/initdb/nls.mk b/src/bin/initdb/nls.mk index fe7bdfc04a..df3287e95b 100644 --- a/src/bin/initdb/nls.mk +++ b/src/bin/initdb/nls.mk @@ -1,6 +1,6 @@ # src/bin/initdb/nls.mk CATALOG_NAME = initdb AVAIL_LANGUAGES = cs de el es fr he it ja ko pl pt_BR ru sv tr uk vi zh_CN -GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) findtimezone.c initdb.c ../../common/exec.c ../../common/fe_memutils.c ../../common/file_utils.c ../../common/pgfnames.c ../../common/restricted_token.c ../../common/rmtree.c ../../common/username.c ../../common/wait_error.c ../../port/dirmod.c +GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) $(wildcard *.c) ../../common/exec.c ../../common/fe_memutils.c ../../common/file_utils.c ../../common/pgfnames.c ../../common/restricted_token.c ../../common/rmtree.c ../../common/username.c ../../common/wait_error.c ../../port/dirmod.c GETTEXT_TRIGGERS = $(FRONTEND_COMMON_GETTEXT_TRIGGERS) simple_prompt GETTEXT_FLAGS= $(FRONTEND_COMMON_GETTEXT_FLAGS) diff --git a/src/bin/pg_basebackup/nls.mk b/src/bin/pg_basebackup/nls.mk index ec7393d321..1750e7ba94 100644 --- a/src/bin/pg_basebackup/nls.mk +++ b/src/bin/pg_basebackup/nls.mk @@ -2,18 +2,7 @@ CATALOG_NAME = pg_basebackup AVAIL_LANGUAGES = cs de es fr he it ja ko pl pt_BR ru sv tr uk vi zh_CN GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) \ - bbstreamer_file.c \ - bbstreamer_gzip.c \ - bbstreamer_inject.c \ - bbstreamer_lz4.c \ - bbstreamer_tar.c \ - bbstreamer_zstd.c \ - pg_basebackup.c \ - pg_receivewal.c \ - pg_recvlogical.c \ - receivelog.c \ - streamutil.c \ - walmethods.c \ + $(wildcard *.c) \ ../../common/compression.c \ ../../common/fe_memutils.c \ ../../common/file_utils.c \ diff --git a/src/bin/pg_dump/nls.mk b/src/bin/pg_dump/nls.mk index dc7360db48..059c862ffb 100644 --- a/src/bin/pg_dump/nls.mk +++ b/src/bin/pg_dump/nls.mk @@ -2,12 +2,7 @@ CATALOG_NAME = pg_dump AVAIL_LANGUAGES = cs de el es fr he it ja ko pl pt_BR ru sv tr uk zh_CN GETTEXT_FILES= $(FRONTEND_COMMON_GETTEXT_FILES) \ - pg_backup_archiver.c pg_backup_db.c pg_backup_custom.c \ - pg_backup_null.c pg_backup_tar.c \ - pg_backup_directory.c dumputils.c compress_io.c \ - pg_dump.c common.c pg_dump_sort.c \ - pg_restore.c pg_dumpall.c \ - parallel.c parallel.h pg_backup_utils.c pg_backup_utils.h \ + $(wildcard *.c) \ ../../common/exec.c ../../common/fe_memutils.c \ ../../common/wait_err
