So if we make the minimum Java 11 for the next release cycle we could use UTF9 for the property files; that sounds like a good idea. -- Jody Garnett
On Mon, 15 Aug 2022 at 09:48, Rahkonen Jukka < jukka.rahko...@maanmittauslaitos.fi> wrote: > Hi, > > > > As a reminder the start of the mailing list thread when we discussed about > starting to translate with Transifex: > https://www.mail-archive.com/geoserver-users@lists.sourceforge.net/msg35112.html. > Title was “Translation workflow improvement” but all mails do not appear in > one thread so best to use also the sort by date view. Before that thread > there was another one “Geoserver 2.20.0, Encoding German Umlaut-Problems”. > Conclusion from that tile > > - The way to use ISO-8859-1 with U-codes is reliable for Java 8 and > above, but it is best to U-encode also some characters which belong to > ISO-8859-1 like “ä” and “ö” to avoid problems with some Java programs. > Weblate seems to do just that. > - UTF-8 is reliable but works only with Java 11 and above without > using some tricks. > > > > For anybody who needs to read or edit the properties file with a pure text > editor the U-coded strings are painful and error prone. Compare > “\u00e4\u00e4r\u00e4” vs. “määrä”. Somehow it would feel ideal to change > into pure UTF-8 when the support for Java 8 is dropped, but a well working > and open-for-all translation tool and automatic workflows are of course > very valuable. > > > > -Jukka Rahkonen- > > > > > > *Lähettäjä:* Jody Garnett <jody.garn...@gmail.com> > *Lähetetty:* maanantai 15. elokuuta 2022 17.10 > *Vastaanottaja:* Alexandre Gacon <alexandre.ga...@gmail.com> > *Kopio:* GeoServer <geoserver-devel@lists.sourceforge.net> > *Aihe:* Re: [Geoserver-devel] Use OSGEO weblate as translation tool - > Development consequences > > > > If you wish to choose weblate I am sure we can work with what it does > > > > On Mon, Aug 15, 2022 at 12:12 AM Alexandre Gacon < > alexandre.ga...@gmail.com> wrote: > > Hi Jody, > > > > I had a look at the git history for the UTF8 file: the file was created by > you by renaming the CN ISO file ( > https://osgeo-org.atlassian.net/browse/GEOS-10282 > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fosgeo-org.atlassian.net%2Fbrowse%2FGEOS-10282&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694649878839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2Bvmo6iinAcc7Mp0oFJYBIekTdA5UOO%2FVE%2BP8TnII6lQ%3D&reserved=0>). > If I understand the JIRA issue correctly, this was related to the encoding > issue in the translation files (the same old story) and the idea was > perhaps to migrate all the translations to UTF-8 encoding. > > > > Weblate UI manages to display the characters in all options: > > - ISO-8859 characters in ISO-8859 encoded file > > - U encoded characters (\u00e9 for example) in ISO-8859 encoded file > > - UTF-8 file > > > > Weblate can read and write either ISO-8859 files or UTF-8 files but inside > for a given component you can have only one of the two options. For > ISO-8859 files, weblate can read either ISO-8859 characters (é) and U > encoded characters (\u00e9) but when you change a translation, it will > always be written in U encoded characters. > > > > Regards > > Alexandre > > > > Le ven. 12 août 2022 à 21:46, Jody Garnett <jody.garn...@gmail.com> a > écrit : > > You may have to search back in the geoserver-devel history or bug tracker > for details. I assume we were provided utf8 translations and wished to > preserve them? > > > > When using the UI in weblate can folks see the native characters > regardless of encoding used? > > > > What I remember is that we needed UTF8 to support some chinese characters; > but the java properties file format is ISO-8859-1 (like the actual format > on disk). > > Wicket had some naming convention where you could append utf8 to the > filename - but no other tools understand this. > > > > If I understand you correctly: > > - Weblate wants to work with UTF8 if it can? > > - Java properties file want ISO-8859-1 > > > > Is there any way to hint to weblate what encoding to use? Looks like yes > https://docs.weblate.org/no/latest/formats.html#java-properties > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.weblate.org%2Fno%2Flatest%2Fformats.html%23java-properties&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694649878839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ftVP2gLjiQn9O8Pc4DOLed05qpqPn7s2MH64m1F7JkU%3D&reserved=0> > > I cannot tell from that page if you can choose the encoding of individual > property files (ie do this manually)? > > > > Following links weblate uses > http://docs.translatehouse.org/projects/translate-toolkit/en/latest/formats/properties.html > <https://eur06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs.translatehouse.org%2Fprojects%2Ftranslate-toolkit%2Fen%2Flatest%2Fformats%2Fproperties.html&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694649878839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=V7ZYl4qJXtpj5X8f9j%2FvTKgW1EWvQGy3xzXwsIsna7g%3D&reserved=0> > > > > Which uses: > > - > http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/prop2po.html > <https://eur06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs.translatehouse.org%2Fprojects%2Ftranslate-toolkit%2Fen%2Flatest%2Fcommands%2Fprop2po.html&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694650660032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2pl4VTdcIsBo0YSN9QGplgDa0w%2FnEZPLHi9ZRR4X4%2BY%3D&reserved=0> > > > > Reading through those docs there is an option "--personality=mozilla" > which would preserve UTF8 characters rather than force them into Latin1 > encoding. > > -- > > Jody > > > > On Fri, 12 Aug 2022 at 04:25, Alexandre Gacon <alexandre.ga...@gmail.com> > wrote: > > I try to keep only the UTF-8 file as the source for Chinese, along with > the other languages in ISO-8859-1: the display of the file content in > weblate is totally broken. I have to define another fake component > configured to support UTF-8 encoding to be able to manage it correctly. > > > > What is the reason to keep the two encoding? > > > > Alexandre > > > > Le ven. 12 août 2022 à 08:26, Alexandre Gacon <alexandre.ga...@gmail.com> > a écrit : > > Hi Jody, > > > > You are guessing well: it is all about the Chinese language : for weblate > it is the same language defined twice so it cannot cope with it. > > > > I have to check how weblate can work with having one of the languages in > UTF-8 whereas the other ones are in ISO-8859-1 : I fear that it will mean > to have two components side by side. > > > > For your last point, it seems to work well for ages: Romanian is mixing > both characters encoding whereas Japanese and Korean are totally with > unicode characters inside a ISO-8859-1 encoded file. > > > > Alexandre > > > > Le jeu. 11 août 2022 à 20:36, Jody Garnett <jody.garn...@gmail.com> a > écrit : > > Weblate is a good idea; we held back perviously because of lack of > hosting. However if OSGeo is able to host :) > > > > I do not understand about two items for the same language: can you provide > links in the github repo? Do you mean two properties files; or two entries > in the same property file. Or two entries in different property files? > > > > I am guessing you mean two property files for the chinese language; where > we followed some wicket convention for having property files in different > encodings. I think we should just use the utf8 encoding? Which is not a > java standard but wicket supports it. > > > > - > https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication.properties > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgeoserver%2Fgeoserver%2Fblob%2Fmain%2Fsrc%2Fweb%2Fwms%2Fsrc%2Fmain%2Fresources%2FGeoServerApplication.properties&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694650660032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=NCQX3UuzLr7k4dFSKBumjfA2h53%2BGtE6uWzHhjMW1Ow%3D&reserved=0> > > - > https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh.properties > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgeoserver%2Fgeoserver%2Fblob%2Fmain%2Fsrc%2Fweb%2Fwms%2Fsrc%2Fmain%2Fresources%2FGeoServerApplication_zh.properties&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694650660032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Qn7cySaRTwOI9n7wvrXXpQqOpaUwGmoG6e9xv%2FDCcJg%3D&reserved=0> > > - > https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh_CN.utf8.properties > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgeoserver%2Fgeoserver%2Fblob%2Fmain%2Fsrc%2Fweb%2Fwms%2Fsrc%2Fmain%2Fresources%2FGeoServerApplication_zh_CN.utf8.properties&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694650660032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WycwdtyR2l2EkxUa6ByE6UNnidQ6jOpOSJ61tqUW6Gw%3D&reserved=0> > > > > My understanding is we should keep the utf8 properties if weblate is > willing to understand utf8 encoding?? > > > > About replacing text with unicode; not sure how that works with java > properties being specified in a in ISO-8859-1 as part of the java api. > > -- > > Jody Garnett > > > > > > On Wed, 10 Aug 2022 at 23:11, Alexandre Gacon <alexandre.ga...@gmail.com> > wrote: > > Hi all, > > > > I am studying if we could migrate the translation tooling from Transifex > to Weblate. I have started this because with the current setup Transifex is > changing a lot of translations when I upload updates of the translation > source, making it difficult to do the synchronization between GitHub and > Transifex. > > > > Weblate is a copyleft libre software and OSGeo is hosting its own > instance, already used by several OSGeo projects (postgis, pgrouting and > grass gis at least). > > > > Thanks to Regina Obe, I have set up a GeoServer project on the OSGeo > instance to study how weblate works and if there is something which can > prevent us from using it. > > > > I have already two points to share with you to get some feedback: > > - First, when you configure a component into weblate, you cannot have > two items for the same language, even if they are in a different encoding. > As a consequence, I cannot directly integrate most of the core components > since they contain 2 files for the Chinese language: is it something which > can be changed? Which one is used by GeoServer? > - Second, when you change the translation of a text in weblate, it > automatically replaces special characters by their equivalent in unicode, > even if the character exists in the ISO-8859-1 encoding. For example: > > org.geoserver.security.GeoServerAuthenticationKeyFilter.name > <https://eur06.safelinks.protection.outlook.com/?url=http%3A%2F%2Forg.geoserver.security.geoserverauthenticationkeyfilter.name%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694650660032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=SDUCKi1ZyvcTHoRmlKYGZOFnxEGVWCar3Y6F3oEv%2Bw8%3D&reserved=0>=Clé > d'authentification > > is replaced by > > org.geoserver.security.GeoServerAuthenticationKeyFilter.name > <https://eur06.safelinks.protection.outlook.com/?url=http%3A%2F%2Forg.geoserver.security.geoserverauthenticationkeyfilter.name%2F&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694650660032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=SDUCKi1ZyvcTHoRmlKYGZOFnxEGVWCar3Y6F3oEv%2Bw8%3D&reserved=0>=Cl\u00E9 > d'authentification > > > > (my own change in the translation was to add a space at the end of the > string, to match the original layout of the source string) > > > > From a technical point of view, it does not break anything but it would > make it more difficult to work on a translation without using weblate. > > > > Do you see any problems around these two points? Anything else to check? > > > > -- > > Alexandre Gacon > > _______________________________________________ > Geoserver-devel mailing list > Geoserver-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geoserver-devel > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fgeoserver-devel&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694650660032%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OkKXIu6NZm%2FoK900xmcqxJ1hTGeVzRJkR2DZVL4mzwY%3D&reserved=0> > > > > -- > > Alexandre Gacon > > > > -- > > Alexandre Gacon > > > > > -- > > Alexandre Gacon > > -- > > -- > > Jody Garnett >
_______________________________________________ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
