Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
Hi, For the UTF-8 file for Chinese, the Transifex update does not apply to it but only to the one in ISO-8859 encoding : could I remove them from the source code? I just gave another try to Transifex and I still have unwanted changes (translations removed without asking it): it will give a lot of work to check things at each update so I am clearly in favor of moving to the osgeo weblate instance (another advantage I discovered recently is that with the GitHub integration 1) it is no longer required to use a local dev environment to synchronize source and translation and 2) administrators are informed of sources to get updated). I will do a last check on the different configuration options to write down which options we would like to take. I will freeze the translations on Transifex meanwhile. Alexandre Le lun. 15 août 2022 à 16:10, Jody Garnett a écrit : > If you wish to choose weblate I am sure we can work with what it does > > On Mon, Aug 15, 2022 at 12:12 AM Alexandre Gacon < > alexandre.ga...@gmail.com> wrote: > >> Hi Jody, >> >> I had a look at the git history for the UTF8 file: the file was created >> by you by renaming the CN ISO file ( >> https://osgeo-org.atlassian.net/browse/GEOS-10282). If I understand the >> JIRA issue correctly, this was related to the encoding issue in the >> translation files (the same old story) and the idea was perhaps to migrate >> all the translations to UTF-8 encoding. >> >> Weblate UI manages to display the characters in all options: >> - ISO-8859 characters in ISO-8859 encoded file >> - U encoded characters (\u00e9 for example) in ISO-8859 encoded file >> - UTF-8 file >> >> Weblate can read and write either ISO-8859 files or UTF-8 files but >> inside for a given component you can have only one of the two options. For >> ISO-8859 files, weblate can read either ISO-8859 characters (é) and U >> encoded characters (\u00e9) but when you change a translation, it will >> always be written in U encoded characters. >> >> Regards >> Alexandre >> >> Le ven. 12 août 2022 à 21:46, Jody Garnett a >> écrit : >> >>> You may have to search back in the geoserver-devel history or bug >>> tracker for details. I assume we were provided utf8 translations and wished >>> to preserve them? >>> >>> When using the UI in weblate can folks see the native characters >>> regardless of encoding used? >>> >>> What I remember is that we needed UTF8 to support some >>> chinese characters; but the java properties file format is ISO-8859-1 (like >>> the actual format on disk). >>> Wicket had some naming convention where you could append utf8 to the >>> filename - but no other tools understand this. >>> >>> If I understand you correctly: >>> - Weblate wants to work with UTF8 if it can? >>> - Java properties file want ISO-8859-1 >>> >>> Is there any way to hint to weblate what encoding to use? Looks like yes >>> https://docs.weblate.org/no/latest/formats.html#java-properties >>> I cannot tell from that page if you can choose the encoding of >>> individual property files (ie do this manually)? >>> >>> Following links weblate uses >>> http://docs.translatehouse.org/projects/translate-toolkit/en/latest/formats/properties.html >>> >>> Which uses: >>> - >>> http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/prop2po.html >>> >>> Reading through those docs there is an option "--personality=mozilla" >>> which would preserve UTF8 characters rather than force them into Latin1 >>> encoding. >>> -- >>> Jody >>> >>> >>> On Fri, 12 Aug 2022 at 04:25, Alexandre Gacon >>> wrote: >>> I try to keep only the UTF-8 file as the source for Chinese, along with the other languages in ISO-8859-1: the display of the file content in weblate is totally broken. I have to define another fake component configured to support UTF-8 encoding to be able to manage it correctly. What is the reason to keep the two encoding? Alexandre Le ven. 12 août 2022 à 08:26, Alexandre Gacon < alexandre.ga...@gmail.com> a écrit : > Hi Jody, > > You are guessing well: it is all about the Chinese language : for > weblate it is the same language defined twice so it cannot cope with it. > > I have to check how weblate can work with having one of the languages > in UTF-8 whereas the other ones are in ISO-8859-1 : I fear that it will > mean to have two components side by side. > > For your last point, it seems to work well for ages: Romanian is > mixing both characters encoding whereas Japanese and Korean are totally > with unicode characters inside a ISO-8859-1 encoded file. > > Alexandre > > Le jeu. 11 août 2022 à 20:36, Jody Garnett a > écrit : > >> Weblate is a good idea; we held back perviously because of lack of >> hosting. However if OSGeo is able to host :) >> >> I do not understand about two items for the same language: can you >> provide links in the github repo? Do
Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
On Tue, Aug 16, 2022 at 1:21 AM Jody Garnett wrote: > "could use UTF9 for the property files" > You're at one bit ahead from the rest of us LOL Cheers Andrea == GeoServer Professional Services from the experts! Visit http://bit.ly/gs-services-us for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions Group phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 https://www.geosolutionsgroup.com/ http://twitter.com/geosolutions_it --- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail ___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
So if we make the minimum Java 11 for the next release cycle we could use UTF9 for the property files; that sounds like a good idea. -- Jody Garnett On Mon, 15 Aug 2022 at 09:48, Rahkonen Jukka < jukka.rahko...@maanmittauslaitos.fi> wrote: > Hi, > > > > As a reminder the start of the mailing list thread when we discussed about > starting to translate with Transifex: > https://www.mail-archive.com/geoserver-users@lists.sourceforge.net/msg35112.html. > Title was “Translation workflow improvement” but all mails do not appear in > one thread so best to use also the sort by date view. Before that thread > there was another one “Geoserver 2.20.0, Encoding German Umlaut-Problems”. > Conclusion from that tile > >- The way to use ISO-8859-1 with U-codes is reliable for Java 8 and >above, but it is best to U-encode also some characters which belong to >ISO-8859-1 like “ä” and “ö” to avoid problems with some Java programs. >Weblate seems to do just that. >- UTF-8 is reliable but works only with Java 11 and above without >using some tricks. > > > > For anybody who needs to read or edit the properties file with a pure text > editor the U-coded strings are painful and error prone. Compare > “\u00e4\u00e4r\u00e4” vs. “määrä”. Somehow it would feel ideal to change > into pure UTF-8 when the support for Java 8 is dropped, but a well working > and open-for-all translation tool and automatic workflows are of course > very valuable. > > > > -Jukka Rahkonen- > > > > > > *Lähettäjä:* Jody Garnett > *Lähetetty:* maanantai 15. elokuuta 2022 17.10 > *Vastaanottaja:* Alexandre Gacon > *Kopio:* GeoServer > *Aihe:* Re: [Geoserver-devel] Use OSGEO weblate as translation tool - > Development consequences > > > > If you wish to choose weblate I am sure we can work with what it does > > > > On Mon, Aug 15, 2022 at 12:12 AM Alexandre Gacon < > alexandre.ga...@gmail.com> wrote: > > Hi Jody, > > > > I had a look at the git history for the UTF8 file: the file was created by > you by renaming the CN ISO file ( > https://osgeo-org.atlassian.net/browse/GEOS-10282 > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fosgeo-org.atlassian.net%2Fbrowse%2FGEOS-10282=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694649878839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=%2Bvmo6iinAcc7Mp0oFJYBIekTdA5UOO%2FVE%2BP8TnII6lQ%3D=0>). > If I understand the JIRA issue correctly, this was related to the encoding > issue in the translation files (the same old story) and the idea was > perhaps to migrate all the translations to UTF-8 encoding. > > > > Weblate UI manages to display the characters in all options: > > - ISO-8859 characters in ISO-8859 encoded file > > - U encoded characters (\u00e9 for example) in ISO-8859 encoded file > > - UTF-8 file > > > > Weblate can read and write either ISO-8859 files or UTF-8 files but inside > for a given component you can have only one of the two options. For > ISO-8859 files, weblate can read either ISO-8859 characters (é) and U > encoded characters (\u00e9) but when you change a translation, it will > always be written in U encoded characters. > > > > Regards > > Alexandre > > > > Le ven. 12 août 2022 à 21:46, Jody Garnett a > écrit : > > You may have to search back in the geoserver-devel history or bug tracker > for details. I assume we were provided utf8 translations and wished to > preserve them? > > > > When using the UI in weblate can folks see the native characters > regardless of encoding used? > > > > What I remember is that we needed UTF8 to support some chinese characters; > but the java properties file format is ISO-8859-1 (like the actual format > on disk). > > Wicket had some naming convention where you could append utf8 to the > filename - but no other tools understand this. > > > > If I understand you correctly: > > - Weblate wants to work with UTF8 if it can? > > - Java properties file want ISO-8859-1 > > > > Is there any way to hint to weblate what encoding to use? Looks like yes > https://docs.weblate.org/no/latest/formats.html#java-properties > <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.weblate.org%2Fno%2Flatest%2Fformats.html%23java-properties=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694649878839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%
Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
Hi, As a reminder the start of the mailing list thread when we discussed about starting to translate with Transifex: https://www.mail-archive.com/geoserver-users@lists.sourceforge.net/msg35112.html. Title was "Translation workflow improvement" but all mails do not appear in one thread so best to use also the sort by date view. Before that thread there was another one "Geoserver 2.20.0, Encoding German Umlaut-Problems". Conclusion from that tile * The way to use ISO-8859-1 with U-codes is reliable for Java 8 and above, but it is best to U-encode also some characters which belong to ISO-8859-1 like "ä" and "ö" to avoid problems with some Java programs. Weblate seems to do just that. * UTF-8 is reliable but works only with Java 11 and above without using some tricks. For anybody who needs to read or edit the properties file with a pure text editor the U-coded strings are painful and error prone. Compare "\u00e4\u00e4r\u00e4" vs. "määrä". Somehow it would feel ideal to change into pure UTF-8 when the support for Java 8 is dropped, but a well working and open-for-all translation tool and automatic workflows are of course very valuable. -Jukka Rahkonen- Lähettäjä: Jody Garnett Lähetetty: maanantai 15. elokuuta 2022 17.10 Vastaanottaja: Alexandre Gacon Kopio: GeoServer Aihe: Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences If you wish to choose weblate I am sure we can work with what it does On Mon, Aug 15, 2022 at 12:12 AM Alexandre Gacon mailto:alexandre.ga...@gmail.com>> wrote: Hi Jody, I had a look at the git history for the UTF8 file: the file was created by you by renaming the CN ISO file (https://osgeo-org.atlassian.net/browse/GEOS-10282<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fosgeo-org.atlassian.net%2Fbrowse%2FGEOS-10282=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694649878839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=%2Bvmo6iinAcc7Mp0oFJYBIekTdA5UOO%2FVE%2BP8TnII6lQ%3D=0>). If I understand the JIRA issue correctly, this was related to the encoding issue in the translation files (the same old story) and the idea was perhaps to migrate all the translations to UTF-8 encoding. Weblate UI manages to display the characters in all options: - ISO-8859 characters in ISO-8859 encoded file - U encoded characters (\u00e9 for example) in ISO-8859 encoded file - UTF-8 file Weblate can read and write either ISO-8859 files or UTF-8 files but inside for a given component you can have only one of the two options. For ISO-8859 files, weblate can read either ISO-8859 characters (é) and U encoded characters (\u00e9) but when you change a translation, it will always be written in U encoded characters. Regards Alexandre Le ven. 12 août 2022 à 21:46, Jody Garnett mailto:jody.garn...@gmail.com>> a écrit : You may have to search back in the geoserver-devel history or bug tracker for details. I assume we were provided utf8 translations and wished to preserve them? When using the UI in weblate can folks see the native characters regardless of encoding used? What I remember is that we needed UTF8 to support some chinese characters; but the java properties file format is ISO-8859-1 (like the actual format on disk). Wicket had some naming convention where you could append utf8 to the filename - but no other tools understand this. If I understand you correctly: - Weblate wants to work with UTF8 if it can? - Java properties file want ISO-8859-1 Is there any way to hint to weblate what encoding to use? Looks like yes https://docs.weblate.org/no/latest/formats.html#java-properties<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.weblate.org%2Fno%2Flatest%2Fformats.html%23java-properties=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694649878839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=ftVP2gLjiQn9O8Pc4DOLed05qpqPn7s2MH64m1F7JkU%3D=0> I cannot tell from that page if you can choose the encoding of individual property files (ie do this manually)? Following links weblate uses http://docs.translatehouse.org/projects/translate-toolkit/en/latest/formats/properties.html<https://eur06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs.translatehouse.org%2Fprojects%2Ftranslate-toolkit%2Fen%2Flatest%2Fformats%2Fproperties.html=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7Cb309015dd8bb4ab101e008da7ec7fb16%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637961694649878839%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=V7ZYl4qJXtpj5X8f9j%2FvTKgW1EWv
Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
If you wish to choose weblate I am sure we can work with what it does On Mon, Aug 15, 2022 at 12:12 AM Alexandre Gacon wrote: > Hi Jody, > > I had a look at the git history for the UTF8 file: the file was created by > you by renaming the CN ISO file ( > https://osgeo-org.atlassian.net/browse/GEOS-10282). If I understand the > JIRA issue correctly, this was related to the encoding issue in the > translation files (the same old story) and the idea was perhaps to migrate > all the translations to UTF-8 encoding. > > Weblate UI manages to display the characters in all options: > - ISO-8859 characters in ISO-8859 encoded file > - U encoded characters (\u00e9 for example) in ISO-8859 encoded file > - UTF-8 file > > Weblate can read and write either ISO-8859 files or UTF-8 files but inside > for a given component you can have only one of the two options. For > ISO-8859 files, weblate can read either ISO-8859 characters (é) and U > encoded characters (\u00e9) but when you change a translation, it will > always be written in U encoded characters. > > Regards > Alexandre > > Le ven. 12 août 2022 à 21:46, Jody Garnett a > écrit : > >> You may have to search back in the geoserver-devel history or bug tracker >> for details. I assume we were provided utf8 translations and wished to >> preserve them? >> >> When using the UI in weblate can folks see the native characters >> regardless of encoding used? >> >> What I remember is that we needed UTF8 to support some >> chinese characters; but the java properties file format is ISO-8859-1 (like >> the actual format on disk). >> Wicket had some naming convention where you could append utf8 to the >> filename - but no other tools understand this. >> >> If I understand you correctly: >> - Weblate wants to work with UTF8 if it can? >> - Java properties file want ISO-8859-1 >> >> Is there any way to hint to weblate what encoding to use? Looks like yes >> https://docs.weblate.org/no/latest/formats.html#java-properties >> I cannot tell from that page if you can choose the encoding of individual >> property files (ie do this manually)? >> >> Following links weblate uses >> http://docs.translatehouse.org/projects/translate-toolkit/en/latest/formats/properties.html >> >> Which uses: >> - >> http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/prop2po.html >> >> Reading through those docs there is an option "--personality=mozilla" >> which would preserve UTF8 characters rather than force them into Latin1 >> encoding. >> -- >> Jody >> >> >> On Fri, 12 Aug 2022 at 04:25, Alexandre Gacon >> wrote: >> >>> I try to keep only the UTF-8 file as the source for Chinese, along with >>> the other languages in ISO-8859-1: the display of the file content in >>> weblate is totally broken. I have to define another fake component >>> configured to support UTF-8 encoding to be able to manage it correctly. >>> >>> What is the reason to keep the two encoding? >>> >>> Alexandre >>> >>> Le ven. 12 août 2022 à 08:26, Alexandre Gacon >>> a écrit : >>> Hi Jody, You are guessing well: it is all about the Chinese language : for weblate it is the same language defined twice so it cannot cope with it. I have to check how weblate can work with having one of the languages in UTF-8 whereas the other ones are in ISO-8859-1 : I fear that it will mean to have two components side by side. For your last point, it seems to work well for ages: Romanian is mixing both characters encoding whereas Japanese and Korean are totally with unicode characters inside a ISO-8859-1 encoded file. Alexandre Le jeu. 11 août 2022 à 20:36, Jody Garnett a écrit : > Weblate is a good idea; we held back perviously because of lack of > hosting. However if OSGeo is able to host :) > > I do not understand about two items for the same language: can you > provide links in the github repo? Do you mean two properties files; or two > entries in the same property file. Or two entries in different property > files? > > I am guessing you mean two property files for the chinese language; > where we followed some wicket convention for having property files in > different encodings. I think we should just use the utf8 encoding? Which > is > not a java standard but wicket supports it. > > - > https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication.properties > - > https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh.properties > - > https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh_CN.utf8.properties > > My understanding is we should keep the utf8 properties if weblate is > willing to understand utf8 encoding?? > > About replacing text with unicode; not sure how that works with java >
Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
Hi Jody, I had a look at the git history for the UTF8 file: the file was created by you by renaming the CN ISO file ( https://osgeo-org.atlassian.net/browse/GEOS-10282). If I understand the JIRA issue correctly, this was related to the encoding issue in the translation files (the same old story) and the idea was perhaps to migrate all the translations to UTF-8 encoding. Weblate UI manages to display the characters in all options: - ISO-8859 characters in ISO-8859 encoded file - U encoded characters (\u00e9 for example) in ISO-8859 encoded file - UTF-8 file Weblate can read and write either ISO-8859 files or UTF-8 files but inside for a given component you can have only one of the two options. For ISO-8859 files, weblate can read either ISO-8859 characters (é) and U encoded characters (\u00e9) but when you change a translation, it will always be written in U encoded characters. Regards Alexandre Le ven. 12 août 2022 à 21:46, Jody Garnett a écrit : > You may have to search back in the geoserver-devel history or bug tracker > for details. I assume we were provided utf8 translations and wished to > preserve them? > > When using the UI in weblate can folks see the native characters > regardless of encoding used? > > What I remember is that we needed UTF8 to support some chinese characters; > but the java properties file format is ISO-8859-1 (like the actual format > on disk). > Wicket had some naming convention where you could append utf8 to the > filename - but no other tools understand this. > > If I understand you correctly: > - Weblate wants to work with UTF8 if it can? > - Java properties file want ISO-8859-1 > > Is there any way to hint to weblate what encoding to use? Looks like yes > https://docs.weblate.org/no/latest/formats.html#java-properties > I cannot tell from that page if you can choose the encoding of individual > property files (ie do this manually)? > > Following links weblate uses > http://docs.translatehouse.org/projects/translate-toolkit/en/latest/formats/properties.html > > Which uses: > - > http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/prop2po.html > > Reading through those docs there is an option "--personality=mozilla" > which would preserve UTF8 characters rather than force them into Latin1 > encoding. > -- > Jody > > > On Fri, 12 Aug 2022 at 04:25, Alexandre Gacon > wrote: > >> I try to keep only the UTF-8 file as the source for Chinese, along with >> the other languages in ISO-8859-1: the display of the file content in >> weblate is totally broken. I have to define another fake component >> configured to support UTF-8 encoding to be able to manage it correctly. >> >> What is the reason to keep the two encoding? >> >> Alexandre >> >> Le ven. 12 août 2022 à 08:26, Alexandre Gacon >> a écrit : >> >>> Hi Jody, >>> >>> You are guessing well: it is all about the Chinese language : for >>> weblate it is the same language defined twice so it cannot cope with it. >>> >>> I have to check how weblate can work with having one of the languages in >>> UTF-8 whereas the other ones are in ISO-8859-1 : I fear that it will mean >>> to have two components side by side. >>> >>> For your last point, it seems to work well for ages: Romanian is mixing >>> both characters encoding whereas Japanese and Korean are totally with >>> unicode characters inside a ISO-8859-1 encoded file. >>> >>> Alexandre >>> >>> Le jeu. 11 août 2022 à 20:36, Jody Garnett a >>> écrit : >>> Weblate is a good idea; we held back perviously because of lack of hosting. However if OSGeo is able to host :) I do not understand about two items for the same language: can you provide links in the github repo? Do you mean two properties files; or two entries in the same property file. Or two entries in different property files? I am guessing you mean two property files for the chinese language; where we followed some wicket convention for having property files in different encodings. I think we should just use the utf8 encoding? Which is not a java standard but wicket supports it. - https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication.properties - https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh.properties - https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh_CN.utf8.properties My understanding is we should keep the utf8 properties if weblate is willing to understand utf8 encoding?? About replacing text with unicode; not sure how that works with java properties being specified in a in ISO-8859-1 as part of the java api. -- Jody Garnett On Wed, 10 Aug 2022 at 23:11, Alexandre Gacon < alexandre.ga...@gmail.com> wrote: > Hi all, > > I am studying if we could migrate the translation
Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
You may have to search back in the geoserver-devel history or bug tracker for details. I assume we were provided utf8 translations and wished to preserve them? When using the UI in weblate can folks see the native characters regardless of encoding used? What I remember is that we needed UTF8 to support some chinese characters; but the java properties file format is ISO-8859-1 (like the actual format on disk). Wicket had some naming convention where you could append utf8 to the filename - but no other tools understand this. If I understand you correctly: - Weblate wants to work with UTF8 if it can? - Java properties file want ISO-8859-1 Is there any way to hint to weblate what encoding to use? Looks like yes https://docs.weblate.org/no/latest/formats.html#java-properties I cannot tell from that page if you can choose the encoding of individual property files (ie do this manually)? Following links weblate uses http://docs.translatehouse.org/projects/translate-toolkit/en/latest/formats/properties.html Which uses: - http://docs.translatehouse.org/projects/translate-toolkit/en/latest/commands/prop2po.html Reading through those docs there is an option "--personality=mozilla" which would preserve UTF8 characters rather than force them into Latin1 encoding. -- Jody On Fri, 12 Aug 2022 at 04:25, Alexandre Gacon wrote: > I try to keep only the UTF-8 file as the source for Chinese, along with > the other languages in ISO-8859-1: the display of the file content in > weblate is totally broken. I have to define another fake component > configured to support UTF-8 encoding to be able to manage it correctly. > > What is the reason to keep the two encoding? > > Alexandre > > Le ven. 12 août 2022 à 08:26, Alexandre Gacon > a écrit : > >> Hi Jody, >> >> You are guessing well: it is all about the Chinese language : for weblate >> it is the same language defined twice so it cannot cope with it. >> >> I have to check how weblate can work with having one of the languages in >> UTF-8 whereas the other ones are in ISO-8859-1 : I fear that it will mean >> to have two components side by side. >> >> For your last point, it seems to work well for ages: Romanian is mixing >> both characters encoding whereas Japanese and Korean are totally with >> unicode characters inside a ISO-8859-1 encoded file. >> >> Alexandre >> >> Le jeu. 11 août 2022 à 20:36, Jody Garnett a >> écrit : >> >>> Weblate is a good idea; we held back perviously because of lack of >>> hosting. However if OSGeo is able to host :) >>> >>> I do not understand about two items for the same language: can you >>> provide links in the github repo? Do you mean two properties files; or two >>> entries in the same property file. Or two entries in different property >>> files? >>> >>> I am guessing you mean two property files for the chinese language; >>> where we followed some wicket convention for having property files in >>> different encodings. I think we should just use the utf8 encoding? Which is >>> not a java standard but wicket supports it. >>> >>> - >>> https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication.properties >>> - >>> https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh.properties >>> - >>> https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh_CN.utf8.properties >>> >>> My understanding is we should keep the utf8 properties if weblate is >>> willing to understand utf8 encoding?? >>> >>> About replacing text with unicode; not sure how that works with java >>> properties being specified in a in ISO-8859-1 as part of the java api. >>> -- >>> Jody Garnett >>> >>> >>> On Wed, 10 Aug 2022 at 23:11, Alexandre Gacon >>> wrote: >>> Hi all, I am studying if we could migrate the translation tooling from Transifex to Weblate. I have started this because with the current setup Transifex is changing a lot of translations when I upload updates of the translation source, making it difficult to do the synchronization between GitHub and Transifex. Weblate is a copyleft libre software and OSGeo is hosting its own instance, already used by several OSGeo projects (postgis, pgrouting and grass gis at least). Thanks to Regina Obe, I have set up a GeoServer project on the OSGeo instance to study how weblate works and if there is something which can prevent us from using it. I have already two points to share with you to get some feedback: - First, when you configure a component into weblate, you cannot have two items for the same language, even if they are in a different encoding. As a consequence, I cannot directly integrate most of the core components since they contain 2 files for the Chinese language: is it something which can be changed? Which one is used by GeoServer? - Second, when
Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
I try to keep only the UTF-8 file as the source for Chinese, along with the other languages in ISO-8859-1: the display of the file content in weblate is totally broken. I have to define another fake component configured to support UTF-8 encoding to be able to manage it correctly. What is the reason to keep the two encoding? Alexandre Le ven. 12 août 2022 à 08:26, Alexandre Gacon a écrit : > Hi Jody, > > You are guessing well: it is all about the Chinese language : for weblate > it is the same language defined twice so it cannot cope with it. > > I have to check how weblate can work with having one of the languages in > UTF-8 whereas the other ones are in ISO-8859-1 : I fear that it will mean > to have two components side by side. > > For your last point, it seems to work well for ages: Romanian is mixing > both characters encoding whereas Japanese and Korean are totally with > unicode characters inside a ISO-8859-1 encoded file. > > Alexandre > > Le jeu. 11 août 2022 à 20:36, Jody Garnett a > écrit : > >> Weblate is a good idea; we held back perviously because of lack of >> hosting. However if OSGeo is able to host :) >> >> I do not understand about two items for the same language: can you >> provide links in the github repo? Do you mean two properties files; or two >> entries in the same property file. Or two entries in different property >> files? >> >> I am guessing you mean two property files for the chinese language; where >> we followed some wicket convention for having property files in different >> encodings. I think we should just use the utf8 encoding? Which is not a >> java standard but wicket supports it. >> >> - >> https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication.properties >> - >> https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh.properties >> - >> https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh_CN.utf8.properties >> >> My understanding is we should keep the utf8 properties if weblate is >> willing to understand utf8 encoding?? >> >> About replacing text with unicode; not sure how that works with java >> properties being specified in a in ISO-8859-1 as part of the java api. >> -- >> Jody Garnett >> >> >> On Wed, 10 Aug 2022 at 23:11, Alexandre Gacon >> wrote: >> >>> Hi all, >>> >>> I am studying if we could migrate the translation tooling from Transifex >>> to Weblate. I have started this because with the current setup >>> Transifex is changing a lot of translations when I upload updates of the >>> translation source, making it difficult to do the synchronization between >>> GitHub and Transifex. >>> >>> Weblate is a copyleft libre software and OSGeo is hosting its own >>> instance, already used by several OSGeo projects (postgis, pgrouting and >>> grass gis at least). >>> >>> Thanks to Regina Obe, I have set up a GeoServer project on the OSGeo >>> instance to study how weblate works and if there is something which can >>> prevent us from using it. >>> >>> I have already two points to share with you to get some feedback: >>> >>>- First, when you configure a component into weblate, you cannot >>>have two items for the same language, even if they are in a different >>>encoding. As a consequence, I cannot directly integrate most of the core >>>components since they contain 2 files for the Chinese language: is it >>>something which can be changed? Which one is used by GeoServer? >>>- Second, when you change the translation of a text in weblate, it >>>automatically replaces special characters by their equivalent in unicode, >>>even if the character exists in the ISO-8859-1 encoding. For example: >>> >>> org.geoserver.security.GeoServerAuthenticationKeyFilter.name=Clé >>> d'authentification >>> is replaced by >>> org.geoserver.security.GeoServerAuthenticationKeyFilter.name=Cl\u00E9 >>> d'authentification >>> >>> (my own change in the translation was to add a space at the end of the >>> string, to match the original layout of the source string) >>> >>> From a technical point of view, it does not break anything but it would >>> make it more difficult to work on a translation without using weblate. >>> >>> Do you see any problems around these two points? Anything else to check? >>> >>> >>> -- >>> Alexandre Gacon >>> ___ >>> Geoserver-devel mailing list >>> Geoserver-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >>> >> > > -- > Alexandre Gacon > -- Alexandre Gacon ___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
Hi Jody, You are guessing well: it is all about the Chinese language : for weblate it is the same language defined twice so it cannot cope with it. I have to check how weblate can work with having one of the languages in UTF-8 whereas the other ones are in ISO-8859-1 : I fear that it will mean to have two components side by side. For your last point, it seems to work well for ages: Romanian is mixing both characters encoding whereas Japanese and Korean are totally with unicode characters inside a ISO-8859-1 encoded file. Alexandre Le jeu. 11 août 2022 à 20:36, Jody Garnett a écrit : > Weblate is a good idea; we held back perviously because of lack of > hosting. However if OSGeo is able to host :) > > I do not understand about two items for the same language: can you provide > links in the github repo? Do you mean two properties files; or two entries > in the same property file. Or two entries in different property files? > > I am guessing you mean two property files for the chinese language; where > we followed some wicket convention for having property files in different > encodings. I think we should just use the utf8 encoding? Which is not a > java standard but wicket supports it. > > - > https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication.properties > - > https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh.properties > - > https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh_CN.utf8.properties > > My understanding is we should keep the utf8 properties if weblate is > willing to understand utf8 encoding?? > > About replacing text with unicode; not sure how that works with java > properties being specified in a in ISO-8859-1 as part of the java api. > -- > Jody Garnett > > > On Wed, 10 Aug 2022 at 23:11, Alexandre Gacon > wrote: > >> Hi all, >> >> I am studying if we could migrate the translation tooling from Transifex >> to Weblate. I have started this because with the current setup Transifex >> is changing a lot of translations when I upload updates of the translation >> source, making it difficult to do the synchronization between GitHub and >> Transifex. >> >> Weblate is a copyleft libre software and OSGeo is hosting its own >> instance, already used by several OSGeo projects (postgis, pgrouting and >> grass gis at least). >> >> Thanks to Regina Obe, I have set up a GeoServer project on the OSGeo >> instance to study how weblate works and if there is something which can >> prevent us from using it. >> >> I have already two points to share with you to get some feedback: >> >>- First, when you configure a component into weblate, you cannot have >>two items for the same language, even if they are in a different encoding. >>As a consequence, I cannot directly integrate most of the core components >>since they contain 2 files for the Chinese language: is it something which >>can be changed? Which one is used by GeoServer? >>- Second, when you change the translation of a text in weblate, it >>automatically replaces special characters by their equivalent in unicode, >>even if the character exists in the ISO-8859-1 encoding. For example: >> >> org.geoserver.security.GeoServerAuthenticationKeyFilter.name=Clé >> d'authentification >> is replaced by >> org.geoserver.security.GeoServerAuthenticationKeyFilter.name=Cl\u00E9 >> d'authentification >> >> (my own change in the translation was to add a space at the end of the >> string, to match the original layout of the source string) >> >> From a technical point of view, it does not break anything but it would >> make it more difficult to work on a translation without using weblate. >> >> Do you see any problems around these two points? Anything else to check? >> >> >> -- >> Alexandre Gacon >> ___ >> Geoserver-devel mailing list >> Geoserver-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >> > -- Alexandre Gacon ___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] Use OSGEO weblate as translation tool - Development consequences
Weblate is a good idea; we held back perviously because of lack of hosting. However if OSGeo is able to host :) I do not understand about two items for the same language: can you provide links in the github repo? Do you mean two properties files; or two entries in the same property file. Or two entries in different property files? I am guessing you mean two property files for the chinese language; where we followed some wicket convention for having property files in different encodings. I think we should just use the utf8 encoding? Which is not a java standard but wicket supports it. - https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication.properties - https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh.properties - https://github.com/geoserver/geoserver/blob/main/src/web/wms/src/main/resources/GeoServerApplication_zh_CN.utf8.properties My understanding is we should keep the utf8 properties if weblate is willing to understand utf8 encoding?? About replacing text with unicode; not sure how that works with java properties being specified in a in ISO-8859-1 as part of the java api. -- Jody Garnett On Wed, 10 Aug 2022 at 23:11, Alexandre Gacon wrote: > Hi all, > > I am studying if we could migrate the translation tooling from Transifex > to Weblate. I have started this because with the current setup Transifex > is changing a lot of translations when I upload updates of the translation > source, making it difficult to do the synchronization between GitHub and > Transifex. > > Weblate is a copyleft libre software and OSGeo is hosting its own > instance, already used by several OSGeo projects (postgis, pgrouting and > grass gis at least). > > Thanks to Regina Obe, I have set up a GeoServer project on the OSGeo > instance to study how weblate works and if there is something which can > prevent us from using it. > > I have already two points to share with you to get some feedback: > >- First, when you configure a component into weblate, you cannot have >two items for the same language, even if they are in a different encoding. >As a consequence, I cannot directly integrate most of the core components >since they contain 2 files for the Chinese language: is it something which >can be changed? Which one is used by GeoServer? >- Second, when you change the translation of a text in weblate, it >automatically replaces special characters by their equivalent in unicode, >even if the character exists in the ISO-8859-1 encoding. For example: > > org.geoserver.security.GeoServerAuthenticationKeyFilter.name=Clé > d'authentification > is replaced by > org.geoserver.security.GeoServerAuthenticationKeyFilter.name=Cl\u00E9 > d'authentification > > (my own change in the translation was to add a space at the end of the > string, to match the original layout of the source string) > > From a technical point of view, it does not break anything but it would > make it more difficult to work on a translation without using weblate. > > Do you see any problems around these two points? Anything else to check? > > > -- > Alexandre Gacon > ___ > Geoserver-devel mailing list > Geoserver-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/geoserver-devel > ___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
