Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
Indrajit Raychaudhuri writes: [...] > FWIW, PropertyResourceBundle uses properties.load(InputStream) which > assumes ISO-8859-1 (Latin1) encoded stream. In Java 6, however, you have > properties.load(Reader) which does not have this limitation. Oh didn't know about that, thanks. Maybe Lift could do some runtime magic to see if it's running on Java. But then again this also provides a new set of possible problems when people run on JDK5 /Jeppe [...] -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
On 12/12/09 4:11 PM, Jeppe Nejsum Madsen wrote: > Indrajit Raychaudhuri writes: > >> On 11/12/09 5:28 PM, Jeppe Nejsum Madsen wrote: >>> Jean-Adrien writes: >>> >>> >>> [...] >>> >>> I have the same issues with the localized properties >>> = Conclusion = The problem here comes from my initial configuration. But in my sense it is not a good idea to store i18n data in another format than UTF-8. >>> >>> Agreed >> >> However, for minimum grief I always use unicode escaping (using \u) in >> locale properties. > > But it is not really an option (I think) to edit files like > this. Sure, for one off changes it's possible, but for an entire > application that needs to be translated into several locales I don't > think this is feasible. Agreed on the inconvenience factor. But not relying on the actual encoding of the file and using \u is what worked most reliably for me (fully localized web applications supporting 10+ languages). FWIW, PropertyResourceBundle uses properties.load(InputStream) which assumes ISO-8859-1 (Latin1) encoded stream. In Java 6, however, you have properties.load(Reader) which does not have this limitation. > >> >> The output of "native2ascii src/main/resources/translate_fr.properties" >> should interest you :) > > I know about this tool, but this means an extra step in the build > process and it's not really supported by IDE's. All in all making the > turnaround time longer... Some build plugin exists: Ant: http://ant.apache.org/manual/OptionalTasks/native2ascii.html Maven MOJO: http://mojo.codehaus.org/native2ascii-maven-plugin/ (as Tim already mentioned) Maven (via antrun): http://docs.codehaus.org/display/MAVENUSER/Maven+and+native2ascii IntelliJ IDEA has first class support for this. Not sure about other IDE support but looks like quite a few exist for Eclipse as well. (I haven't used Eclipse or Netbeans for quite sometime now.) > >> You don't have to use the xml format really. Just escape the characters >> with unicode encoding (\u) and you should be covered. > > See above :-) > > > /Jeppe > > -- > > You received this message because you are subscribed to the Google Groups > "Lift" group. > To post to this group, send email to lift...@googlegroups.com. > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
Have you seen: http://mojo.codehaus.org/native2ascii-maven-plugin/ Looks like you could automate that into your build process, no? Cheers, Tim On 12 Dec 2009, at 10:41, Jeppe Nejsum Madsen wrote: > But it is not really an option (I think) to edit files like > this. Sure, for one off changes it's possible, but for an entire > application that needs to be translated into several locales I don't > think this is feasible. > >> >> The output of "native2ascii src/main/resources/translate_fr.properties" >> should interest you :) > > I know about this tool, but this means an extra step in the build > process and it's not really supported by IDE's. All in all making the > turnaround time longer... > >> You don't have to use the xml format really. Just escape the characters >> with unicode encoding (\u) and you should be covered. > > See above :-) -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
Indrajit Raychaudhuri writes: > On 11/12/09 5:28 PM, Jeppe Nejsum Madsen wrote: >> Jean-Adrien writes: >> >> >> [...] >> >> I have the same issues with the localized properties >> >>> = Conclusion = >>> >>> The problem here comes from my initial configuration. But in my sense >>> it is not a good idea to store i18n data in another format than UTF-8. >> >> Agreed > > However, for minimum grief I always use unicode escaping (using \u) in > locale properties. But it is not really an option (I think) to edit files like this. Sure, for one off changes it's possible, but for an entire application that needs to be translated into several locales I don't think this is feasible. > > The output of "native2ascii src/main/resources/translate_fr.properties" > should interest you :) I know about this tool, but this means an extra step in the build process and it's not really supported by IDE's. All in all making the turnaround time longer... > You don't have to use the xml format really. Just escape the characters > with unicode encoding (\u) and you should be covered. See above :-) /Jeppe -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
On 11/12/09 5:28 PM, Jeppe Nejsum Madsen wrote: > Jean-Adrien writes: > > > [...] > > I have the same issues with the localized properties > >> = Conclusion = >> >> The problem here comes from my initial configuration. But in my sense >> it is not a good idea to store i18n data in another format than UTF-8. > > Agreed However, for minimum grief I always use unicode escaping (using \u) in locale properties. The output of "native2ascii src/main/resources/translate_fr.properties" should interest you :) > >> Maybe lift should not use java.util.Properties to handle i18n strings. > > Could be nice, but not java standard, so there may be interop > problems. But maybe a LiftRule setting for a UTF8 property reader > >> Moreover keys in properties file does not allow spaces, and the >> feature to use S ? "a key with space that will be used as a default >> value" does not work. > > You can have spaces in keys: > > Last\ Name = Efternavn > >> Another approach would be to use java.util.Properties in its xml form: >> value this one supports UTF-8, but once can >> find it too versatile. > > Oh no! I definitely don't want to go that route...much too verbose. You don't have to use the xml format really. Just escape the characters with unicode encoding (\u) and you should be covered. > > /Jeppe > > -- > > You received this message because you are subscribed to the Google Groups > "Lift" group. > To post to this group, send email to lift...@googlegroups.com. > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
Jean-Adrien writes: [...] I have the same issues with the localized properties > = Conclusion = > > The problem here comes from my initial configuration. But in my sense > it is not a good idea to store i18n data in another format than UTF-8. Agreed > Maybe lift should not use java.util.Properties to handle i18n strings. Could be nice, but not java standard, so there may be interop problems. But maybe a LiftRule setting for a UTF8 property reader > Moreover keys in properties file does not allow spaces, and the > feature to use S ? "a key with space that will be used as a default > value" does not work. You can have spaces in keys: Last\ Name = Efternavn > Another approach would be to use java.util.Properties in its xml form: > value this one supports UTF-8, but once can > find it too versatile. Oh no! I definitely don't want to go that route...much too verbose. /Jeppe -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
On Tue, Dec 8, 2009 at 6:27 AM, Indrajit Raychaudhuri wrote: > Good catch! I'll take a look. > > Jean, can you please send a zip of your project (just enough to recreate > the scenario, no sensitive code necessary)? > > And perhaps something that we can add as a test to the Lift build process. ;-) > Cheers, Indrajit > > > On 08/12/09 3:48 PM, Timothy Perrett wrote: > > Ahhh! That is interesting... i'd not thought of that :-) > > > > You'll need to specify a proper encoding type for sure otherwise maven > will just assume your platform default... for example, MacRoman. Perhaps IRC > or Josh will be able to advise... > > > > Cheers, Tim > > > > On 8 Dec 2009, at 10:02, Jean-Adrien wrote: > > > >> Ah ! > >> > >> In fact the problem did not appear when I upgraded to 1.1-SNAPSHOT but > >> when I enabled maven resource filtering (which assumes all files use > >> UTF-8). > >> Anyway I'll look for a correct setting to have both ISO 8859-1 > >> properties file and maven resource filtering enabled. > >> > >> > >> On Dec 8, 10:41 am, Jean-Adrien wrote: > >>> Hello, > >>> > >>> I have a char encoding problem, using the localization feature of > >>> lift. > >>> I use the ? method of the S object in order to translate strings. The > >>> localized values are in a translate_fr.properties file in the > >>> resources of the project. > >>> > >>> Until I upgrade from lift 1.1-M6 to 1.1-SNAPSHOT everything was ok, > >>> but now it seems that the following happens: > >>> As specified in the java doc, the properties file is encoded using ISO > >>> 8859-1. All xhtml templates in lift use UTF-8, and are interpreted in > >>> UTF-8 by the browser. But it seems that the the string loaded from > >>> properties file are badly translated to UTF-8 (?) > >>> > >>> I.e. "é" is 0xE9 in ISO 8859-1 and should turn when loaded in a String > >>> by java properties into the UTF-8 bytes 0xC3 0xA9. But it is > >>> interpreted some way in � (0x EF 0xBF 0xBD) > >>> > >>> I tried to encode my properties file in UTF-8 but I have the UTF > >>> encoding (0xC3 0xA9) interpreted in ISO 8859-1. "é" become é (famous > >>> French letter) in the webpage. Behaviour I understand since java loads > >>> properties String using ISO-8859-1 charset > >>> > >>> Does anyone observed the same ? > >> > >> -- > >> > >> You received this message because you are subscribed to the Google > Groups "Lift" group. > >> To post to this group, send email to lift...@googlegroups.com. > >> To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com > . > >> For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > >> > >> > >> > > > > -- > > > > You received this message because you are subscribed to the Google Groups > "Lift" group. > > To post to this group, send email to lift...@googlegroups.com. > > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com > . > > For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > "Lift" group. > To post to this group, send email to lift...@googlegroups.com. > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > > > -- Lift, the simply functional web framework http://liftweb.net Beginning Scala http://www.apress.com/book/view/1430219890 Follow me: http://twitter.com/dpp Surf the harmonics -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
Good catch! I'll take a look. Jean, can you please send a zip of your project (just enough to recreate the scenario, no sensitive code necessary)? Cheers, Indrajit On 08/12/09 3:48 PM, Timothy Perrett wrote: > Ahhh! That is interesting... i'd not thought of that :-) > > You'll need to specify a proper encoding type for sure otherwise maven will > just assume your platform default... for example, MacRoman. Perhaps IRC or > Josh will be able to advise... > > Cheers, Tim > > On 8 Dec 2009, at 10:02, Jean-Adrien wrote: > >> Ah ! >> >> In fact the problem did not appear when I upgraded to 1.1-SNAPSHOT but >> when I enabled maven resource filtering (which assumes all files use >> UTF-8). >> Anyway I'll look for a correct setting to have both ISO 8859-1 >> properties file and maven resource filtering enabled. >> >> >> On Dec 8, 10:41 am, Jean-Adrien wrote: >>> Hello, >>> >>> I have a char encoding problem, using the localization feature of >>> lift. >>> I use the ? method of the S object in order to translate strings. The >>> localized values are in a translate_fr.properties file in the >>> resources of the project. >>> >>> Until I upgrade from lift 1.1-M6 to 1.1-SNAPSHOT everything was ok, >>> but now it seems that the following happens: >>> As specified in the java doc, the properties file is encoded using ISO >>> 8859-1. All xhtml templates in lift use UTF-8, and are interpreted in >>> UTF-8 by the browser. But it seems that the the string loaded from >>> properties file are badly translated to UTF-8 (?) >>> >>> I.e. "é" is 0xE9 in ISO 8859-1 and should turn when loaded in a String >>> by java properties into the UTF-8 bytes 0xC3 0xA9. But it is >>> interpreted some way in � (0x EF 0xBF 0xBD) >>> >>> I tried to encode my properties file in UTF-8 but I have the UTF >>> encoding (0xC3 0xA9) interpreted in ISO 8859-1. "é" become é (famous >>> French letter) in the webpage. Behaviour I understand since java loads >>> properties String using ISO-8859-1 charset >>> >>> Does anyone observed the same ? >> >> -- >> >> You received this message because you are subscribed to the Google Groups >> "Lift" group. >> To post to this group, send email to lift...@googlegroups.com. >> To unsubscribe from this group, send email to >> liftweb+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/liftweb?hl=en. >> >> >> > > -- > > You received this message because you are subscribed to the Google Groups > "Lift" group. > To post to this group, send email to lift...@googlegroups.com. > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
you could select which file should be filtering, else every resources are filtering and "converted" (to UTF-8) /davidB On Tue, Dec 8, 2009 at 11:18, Timothy Perrett wrote: > Ahhh! That is interesting... i'd not thought of that :-) > > You'll need to specify a proper encoding type for sure otherwise maven will > just assume your platform default... for example, MacRoman. Perhaps IRC or > Josh will be able to advise... > > Cheers, Tim > > On 8 Dec 2009, at 10:02, Jean-Adrien wrote: > > > Ah ! > > > > In fact the problem did not appear when I upgraded to 1.1-SNAPSHOT but > > when I enabled maven resource filtering (which assumes all files use > > UTF-8). > > Anyway I'll look for a correct setting to have both ISO 8859-1 > > properties file and maven resource filtering enabled. > > > > > > On Dec 8, 10:41 am, Jean-Adrien wrote: > >> Hello, > >> > >> I have a char encoding problem, using the localization feature of > >> lift. > >> I use the ? method of the S object in order to translate strings. The > >> localized values are in a translate_fr.properties file in the > >> resources of the project. > >> > >> Until I upgrade from lift 1.1-M6 to 1.1-SNAPSHOT everything was ok, > >> but now it seems that the following happens: > >> As specified in the java doc, the properties file is encoded using ISO > >> 8859-1. All xhtml templates in lift use UTF-8, and are interpreted in > >> UTF-8 by the browser. But it seems that the the string loaded from > >> properties file are badly translated to UTF-8 (?) > >> > >> I.e. "é" is 0xE9 in ISO 8859-1 and should turn when loaded in a String > >> by java properties into the UTF-8 bytes 0xC3 0xA9. But it is > >> interpreted some way in � (0x EF 0xBF 0xBD) > >> > >> I tried to encode my properties file in UTF-8 but I have the UTF > >> encoding (0xC3 0xA9) interpreted in ISO 8859-1. "é" become é (famous > >> French letter) in the webpage. Behaviour I understand since java loads > >> properties String using ISO-8859-1 charset > >> > >> Does anyone observed the same ? > > > > -- > > > > You received this message because you are subscribed to the Google Groups > "Lift" group. > > To post to this group, send email to lift...@googlegroups.com. > > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com > . > > For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > "Lift" group. > To post to this group, send email to lift...@googlegroups.com. > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.
Re: [Lift] Re: (Maven problem?) Char encoding problem using S.?
Ahhh! That is interesting... i'd not thought of that :-) You'll need to specify a proper encoding type for sure otherwise maven will just assume your platform default... for example, MacRoman. Perhaps IRC or Josh will be able to advise... Cheers, Tim On 8 Dec 2009, at 10:02, Jean-Adrien wrote: > Ah ! > > In fact the problem did not appear when I upgraded to 1.1-SNAPSHOT but > when I enabled maven resource filtering (which assumes all files use > UTF-8). > Anyway I'll look for a correct setting to have both ISO 8859-1 > properties file and maven resource filtering enabled. > > > On Dec 8, 10:41 am, Jean-Adrien wrote: >> Hello, >> >> I have a char encoding problem, using the localization feature of >> lift. >> I use the ? method of the S object in order to translate strings. The >> localized values are in a translate_fr.properties file in the >> resources of the project. >> >> Until I upgrade from lift 1.1-M6 to 1.1-SNAPSHOT everything was ok, >> but now it seems that the following happens: >> As specified in the java doc, the properties file is encoded using ISO >> 8859-1. All xhtml templates in lift use UTF-8, and are interpreted in >> UTF-8 by the browser. But it seems that the the string loaded from >> properties file are badly translated to UTF-8 (?) >> >> I.e. "é" is 0xE9 in ISO 8859-1 and should turn when loaded in a String >> by java properties into the UTF-8 bytes 0xC3 0xA9. But it is >> interpreted some way in � (0x EF 0xBF 0xBD) >> >> I tried to encode my properties file in UTF-8 but I have the UTF >> encoding (0xC3 0xA9) interpreted in ISO 8859-1. "é" become é (famous >> French letter) in the webpage. Behaviour I understand since java loads >> properties String using ISO-8859-1 charset >> >> Does anyone observed the same ? > > -- > > You received this message because you are subscribed to the Google Groups > "Lift" group. > To post to this group, send email to lift...@googlegroups.com. > To unsubscribe from this group, send email to > liftweb+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/liftweb?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "Lift" group. To post to this group, send email to lift...@googlegroups.com. To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/liftweb?hl=en.