[appengine-java] Re: Unicode problems
Hello, I have the same issue with Czech characters. It is fine on local SDK but not when I send it to appengine. See my testing application on http://lskokan.appspot.com. Should I create a bug for this? Ladislav. On 15 zář, 10:58, George Moschovitis wrote: > I have created a simple application that usesUnicodehtml files andUnicodedata > in the Datastore. To be more specific I use Greek text. > The application runs fine in the development server. However, when I > deployed the application to GAE all theUnicode(Greek) strings show > up as '?'. > > This problem affects both strings in html templates and strings > fetched from the Datastore. > > Any ideas? > -g. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
Very interesting, I don't have any problems with Polish characters. Firstly look at your tables via data-viewer in the production environment. I see Polish characters there. You have to narrow the problem - the characters are lost during sending, persisting or retrieving. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
I dont see what is posted to app engine, but I dont see proper characters using data viewer. If I correct the data in Data viewer, these are correctelz displayed in website. This leads to sending problem. What is your local setting? Polish or English? L. On 16 zář, 11:45, leszek wrote: > Very interesting, I don't have any problems with Polish characters. > Firstly look at your tables via data-viewer in the production > environment. I see Polish characters there. You have to narrow the > problem - the characters are lost during sending, persisting or > retrieving. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
Hmm, I see correct (Greek) characters in the DataViewer. But I get s in my app. The strange thing is that everything works correctly in the development server. -g. On Sep 16, 12:45 pm, leszek wrote: > Very interesting, I don't have any problems with Polish characters. > Firstly look at your tables via data-viewer in the production > environment. I see Polish characters there. You have to narrow the > problem - the characters are lost during sending, persisting or > retrieving. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
Yes, I o have this tag in all html files. But I am uploading .groovy files (Groovy source code) and these dont have this tag. These are source code UTF-8 encoded. L. On 16 zář, 12:28, Gianni Mariani wrote: > Make sure you have this meta tag in your generated html file. > > > Also, you can set a response HTTP header with the same string. > > On Wed, Sep 16, 2009 at 8:06 PM, George Moschovitis < > > > > > > george.moschovi...@gmail.com> wrote: > > > Hmm, I see correct (Greek) characters in the DataViewer. > > But I get s in my app. > > The strange thing is that everything works correctly in the > > development server. > > > -g. > > > On Sep 16, 12:45 pm, leszek wrote: > > > Very interesting, I don't have any problems with Polish characters. > > > Firstly look at your tables via data-viewer in the production > > > environment. I see Polish characters there. You have to narrow the > > > problem - the characters are lost during sending, persisting or > > > retrieving. > > -- > Gianni Mariani > Google, Sydney --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
> Make sure you have this meta tag in your generated html file. > I *do* have this tag in my html! > Also, you can set a response HTTP header with the same string. OK, I will try to set the HTTP header, but I don't think this is the problem. -g. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
These in the header - grey background - a stored a s constant in a java String in a source. I am using Groovy a Groovy templates that translates all to Java and html in runtime. Java stores everything like UTF-8. On my local GAE SDK its displayed correctly. These below with white background are from database. I am using JPA. If I import data from XML to database, special characters are damaged. XML contains UTF-8 representation. But when I correct database by GAE data viewer, it is displayed properly. So it seams, that encoding is somehow mixed up while transfering from local SDK to GAE. Ladislav. On 16 zář, 13:42, Gianni Mariani wrote: > Looking athttp://lskokan.appspot.com, somehow the strings are being > rendered incorrectly in the application. > > It would be nice to know what the rendering code looks like. > > On Wed, Sep 16, 2009 at 9:08 PM, George Moschovitis < > > > > > > george.moschovi...@gmail.com> wrote: > > > > Make sure you have this meta tag in your generated html file. > > > > > > I *do* have this tag in my html! > > > > Also, you can set a response HTTP header with the same string. > > > OK, I will try to set the HTTP header, but I don't think this is the > > problem. > > > -g. > > -- > Gianni Mariani > Google, Sydney --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
Hello I have found out how to resolve the issue. If I use any special local characters inside dir "war\..." and it is in .groovy script these are interpretted incorrectly. It looks like GAE does not UTF-8 encoding here. But if the file with local characters is inside "src\..." dir, there is everythign fine. This is sufficient for me. Thank you, Ladislav. On 16 zář, 16:09, Laco Skokan wrote: > These in the header - grey background - a stored a s constant in a > java String in a source. I am using Groovy a Groovy templates that > translates all to Java and html in runtime. Java stores everything > like UTF-8. On my local GAE SDK its displayed correctly. > > These below with white background are from database. I am using JPA. > If I import data from XML to database, special characters are damaged. > XML contains UTF-8 representation. But when I correct database by GAE > data viewer, it is displayed properly. > > So it seams, that encoding is somehow mixed up while transfering from > local SDK to GAE. > > Ladislav. > > On 16 zář, 13:42, Gianni Mariani wrote: > > > > > Looking athttp://lskokan.appspot.com, somehow the strings are being > > rendered incorrectly in the application. > > > It would be nice to know what the rendering code looks like. > > > On Wed, Sep 16, 2009 at 9:08 PM, George Moschovitis < > > > george.moschovi...@gmail.com> wrote: > > > > > Make sure you have this meta tag in your generated html file. > > > > > > > > I *do* have this tag in my html! > > > > > Also, you can set a response HTTP header with the same string. > > > > OK, I will try to set the HTTP header, but I don't think this is the > > > problem. > > > > -g. > > > -- > > Gianni Mariani > > Google, Sydney --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
One more note. The issue here is, that this behaves different way on local and deployed version. L. On 17 zář, 12:17, Laco Skokan wrote: > Hello > > I have found out how to resolve the issue. > > If I use any special local characters inside dir "war\..." and it is > in .groovy script these are interpretted incorrectly. It looks like > GAE does not UTF-8 encoding here. > > But if the file with local characters is inside "src\..." dir, there > is everythign fine. This is sufficient for me. > > Thank you, Ladislav. > > On 16 zář, 16:09, Laco Skokan wrote: > > > > > These in the header - grey background - a stored a s constant in a > > java String in a source. I am using Groovy a Groovy templates that > > translates all to Java and html in runtime. Java stores everything > > like UTF-8. On my local GAE SDK its displayed correctly. > > > These below with white background are from database. I am using JPA. > > If I import data from XML to database, special characters are damaged. > > XML contains UTF-8 representation. But when I correct database by GAE > > data viewer, it is displayed properly. > > > So it seams, that encoding is somehow mixed up while transfering from > > local SDK to GAE. > > > Ladislav. > > > On 16 zář, 13:42, Gianni Mariani wrote: > > > > Looking athttp://lskokan.appspot.com, somehow the strings are being > > > rendered incorrectly in the application. > > > > It would be nice to know what the rendering code looks like. > > > > On Wed, Sep 16, 2009 at 9:08 PM, George Moschovitis < > > > > george.moschovi...@gmail.com> wrote: > > > > > > Make sure you have this meta tag in your generated html file. > > > > > > > > > > I *do* have this tag in my html! > > > > > > Also, you can set a response HTTP header with the same string. > > > > > OK, I will try to set the HTTP header, but I don't think this is the > > > > problem. > > > > > -g. > > > > -- > > > Gianni Mariani > > > Google, Sydney --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
I'm using GWT on the client side. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
> OK, I will try to set the HTTP header, but I don't think this is the > problem. I have set the header but the problem is *not* fixed! Any other ideas? This is a show-stopper bug :( :( -g. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
> But if the file with local characters is inside "src\..." dir, there
> is everythign fine. This is sufficient for me.
saddly, this is not sufficient for me :(
I am still looking for a way to fix the encoding problem in files that
reside in war/
quite interestingly in my HTML files I have some friend-connect code
for signin etc. While the FC code renders correct Greek chars on the
development server,
again it renders garbage ('??' strings) on AppEngine.
-g.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
Hi George. Which version of the SDK are you using? Just to clarify, you're
seeing this in all HTML files under war/, not your JSPs? Would it be
possible for you to post a sample HTML file that I can use to reproduce the
issue? I've tried with UTF-8 encoding in my local development environment,
but everything is rendering correctly even after being deployed.
Thank you,
- Jason
On Fri, Sep 18, 2009 at 1:12 AM, George Moschovitis <
george.moschovi...@gmail.com> wrote:
>
> > But if the file with local characters is inside "src\..." dir, there
> > is everythign fine. This is sufficient for me.
>
> saddly, this is not sufficient for me :(
> I am still looking for a way to fix the encoding problem in files that
> reside in war/
>
> quite interestingly in my HTML files I have some friend-connect code
> for signin etc. While the FC code renders correct Greek chars on the
> development server,
> again it renders garbage ('??' strings) on AppEngine.
>
> -g.
> >
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
[appengine-java] Re: Unicode problems
FYI, I fixed the problem with a little patch in narwhal (the commonjs
implementation used by Jack):
narwhal/engines/rhino/lib/binary-engine.js
exports.B_DECODE_DEFAULT = function(bytes, offset, length) {
return String(new Packages.java.lang.String(bytes, offset, length,
"UTF-8"));
}
exports.B_ENCODE_DEFAULT = function(string) {
return new Packages.java.lang.String(string).getBytes("UTF-8");
}
regards,
George.
On Oct 14, 9:25 pm, "Jason (Google)" wrote:
> Hi Prashant. I'm still waiting on a .html or .txt file to test with using
> Helma NG/Jack in particular. If you're seeing something similar in a
> different environment, please post a new thread with more details. I do
> recall that there was a known issue with XMPP character encoding in version
> 1.2.5 -- can you check if this is fixed with yesterday's 1.2.6 release?
> - Jason
>
>
>
> On Sun, Oct 11, 2009 at 11:40 AM, Prashant wrote:
> > is this issue solved?
>
> > i am having similar problem when i store some message, received via xmpp,
> > to datastore.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
Re: [giannim] [appengine-java] Re: Unicode problems
Make sure you have this meta tag in your generated html file. Also, you can set a response HTTP header with the same string. On Wed, Sep 16, 2009 at 8:06 PM, George Moschovitis < george.moschovi...@gmail.com> wrote: > > > Hmm, I see correct (Greek) characters in the DataViewer. > But I get s in my app. > The strange thing is that everything works correctly in the > development server. > > -g. > > On Sep 16, 12:45 pm, leszek wrote: > > Very interesting, I don't have any problems with Polish characters. > > Firstly look at your tables via data-viewer in the production > > environment. I see Polish characters there. You have to narrow the > > problem - the characters are lost during sending, persisting or > > retrieving. > > > > > -- Gianni Mariani Google, Sydney --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
Re: [giannim] [appengine-java] Re: Unicode problems
Looking at http://lskokan.appspot.com, somehow the strings are being rendered incorrectly in the application. It would be nice to know what the rendering code looks like. On Wed, Sep 16, 2009 at 9:08 PM, George Moschovitis < george.moschovi...@gmail.com> wrote: > > > > Make sure you have this meta tag in your generated html file. > > > > I *do* have this tag in my html! > > > Also, you can set a response HTTP header with the same string. > > OK, I will try to set the HTTP header, but I don't think this is the > problem. > > -g. > > > > > > -- Gianni Mariani Google, Sydney --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
