Re: [Trinidad] forced UTF-8 in PPR responses?
According to the W3C specification, XML http responses should always use UTF-8 encoding (requests too actually) http://www.w3.org/TR/XMLHttpRequest/ "Authors are strongly encouraged to encode their resources using UTF-8" http://erik.eae.net/archives/2005/05/27/18.55.22/: "UTF-8 is the standard encoding for XML files, so it MSXML probably assumes that all files have that encoding if none is set." http://www.w3.org/TR/2006/WD-XMLHttpRequest-20060405/ "responseText of type DOMString If the readyState attribute has a value other than 3 (Receiving) or 4 (Loaded), it must be the empty string. Otherwise, it must be the the body of the data received so far, interpreted using the character encoding specified in the response, or UTF-8 if no character encoding was specified. Invalid bytes must be converted to U+FFFD REPLACEMENT CHARACTER." "If the method is POST or PUT, then the data passed to the send() method must be used for the entity body. If data is a string, the data must be encoded as UTF-8 for transmission. If the data is a Document, then the document must be serialised using the encoding given by data.xmlEncoding, if specified, or UTF-8 otherwise [DOM3]. If data is not a Document or a DOMString the host language must use the stringification mechanisms on the argument that was passed." Basically from what I have seen in the google results, UTF-8 is the XML standard and browsers are expecting AJAX to use UTF-8 for both request and responses. It appears that Trinidad is honoring these guidelines by forcing UTF-8 for XML responses (and other responses, like file-download) My question is why you are using windows-1252 encoding? What is the "unavoidable reason"? -Andrew On Thu, Feb 11, 2010 at 6:07 AM, Cédric Durmont wrote: > Hi there, > > I was wondering : what's the reason why XmlHttpServletResponse forces > the response to UTF-8, explicitly ignoring the page's encoding ? > I had a project in utf-8 that ran just fine (even with all accents and > fancy stuff we have here in France), but I have to switch it to > windows-1252 for some unavoidable reason. Everything has been > converted to windows-1252, including the filter I use to force > encoding in http requests. The only non-working things are PPR calls. > > I tracked modifications of http Response objects down to > XmlHttpServletResponse : > > .. > _contentType = "text/xml;charset=utf-8"; > .. > > So, did I miss something, or PPR actually only works for iso8859-1 / > utf-8 apps ? > > > Regards, > Cedric Durmont >
Re: [Trinidad] forced UTF-8 in PPR responses?
Thanks for the answer. So Trinidad is fine, which is good news, but I have to find another explanation as why it fails on my app. If I'm not mistaking, the browser should make the conversion on-the-fly before putting the new content into the page. But it does dot, and I have  © characters, meaning that those are utf8 chars interpreted as windows-1252. About the reason that I am using win1252 instead of utf-8 : I'd be glad to hear that it's not totally "unavoidable". We're developping a new application that will come as a complement of our older ones. One prerequisite is to re-use some database tables (say, customers, cities/countries, and the likes). The database used is... Oracle 10 XE (yeah, I know, don't get me started), which has no i18n support, and only knows windows-1252. I'm not the Master of Charset Encoding, especially when it comes to databases, but I was told that in this special case, I cannot instruct the Oracle server that I'm a utf8-speaking client, so I have to use the default, which is windows-1252. I'd take ANY solution to this problem, as far as it doesn't break the compatibility with the other apps using the same database. If there's a way to have xmhHttpRequest responses correctly displayed as windows1252, then I'm fine. If I can instead keep utf8 and still use the existing databases as-is, it'll make my day :o) Again, thanks for answering me Regards, Cedric Durmont 2010/2/11 Andrew Robinson : > According to the W3C specification, XML http responses should always > use UTF-8 encoding (requests too actually) > http://www.w3.org/TR/XMLHttpRequest/ > > "Authors are strongly encouraged to encode their resources using UTF-8" > > http://erik.eae.net/archives/2005/05/27/18.55.22/: > "UTF-8 is the standard encoding for XML files, so it MSXML probably > assumes that all files have that encoding if none is set." > > http://www.w3.org/TR/2006/WD-XMLHttpRequest-20060405/ > "responseText of type DOMString > If the readyState attribute has a value other than 3 (Receiving) or 4 > (Loaded), it must be the empty string. Otherwise, it must be the the > body of the data received so far, interpreted using the character > encoding specified in the response, or UTF-8 if no character encoding > was specified. Invalid bytes must be converted to U+FFFD REPLACEMENT > CHARACTER." > "If the method is POST or PUT, then the data passed to the send() > method must be used for the entity body. If data is a string, the data > must be encoded as UTF-8 for transmission. If the data is a Document, > then the document must be serialised using the encoding given by > data.xmlEncoding, if specified, or UTF-8 otherwise [DOM3]. If data is > not a Document or a DOMString the host language must use the > stringification mechanisms on the argument that was passed." > > Basically from what I have seen in the google results, UTF-8 is the > XML standard and browsers are expecting AJAX to use UTF-8 for both > request and responses. It appears that Trinidad is honoring these > guidelines by forcing UTF-8 for XML responses (and other responses, > like file-download) > > My question is why you are using windows-1252 encoding? What is the > "unavoidable reason"? > > -Andrew > > > On Thu, Feb 11, 2010 at 6:07 AM, Cédric Durmont wrote: >> Hi there, >> >> I was wondering : what's the reason why XmlHttpServletResponse forces >> the response to UTF-8, explicitly ignoring the page's encoding ? >> I had a project in utf-8 that ran just fine (even with all accents and >> fancy stuff we have here in France), but I have to switch it to >> windows-1252 for some unavoidable reason. Everything has been >> converted to windows-1252, including the filter I use to force >> encoding in http requests. The only non-working things are PPR calls. >> >> I tracked modifications of http Response objects down to >> XmlHttpServletResponse : >> >> .. >> _contentType = "text/xml;charset=utf-8"; >> .. >> >> So, did I miss something, or PPR actually only works for iso8859-1 / >> utf-8 apps ? >> >> >> Regards, >> Cedric Durmont >> >
Re: [Trinidad] forced UTF-8 in PPR responses?
Hi, we were having the same problem some time ago. We added a filter to our webapp that checks for the type of request and change the encoding properly. This is, you can call CoreRenderKit.isAjaxRequest(externalContext) to check if it is a PPR request and in that case, you can force the encoding to be UTF-8, avoiding the wrong conversion. HTH, -- Rafa On Fri, Feb 12, 2010 at 12:38 PM, Cédric Durmont wrote: > Thanks for the answer. So Trinidad is fine, which is good news, but I > have to find another explanation as why it fails on my app. If I'm not > mistaking, the browser should make the conversion on-the-fly before > putting the new content into the page. But it does dot, and I have  © > characters, meaning that those are utf8 chars interpreted as > windows-1252. > > About the reason that I am using win1252 instead of utf-8 : I'd be > glad to hear that it's not totally "unavoidable". > We're developping a new application that will come as a complement of > our older ones. One prerequisite is to re-use some database tables > (say, customers, cities/countries, and the likes). The database used > is... Oracle 10 XE (yeah, I know, don't get me started), which has no > i18n support, and only knows windows-1252. > > I'm not the Master of Charset Encoding, especially when it comes to > databases, but I was told that in this special case, I cannot instruct > the Oracle server that I'm a utf8-speaking client, so I have to use > the default, which is windows-1252. > > I'd take ANY solution to this problem, as far as it doesn't break the > compatibility with the other apps using the same database. If there's > a way to have xmhHttpRequest responses correctly displayed as > windows1252, then I'm fine. If I can instead keep utf8 and still use > the existing databases as-is, it'll make my day :o) > > Again, thanks for answering me > Regards, > Cedric Durmont > > 2010/2/11 Andrew Robinson : > > According to the W3C specification, XML http responses should always > > use UTF-8 encoding (requests too actually) > > http://www.w3.org/TR/XMLHttpRequest/ > > > > "Authors are strongly encouraged to encode their resources using UTF-8" > > > > http://erik.eae.net/archives/2005/05/27/18.55.22/: > > "UTF-8 is the standard encoding for XML files, so it MSXML probably > > assumes that all files have that encoding if none is set." > > > > http://www.w3.org/TR/2006/WD-XMLHttpRequest-20060405/ > > "responseText of type DOMString > > If the readyState attribute has a value other than 3 (Receiving) or 4 > > (Loaded), it must be the empty string. Otherwise, it must be the the > > body of the data received so far, interpreted using the character > > encoding specified in the response, or UTF-8 if no character encoding > > was specified. Invalid bytes must be converted to U+FFFD REPLACEMENT > > CHARACTER." > > "If the method is POST or PUT, then the data passed to the send() > > method must be used for the entity body. If data is a string, the data > > must be encoded as UTF-8 for transmission. If the data is a Document, > > then the document must be serialised using the encoding given by > > data.xmlEncoding, if specified, or UTF-8 otherwise [DOM3]. If data is > > not a Document or a DOMString the host language must use the > > stringification mechanisms on the argument that was passed." > > > > Basically from what I have seen in the google results, UTF-8 is the > > XML standard and browsers are expecting AJAX to use UTF-8 for both > > request and responses. It appears that Trinidad is honoring these > > guidelines by forcing UTF-8 for XML responses (and other responses, > > like file-download) > > > > My question is why you are using windows-1252 encoding? What is the > > "unavoidable reason"? > > > > -Andrew > > > > > > On Thu, Feb 11, 2010 at 6:07 AM, Cédric Durmont > wrote: > >> Hi there, > >> > >> I was wondering : what's the reason why XmlHttpServletResponse forces > >> the response to UTF-8, explicitly ignoring the page's encoding ? > >> I had a project in utf-8 that ran just fine (even with all accents and > >> fancy stuff we have here in France), but I have to switch it to > >> windows-1252 for some unavoidable reason. Everything has been > >> converted to windows-1252, including the filter I use to force > >> encoding in http requests. The only non-working things are PPR calls. > >> > >> I tracked modifications of http Response objects down to > >> XmlHttpServletResponse : > >> > >> .. > >> _contentType = "text/xml;charset=utf-8"; > >> .. > >> > >> So, did I miss something, or PPR actually only works for iso8859-1 / > >> utf-8 apps ? > >> > >> > >> Regards, > >> Cedric Durmont > >> > > >
Re: [Trinidad] forced UTF-8 in PPR responses?
I recommend trying to set the encoding in the web container's configuration (eg: server.xml). Mike Quentel Senior Geospatial Software Developer 4DM Inc. 671 Danforth Avenue Suite 305 Toronto, Ontario M4J 1L3 Ph/Fax 416 - 410-7569 www.4dm-inc.com Providing solutions through mapping technology -Original Message- From: Cédric Durmont Date: Fri, 12 Feb 2010 12:38:39 To: MyFaces Discussion Subject: Re: [Trinidad] forced UTF-8 in PPR responses? Thanks for the answer. So Trinidad is fine, which is good news, but I have to find another explanation as why it fails on my app. If I'm not mistaking, the browser should make the conversion on-the-fly before putting the new content into the page. But it does dot, and I have  © characters, meaning that those are utf8 chars interpreted as windows-1252. About the reason that I am using win1252 instead of utf-8 : I'd be glad to hear that it's not totally "unavoidable". We're developping a new application that will come as a complement of our older ones. One prerequisite is to re-use some database tables (say, customers, cities/countries, and the likes). The database used is... Oracle 10 XE (yeah, I know, don't get me started), which has no i18n support, and only knows windows-1252. I'm not the Master of Charset Encoding, especially when it comes to databases, but I was told that in this special case, I cannot instruct the Oracle server that I'm a utf8-speaking client, so I have to use the default, which is windows-1252. I'd take ANY solution to this problem, as far as it doesn't break the compatibility with the other apps using the same database. If there's a way to have xmhHttpRequest responses correctly displayed as windows1252, then I'm fine. If I can instead keep utf8 and still use the existing databases as-is, it'll make my day :o) Again, thanks for answering me Regards, Cedric Durmont 2010/2/11 Andrew Robinson : > According to the W3C specification, XML http responses should always > use UTF-8 encoding (requests too actually) > http://www.w3.org/TR/XMLHttpRequest/ > > "Authors are strongly encouraged to encode their resources using UTF-8" > > http://erik.eae.net/archives/2005/05/27/18.55.22/: > "UTF-8 is the standard encoding for XML files, so it MSXML probably > assumes that all files have that encoding if none is set." > > http://www.w3.org/TR/2006/WD-XMLHttpRequest-20060405/ > "responseText of type DOMString > If the readyState attribute has a value other than 3 (Receiving) or 4 > (Loaded), it must be the empty string. Otherwise, it must be the the > body of the data received so far, interpreted using the character > encoding specified in the response, or UTF-8 if no character encoding > was specified. Invalid bytes must be converted to U+FFFD REPLACEMENT > CHARACTER." > "If the method is POST or PUT, then the data passed to the send() > method must be used for the entity body. If data is a string, the data > must be encoded as UTF-8 for transmission. If the data is a Document, > then the document must be serialised using the encoding given by > data.xmlEncoding, if specified, or UTF-8 otherwise [DOM3]. If data is > not a Document or a DOMString the host language must use the > stringification mechanisms on the argument that was passed." > > Basically from what I have seen in the google results, UTF-8 is the > XML standard and browsers are expecting AJAX to use UTF-8 for both > request and responses. It appears that Trinidad is honoring these > guidelines by forcing UTF-8 for XML responses (and other responses, > like file-download) > > My question is why you are using windows-1252 encoding? What is the > "unavoidable reason"? > > -Andrew > > > On Thu, Feb 11, 2010 at 6:07 AM, Cédric Durmont wrote: >> Hi there, >> >> I was wondering : what's the reason why XmlHttpServletResponse forces >> the response to UTF-8, explicitly ignoring the page's encoding ? >> I had a project in utf-8 that ran just fine (even with all accents and >> fancy stuff we have here in France), but I have to switch it to >> windows-1252 for some unavoidable reason. Everything has been >> converted to windows-1252, including the filter I use to force >> encoding in http requests. The only non-working things are PPR calls. >> >> I tracked modifications of http Response objects down to >> XmlHttpServletResponse : >> >> .. >> _contentType = "text/xml;charset=utf-8"; >> .. >> >> So, did I miss something, or PPR actually only works for iso8859-1 / >> utf-8 apps ? >> >> >> Regards, >> Cedric Durmont >> >
Re: [Trinidad] forced UTF-8 in PPR responses?
Thanks for the tip, Rafa, now it works like a charm :o)
Could not find how to get an ExternalContext from inside a filter,
however. So my test is :
if ("true".equals(((HttpServletRequest)(request)).getHeader("Tr-XHR-Message")) )
... because it's basically what isAjaxRequest and ExternalContext actually do.
Thanks everyone for your help :o)
Regards,
Cedric Durmont
2010/2/12 Mike Quentel (4DM) :
> I recommend trying to set the encoding in the web container's configuration
> (eg: server.xml).
>
>
> Mike Quentel
> Senior Geospatial Software Developer
> 4DM Inc.
> 671 Danforth Avenue Suite 305
> Toronto, Ontario
> M4J 1L3
> Ph/Fax 416 - 410-7569
> www.4dm-inc.com
> Providing solutions through mapping technology
>
> -Original Message-
> From: Cédric Durmont
> Date: Fri, 12 Feb 2010 12:38:39
> To: MyFaces Discussion
> Subject: Re: [Trinidad] forced UTF-8 in PPR responses?
>
> Thanks for the answer. So Trinidad is fine, which is good news, but I
> have to find another explanation as why it fails on my app. If I'm not
> mistaking, the browser should make the conversion on-the-fly before
> putting the new content into the page. But it does dot, and I have  ©
> characters, meaning that those are utf8 chars interpreted as
> windows-1252.
>
> About the reason that I am using win1252 instead of utf-8 : I'd be
> glad to hear that it's not totally "unavoidable".
> We're developping a new application that will come as a complement of
> our older ones. One prerequisite is to re-use some database tables
> (say, customers, cities/countries, and the likes). The database used
> is... Oracle 10 XE (yeah, I know, don't get me started), which has no
> i18n support, and only knows windows-1252.
>
> I'm not the Master of Charset Encoding, especially when it comes to
> databases, but I was told that in this special case, I cannot instruct
> the Oracle server that I'm a utf8-speaking client, so I have to use
> the default, which is windows-1252.
>
> I'd take ANY solution to this problem, as far as it doesn't break the
> compatibility with the other apps using the same database. If there's
> a way to have xmhHttpRequest responses correctly displayed as
> windows1252, then I'm fine. If I can instead keep utf8 and still use
> the existing databases as-is, it'll make my day :o)
>
> Again, thanks for answering me
> Regards,
> Cedric Durmont
>
> 2010/2/11 Andrew Robinson :
>> According to the W3C specification, XML http responses should always
>> use UTF-8 encoding (requests too actually)
>> http://www.w3.org/TR/XMLHttpRequest/
>>
>> "Authors are strongly encouraged to encode their resources using UTF-8"
>>
>> http://erik.eae.net/archives/2005/05/27/18.55.22/:
>> "UTF-8 is the standard encoding for XML files, so it MSXML probably
>> assumes that all files have that encoding if none is set."
>>
>> http://www.w3.org/TR/2006/WD-XMLHttpRequest-20060405/
>> "responseText of type DOMString
>> If the readyState attribute has a value other than 3 (Receiving) or 4
>> (Loaded), it must be the empty string. Otherwise, it must be the the
>> body of the data received so far, interpreted using the character
>> encoding specified in the response, or UTF-8 if no character encoding
>> was specified. Invalid bytes must be converted to U+FFFD REPLACEMENT
>> CHARACTER."
>> "If the method is POST or PUT, then the data passed to the send()
>> method must be used for the entity body. If data is a string, the data
>> must be encoded as UTF-8 for transmission. If the data is a Document,
>> then the document must be serialised using the encoding given by
>> data.xmlEncoding, if specified, or UTF-8 otherwise [DOM3]. If data is
>> not a Document or a DOMString the host language must use the
>> stringification mechanisms on the argument that was passed."
>>
>> Basically from what I have seen in the google results, UTF-8 is the
>> XML standard and browsers are expecting AJAX to use UTF-8 for both
>> request and responses. It appears that Trinidad is honoring these
>> guidelines by forcing UTF-8 for XML responses (and other responses,
>> like file-download)
>>
>> My question is why you are using windows-1252 encoding? What is the
>> "unavoidable reason"?
>>
>> -Andrew
>>
>>
>> On Thu, Feb 11, 2010 at 6:07 AM, Cédric Durmont wrote:
>>> Hi there,
>>>
>>> I was wondering : what's the reason why XmlHttpServletResponse forces
>>> the response to UTF-8, explicitly ignoring the page's
Re: [Trinidad] forced UTF-8 in PPR responses?
Good! :D
On Fri, Feb 12, 2010 at 4:57 PM, Cédric Durmont wrote:
> Thanks for the tip, Rafa, now it works like a charm :o)
> Could not find how to get an ExternalContext from inside a filter,
> however. So my test is :
> if
> ("true".equals(((HttpServletRequest)(request)).getHeader("Tr-XHR-Message"))
> )
>
> ... because it's basically what isAjaxRequest and ExternalContext actually
> do.
>
> Thanks everyone for your help :o)
> Regards,
> Cedric Durmont
>
>
> 2010/2/12 Mike Quentel (4DM) :
> > I recommend trying to set the encoding in the web container's
> configuration (eg: server.xml).
> >
> >
> > Mike Quentel
> > Senior Geospatial Software Developer
> > 4DM Inc.
> > 671 Danforth Avenue Suite 305
> > Toronto, Ontario
> > M4J 1L3
> > Ph/Fax 416 - 410-7569
> > www.4dm-inc.com
> > Providing solutions through mapping technology....
> >
> > -Original Message-
> > From: Cédric Durmont
> > Date: Fri, 12 Feb 2010 12:38:39
> > To: MyFaces Discussion
> > Subject: Re: [Trinidad] forced UTF-8 in PPR responses?
> >
> > Thanks for the answer. So Trinidad is fine, which is good news, but I
> > have to find another explanation as why it fails on my app. If I'm not
> > mistaking, the browser should make the conversion on-the-fly before
> > putting the new content into the page. But it does dot, and I have  ©
> > characters, meaning that those are utf8 chars interpreted as
> > windows-1252.
> >
> > About the reason that I am using win1252 instead of utf-8 : I'd be
> > glad to hear that it's not totally "unavoidable".
> > We're developping a new application that will come as a complement of
> > our older ones. One prerequisite is to re-use some database tables
> > (say, customers, cities/countries, and the likes). The database used
> > is... Oracle 10 XE (yeah, I know, don't get me started), which has no
> > i18n support, and only knows windows-1252.
> >
> > I'm not the Master of Charset Encoding, especially when it comes to
> > databases, but I was told that in this special case, I cannot instruct
> > the Oracle server that I'm a utf8-speaking client, so I have to use
> > the default, which is windows-1252.
> >
> > I'd take ANY solution to this problem, as far as it doesn't break the
> > compatibility with the other apps using the same database. If there's
> > a way to have xmhHttpRequest responses correctly displayed as
> > windows1252, then I'm fine. If I can instead keep utf8 and still use
> > the existing databases as-is, it'll make my day :o)
> >
> > Again, thanks for answering me
> > Regards,
> > Cedric Durmont
> >
> > 2010/2/11 Andrew Robinson :
> >> According to the W3C specification, XML http responses should always
> >> use UTF-8 encoding (requests too actually)
> >> http://www.w3.org/TR/XMLHttpRequest/
> >>
> >> "Authors are strongly encouraged to encode their resources using UTF-8"
> >>
> >> http://erik.eae.net/archives/2005/05/27/18.55.22/:
> >> "UTF-8 is the standard encoding for XML files, so it MSXML probably
> >> assumes that all files have that encoding if none is set."
> >>
> >> http://www.w3.org/TR/2006/WD-XMLHttpRequest-20060405/
> >> "responseText of type DOMString
> >> If the readyState attribute has a value other than 3 (Receiving) or 4
> >> (Loaded), it must be the empty string. Otherwise, it must be the the
> >> body of the data received so far, interpreted using the character
> >> encoding specified in the response, or UTF-8 if no character encoding
> >> was specified. Invalid bytes must be converted to U+FFFD REPLACEMENT
> >> CHARACTER."
> >> "If the method is POST or PUT, then the data passed to the send()
> >> method must be used for the entity body. If data is a string, the data
> >> must be encoded as UTF-8 for transmission. If the data is a Document,
> >> then the document must be serialised using the encoding given by
> >> data.xmlEncoding, if specified, or UTF-8 otherwise [DOM3]. If data is
> >> not a Document or a DOMString the host language must use the
> >> stringification mechanisms on the argument that was passed."
> >>
> >> Basically from what I have seen in the google results, UTF-8 is the
> >> XML standard and browsers are expecting AJAX to use UTF-8 for both
> >> request and responses. It appears that Trinidad is honoring
