Re: TextDecode JSON array
On 1/22/17 1:26 AM, Peter TB Brett via use-livecode wrote:
On 22/01/2017 03:22, J. Landman Gay via use-livecode wrote:
Here's a test sample of some UTF8 I get back from a server:
{"UserID":48,"UserName":"Eduardo Ba\u00f1uls","UserLoginName":"ebanu"}
Hi Jacque,
This is valid JSON (and also valid ASCII). In JSON, any character in a
string may be encoded in the form \u where is the 4-digit
hexadecimal representation of a Unicode codepoint. No textDecode()
operation is required.
JSONImport() handles this correctly.
1) Create a stack with a field and a button
2) Put the text above into the field
3) Set the script of the button to:
local tJson
put JSONImport(field 1) into tJson
answer tJson["UserName"]
4) Enter browse mode and click the button
You will get an answer dialog displaying "Eduardo Bañuls".
Thank you, and Trevor also, for this. I hadn't heard of importJSON()
before but it seems to be what I need. I'll take Trevor's advice and
textDecode the JSON before running it through JSONImport(). All the
incoming data is being decoded already anyway so I'll just leave that in
place.
--
Jacqueline Landman Gay | jac...@hyperactivesw.com
HyperActive Software | http://www.hyperactivesw.com
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: TextDecode JSON array
On Sun, Jan 22, 2017 at 1:26 AM, Peter TB Brett via use-livecode <
use-livecode@lists.runrev.com> wrote:
> On 22/01/2017 03:22, J. Landman Gay via use-livecode wrote:
>
>> Here's a test sample of some UTF8 I get back from a server:
>>
>> {"UserID":48,"UserName":"Eduardo Ba\u00f1uls","UserLoginName":"ebanu"}
>
>
> This is valid JSON (and also valid ASCII). In JSON, any character in a
> string may be encoded in the form \u where is the 4-digit
> hexadecimal representation of a Unicode codepoint. No textDecode()
> operation is required.
>
> JSONImport() handles this correctly.
>
Let me add a warning about this. I was going to write this in my original
response but I was writing on my phone when I responded.
While it is true that you don't need to run textDecode() on the JSON and
that JSONImprt() will handle it correctly, you may want to do it anyway if
the JSON is coming from a web application with a Content-Type header that
says it is UTF-8 encoded. I say that based upon the two days my team and I
spent troubleshooting a related issue just two weeks ago.
We recently completed a major update of our web application which involved
upgrading the version of Ruby on Rails that we use. We switch from version
3 to version 4. The `#to_json` method in Ruby on Rails 3 returned JSON data
that was encoded using escape sequences (\u) with the header
`Content-Type: application/json; charset=utf-8`. As Peter points out, the
data is actually valid ASCII. When I originally wrote the LiveCode code
that parsed the JSON responses I used JSONImport() and everything worked as
I expected. Based on the UTF-8 headers I was seeing and the fact that my
code was handling unicode characters properly, I assumed that libURL was
transparently decoding the data returned by a url in LC >= 7.
After deploying our app upgrade two weeks ago we started getting reports of
weird characters showing up in the desktop software. What we found out is
that the `#to_json` method in Ruby on Rails 4 stopped using escape
sequences and started returning UTF-8 encoded JSON. It turns out that
libURL only decodes SOME of data that passes through it [1]. Since my
LiveCode app was not decoding the UTF-8 data that the server was returning
to the internal LiveCode text format any characters with a codepoint > 127
would display as an odd character in our app.
Tracking down the cause took a lot of time. We added a monkey patch on our
server that switched back to returning the JSON to the escape sequence
format. That way existing desktop software would continue to work. I've
updated the custom version of libURL that we use so that it looks at the
Content-Type header and runs textDecode() on data passing through it. We
will remove the monkey patch on the server after we have pushed the update
desktop software out to everyone.
Moral of the story for people using LC >= 7? If the Content-Type header of
your JSON responses says it is supposed to be UTF-8 encoded then decode the
data when it comes into LiveCode. That way you know it is in the correct
encoding before you start working with it.
[1] There is a ulDecodeData() function in versions of libURL that ship with
LC >= 7. It will decode HTML data with a http://quality.livecode.com/show_bug.cgi?id=19085.
--
Trevor DeVore
Outcome & ScreenSteps
www.outcomeapp.io - www.screensteps.com
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: TextDecode JSON array
On 22/01/2017 03:22, J. Landman Gay via use-livecode wrote:
Here's a test sample of some UTF8 I get back from a server:
{"UserID":48,"UserName":"Eduardo Ba\u00f1uls","UserLoginName":"ebanu"}
Hi Jacque,
This is valid JSON (and also valid ASCII). In JSON, any character in a
string may be encoded in the form \u where is the 4-digit
hexadecimal representation of a Unicode codepoint. No textDecode()
operation is required.
JSONImport() handles this correctly.
1) Create a stack with a field and a button
2) Put the text above into the field
3) Set the script of the button to:
> local tJson
> put JSONImport(field 1) into tJson
> answer tJson["UserName"]
4) Enter browse mode and click the button
You will get an answer dialog displaying "Eduardo Bañuls".
Peter
--
Dr Peter Brett
LiveCode Technical Project Manager
lcb-mode for Emacs: https://github.com/peter-b/lcb-mode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: TextDecode JSON array
That is UTF-8. I've seen older versions of Ruby on Rails would use escape
sequences like that in JSON responses.
--
Trevor DeVore
On Sat, Jan 21, 2017 at 9:22 PM J. Landman Gay via use-livecode <
use-livecode@lists.runrev.com> wrote:
> Here's a test sample of some UTF8 I get back from a server:
>
>
>
>{"UserID":48,"UserName":"Eduardo Ba\u00f1uls","UserLoginName":"ebanu"}
>
>
>
>
>
> If I run textDecode(,"UTF8") on it, I get it back unaltered.
>
> Does anyone recognize the text encoding? It doesn't seem to be UTF8.
>
>
>
> If I then run JSONToArray on it, I get a LC array and the name is
>
> changed to this:
>
>
>
>Eduardo Bañuls
>
>
>
> And when I run textDecode on *that* it comes out right:
>
>
>
>Eduardo Bañuls
>
>
>
> So what's the first string? JSONtoArray seems to recognize it. I'd like
>
> to decode the JSON array completely before passing it to JSONtoArray. If
>
> I can do that, I don't have to loop through all the array keys, decoding
>
> each one individually.
>
>
>
> --
>
> Jacqueline Landman Gay | jac...@hyperactivesw.com
>
> HyperActive Software | http://www.hyperactivesw.com
>
>
>
>
>
> ___
>
> use-livecode mailing list
>
> use-livecode@lists.runrev.com
>
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
>
> http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: TextDecode JSON array
So LC should recognize it without alteration I'd assume, but it doesn't.
Probably the leading "\u"? I'm still not sure how to handle that.
On 1/21/17 9:29 PM, Mike Bonner via use-livecode wrote:
Looks like utf16 based on this info I found here:
https://codepoints.net/U+00F1?lang=en
SystemRepresentation
Nº 241
UTF-8 C3 B1
UTF-16 00 F1
On Sat, Jan 21, 2017 at 8:22 PM, J. Landman Gay via use-livecode <
use-livecode@lists.runrev.com> wrote:
Here's a test sample of some UTF8 I get back from a server:
{"UserID":48,"UserName":"Eduardo Ba\u00f1uls","UserLoginName":"ebanu"}
If I run textDecode(,"UTF8") on it, I get it back unaltered. Does
anyone recognize the text encoding? It doesn't seem to be UTF8.
If I then run JSONToArray on it, I get a LC array and the name is changed
to this:
Eduardo Bañuls
And when I run textDecode on *that* it comes out right:
Eduardo Bañuls
So what's the first string? JSONtoArray seems to recognize it. I'd like to
decode the JSON array completely before passing it to JSONtoArray. If I can
do that, I don't have to loop through all the array keys, decoding each one
individually.
--
Jacqueline Landman Gay | jac...@hyperactivesw.com
HyperActive Software | http://www.hyperactivesw.com
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
--
Jacqueline Landman Gay | jac...@hyperactivesw.com
HyperActive Software | http://www.hyperactivesw.com
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: TextDecode JSON array
On 1/21/17 9:22 PM, J. Landman Gay via use-livecode wrote:
Here's a test sample of some UTF8 I get back from a server:
{"UserID":48,"UserName":"Eduardo Ba\u00f1uls","UserLoginName":"ebanu"}
If I run textDecode(,"UTF8") on it, I get it back unaltered.
Does anyone recognize the text encoding? It doesn't seem to be UTF8.
If I then run JSONToArray on it, I get a LC array and the name is
changed to this:
Eduardo Bañuls
And when I run textDecode on *that* it comes out right:
Eduardo Bañuls
So what's the first string? JSONtoArray seems to recognize it. I'd like
to decode the JSON array completely before passing it to JSONtoArray. If
I can do that, I don't have to loop through all the array keys, decoding
each one individually.
Hm. Google says it's C/C++/Java encoding. Not sure what to do with that.
--
Jacqueline Landman Gay | jac...@hyperactivesw.com
HyperActive Software | http://www.hyperactivesw.com
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Re: TextDecode JSON array
Looks like utf16 based on this info I found here:
https://codepoints.net/U+00F1?lang=en
SystemRepresentation
Nº 241
UTF-8 C3 B1
UTF-16 00 F1
On Sat, Jan 21, 2017 at 8:22 PM, J. Landman Gay via use-livecode <
use-livecode@lists.runrev.com> wrote:
> Here's a test sample of some UTF8 I get back from a server:
>
> {"UserID":48,"UserName":"Eduardo Ba\u00f1uls","UserLoginName":"ebanu"}
>
>
> If I run textDecode(,"UTF8") on it, I get it back unaltered. Does
> anyone recognize the text encoding? It doesn't seem to be UTF8.
>
> If I then run JSONToArray on it, I get a LC array and the name is changed
> to this:
>
> Eduardo Bañuls
>
> And when I run textDecode on *that* it comes out right:
>
> Eduardo Bañuls
>
> So what's the first string? JSONtoArray seems to recognize it. I'd like to
> decode the JSON array completely before passing it to JSONtoArray. If I can
> do that, I don't have to loop through all the array keys, decoding each one
> individually.
>
> --
> Jacqueline Landman Gay | jac...@hyperactivesw.com
> HyperActive Software | http://www.hyperactivesw.com
>
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
TextDecode JSON array
Here's a test sample of some UTF8 I get back from a server:
{"UserID":48,"UserName":"Eduardo Ba\u00f1uls","UserLoginName":"ebanu"}
If I run textDecode(,"UTF8") on it, I get it back unaltered.
Does anyone recognize the text encoding? It doesn't seem to be UTF8.
If I then run JSONToArray on it, I get a LC array and the name is
changed to this:
Eduardo Bañuls
And when I run textDecode on *that* it comes out right:
Eduardo Bañuls
So what's the first string? JSONtoArray seems to recognize it. I'd like
to decode the JSON array completely before passing it to JSONtoArray. If
I can do that, I don't have to loop through all the array keys, decoding
each one individually.
--
Jacqueline Landman Gay | jac...@hyperactivesw.com
HyperActive Software | http://www.hyperactivesw.com
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
