On 2014/02/17 19:01, Stephan Beal wrote:
On Mon, Feb 17, 2014 at 5:57 PM, Tim Streater <t...@clothears.org.uk> wrote:
complete in a browser window, which data is then gathered up and sent
using ajax to be processed by a PHP script, which writes it to an sqlite
db. The user complains that some of this data doesn't make it, so I want to
pin down where in the chain this might be failing. Hence my Q.
FWIW, i have seen a similar problem in a legacy app which uses latin1
encoding in the DB. Latin1 doesn't always survive round-trip through PHP's
JSON APIs. My case was similar to yours, and we eventually determined that
the fields which got "lost" (set to null or empty values) were those which
came out of the latin1-encoded MySQL db containing invalid UTF8 characters
- the whole values were getting dropped upon transforming to JSON.
TL;DR: double-check all encodings.
Yeah, I too have had real problems with this - It isn't limited to Latin1 either - but in my case I found one invalid character that
doesn't conform to the specified encoding would drop the entire string in any encoding. This was hard to trace because 99
conversions would succeed with all kinds of weird and wonderful encoded characters, and then one suddenly returns an empty string.
I eventually traced this down to the OS's C API for encoding conversions which seems to simply drop the entire value and not even
cause an error - just returns empty string. Peculiar behaviour if you ask me (though that is probably meant to prompt an error from
the high-level code's design, but nobody told me...). This seems the case for both Windows and OSX, I can't speak for Linux,
Android, etc. maybe someone else knows?
Anyway, adding a conversion check is the key, but the point is moot if you rely on PHP or such to do the conversions for you.
Scripts don't have GUIs... they cant really do much about it. Maybe an error log somewhere contains some info?
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
