On Aug 23, 2019, at 9:05 AM, Paul Dupuis via use-livecode
<use-livecode@lists.runrev.com> wrote:
On 8/23/2019 10:30 AM, Dar Scott Consulting via use-livecode wrote:
SHELL()
Related: https://quality.livecode.com/show_bug.cgi?id=22334
<https://quality.livecode.com/show_bug.cgi?id=22334>
I was on the fence concerning whether shell() returns a text string or binary data. I have fallen to one side: BINARY.
I concur with the general idea of returning a 8-bit byte stream from Shell
(i.e. binary)
Here is a summary of my position. Tomatoes are welcome.
The shell() function is on the edge of LiveCode. All I/O on the edge is binary
and is either used that way or is text decoded. The text decoding can be part
of the I/O operation or explicit. I see shell() this way. LiveCode has no
control over what command line tools will do, or what the executable in
shellCommand will do.
Some I/O operations have optional text decoding built-in. This can be added to
shell(), but I don't think it is needed.
LEGACY: For my part, I have used shell() with an underlying assumption of
ASCII. This assumption works OK with the ASCII subset of Latin-1, UTF-8 and
most Windows codepages. I think most uses of shell() make this assumption plus
the assumption that a text decoding is not needed. For this to be preserved,
the text interpretation of a binary data should continue to be a sequence of
single byte characters from an ASCII superset, and the default values for
shellCommand should ensure that for each OS.
Well this is why I advocate that the encoding should be UTF-8 rather than UTF-16. (1) this
matches shell() on OSX which returns results from the shell command UTF-8 encoded, or would you
suggest changing OSX?; (2) UTF8 encoding means that people expecting ASCII can just work with
the returned results as they always have. People expecting Unicode, will need to perform a
textDecode(shell(<command>),"UTF-8") and if the results were just ASCII, they
they are still fine, since UTF8 decoding ASCII = ASCII.
My recommendation:
shell(x) is strictly binary data
A textDecode example is provided in dict for shell.
shellCommand is reference in dict for shell.
The "cmd.exe /u" example is provided in dict for shellCommand
Correct the default for shellCommand in dictionary.
My recommendation:
Fix Windows shell() to UTF8 encode results
Add a textDecode example to the Dictionary entry for BOTH Windows and OSX (and
Linux, where I have no idea what Linux does)
Correct the CMD.EXE vs Command.exe reference in the Dictionary as well
Dar Scott
Mad Scientist
Paul Dupuis
Mad Wizard
:-)
On Aug 22, 2019, at 9:37 AM, Paul Dupuis via use-livecode
<use-livecode@lists.runrev.com> wrote:
I have filed two bug reports that are in LC905rc1 and go back to 7.0 where LC
functions that should deal with Unicode properly do not.
These are:
https://quality.livecode.com/show_bug.cgi?id=22213 -- The "detailed files"
function fails for any files with Unicode in the name, returning the filename with %3F
(?) instead of the Unicode characters properly URL encoded (they should be UFT8 encoded
and then URL encoded)
and
https://quality.livecode.com/show_bug.cgi?id=22334 -- the shell command is not
Unicode aware in returning it's results. On OSX, the results are UTF8 encoded
(discovered by accident) and so an extra step is needed to text decode them,
but on Windows it is a complete failure and any Unicode results of the command
line - SHOWN 100% correctly is executed in the command line - are returned
NATIVE encoded, causing all Unicode characters to become question marks.
I have written work-around for both of these bugs that can be found in the bug reports.
My work-around for the "details files" is slow, due to repeated calls to shell
to fetch file properties one at a time.
If anyone else out there has run into these bugs in your own code and developed a faster
work-around for the "detailed files" and would care to share, I would welcome a
faster fix.
Of course, I'd welcome a fix from LiveCode, Ltd. to these bugs even more!
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
