[Issue 15845] Windows console cannot read properly UTF-8 lines

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15845

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P1  |P3

--

[Issue 15845] Windows console cannot read properly UTF-8 lines

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15845

Martin Krejcirik  changed:

   What|Removed |Added

 CC||m...@krej.cz
   See Also||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=1448,
   ||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=15761

--

[Issue 15845] Windows console cannot read properly UTF-8 lines

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15845

--- Comment #4 from ag0ae...@gmail.com ---
(In reply to ag0aep6g from comment #2)
> The issue does not manifest with a pipe: `echo ä | test` works.

It seems to be a problem with input from a TTY. Normal stdin is a TTY, but when
a pipe is used it's not. According to _isatty [1], stdin is also not a TTY when
I run things in the bash from Git for Windows. And indeed: no UTF-8 problems
there.


[1] https://msdn.microsoft.com/en-us/library/f4s0ddew.aspx

--

[Issue 15845] Windows console cannot read properly UTF-8 lines

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15845

--- Comment #3 from ag0ae...@gmail.com ---
(In reply to ag0aep6g from comment #2)
> For -m32 (DIGITAL_MARS_STDIO) it seems to come down to this (with `chcp
> 65001` in the console):
[...]
> That is, Digital Mars's FGETC (_fgetc_nlock) returns -1 for non-ASCII
> characters.
> 
> The issue does not manifest with a pipe: `echo ä | test` works.

The same happens with -m64, and with a simple C++ program (just `printf("%d\n",
fgetc(stdin));`). So apparently `chcp 65001` is not enough to make UTF-8 input
work from the console. I'm not much of a Windows programmer, though, so I have
no idea what's missing.

--

[Issue 15845] Windows console cannot read properly UTF-8 lines

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15845

ag0ae...@gmail.com changed:

   What|Removed |Added

 CC||ag0ae...@gmail.com

--- Comment #2 from ag0ae...@gmail.com ---
For -m32 (DIGITAL_MARS_STDIO) it seems to come down to this (with `chcp 65001`
in the console):


import std.stdio;

void main()
{
FILE* fps = core.stdc.stdio.stdin;
FLOCK(fps);
scope(exit) FUNLOCK(fps);

auto fp = cast(_iobuf*)fps;
assert(!(__fhnd_info[fp._file] & FHND_WCHAR)); /* passes; no wide
characters */
assert(!(fp._flag & _IOTRAN)); /* passes; no translated mode */

int c = FGETC(fp);
assert(c != -1); /* passes with 'a'; fails with 'ä' */
}


That is, Digital Mars's FGETC (_fgetc_nlock) returns -1 for non-ASCII
characters.

The issue does not manifest with a pipe: `echo ä | test` works.

--

[Issue 15845] Windows console cannot read properly UTF-8 lines

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15845

--- Comment #1 from Steven Schveighoffer  ---
*** Issue 15846 has been marked as a duplicate of this issue. ***

--