Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Mon, 27 Oct 2014 17:36:55, "Eric Blake (cygwin)" wrote: > A new release of dash, 0.5.8-3, has been uploaded and will soon reach a > mirror near you; replacing 0.5.8-2, and leaving the previous version at > 0.5.7-1 on 32-bit and 0.5.7-4 on 64-bit. A note to let you know ... The dash interpreter has a minor problem with multi-byte characters ... .. at least on Cygwin, NOT on Fedora 24. When the omega symbol is entered, followed by a backspace, the input buffer is NOT correctly cleared (or so it appears to me). Executed in Windows Console: (but similar behaviour when executed using mintty) 64-@@ uname -a CYGWIN_NT-6.1 Seven 2.7.0(0.306/5/3) x86_64 Cygwin (but same behaviour on x86) 64-@@ dash # latest package (0.5.8-3) # enter alt-234 (omega symbol), followed by a backspace $ dash: 1: â: not found # enter alt-234 $ echo Ω | xxd : cea9 0a ... # enter alt-234, followed by a backspace $ echo | xxd : ce0a .. NOTE: this behaviour already existed BEFORE Corinna made her modification to cygwin1.dll in order to "please" Steven P. Regards, Henri = -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Tue, 24 Jan 2017 16:58:39, Houder wrote: > When the omega symbol is entered, followed by a backspace, the input buffer > is NOT correctly cleared (or so it appears to me). Working fine here. Tested with: $ cygcheck -sv | awk '$1~/^(dash|cygwin|"cygwin1.dll")$/' "cygwin1.dll" v0.0 ts=2016-12-16 04:55 cygwin 2.6.1-1OK dash0.5.8-3OK and: $ cygcheck -sv | awk '$1~/^(dash|cygwin|"cygwin1.dll")$/' "cygwin1.dll" v0.0 ts=2017-01-19 15:16 cygwin 2.6.1-1OK dash0.5.8-3OK > NOTE: this behaviour already existed BEFORE Corinna made her > modification to cygwin1.dll in order to "please" Steven P. You may not understand the importance of UTF-8, but we do. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Tue, 24 Jan 2017 17:28:13, Steven Penny wrote: > On Tue, 24 Jan 2017 16:58:39, Houder wrote: > > When the omega symbol is entered, followed by a backspace, the input buffer > > is NOT correctly cleared (or so it appears to me). > > Working fine here. Tested with: Peculiar ... must be "Henri" problem then :-P Installed Cygwin afresh ... (Erik, using the official tool! - setup-x86_64.exe) - I get the same result (for both the current cygwin1.dll and the modified one) - enter alt-234, followed by a backspace, followed by a linefeed, results in dash complaining - enter alt-234, followed by TWO backspaces (which eats part of the prompt), followed by a linefeed, leaves dash at peace ... Regards, Henri = Test1: ... using a Windows shortcut to start dash with option -l: $ uname -a CYGWIN_NT-6.1 Seven 2.6.1(0.305/5/3) 2016-12-16 11:55 x86_64 Cygwin $ locale LANG=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_ALL= $ ls -l /bin/cygwin1* -rwxr-xr-x 1 Henri None 3316345 Dec 16 11:57 /bin/cygwin1.dll -rwxr-xr-x 1 Henri None 3316345 Dec 16 11:57 /bin/cygwin1-2.6.1-1.X -rwxr-xr-x 1 Henri None 3318753 Jan 19 20:01 /bin/cygwin1-64-20170119.X -rwxr-xr-x 1 Henri None 3319106 Jan 25 13:32 /bin/cygwin1-64-20170119-2nd.X $ cygcheck -sv | awk '$1~/^(dash|cygwin|"cygwin1.dll")$/' "cygwin1.dll" v0.0 ts=2016-12-16 10:55 cygwin 2.6.1-1OK dash 0.5.8-3OK # enter alt-234 (omega symbol), followed by a backspace $ /usr/bin/dash: 5: â: not found # enter alt-234 (omega symbol), followed by TWO backspaces $ # enter alt-234 (omega symbol), followed by a backspace $ echo | od -Ax -tx1z 00 ce 0a>..< 02 $ Test2: ... using a Windows shortcut to start dash with option -l: $ uname -a CYGWIN_NT-6.1 Seven 2.7.0(0.306/5/3) x86_64 Cygwin $ locale [snip] # same as above ... $ ls -l /bin/cygwin1* -rwxr-xr-x 1 Henri None 3319106 Jan 25 13:32 /bin/cygwin1.dll -rwxr-xr-x 1 Henri None 3316345 Dec 16 11:57 /bin/cygwin1-2.6.1-1.X -rwxr-xr-x 1 Henri None 3318753 Jan 19 20:01 /bin/cygwin1-64-20170119.X -rwxr-xr-x 1 Henri None 3319106 Jan 25 13:32 /bin/cygwin1-64-20170119-2nd.X $ cygcheck -sv | awk '$1~/^(dash|cygwin|"cygwin1.dll")$/' "cygwin1.dll" v0.0 ts=2017-01-19 21:16 cygwin 2.6.1-1OK dash 0.5.8-3OK # enter alt-234 (omega symbol), followed by a backspace $ /usr/bin/dash: 5: â: not found # enter alt-234 (omega symbol), followed by TWO backspaces $ # enter alt-234 (omega symbol), followed by a backspace $ echo | od -Ax -tx1z 00 ce 0a>..< 02 $ = -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On 1/25/2017 8:37 AM, Houder wrote: > On Tue, 24 Jan 2017 17:28:13, Steven Penny wrote: >> On Tue, 24 Jan 2017 16:58:39, Houder wrote: >>> When the omega symbol is entered, followed by a backspace, the input buffer >>> is NOT correctly cleared (or so it appears to me). >> >> Working fine here. Tested with: > > Peculiar ... must be "Henri" problem then :-P > > Installed Cygwin afresh ... (Erik, using the official tool! - > setup-x86_64.exe) > > - I get the same result (for both the current cygwin1.dll and the modified > one) > - enter alt-234, >followed by a backspace, followed by a linefeed, results in dash > complaining > - enter alt-234, >followed by TWO backspaces (which eats part of the prompt), followed by >a linefeed, leaves dash at peace ... > Could your issues be relevant to differing stty settings? -- cyg Simple -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Wed, 25 Jan 2017 14:37:00, Houder wrote: > - I get the same result (for both the current cygwin1.dll and the modified > one) > - enter alt-234, >followed by a backspace, followed by a linefeed Ok, I can dup this. In your previous email you did not mention the final newline. Here is my similar test: 1. echo 2. Alt 234 3. Backspace Dash result: $ echo | od -tcx1 000 316 \n ce 0a Bash result: $ echo | od -tcx1 000 \n 0a Obviously Bash is not the problem, nor readline as Dash doesnt use readline. So it appears the issue this time is again with cygwin1.dll, or perhaps the Dash package. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Wed, 25 Jan 2017 16:14:00, Steven Penny wrote: > Obviously Bash is not the problem, nor readline as Dash doesnt use readline. > So > it appears the issue this time is again with cygwin1.dll, or perhaps the Dash > package. .. uhm, it appears to me that Windows is the issue here. As those in the know do not feel inclined to respond, I will provide some guesses that are my own: - in terms of input buffer management, utf-8 encoded characters will not be recognized in case of bash and dash ... (they are under Fedora) - see the output of stty -a: iutf8 is not present (it is under Fedora) - readline provides bash with input buffer management for utf-8 encoded characters on Windows (that is why it 'works' in case of bash) - bash has support for utf-8 encoded characters ... (e.g. ls -l ? will include one-character filenames in case the name is made up of only one multi-byte character) - dash has no such support ... [1][2] Consequently, dash is only partly useful, even more so on Windows (as it would require an additional "helper" on Windows in order to obtain proper line-editing). Helper? readline, libedit ... However, I am only guessing ... (only Erik and Corinna can provide expert details here). Regards, Henri 1. https://en.wikipedia.org/wiki/Almquist_shell see Debian Almquist Shell 2. http://www.mail-archive.com/dash@vger.kernel.org/msg01072.html reply by Hertbert Xu, project owner of dash = -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Wed, 25 Jan 2017 15:31:09, cyg Simple wrote: > On 1/25/2017 8:37 AM, Houder wrote: > > On Tue, 24 Jan 2017 17:28:13, Steven Penny wrote: > >> On Tue, 24 Jan 2017 16:58:39, Houder wrote: > >>> When the omega symbol is entered, followed by a backspace, the input > >>> buffer > >>> is NOT correctly cleared (or so it appears to me). > >> > >> Working fine here. Tested with: > > > > Peculiar ... must be "Henri" problem then :-P > > > > Installed Cygwin afresh ... (Erik, using the official tool! - > > setup-x86_64.exe) > > > > - I get the same result (for both the current cygwin1.dll and the modified > > one) > > - enter alt-234, > >followed by a backspace, followed by a linefeed, results in dash > > complaining > > - enter alt-234, > >followed by TWO backspaces (which eats part of the prompt), followed by > >a linefeed, leaves dash at peace ... > > > > Could your issues be relevant to differing stty settings? Yes, that would have been a possibility ... So, I investigated. I also compared the stty settings with the one I got using Fedora 24. That made me aware of the 'iutf8' setting (on Linux). This setting I could not switch on using Cygwin/Windows. Regards, Henri = -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Jan 28 14:44, Houder wrote: > On Wed, 25 Jan 2017 16:14:00, Steven Penny wrote: > > Obviously Bash is not the problem, nor readline as Dash doesnt use > > readline. So > > it appears the issue this time is again with cygwin1.dll, or perhaps the > > Dash > > package. > > .. uhm, it appears to me that Windows is the issue here. > > As those in the know do not feel inclined to respond, I will provide some > guesses that are my own: > > - in terms of input buffer management, utf-8 encoded characters will not >be recognized in case of bash and dash ... (they are under Fedora) > - see the output of stty -a: iutf8 is not present (it is under Fedora) > - readline provides bash with input buffer management for utf-8 encoded >characters on Windows (that is why it 'works' in case of bash) > - bash has support for utf-8 encoded characters ... >(e.g. ls -l ? will include one-character filenames in case the name is > made up of only one multi-byte character) > - dash has no such support ... [1][2] > > Consequently, dash is only partly useful, even more so on Windows (as it > would require an additional "helper" on Windows in order to obtain proper > line-editing). Helper? readline, libedit ... > > However, I am only guessing ... (only Erik and Corinna can provide expert > details here). I'm not quite sure yet but apparently the problem is in the handling of VERASE in the termios implementation. In cooked mode it fills a char buffer with what has been typed. The code doesn't know if the bytes in the buffer are UTF-8 chars or just random bytes. So VERASE erases exactly one byte, which means, in case of UTF-8 chars it only erases the last byte of of a mulitbyte character. It seems the Linux termios implementation is different in that it still knows which bytes constitute a single keypress and thus knows how much byte it has to erase. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Jan 31 11:04, Corinna Vinschen wrote: > On Jan 28 14:44, Houder wrote: > > On Wed, 25 Jan 2017 16:14:00, Steven Penny wrote: > > > Obviously Bash is not the problem, nor readline as Dash doesnt use > > > readline. So > > > it appears the issue this time is again with cygwin1.dll, or perhaps the > > > Dash > > > package. > > > > .. uhm, it appears to me that Windows is the issue here. > > > > As those in the know do not feel inclined to respond, I will provide some > > guesses that are my own: > > > > - in terms of input buffer management, utf-8 encoded characters will not > >be recognized in case of bash and dash ... (they are under Fedora) > > - see the output of stty -a: iutf8 is not present (it is under Fedora) > > - readline provides bash with input buffer management for utf-8 encoded > >characters on Windows (that is why it 'works' in case of bash) > > - bash has support for utf-8 encoded characters ... > >(e.g. ls -l ? will include one-character filenames in case the name is > > made up of only one multi-byte character) > > - dash has no such support ... [1][2] > > > > Consequently, dash is only partly useful, even more so on Windows (as it > > would require an additional "helper" on Windows in order to obtain proper > > line-editing). Helper? readline, libedit ... > > > > However, I am only guessing ... (only Erik and Corinna can provide expert > > details here). > > I'm not quite sure yet but apparently the problem is in the handling of > VERASE in the termios implementation. In cooked mode it fills a char > buffer with what has been typed. The code doesn't know if the bytes in > the buffer are UTF-8 chars or just random bytes. So VERASE erases > exactly one byte, which means, in case of UTF-8 chars it only erases the > last byte of of a mulitbyte character. > > It seems the Linux termios implementation is different in that it > still knows which bytes constitute a single keypress and thus knows > how much byte it has to erase. Ok, here's what happens on Linux: The termios code support a flag IUTF8. This flag determines if the termios code checks for UTF8 characters in the input when performing an ERASE. It checks if the IUTF8 flag is set and if so, it checks in a loop if the just erased byte is a UTF-8 continuation character. If so, it erases another byte. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Tue, 31 Jan 2017 14:16:16, Corinna Vinschen wrote: [snip] > > I'm not quite sure yet but apparently the problem is in the handling of > > VERASE in the termios implementation. In cooked mode it fills a char > > buffer with what has been typed. The code doesn't know if the bytes in > > the buffer are UTF-8 chars or just random bytes. So VERASE erases > > exactly one byte, which means, in case of UTF-8 chars it only erases the > > last byte of of a mulitbyte character. > >=20 > > It seems the Linux termios implementation is different in that it > > still knows which bytes constitute a single keypress and thus knows > > how much byte it has to erase. > > Ok, here's what happens on Linux: The termios code support a flag > IUTF8. This flag determines if the termios code checks for UTF8 > characters in the input when performing an ERASE. It checks if the > IUTF8 flag is set and if so, it checks in a loop if the just erased byte > is a UTF-8 continuation character. If so, it erases another byte. (Thank you for responding -- and your effort thus far). Agreed. One byte or more, depending on the "character" ... (which is not a problem in case of UTF-8 encoding -- continuation bit). Of course, the terminal driver must receive the characters encoded in UTF-8. Therefore the question is: 'can the same situation be created under under Windows?' (does Windows provide the required support?) A second question is: 'is it worth the effort?' Given that bash can recognize UTF-8 (using readline) ... .. and the fact that dash does not support multi-byte characters. Regards, Henri = -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Jan 31 16:01, Houder wrote: > On Tue, 31 Jan 2017 14:16:16, Corinna Vinschen wrote: > > [snip] > > > > I'm not quite sure yet but apparently the problem is in the handling of > > > VERASE in the termios implementation. In cooked mode it fills a char > > > buffer with what has been typed. The code doesn't know if the bytes in > > > the buffer are UTF-8 chars or just random bytes. So VERASE erases > > > exactly one byte, which means, in case of UTF-8 chars it only erases the > > > last byte of of a mulitbyte character. > > >=20 > > > It seems the Linux termios implementation is different in that it > > > still knows which bytes constitute a single keypress and thus knows > > > how much byte it has to erase. > > > > Ok, here's what happens on Linux: The termios code support a flag > > IUTF8. This flag determines if the termios code checks for UTF8 > > characters in the input when performing an ERASE. It checks if the > > IUTF8 flag is set and if so, it checks in a loop if the just erased byte > > is a UTF-8 continuation character. If so, it erases another byte. > > (Thank you for responding -- and your effort thus far). > > Agreed. One byte or more, depending on the "character" ... (which is > not a problem in case of UTF-8 encoding -- continuation bit). > > Of course, the terminal driver must receive the characters encoded in > UTF-8. > > Therefore the question is: 'can the same situation be created under > under Windows?' (does Windows provide the required support?) This has nothing to do with Windows. It's the termios implementation inside Cygwin. I created a patch introducing the IUTF8 flag as on Linux as well as a code snippet trying to remove entire utf-8 characters from the input if the IUTF8 flag is set. And it's set now by default since we default to UTF-8 anyway. Thomas, you may want to check for the IUTF8 flag in upcoming mintty versions and unset it if character set configured in the mintty options dialog is != UTF-8. I uploaded new developer snapshots to https://cygwin.com/snapshots/ with this patch. I'm also going to release a Cygwin test version later today. HTH, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On 01/31/2017 09:32 AM, Corinna Vinschen wrote: > This has nothing to do with Windows. It's the termios implementation > inside Cygwin. I created a patch introducing the IUTF8 flag as on Linux > as well as a code snippet trying to remove entire utf-8 characters from > the input if the IUTF8 flag is set. And it's set now by default since > we default to UTF-8 anyway. > > Thomas, you may want to check for the IUTF8 flag in upcoming mintty > versions and unset it if character set configured in the mintty options > dialog is != UTF-8. > > I uploaded new developer snapshots to https://cygwin.com/snapshots/ > with this patch. I'm also going to release a Cygwin test version > later today. And I will be providing a test build of coreutils (for stty) that exposes IUTF8 to the command line (to be promoted to current once the cygwin release is). dash and readline do NOT need to be rebuilt to take advantage of it (since it is the terminal, not the shell, that is handling input), although I need to rebuild readline soon because of some upstream patches that need to be folded in. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Tue, 31 Jan 2017 16:32:45, Corinna Vinschen wrote: [snip] > > Therefore the question is: 'can the same situation be created under > > under Windows?' (does Windows provide the required support?) > > This has nothing to do with Windows. It's the termios implementation > inside Cygwin. I created a patch introducing the IUTF8 flag as on Linux > as well as a code snippet trying to remove entire utf-8 characters from > the input if the IUTF8 flag is set. And it's set now by default since > we default to UTF-8 anyway. Downloaded the snapshots (both x86 and x86_64). Replaced cygwin1.dll. Repeated my test for both - bash --noediting, and - dash on both snapshots. Now I get the expected result! (UTF-8 characters are being erased) Thank you. Henri. = -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Tue, 31 Jan 2017 09:42:19, Eric Blake wrote: > And I will be providing a test build of coreutils (for stty) that > exposes IUTF8 to the command line (to be promoted to current once the > cygwin release is). dash and readline do NOT need to be rebuilt to take > advantage of it (since it is the terminal, not the shell, that is > handling input), although I need to rebuild readline soon because of > some upstream patches that need to be folded in. Eric, my apologies for mispelling your name (twice!). Regards, Henri = -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Jan 31 18:54, Houder wrote: > On Tue, 31 Jan 2017 16:32:45, Corinna Vinschen wrote: > > [snip] > > > > Therefore the question is: 'can the same situation be created under > > > under Windows?' (does Windows provide the required support?) > > > > This has nothing to do with Windows. It's the termios implementation > > inside Cygwin. I created a patch introducing the IUTF8 flag as on Linux > > as well as a code snippet trying to remove entire utf-8 characters from > > the input if the IUTF8 flag is set. And it's set now by default since > > we default to UTF-8 anyway. > > Downloaded the snapshots (both x86 and x86_64). Replaced cygwin1.dll. > > Repeated my test for both > > - bash --noediting, and > - dash > > on both snapshots. > > Now I get the expected result! (UTF-8 characters are being erased) > > Thank you. > > Henri. Thank you for testing. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
Am 31.01.2017 um 16:32 schrieb Corinna Vinschen: On Jan 31 16:01, Houder wrote: On Tue, 31 Jan 2017 14:16:16, Corinna Vinschen wrote: [snip] I'm not quite sure yet but apparently the problem is in the handling of VERASE in the termios implementation. In cooked mode it fills a char buffer with what has been typed. The code doesn't know if the bytes in the buffer are UTF-8 chars or just random bytes. So VERASE erases exactly one byte, which means, in case of UTF-8 chars it only erases the last byte of of a mulitbyte character. ... Ok, here's what happens on Linux: The termios code support a flag IUTF8. This flag determines if the termios code checks for UTF8 characters in the input when performing an ERASE. It checks if the IUTF8 flag is set and if so, it checks in a loop if the just erased byte is a UTF-8 continuation character. If so, it erases another byte. Agreed. One byte or more, depending on the "character" ... (which is not a problem in case of UTF-8 encoding -- continuation bit). Of course, the terminal driver must receive the characters encoded in UTF-8. ... ... It's the termios implementation inside Cygwin. I created a patch introducing the IUTF8 flag as on Linux as well as a code snippet trying to remove entire utf-8 characters from the input if the IUTF8 flag is set. And it's set now by default since we default to UTF-8 anyway. Thomas, you may want to check for the IUTF8 flag in upcoming mintty versions and unset it if character set configured in the mintty options dialog is != UTF-8. So the flag is always set initially? Also on Linux? Does it (on Linux) also have an effect for non-UTF-8 multibyte encodings? And cannot the Cygwin DLL set the flag to match the locale setting when it was invoked? I can (and will if appropriate) handle the flag in mintty as needed, but what if someone calls LC_ALL=.other_encoding dash later within the terminal session? I guess the more consistent solution would be to handle this in the cygwin DLL. -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Feb 13 23:03, Thomas Wolff wrote: > Am 31.01.2017 um 16:32 schrieb Corinna Vinschen: > > On Jan 31 16:01, Houder wrote: > > > On Tue, 31 Jan 2017 14:16:16, Corinna Vinschen wrote: > > > [snip] > > > > Ok, here's what happens on Linux: The termios code support a flag > > > > IUTF8. This flag determines if the termios code checks for UTF8 > > > > characters in the input when performing an ERASE. It checks if the > > > > IUTF8 flag is set and if so, it checks in a loop if the just erased byte > > > > is a UTF-8 continuation character. If so, it erases another byte. > > > Agreed. One byte or more, depending on the "character" ... (which is > > > not a problem in case of UTF-8 encoding -- continuation bit). > > > > > > Of course, the terminal driver must receive the characters encoded in > > > UTF-8. > > > > > > ... > > ... It's the termios implementation > > inside Cygwin. I created a patch introducing the IUTF8 flag as on Linux > > as well as a code snippet trying to remove entire utf-8 characters from > > the input if the IUTF8 flag is set. And it's set now by default since > > we default to UTF-8 anyway. > > > > Thomas, you may want to check for the IUTF8 flag in upcoming mintty > > versions and unset it if character set configured in the mintty options > > dialog is != UTF-8. > So the flag is always set initially? Also on Linux? Does it (on Linux) also > have an effect for non-UTF-8 multibyte encodings? Yes, yes, and yes. > And cannot the Cygwin DLL set the flag to match the locale setting when it > was invoked? > > I can (and will if appropriate) handle the flag in mintty as needed, but > what if someone calls LC_ALL=.other_encoding dash later within the terminal > session? I guess the more consistent solution would be to handle this in the No. We're talking about a function in the master side of the tty, while the applications started in the terminal are on the slave side. iutf8 is set in Linux by default and by most terminal applications ionly reset if the LC_CTYPE setting in the environment of the terminal application is not set to the utf8 codeset. This is determined at terminal startup, not by the inferior processes runnin in the terminal. The applications still can set iutf8 via termios control (or stty(1)). For mintty I just thought it might be helpful to honor the character set setting in its options and to default to iutf8 if it's not set. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
Am 14.02.2017 um 09:45 schrieb Corinna Vinschen: On Feb 13 23:03, Thomas Wolff wrote: Am 31.01.2017 um 16:32 schrieb Corinna Vinschen: On Jan 31 16:01, Houder wrote: On Tue, 31 Jan 2017 14:16:16, Corinna Vinschen wrote: [snip] Ok, here's what happens on Linux: The termios code support a flag IUTF8. This flag determines if the termios code checks for UTF8 characters in the input when performing an ERASE. It checks if the IUTF8 flag is set and if so, it checks in a loop if the just erased byte is a UTF-8 continuation character. If so, it erases another byte. Agreed. One byte or more, depending on the "character" ... (which is not a problem in case of UTF-8 encoding -- continuation bit). Of course, the terminal driver must receive the characters encoded in UTF-8. ... ... It's the termios implementation inside Cygwin. I created a patch introducing the IUTF8 flag as on Linux as well as a code snippet trying to remove entire utf-8 characters from the input if the IUTF8 flag is set. And it's set now by default since we default to UTF-8 anyway. Thomas, you may want to check for the IUTF8 flag in upcoming mintty versions and unset it if character set configured in the mintty options dialog is != UTF-8. So the flag is always set initially? Also on Linux? Does it (on Linux) also have an effect for non-UTF-8 multibyte encodings? Yes, yes, and yes. And cannot the Cygwin DLL set the flag to match the locale setting when it was invoked? I can (and will if appropriate) handle the flag in mintty as needed, but what if someone calls LC_ALL=.other_encoding dash later within the terminal session? I guess the more consistent solution would be to handle this in the No. We're talking about a function in the master side of the tty, while the applications started in the terminal are on the slave side. I am not familiar with the concept of setting termios properties on either the master or slave side of a pty. I've only ever set them in the client application, including my tests about IUTF8 which worked. Would setting on the master side imply it's set for the clients implicitly, and can it be changed later, e.g. when mintty character encoding is being changed from the Options dialog? And you say the function of erasing characters on BS is in the master side? To be honest, this confuses me. I thought it's a client function, like readline() would perform if used (apparently not by dash), which is kind of an enhanced version of the tty cooked mode and used to work even without the new flag, right? iutf8 is set in Linux by default and by most terminal applications ionly reset if the LC_CTYPE setting in the environment of the terminal application is not set to the utf8 codeset. This is determined at terminal startup, not by the inferior processes runnin in the terminal. The applications still can set iutf8 via termios control (or stty(1)). Will you patch stty as well to address the new flag? For mintty I just thought it might be helpful to honor the character set setting in its options and to default to iutf8 if it's not set. Sure, but it would be better to find a solution that implicitly works in all terminals. Isn't it possible to handle this in forkpty()/openpty()? -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On 02/14/2017 01:40 PM, Thomas Wolff wrote: >> No. We're talking about a function in the master side of the tty, while >> the applications started in the terminal are on the slave side. > I am not familiar with the concept of setting termios properties on > either the master or slave side of a pty. I've only ever set them in the > client application, including my tests about IUTF8 which worked. Would > setting on the master side imply it's set for the clients implicitly, > and can it be changed later, e.g. when mintty character encoding is > being changed from the Options dialog? > And you say the function of erasing characters on BS is in the master > side? To be honest, this confuses me. I thought it's a client function, > like readline() would perform if used (apparently not by dash), which is > kind of an enhanced version of the tty cooked mode and used to work even > without the new flag, right? The readline source code does not mention IUTF8; and neither bash nor dash need to reference it, because if the tty handling code sets it correctly for what the terminal is going to display, then the clients that are read()ing from the tty never even see BS in cooked mode (the master side of the terminal handles BS before the read() completes in the slave, if I'm understanding it correctly). >> iutf8 is set in Linux by default and by most terminal applications ionly >> reset if the LC_CTYPE setting in the environment of the terminal >> application is not set to the utf8 codeset. This is determined at >> terminal startup, not by the inferior processes runnin in the terminal. >> The applications still can set iutf8 via termios control (or stty(1)). > Will you patch stty as well to address the new flag? Already patched; coreutils-8.26-2 was promoted to current yesterday. > >> For mintty I just thought it might be helpful to honor the character set >> setting in its options and to default to iutf8 if it's not set. > Sure, but it would be better to find a solution that implicitly works in > all terminals. Isn't it possible to handle this in forkpty()/openpty()? Does forkpty()/openpty() currently pay attention to environment variables to even know what encoding is currently in use? And what's to say that the environment used to fork the master side will match the locale settings of the slave process that connects to the pty, so how do we know whether to default IUTF8 on or off based solely on the slave's environment, when it is the master that is handling BS and therefore the master's character encoding that matters for how much BS should erase ? -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
Am 14.02.2017 um 20:56 schrieb Eric Blake: On 02/14/2017 01:40 PM, Thomas Wolff wrote: No. We're talking about a function in the master side of the tty, while the applications started in the terminal are on the slave side. I am not familiar with the concept of setting termios properties on either the master or slave side of a pty. I've only ever set them in the client application, including my tests about IUTF8 which worked. Would setting on the master side imply it's set for the clients implicitly, and can it be changed later, e.g. when mintty character encoding is being changed from the Options dialog? And you say the function of erasing characters on BS is in the master side? To be honest, this confuses me. I thought it's a client function, like readline() would perform if used (apparently not by dash), which is kind of an enhanced version of the tty cooked mode and used to work even without the new flag, right? The readline source code does not mention IUTF8; and neither bash nor dash need to reference it, because if the tty handling code sets it correctly for what the terminal is going to display, then the clients that are read()ing from the tty never even see BS in cooked mode (the master side of the terminal handles BS before the read() completes in the slave, if I'm understanding it correctly). This does not comply with my (limited) understanding of pty stuff. In mintty, forkpty will create a master/slave pty; mintty feeds it on the master side, while the client program (usually a shell) reads from the slave side. Mintty never handles BS for input, it simply feeds it into the pty. "Line disciplines" like cooked mode must be handled on the slave side. iutf8 is set in Linux by default and by most terminal applications ionly reset if the LC_CTYPE setting in the environment of the terminal application is not set to the utf8 codeset. This is determined at terminal startup, not by the inferior processes runnin in the terminal. The applications still can set iutf8 via termios control (or stty(1)). Will you patch stty as well to address the new flag? Already patched; coreutils-8.26-2 was promoted to current yesterday. For mintty I just thought it might be helpful to honor the character set setting in its options and to default to iutf8 if it's not set. Sure, but it would be better to find a solution that implicitly works in all terminals. Isn't it possible to handle this in forkpty()/openpty()? Does forkpty()/openpty() currently pay attention to environment variables to even know what encoding is currently in use? Don't know, but maybe it simply should for this purpose, for a widely-useful solution. And what's to say that the environment used to fork the master side will match the locale settings of the slave process that connects to the pty, so how do we know whether to default IUTF8 on or off based solely on the slave's environment, when it is the master that is handling BS and therefore the master's character encoding that matters for how much BS should erase ? See above; the master isn't handling BS. But there should be no such inconsistency in the case of mintty because master and slave are forked from common initial code. I think this consideration is only relevant for reattaching programs like screen. -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
Am 14.02.2017 um 21:29 schrieb Thomas Wolff: Am 14.02.2017 um 20:56 schrieb Eric Blake: On 02/14/2017 01:40 PM, Thomas Wolff wrote: No. We're talking about a function in the master side of the tty, while the applications started in the terminal are on the slave side. I am not familiar with the concept of setting termios properties on either the master or slave side of a pty. I've only ever set them in the client application, including my tests about IUTF8 which worked. Would setting on the master side imply it's set for the clients implicitly, and can it be changed later, e.g. when mintty character encoding is being changed from the Options dialog? And you say the function of erasing characters on BS is in the master side? To be honest, this confuses me. I thought it's a client function, like readline() would perform if used (apparently not by dash), which is kind of an enhanced version of the tty cooked mode and used to work even without the new flag, right? The readline source code does not mention IUTF8; and neither bash nor dash need to reference it, because if the tty handling code sets it correctly for what the terminal is going to display, then the clients that are read()ing from the tty never even see BS in cooked mode (the master side of the terminal handles BS before the read() completes in the slave, if I'm understanding it correctly). This does not comply with my (limited) understanding of pty stuff. In mintty, forkpty will create a master/slave pty; mintty feeds it on the master side, while the client program (usually a shell) reads from the slave side. Mintty never handles BS for input, it simply feeds it into the pty. "Line disciplines" like cooked mode must be handled on the slave side. Also, I've tried both options in mintty. Setting the flag on the master side has weird effects, initially blocking the terminal process. Setting it on the slave side works fine. I've uploaded that to the git repository. -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
Am 14.02.2017 um 21:35 schrieb Thomas Wolff: Am 14.02.2017 um 21:29 schrieb Thomas Wolff: Am 14.02.2017 um 20:56 schrieb Eric Blake: On 02/14/2017 01:40 PM, Thomas Wolff wrote: No. We're talking about a function in the master side of the tty, while the applications started in the terminal are on the slave side. I am not familiar with the concept of setting termios properties on either the master or slave side of a pty. I've only ever set them in the client application, including my tests about IUTF8 which worked. Would setting on the master side imply it's set for the clients implicitly, and can it be changed later, e.g. when mintty character encoding is being changed from the Options dialog? And you say the function of erasing characters on BS is in the master side? To be honest, this confuses me. I thought it's a client function, like readline() would perform if used (apparently not by dash), which is kind of an enhanced version of the tty cooked mode and used to work even without the new flag, right? The readline source code does not mention IUTF8; and neither bash nor dash need to reference it, because if the tty handling code sets it correctly for what the terminal is going to display, then the clients that are read()ing from the tty never even see BS in cooked mode (the master side of the terminal handles BS before the read() completes in the slave, if I'm understanding it correctly). This does not comply with my (limited) understanding of pty stuff. In mintty, forkpty will create a master/slave pty; mintty feeds it on the master side, while the client program (usually a shell) reads from the slave side. Mintty never handles BS for input, it simply feeds it into the pty. "Line disciplines" like cooked mode must be handled on the slave side. Also, I've tried both options in mintty. Setting the flag on the master side has weird effects, initially blocking the terminal process. Setting it on the slave side works fine. That was a mistake (got something wrong when testing). It works from either side alike. I've now patched mintty to keep the flag in sync with the character encoding, including on later changes (from Options menu or by escape sequence). -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Feb 15 23:19, Thomas Wolff wrote: > Am 14.02.2017 um 21:35 schrieb Thomas Wolff: > > Am 14.02.2017 um 21:29 schrieb Thomas Wolff: > > > Am 14.02.2017 um 20:56 schrieb Eric Blake: > > > > On 02/14/2017 01:40 PM, Thomas Wolff wrote: > > > > > > No. We're talking about a function in the master side > > > > > > of the tty, while > > > > > > the applications started in the terminal are on the slave side. > > > > > I am not familiar with the concept of setting termios properties on > > > > > either the master or slave side of a pty. I've only ever set > > > > > them in the > > > > > client application, including my tests about IUTF8 which worked. Would > > > > > setting on the master side imply it's set for the clients implicitly, > > > > > and can it be changed later, e.g. when mintty character encoding is > > > > > being changed from the Options dialog? > > > > > And you say the function of erasing characters on BS is in the master > > > > > side? To be honest, this confuses me. I thought it's a > > > > > client function, > > > > > like readline() would perform if used (apparently not by > > > > > dash), which is > > > > > kind of an enhanced version of the tty cooked mode and used > > > > > to work even > > > > > without the new flag, right? > > > > The readline source code does not mention IUTF8; and neither bash nor > > > > dash need to reference it, because if the tty handling code sets it > > > > correctly for what the terminal is going to display, then the clients > > > > that are read()ing from the tty never even see BS in cooked mode (the > > > > master side of the terminal handles BS before the read() completes in > > > > the slave, if I'm understanding it correctly). > > > This does not comply with my (limited) understanding of pty stuff. > > > In mintty, forkpty will create a master/slave pty; mintty feeds it > > > on the master side, while the client program (usually a shell) reads > > > from the slave side. Mintty never handles BS for input, it simply > > > feeds it into the pty. "Line disciplines" like cooked mode must be > > > handled on the slave side. > > Also, I've tried both options in mintty. Setting the flag on the master > > side has weird effects, initially blocking the terminal process. > > Setting it on the slave side works fine. > That was a mistake (got something wrong when testing). It works from either > side alike. > I've now patched mintty to keep the flag in sync with the character > encoding, including on later changes (from Options menu or by escape > sequence). There's an ESC sequence to change the codeset? Do you mean the alternate codeset sequence \e[10m / \e[11m or is there something more sophisticated? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
Am 16.02.2017 um 13:49 schrieb Corinna Vinschen: On Feb 15 23:19, Thomas Wolff wrote: Am 14.02.2017 um 21:35 schrieb Thomas Wolff: Am 14.02.2017 um 21:29 schrieb Thomas Wolff: Am 14.02.2017 um 20:56 schrieb Eric Blake: On 02/14/2017 01:40 PM, Thomas Wolff wrote: No. We're talking about a function in the master side of the tty, while the applications started in the terminal are on the slave side. I am not familiar with the concept of setting termios properties on either the master or slave side of a pty. I've only ever set them in the client application, including my tests about IUTF8 which worked. Would setting on the master side imply it's set for the clients implicitly, and can it be changed later, e.g. when mintty character encoding is being changed from the Options dialog? And you say the function of erasing characters on BS is in the master side? To be honest, this confuses me. I thought it's a client function, like readline() would perform if used (apparently not by dash), which is kind of an enhanced version of the tty cooked mode and used to work even without the new flag, right? The readline source code does not mention IUTF8; and neither bash nor dash need to reference it, because if the tty handling code sets it correctly for what the terminal is going to display, then the clients that are read()ing from the tty never even see BS in cooked mode (the master side of the terminal handles BS before the read() completes in the slave, if I'm understanding it correctly). This does not comply with my (limited) understanding of pty stuff. In mintty, forkpty will create a master/slave pty; mintty feeds it on the master side, while the client program (usually a shell) reads from the slave side. Mintty never handles BS for input, it simply feeds it into the pty. "Line disciplines" like cooked mode must be handled on the slave side. Also, I've tried both options in mintty. Setting the flag on the master side has weird effects, initially blocking the terminal process. Setting it on the slave side works fine. That was a mistake (got something wrong when testing). It works from either side alike. I've now patched mintty to keep the flag in sync with the character encoding, including on later changes (from Options menu or by escape sequence). There's an ESC sequence to change the codeset? Do you mean the alternate codeset sequence \e[10m / \e[11m Oh, that one! Thanks for mentioning, I had overlooked it and fixed mintty now to consider it. or is there something more sophisticated? I actually meant to adress https://github.com/mintty/mintty/wiki/CtrlSeqs#locale and there is also \e%G and \e%@. I just notice that later changing of the IUTF8 flag from the master side does not seem to work on a Window 10 system (although it works initially) while it does work on a Windows 7 system. Weird. -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
Am 16.02.2017 um 21:32 schrieb Thomas Wolff: Am 16.02.2017 um 13:49 schrieb Corinna Vinschen: On Feb 15 23:19, Thomas Wolff wrote: Am 14.02.2017 um 21:35 schrieb Thomas Wolff: Am 14.02.2017 um 21:29 schrieb Thomas Wolff: Am 14.02.2017 um 20:56 schrieb Eric Blake: On 02/14/2017 01:40 PM, Thomas Wolff wrote: No. We're talking about a function in the master side of the tty, while the applications started in the terminal are on the slave side. I am not familiar with the concept of setting termios properties on either the master or slave side of a pty. I've only ever set them in the client application, including my tests about IUTF8 which worked. Would setting on the master side imply it's set for the clients implicitly, and can it be changed later, e.g. when mintty character encoding is being changed from the Options dialog? And you say the function of erasing characters on BS is in the master side? To be honest, this confuses me. I thought it's a client function, like readline() would perform if used (apparently not by dash), which is kind of an enhanced version of the tty cooked mode and used to work even without the new flag, right? The readline source code does not mention IUTF8; and neither bash nor dash need to reference it, because if the tty handling code sets it correctly for what the terminal is going to display, then the clients that are read()ing from the tty never even see BS in cooked mode (the master side of the terminal handles BS before the read() completes in the slave, if I'm understanding it correctly). This does not comply with my (limited) understanding of pty stuff. In mintty, forkpty will create a master/slave pty; mintty feeds it on the master side, while the client program (usually a shell) reads from the slave side. Mintty never handles BS for input, it simply feeds it into the pty. "Line disciplines" like cooked mode must be handled on the slave side. Also, I've tried both options in mintty. Setting the flag on the master side has weird effects, initially blocking the terminal process. Setting it on the slave side works fine. That was a mistake (got something wrong when testing). It works from either side alike. I've now patched mintty to keep the flag in sync with the character encoding, including on later changes (from Options menu or by escape sequence). There's an ESC sequence to change the codeset? Do you mean the alternate codeset sequence \e[10m / \e[11m Oh, that one! Thanks for mentioning, I had overlooked it and fixed mintty now to consider it. or is there something more sophisticated? I actually meant to adress https://github.com/mintty/mintty/wiki/CtrlSeqs#locale and there is also \e%G and \e%@. I just notice that later changing of the IUTF8 flag from the master side does not seem to work on a Window 10 system (although it works initially) while it does work on a Windows 7 system. Weird. Now tested on 2 Windows 7 systems and 2 Windows 10 systems. Does not work on Windows 10. Any idea? -- Thomas --- Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft. https://www.avast.com/antivirus -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Feb 17 08:36, Thomas Wolff wrote: > Am 16.02.2017 um 21:32 schrieb Thomas Wolff: > > Am 16.02.2017 um 13:49 schrieb Corinna Vinschen: > > > There's an ESC sequence to change the codeset? Do you mean the > > > alternate codeset sequence \e[10m / \e[11m > > Oh, that one! Thanks for mentioning, I had overlooked it and fixed > > mintty now to consider it. > > > or is there something more sophisticated? > > I actually meant to adress > > https://github.com/mintty/mintty/wiki/CtrlSeqs#locale and there is > > also \e%G and \e%@. > > > > I just notice that later changing of the IUTF8 flag from the master > > side does not seem to work on a Window 10 system (although it works > > initially) while it does work on a Windows 7 system. Weird. > Now tested on 2 Windows 7 systems and 2 Windows 10 systems. Does not > work on Windows 10. > Any idea? Whatever you're observing, there's nothing Windows version-specific here. The tcsetattr function indiscriminately copies the incoming termios structure over. Maybe the bg_check function fails for some reason? Can you strace setting the flag via tcsetattr? There should be some output from tcsetattr as well as from fhandler_termios::bg_check. Can you check? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
Am 17.02.2017 um 23:29 schrieb Thomas Wolff: Am 17.02.2017 um 10:43 schrieb Corinna Vinschen: On Feb 17 08:36, Thomas Wolff wrote: Am 16.02.2017 um 21:32 schrieb Thomas Wolff: Am 16.02.2017 um 13:49 schrieb Corinna Vinschen: There's an ESC sequence to change the codeset? Do you mean the alternate codeset sequence \e[10m / \e[11m Oh, that one! Thanks for mentioning, I had overlooked it and fixed mintty now to consider it. or is there something more sophisticated? I actually meant to adress https://github.com/mintty/mintty/wiki/CtrlSeqs#locale and there is also \e%G and \e%@. I just notice that later changing of the IUTF8 flag from the master side does not seem to work on a Window 10 system (although it works initially) while it does work on a Windows 7 system. Weird. Now tested on 2 Windows 7 systems and 2 Windows 10 systems. Does not work on Windows 10. Any idea? Actually, I'm not sure but I think this problem only occurs if mintty is started in a non-UTF-8 locale. If it's started in UTF-8, later switching seems to work. The strace shows that errno is set to 88 ENOSYS at some place (but I don't know where). Whatever you're observing, there's nothing Windows version-specific here. The tcsetattr function indiscriminately copies the incoming termios structure over. Maybe the bg_check function fails for some reason? Can you strace setting the flag via tcsetattr? Attached. There are lots of strace output even without interaction. I've tried to isolate the switching moment tightly. There should be some output from tcsetattr as well as from fhandler_termios::bg_check. Can you check? You mean the termios_printf trace? How would I activate it (without recompiling cygwin)? -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Feb 17 23:29, Thomas Wolff wrote: > Am 17.02.2017 um 10:43 schrieb Corinna Vinschen: > > On Feb 17 08:36, Thomas Wolff wrote: > > > Am 16.02.2017 um 21:32 schrieb Thomas Wolff: > > > > Am 16.02.2017 um 13:49 schrieb Corinna Vinschen: > > > > > There's an ESC sequence to change the codeset? Do you mean the > > > > > alternate codeset sequence \e[10m / \e[11m > > > > Oh, that one! Thanks for mentioning, I had overlooked it and fixed > > > > mintty now to consider it. > > > > > or is there something more sophisticated? > > > > I actually meant to adress > > > > https://github.com/mintty/mintty/wiki/CtrlSeqs#locale and there is > > > > also \e%G and \e%@. > > > > > > > > I just notice that later changing of the IUTF8 flag from the master > > > > side does not seem to work on a Window 10 system (although it works > > > > initially) while it does work on a Windows 7 system. Weird. > > > Now tested on 2 Windows 7 systems and 2 Windows 10 systems. Does not > > > work on Windows 10. > > > Any idea? > > Whatever you're observing, there's nothing Windows version-specific > > here. The tcsetattr function indiscriminately copies the incoming > > termios structure over. Maybe the bg_check function fails for some > > reason? Can you strace setting the flag via tcsetattr? > Attached. There are lots of strace output even without interaction. I've > tried to isolate the switching moment tightly. > > There should be some output from tcsetattr as well as from > > fhandler_termios::bg_check. > > > > Can you check? > You mean the termios_printf trace? How would I activate it (without > recompiling cygwin)? Just use the -m option of strace. Have a look into strace --help, it's explained quite thoroughly. > 861 40124163 [main] bash 7384 fhandler_pty_slave::tcsetattr: (900): pty > output_mutex (0x4AC): waiting -1 ms >54 40124217 [main] bash 7384 fhandler_pty_slave::tcsetattr: (900): pty > output_mutex: acquired >48 40124265 [main] bash 7384 fhandler_pty_slave::tcsetattr: (902): pty > output_mutex(0x4AC) released >54 40124319 [main] bash 7384 __set_errno: int tcsetattr(int, int, const > termios*):158 setting errno 0 >58 40124377 [main] bash 7384 tcsetattr: iflag 0x850A, oflag 0x9, cflag > 0xBF, lflag 0xD1F, VMIN 1, VTIME 0 >58 40124435 [main] bash 7384 tcsetattr: 0 = tcsetattr(0, 3, 0x6B1F1240) Ok, so here you're setting iflags & ~IUTF8 >34 40131547 [main] bash 7384 tcgetattr: iflag 0x850A, oflag 0x9, cflag > 0xBF, lflag 0xD1F, VMIN 1, VTIME 0 confirmed by bash >35 40131582 [main] bash 7384 fhandler_pty_slave::tcsetattr: (900): pty > output_mutex (0x4AC): waiting -1 ms >33 40131615 [main] bash 7384 fhandler_pty_slave::tcsetattr: (900): pty > output_mutex: acquired >34 40131649 [main] bash 7384 fhandler_pty_slave::tcsetattr: (902): pty > output_mutex(0x4AC) released >32 40131681 [main] bash 7384 __set_errno: int tcsetattr(int, int, const > termios*):158 setting errno 0 >33 40131714 [main] bash 7384 tcsetattr: iflag 0x840A, oflag 0x9, cflag > 0xBF, lflag 0xD19, VMIN 1, VTIME 0 >32 40131746 [main] bash 7384 tcsetattr: 0 = tcsetattr(0, 3, 0x6DC0D4) And then bash sets iflags again, just to the same values as mintty had set them before. > 145 43721757 [main] mintty 9348 tcgetattr: iflag 0x840A, oflag 0x9, cflag > 0xBF, lflag 0xD19, VMIN 1, VTIME 0 >33 43721790 [main] mintty 9348 __set_errno: int tcsetattr(int, int, const > termios*):158 setting errno 88 >36 43721826 [main] mintty 9348 tcsetattr: iflag 0x2840A, oflag 0x9, cflag > 0xBF, lflag 0xD19, VMIN 1, VTIME 0 >36 43721862 [main] mintty 9348 tcsetattr: 0 = tcsetattr(3, 2, 0x66C774) >36 43721898 [main] mintty 9348 tcgetattr: iflag 0x2840A, oflag 0x9, cflag > 0xBF, lflag 0xD19, VMIN 1, VTIME 0 And now mintty sets iflags | IUTF8. Hmm. Why does it do that here? It's certainly too late for bash. I can't see more, but setting and resetting the flag works as expected. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
On Feb 18 23:45, Thomas Wolff wrote: > Am 17.02.2017 um 23:29 schrieb Thomas Wolff: > > Am 17.02.2017 um 10:43 schrieb Corinna Vinschen: > > > On Feb 17 08:36, Thomas Wolff wrote: > > > > Am 16.02.2017 um 21:32 schrieb Thomas Wolff: > > > > > Am 16.02.2017 um 13:49 schrieb Corinna Vinschen: > > > > > > There's an ESC sequence to change the codeset? Do you mean the > > > > > > alternate codeset sequence \e[10m / \e[11m > > > > > Oh, that one! Thanks for mentioning, I had overlooked it and fixed > > > > > mintty now to consider it. > > > > > > or is there something more sophisticated? > > > > > I actually meant to adress > > > > > https://github.com/mintty/mintty/wiki/CtrlSeqs#locale and there is > > > > > also \e%G and \e%@. > > > > > > > > > > I just notice that later changing of the IUTF8 flag from the master > > > > > side does not seem to work on a Window 10 system (although it works > > > > > initially) while it does work on a Windows 7 system. Weird. > > > > Now tested on 2 Windows 7 systems and 2 Windows 10 systems. Does not > > > > work on Windows 10. > > > > Any idea? > Actually, I'm not sure but I think this problem only occurs if mintty is > started in a non-UTF-8 locale. > If it's started in UTF-8, later switching seems to work. > The strace shows that errno is set to 88 ENOSYS at some place (but I don't > know where). I wonder. Can't you just start mintty under GDB and debug what happens in the places it calls tcgetattr/tcsetattr? You can also see what tcsetattr actually doe, which is, almost nothing. At the core it chooses the tty and just overwrites the termios structure of the tty. That's all. You can also see what function call returns ENOSYS then. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: [ANNOUNCEMENT] Updated: dash-0.5.8-3
Am 20.02.2017 um 10:11 schrieb Corinna Vinschen: On Feb 18 23:45, Thomas Wolff wrote: Am 17.02.2017 um 23:29 schrieb Thomas Wolff: Am 17.02.2017 um 10:43 schrieb Corinna Vinschen: On Feb 17 08:36, Thomas Wolff wrote: Am 16.02.2017 um 21:32 schrieb Thomas Wolff: Am 16.02.2017 um 13:49 schrieb Corinna Vinschen: There's an ESC sequence to change the codeset? Do you mean the alternate codeset sequence \e[10m / \e[11m Oh, that one! Thanks for mentioning, I had overlooked it and fixed mintty now to consider it. or is there something more sophisticated? I actually meant to adress https://github.com/mintty/mintty/wiki/CtrlSeqs#locale and there is also \e%G and \e%@. I just notice that later changing of the IUTF8 flag from the master side does not seem to work on a Window 10 system (although it works initially) while it does work on a Windows 7 system. Weird. Now tested on 2 Windows 7 systems and 2 Windows 10 systems. Does not work on Windows 10. Any idea? Actually, I'm not sure but I think this problem only occurs if mintty is started in a non-UTF-8 locale. If it's started in UTF-8, later switching seems to work. The strace shows that errno is set to 88 ENOSYS at some place (but I don't know where). I wonder. Can't you just start mintty under GDB and debug what happens in the places it calls tcgetattr/tcsetattr? You can also see what tcsetattr actually doe, which is, almost nothing. At the core it chooses the tty and just overwrites the termios structure of the tty. That's all. You can also see what function call returns ENOSYS then. Before fiddling around with gdb (which I'm not familiar with, sorry), I've made additional observations. First, the issue occurs regardless of how the terminal is started. Then, it seems to occur in interactive bash or tcsh only, not in dash, or within the following loop started from bash: while true; do sleep 1; stty -a; done In the latter cases everything works as expected. Maybe it's readline() trying to keep the flag consistent with its locale? (If that's the case, it still doesn't explain why it only fails on Windows 10.) -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
