Re: testsuite under wine

[Jerry D via Gcc]

On 1/5/23 7:33 PM, Jacob Bachmeyer via Fortran wrote:

NightStrike wrote:
On Fri, Dec 23, 2022 at 11:00 PM Jacob Bachmeyer  
wrote:

NightStrike wrote:
On Wed, Dec 21, 2022 at 11:37 PM Jacob Bachmeyer 
 wrote:

[...]

So at least we know for sure that this particular instance of extra
characters is coming from Wine.  Maybe Wine can be smart enough to
only translate \n into \r\n instead of translating \r\n into \r\r\n.
Jacek / Eric, comments here?  I'm happy to try another patch, the
first one was great.


I doubt that Wine is doing that translation.  MinGW libc produces output
conformant to Windows conventions, so printf("\n") on a text handle
emits "\r\n", which Wine passes along.  POSIX convention is that "\n" is
translated to "\r\n" in the kernel terminal driver upon output, so the
kernel translates the "\n" in the "\r\n" into /another/ "\r\n", yielding
"\r\r\n" at the pty master end.  This is why DejaGnu testsuites must be
prepared to discard excess carriage returns.  The first CR came from
MinGW libc; the second CR came from the kernel terminal driver; the LF
was ultimately passed through.


Jacek and I have been digging into this on IRC, and he's been very
helpful in trying to get further, but we're still stuck.  We tried to
be more introspective, inserting strace both as "strace script wine"
and as "script strace wine".  We tried running just "wine a.exe"
without any extra glue, and logging the raw SSH packets from putty.
After many iterations on these and other tests, Jacek finally had the
idea to try removing Windows entirely from the equation, and we ran
with a purely unix program / compiler combination:

#include 

int main()
{
    write(1, "test\r\n", 6);
    return 0;
}

(and also as "test\n", 5)

In both versions, the following was observed:

case 1) ./a.out | xxd
case 2) script -c ./a.out out; xxd out
case 3) enable putting logging, ./a.out

In case 1, xxd showed no extra \r's.  In cases 2 and 3, there was an
extra \r (either 0d 0d 0a for test\r\n, or 0d 0a for test\n).

So, is it possible after all of this back and forth regarding mingw,
wine, and others, that it's down to the write() system call that's
inserting extra \r's?  Is this expected?


"This is why DejaGnu testsuites must be prepared to discard excess 
carriage returns."


The write(2) system call inserts nothing and simply hands off the buffer 
to the relevant part of the kernel I/O subsystem.  (The kernel in POSIX 
is *not* a monolithic black box.)  When stdout for your test program is 
a pty slave, that relevant part is the kernel terminal driver.  The 
kernel terminal driver is converting "\n" to "\r\n" upon output to the 
associated port, since hardware terminals typically *do* require CRLF. 
The associated port in this case is virtual and part of the kernel pty 
subsystem, which presents octets written to that port to its associated 
pty master device.  The user-visible pty slave device acts just like a 
serial terminal, including all translations normally done for handling 
serial terminals.


A pty is conceptually a null-modem cable connected between two 
infinitely-fast serial ports on the same machine, although the slave 
will still report an actual baud rate if queried.  (Run "stty" with no 
arguments under script(1), an ssh session, or an X11 terminal emulator 
to see what a pty slave looks like on your machine.)


In your case 1, the pty subsystem is not used and output is collected 
over a pipe.  Using "./a.out > out; xxd out" would produce the same 
results.  In cases 2 and 3, there is a pty involved, either set up by 
script(1) or by sshd (assuming you meant "enable putty logging" in case 
3) that performs the standard terminal translations.  In all cases, 
strace(1) will show the exact string written to the pty slave device, 
which will not include any extra CRs because *those* *are* *inserted* 
*by* *the* *kernel* *terminal* *driver* as the data is transferred to 
the pty master device's read queue.


This insertion of carriage returns is expected and standardized behavior 
in POSIX and is the reason Unix could use bare LF as end-of-line even 
though hardware terminals always needed CRLF.  CP/M (and therefore 
MS-DOS which began its existence as a cheap CP/M knockoff) did not have 
this translation layer and instead dumped the complexity of a two-octet 
end-of-line sequence on user programs, leading to much confusion even 
today.  This is not a Wine issue, although the terminal escape sequences 
in your original issue *were* from Wine.  Note that the number of excess 
carriage returns that a DejaGnu testsuite must be prepared to discard is 
unspecified because running tests on remote targets may result in *any* 
*number* of CRs preceding each LF by the time the results reach the test 
driver machine in more complex testing lab environments.



-- Jacob


Agree

Re: testsuite under wine

[Jerry D via Gcc]

On 1/4/23 6:50 PM, NightStrike via Fortran wrote:

On Fri, Dec 23, 2022 at 11:00 PM Jacob Bachmeyer  wrote:

NightStrike wrote:

On Wed, Dec 21, 2022 at 11:37 PM Jacob Bachmeyer  wrote:

NightStrike wrote:


[...]
Second, the problems with extra \r's still remain, but I think we've
generally come to think that that part isn't Wine and is instead
either the testsuite or deja.  So I'll keep those replies to Jacob's
previous message.



Most likely, it is a combination of the MinGW libc (which emits "\r\n"
for end-of-line in accordance with Windows convention) and the kernel
terminal driver (which passes "\r" and translates "\n" to "\r\n" in
accordance with POSIX convention).  Wine, short of trying to translate
"\r\n" back to "\n" in accordance with POSIX conventions (and likely
making an even bigger mess---does Wine know if a handle is supposed to
be text or binary?) cannot really fix this, so the testsuite needs to
handle non-POSIX-standard line endings.  (The Rust tests probably have
an outright bug if the newlines are being duplicated.)



You may be onto something here.  I ran wine under script as `script -c
"wine64 ./a.exe" out` (thanks, Arsen!), and it had the same extra \r
prepended to the \r\n.  I was making the mistake previously of running
wine manually and capturing it to a file as `wine64 ./a.exe > out`,
which as several have pointed out in this thread, that would disable
the quirk, so of course it didn't reveal any problems.  I'm behind,
but I'll catch up to you guys eventually :)



So close, and yet so far...  script(1) /also/ uses a pty, so it is
getting the same translations as Expect and therefore DejaGnu.


So at least we know for sure that this particular instance of extra
characters is coming from Wine.  Maybe Wine can be smart enough to
only translate \n into \r\n instead of translating \r\n into \r\r\n.
Jacek / Eric, comments here?  I'm happy to try another patch, the
first one was great.



I doubt that Wine is doing that translation.  MinGW libc produces output
conformant to Windows conventions, so printf("\n") on a text handle
emits "\r\n", which Wine passes along.  POSIX convention is that "\n" is
translated to "\r\n" in the kernel terminal driver upon output, so the
kernel translates the "\n" in the "\r\n" into /another/ "\r\n", yielding
"\r\r\n" at the pty master end.  This is why DejaGnu testsuites must be
prepared to discard excess carriage returns.  The first CR came from
MinGW libc; the second CR came from the kernel terminal driver; the LF
was ultimately passed through.


Jacek and I have been digging into this on IRC, and he's been very
helpful in trying to get further, but we're still stuck.  We tried to
be more introspective, inserting strace both as "strace script wine"
and as "script strace wine".  We tried running just "wine a.exe"
without any extra glue, and logging the raw SSH packets from putty.
After many iterations on these and other tests, Jacek finally had the
idea to try removing Windows entirely from the equation, and we ran
with a purely unix program / compiler combination:

#include 

int main()
{
 write(1, "test\r\n", 6);
 return 0;
}

(and also as "test\n", 5)

In both versions, the following was observed:

case 1) ./a.out | xxd
case 2) script -c ./a.out out; xxd out
case 3) enable putting logging, ./a.out

In case 1, xxd showed no extra \r's.  In cases 2 and 3, there was an
extra \r (either 0d 0d 0a for test\r\n, or 0d 0a for test\n).

So, is it possible after all of this back and forth regarding mingw,
wine, and others, that it's down to the write() system call that's
inserting extra \r's?  Is this expected?


I have reproduced this test with C.  I suspect that the 'write' function 
was written to accommodate non standard behavior of windows which 
expects a CR-LF.  This means that POSIX compliant code is adjusted by 
the C or gfortran libraries to emit a extra CR so that on Windows it 
will just work ok.  So the test is doing exactly what you are telling it 
to do.  An LF causes the run times to add a CR in front.


With libgfortran I remember implementing some of this code myself and 
this is the reason, to keep applications happy on Windows.


So the gfortran tests we wrote to accept either a CR or a CR-LF, and in 
the test code to only issue a normal line ending which on UNIX will be 
an LF and Windows an CR-LF.


I lose track of details in between looking ta the test cases. let me 
know one of them to study that is gfortran side and will see what it is 
doing.


Jerry

Re: testsuite under wine

[Jacob Bachmeyer via Gcc]

NightStrike wrote:

On Fri, Dec 23, 2022 at 11:00 PM Jacob Bachmeyer  wrote:
  

NightStrike wrote:


On Wed, Dec 21, 2022 at 11:37 PM Jacob Bachmeyer  wrote:
  

[...]

So at least we know for sure that this particular instance of extra
characters is coming from Wine.  Maybe Wine can be smart enough to
only translate \n into \r\n instead of translating \r\n into \r\r\n.
Jacek / Eric, comments here?  I'm happy to try another patch, the
first one was great.

  

I doubt that Wine is doing that translation.  MinGW libc produces output
conformant to Windows conventions, so printf("\n") on a text handle
emits "\r\n", which Wine passes along.  POSIX convention is that "\n" is
translated to "\r\n" in the kernel terminal driver upon output, so the
kernel translates the "\n" in the "\r\n" into /another/ "\r\n", yielding
"\r\r\n" at the pty master end.  This is why DejaGnu testsuites must be
prepared to discard excess carriage returns.  The first CR came from
MinGW libc; the second CR came from the kernel terminal driver; the LF
was ultimately passed through.



Jacek and I have been digging into this on IRC, and he's been very
helpful in trying to get further, but we're still stuck.  We tried to
be more introspective, inserting strace both as "strace script wine"
and as "script strace wine".  We tried running just "wine a.exe"
without any extra glue, and logging the raw SSH packets from putty.
After many iterations on these and other tests, Jacek finally had the
idea to try removing Windows entirely from the equation, and we ran
with a purely unix program / compiler combination:

#include 

int main()
{
write(1, "test\r\n", 6);
return 0;
}

(and also as "test\n", 5)

In both versions, the following was observed:

case 1) ./a.out | xxd
case 2) script -c ./a.out out; xxd out
case 3) enable putting logging, ./a.out

In case 1, xxd showed no extra \r's.  In cases 2 and 3, there was an
extra \r (either 0d 0d 0a for test\r\n, or 0d 0a for test\n).

So, is it possible after all of this back and forth regarding mingw,
wine, and others, that it's down to the write() system call that's
inserting extra \r's?  Is this expected?
  


"This is why DejaGnu testsuites must be prepared to discard excess 
carriage returns."


The write(2) system call inserts nothing and simply hands off the buffer 
to the relevant part of the kernel I/O subsystem.  (The kernel in POSIX 
is *not* a monolithic black box.)  When stdout for your test program is 
a pty slave, that relevant part is the kernel terminal driver.  The 
kernel terminal driver is converting "\n" to "\r\n" upon output to the 
associated port, since hardware terminals typically *do* require CRLF.  
The associated port in this case is virtual and part of the kernel pty 
subsystem, which presents octets written to that port to its associated 
pty master device.  The user-visible pty slave device acts just like a 
serial terminal, including all translations normally done for handling 
serial terminals.


A pty is conceptually a null-modem cable connected between two 
infinitely-fast serial ports on the same machine, although the slave 
will still report an actual baud rate if queried.  (Run "stty" with no 
arguments under script(1), an ssh session, or an X11 terminal emulator 
to see what a pty slave looks like on your machine.)


In your case 1, the pty subsystem is not used and output is collected 
over a pipe.  Using "./a.out > out; xxd out" would produce the same 
results.  In cases 2 and 3, there is a pty involved, either set up by 
script(1) or by sshd (assuming you meant "enable putty logging" in case 
3) that performs the standard terminal translations.  In all cases, 
strace(1) will show the exact string written to the pty slave device, 
which will not include any extra CRs because *those* *are* *inserted* 
*by* *the* *kernel* *terminal* *driver* as the data is transferred to 
the pty master device's read queue.


This insertion of carriage returns is expected and standardized behavior 
in POSIX and is the reason Unix could use bare LF as end-of-line even 
though hardware terminals always needed CRLF.  CP/M (and therefore 
MS-DOS which began its existence as a cheap CP/M knockoff) did not have 
this translation layer and instead dumped the complexity of a two-octet 
end-of-line sequence on user programs, leading to much confusion even 
today.  This is not a Wine issue, although the terminal escape sequences 
in your original issue *were* from Wine.  Note that the number of excess 
carriage returns that a DejaGnu testsuite must be prepared to discard is 
unspecified because running tests on remote targets may result in *any* 
*number* of CRs preceding each LF by the time the results reach the test 
driver machine in more complex testing lab environments.



-- Jacob

Re: RISC-V V C Intrinsic API v1.0 release meeting reminder (January 05th, 2023)

[Eop Chen]

The meeting minutes in added in the note. 


You can also find it under riscv-admin/rvv-intrinsics. 


> Eop Chen  於 2023年1月4日 下午11:18 寫道：
> 
> 
> Hi all,
> 
> A reminder that the next open meeting to discuss on the RISC-V V C Intrinsic 
> API v1.0 release is going to
> be held on  2023/01/05 6AM (GMT -7) / 11PM (GMT +8).
> 
> The agenda can be found in the second page of the meeting slides (link 
> ).
> Please join the calendar to be constantly notified - Google calender link 
> ,
>  ICal 
> 
> We also have a mailing list now hosted by RISC-V International (link 
> ).
> 
> Regards,
> 
> eop Chen
> 

gcc-10-20230105 is now available

[GCC Administrator via Gcc]

Snapshot gcc-10-20230105 is now available on
  https://gcc.gnu.org/pub/gcc/snapshots/10-20230105/
and on various mirrors, see http://gcc.gnu.org/mirrors.html for details.

This snapshot has been generated from the GCC 10 git branch
with the following options: git://gcc.gnu.org/git/gcc.git branch 
releases/gcc-10 revision 32a5510d5f419db88a0e14f3c6e4f708c0bc776d

You'll find:

 gcc-10-20230105.tar.xz   Complete GCC

  SHA256=821117abd903ecb69fce6c1251ffe43a6f24f63297e6cc9cd8e11d3e0b1fc716
  SHA1=43e5459389c0a1f51dfbe89ac37dcb12e4b8ae0d

Diffs from 10-20221229 are available in the diffs/ subdirectory.

When a particular snapshot is ready for public consumption the LATEST-10
link is updated and a message is sent to the gcc list.  Please do not use
a snapshot before it has been announced that way.