Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On Sep 6 21:34, Brian Inglis wrote:
> On 2021-09-06 15:24, Ken Brown via Cygwin wrote:
> > On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote:
> > > Hi there,
> > >
> > > On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote:
> > > > > No, wait. I get what you say. The optimzation settings of the test
> > > > > case should have no influence on the code inside the DLL. That
> > > > > doesn't
> > > > > make sense for sure. However, I ran the testcase under GDB, I could
> > > > > reproduce the issue, and I could fix it by setting mmap_ext.Reserved
> > > > > = 0;
> > > > > Go figure!
> > > >
> > > > I don't get it, but I can confirm that the problem is fixed.
> > >
> > > That sounds a bit like a voodoo fix, that could quickly regress again.
> > >
> > > Here is my 2 cents:
> > >
> > > Currently the mmap_ext structure is setup like this:
> > >
> > > 215 MEM_EXTENDED_PARAMETER mmap_ext = {
> > > 216 .Type = MemExtendedParameterAddressRequirements,
> > > 217 .Pointer = (PVOID) _req
> > > 218 };
> > >
> > > This means that all other entries in the struct are zero at
> > > initialization as described here:
> > > https://en.cppreference.com/w/c/language/struct_initialization
> > >
> > > So if you set "mmap_ext.Reserved = 0" again after that its a double
> > > failure.
> >
> > You're looking at the wrong source code. The bug didn't occur until the
> > code was changed to do the following:
> >
> > /* g++ 11.2 workaround: don't use initializer */
> > MEM_EXTENDED_PARAMETER mmap_ext;
> > mmap_ext.Type = MemExtendedParameterAddressRequirements;
> > mmap_ext.Pointer = (PVOID) _req;
> >
> > This left mmap_ext.Reserved uninitialized, which Corinna has now fixed.
>
> With undocumented structure member initialization an issue, maybe better to
> future proof using e.g.
>
> MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero
> ...
You're right. The bits in Reserved are just that, reserved. So MSFT
may decide to use some of the bits for other purposes, should the need
arise. I've fixed that in git master.
Thanks,
Corinna
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 07/09/2021 23:44, Ken Brown via Cygwin wrote:
>
> MS can't add a new named field to a documented struct without
breaking a lot of code. I think it's extremely unlikely that they would
do that. On the other hand, I think it's very likely that a reader of
the Cygwin code would be confused by code that initializes a struct to 0
prior to assigning a value to every member.
Hi Ken.
It is common practice when following a coding standard (especially those
for safety or security sensitive code) to initialize each and every
automatic C/C++ variable at the point of definition even if it is later
assigned a value. The '= {0}' mechanism for structures is a common way
of ensuring this is the case for all fields in a structure and should be
familiar to most developers.
This ensures deterministic behaviour instead of bugs that come and go
depending upon what was on the stack or in the register before the
variable is used. It also helps when reviewing code.
A compiler or static analyser cannot always determine whether a variable
is used before it's assigned so getting into the habit of doing this
saves a lot of grief down the line, as demonstrated by this thread!
--
Sam "Here Endeth The Lesson" Edge
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/7/2021 5:52 PM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] wrote:
With undocumented structure member initialization an issue, maybe better to
future proof using e.g.
MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero
I don't see what this would accomplish. We're already initializing every member
after Corinna's last patch.
Well, if one day MS decides to use some of the Reserved field by splinting off
a new
named field out of that bulk of bits, the code will be in trouble again...
While with memset, bzero or C-style initializer, everything is guaranteed to be
zapped with 0.
MS can't add a new named field to a documented struct without breaking a lot of
code. I think it's extremely unlikely that they would do that. On the other
hand, I think it's very likely that a reader of the Cygwin code would be
confused by code that initializes a struct to 0 prior to assigning a value to
every member.
Ken
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
RE: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
> >
> > With undocumented structure member initialization an issue, maybe better to
> > future proof using e.g.
> >
> > MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero
>
> I don't see what this would accomplish. We're already initializing every
> member
> after Corinna's last patch.
>
Well, if one day MS decides to use some of the Reserved field by splinting off
a new
named field out of that bulk of bits, the code will be in trouble again...
While with memset, bzero or C-style initializer, everything is guaranteed to be
zapped with 0.
Anton Lavrentiev
Contractor NIH/NLM/NCBI
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
Hi Ken, On Mon, 2021-09-06 at 17:24 -0400, Ken Brown via Cygwin wrote: > You're looking at the wrong source code. The bug didn't occur until > the code > was changed to do the following: You are right. I do not know why i looked at an old checkout of the code. Shame on me! Sorry for wasting your time. Maybe i should stop looking at code after 24:00. Bad idea! :-). So with my new shiny view, the fix makes perfectly sense. Great! /pedro -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/6/2021 11:34 PM, Brian Inglis wrote:
On 2021-09-06 15:24, Ken Brown via Cygwin wrote:
On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote:
Hi there,
On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote:
No, wait. I get what you say. The optimzation settings of the test
case should have no influence on the code inside the DLL. That
doesn't
make sense for sure. However, I ran the testcase under GDB, I could
reproduce the issue, and I could fix it by setting mmap_ext.Reserved
= 0;
Go figure!
I don't get it, but I can confirm that the problem is fixed.
That sounds a bit like a voodoo fix, that could quickly regress again.
Here is my 2 cents:
Currently the mmap_ext structure is setup like this:
215 MEM_EXTENDED_PARAMETER mmap_ext = {
216 .Type = MemExtendedParameterAddressRequirements,
217 .Pointer = (PVOID) _req
218 };
This means that all other entries in the struct are zero at
initialization as described here:
https://en.cppreference.com/w/c/language/struct_initialization
So if you set "mmap_ext.Reserved = 0" again after that its a double
failure.
You're looking at the wrong source code. The bug didn't occur until the code
was changed to do the following:
/* g++ 11.2 workaround: don't use initializer */
MEM_EXTENDED_PARAMETER mmap_ext;
mmap_ext.Type = MemExtendedParameterAddressRequirements;
mmap_ext.Pointer = (PVOID) _req;
This left mmap_ext.Reserved uninitialized, which Corinna has now fixed.
With undocumented structure member initialization an issue, maybe better to
future proof using e.g.
MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero
I don't see what this would accomplish. We're already initializing every member
after Corinna's last patch.
Ken
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 2021-09-06 15:24, Ken Brown via Cygwin wrote:
On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote:
Hi there,
On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote:
No, wait. I get what you say. The optimzation settings of the test
case should have no influence on the code inside the DLL. That
doesn't
make sense for sure. However, I ran the testcase under GDB, I could
reproduce the issue, and I could fix it by setting mmap_ext.Reserved
= 0;
Go figure!
I don't get it, but I can confirm that the problem is fixed.
That sounds a bit like a voodoo fix, that could quickly regress again.
Here is my 2 cents:
Currently the mmap_ext structure is setup like this:
215 MEM_EXTENDED_PARAMETER mmap_ext = {
216 .Type = MemExtendedParameterAddressRequirements,
217 .Pointer = (PVOID) _req
218 };
This means that all other entries in the struct are zero at
initialization as described here:
https://en.cppreference.com/w/c/language/struct_initialization
So if you set "mmap_ext.Reserved = 0" again after that its a double
failure.
You're looking at the wrong source code. The bug didn't occur until the
code was changed to do the following:
/* g++ 11.2 workaround: don't use initializer */
MEM_EXTENDED_PARAMETER mmap_ext;
mmap_ext.Type = MemExtendedParameterAddressRequirements;
mmap_ext.Pointer = (PVOID) _req;
This left mmap_ext.Reserved uninitialized, which Corinna has now fixed.
With undocumented structure member initialization an issue, maybe better
to future proof using e.g.
MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero
...
--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/6/2021 5:24 PM, Ken Brown via Cygwin wrote:
On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote:
Hi there,
On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote:
No, wait. I get what you say. The optimzation settings of the test
case should have no influence on the code inside the DLL. That
doesn't
make sense for sure. However, I ran the testcase under GDB, I could
reproduce the issue, and I could fix it by setting mmap_ext.Reserved
= 0;
Go figure!
I don't get it, but I can confirm that the problem is fixed.
That sounds a bit like a voodoo fix, that could quickly regress again.
Here is my 2 cents:
Currently the mmap_ext structure is setup like this:
215 MEM_EXTENDED_PARAMETER mmap_ext = {
216 .Type = MemExtendedParameterAddressRequirements,
217 .Pointer = (PVOID) _req
218 };
This means that all other entries in the struct are zero at
initialization as described here:
https://en.cppreference.com/w/c/language/struct_initialization
So if you set "mmap_ext.Reserved = 0" again after that its a double
failure.
You're looking at the wrong source code. The bug didn't occur until the code
was changed to do the following:
/* g++ 11.2 workaround: don't use initializer */
MEM_EXTENDED_PARAMETER mmap_ext;
mmap_ext.Type = MemExtendedParameterAddressRequirements;
mmap_ext.Pointer = (PVOID) _req;
This left mmap_ext.Reserved uninitialized, which Corinna has now fixed.
I should add that when I said "I don't get it", I wasn't referring to Corinna's
fix. I was referring to the fact that the bug did *not* occur when the
unoptimized build was run under gdb.
Ken
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote:
Hi there,
On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote:
No, wait. I get what you say. The optimzation settings of the test
case should have no influence on the code inside the DLL. That
doesn't
make sense for sure. However, I ran the testcase under GDB, I could
reproduce the issue, and I could fix it by setting mmap_ext.Reserved
= 0;
Go figure!
I don't get it, but I can confirm that the problem is fixed.
That sounds a bit like a voodoo fix, that could quickly regress again.
Here is my 2 cents:
Currently the mmap_ext structure is setup like this:
215 MEM_EXTENDED_PARAMETER mmap_ext = {
216 .Type = MemExtendedParameterAddressRequirements,
217 .Pointer = (PVOID) _req
218 };
This means that all other entries in the struct are zero at
initialization as described here:
https://en.cppreference.com/w/c/language/struct_initialization
So if you set "mmap_ext.Reserved = 0" again after that its a double
failure.
You're looking at the wrong source code. The bug didn't occur until the code
was changed to do the following:
/* g++ 11.2 workaround: don't use initializer */
MEM_EXTENDED_PARAMETER mmap_ext;
mmap_ext.Type = MemExtendedParameterAddressRequirements;
mmap_ext.Pointer = (PVOID) _req;
This left mmap_ext.Reserved uninitialized, which Corinna has now fixed.
Ken
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
Hi there,
On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote:
> > No, wait. I get what you say. The optimzation settings of the test
> > case should have no influence on the code inside the DLL. That
> > doesn't
> > make sense for sure. However, I ran the testcase under GDB, I could
> > reproduce the issue, and I could fix it by setting mmap_ext.Reserved
> > = 0;
> > Go figure!
>
> I don't get it, but I can confirm that the problem is fixed.
That sounds a bit like a voodoo fix, that could quickly regress again.
Here is my 2 cents:
Currently the mmap_ext structure is setup like this:
215 MEM_EXTENDED_PARAMETER mmap_ext = {
216 .Type = MemExtendedParameterAddressRequirements,
217 .Pointer = (PVOID) _req
218 };
This means that all other entries in the struct are zero at
initialization as described here:
https://en.cppreference.com/w/c/language/struct_initialization
So if you set "mmap_ext.Reserved = 0" again after that its a double
failure.
1) The compiler should ignore it as it is redundent.
2) It makes no sense, as it is already zero :-)
Since it is not ignored, the compiler clearly puts in code to to
reinitialize the variable (which some compilers do not optimize for).
The reason this makes it "work" it probably because of some other stack
curruption that is going on which then disappears because of the code
moving around a bit due to the new line. My best guess would be that
other fun stuff in the same location would also "fix" the problem.
These are not the droids you are looking for. The real problem is
elsewhere, and is likely due to some stack-smashing going on. This is
also likely why recompiling the test program makes a difference as that
changes what goes on the variable stack. When the code moves around
again (e.g. new compiler version), it could break again.
Looking at the test program -02 vs -O0:
pushq %rbp
.seh_pushreg%rbp
movq%rsp, %rbp
.seh_setframe %rbp, 0
subq$64, %rsp
.seh_stackalloc 64
.seh_endprologue
vs
subq$56, %rsp
.seh_stackalloc 56
Which gives a different stack layout. I think the problem must be in
the start of mmap() or subsequent calls like CreateMapping() and
MapView(). Something smashes or affects the stack.
Thanks,
/pedro
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
Hi there,
On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote:
> > No, wait. I get what you say. The optimzation settings of the test
> > case should have no influence on the code inside the DLL. That
> > doesn't
> > make sense for sure. However, I ran the testcase under GDB, I could
> > reproduce the issue, and I could fix it by setting mmap_ext.Reserved
> > = 0;
> > Go figure!
>
> I don't get it, but I can confirm that the problem is fixed.
That sounds a bit like a voodoo fix, that could quickly regress again.
Here is my 2 cents:
Currently the mmap_ext structure is setup like this:
215 MEM_EXTENDED_PARAMETER mmap_ext = {
216 .Type = MemExtendedParameterAddressRequirements,
217 .Pointer = (PVOID) _req
218 };
This means that all other entries in the struct are zero at
initialization as described here:
https://en.cppreference.com/w/c/language/struct_initialization
So if you set "mmap_ext.Reserved = 0" again after that its a double
failure.
1) The compiler should ignore it as it is redundent.
2) It makes no sense, as it is already zero :-)
Since it is not ignored, the compiler clearly puts in code to to
reinitialize the variable (which some compilers do not optimize for).
The reason this makes it "work" it probably because of some other stack
curruption that is going on which then disappears because of the code
moving around a bit due to the new line. My best guess would be that
other fun stuff in the same location would also "fix" the problem.
These are not the droids you are looking for. The real problem is
elsewhere, and is likely due to some stack-smashing going on. This is
also likely why recompiling the test program makes a difference as that
changes what goes on the variable stack. When the code moves around
again (e.g. new compiler version), it could break again.
Looking at the test program -02 vs -O0:
pushq %rbp
.seh_pushreg%rbp
movq%rsp, %rbp
.seh_setframe %rbp, 0
subq$64, %rsp
.seh_stackalloc 64
.seh_endprologue
vs
subq$56, %rsp
.seh_stackalloc 56
Which gives a different stack layout. I think the problem must be in
the start of mmap() or subsequent calls like CreateMapping() and
MapView(). Something smashes or affects the stack.
Thanks,
/pedro
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/6/2021 2:07 PM, Corinna Vinschen via Cygwin wrote: On Sep 6 19:59, Corinna Vinschen via Cygwin wrote: On Sep 6 13:38, Ken Brown via Cygwin wrote: On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: On Sep 5 09:24, Ken Brown via Cygwin wrote: On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: Here are the correct commits: 8169e39ab Cygwin: C++17: register keyword is deprecated 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 bdb7991db Cygwin: workaround a g++ 11.2 initialization bug 801120c1f Cygwin: loader script: add DWARF 5 sections d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning c2fe205b5 strstr: avoid warnings 76c2c7a89 ldexp/ldexpf: avoid assembler warning eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString So there appears to be something wrong with cygwin1.dll built with the current build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant). Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER initialization problem that was dealt with in commit bdb7991db. More data: When I run the test case under gdb, it succeeds. When I run it under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with windows error 87. Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, perhaps? I tried removing them, and I got the same error. I also tried removing static, and I tried removing both static and const. BTW, when I reported that the test case succeeds under gdb, that only happens when I build the test case without optimization. If I build with -O2, it fails under gdb also. [In all my tests, I built cygwin1.dll without optimization.] This makes no sense to me at all. Good hint. I found the culprit. With optimization, the code doesn't set the "Reserved" bits in the first struct of MEM_EXTENDED_PARAMETER to 0. No, wait. I get what you say. The optimzation settings of the test case should have no influence on the code inside the DLL. That doesn't make sense for sure. However, I ran the testcase under GDB, I could reproduce the issue, and I could fix it by setting mmap_ext.Reserved = 0; Go figure! I don't get it, but I can confirm that the problem is fixed. Thanks. Ken -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On Sep 6 19:59, Corinna Vinschen via Cygwin wrote: > On Sep 6 13:38, Ken Brown via Cygwin wrote: > > On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: > > > On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: > > > > On Sep 5 09:24, Ken Brown via Cygwin wrote: > > > > > On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: > > > > > > On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: > > > > > > > Here are the correct commits: > > > > > > > > > > > > > > 8169e39ab Cygwin: C++17: register keyword is deprecated > > > > > > > 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 > > > > > > > bdb7991db Cygwin: workaround a g++ 11.2 initialization bug > > > > > > > 801120c1f Cygwin: loader script: add DWARF 5 sections > > > > > > > d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning > > > > > > > c2fe205b5 strstr: avoid warnings > > > > > > > 76c2c7a89 ldexp/ldexpf: avoid assembler warning > > > > > > > eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString > > > > > > > > > > > > > > > > > > > > > > > > So there appears to be something wrong with cygwin1.dll > > > > > > > > > built with the current build tools (gcc 11.2.0, binutils > > > > > > > > > 2.37, not sure what else is relevant). > > > > > > > > > > > > Wait a minute...I'll bet this is related to the > > > > > > MEM_EXTENDED_PARAMETER > > > > > > initialization problem that was dealt with in commit bdb7991db. > > > > > > > > > > More data: When I run the test case under gdb, it succeeds. When I > > > > > run it > > > > > under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing > > > > > with > > > > > windows error 87. > > > > > > > > Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, > > > > perhaps? > > > > > > I tried removing them, and I got the same error. I also tried removing > > > static, and I tried removing both static and const. > > > > BTW, when I reported that the test case succeeds under gdb, that only > > happens when I build the test case without optimization. If I build with > > -O2, it fails under gdb also. [In all my tests, I built cygwin1.dll without > > optimization.] This makes no sense to me at all. > > Good hint. I found the culprit. With optimization, the code doesn't > set the "Reserved" bits in the first struct of MEM_EXTENDED_PARAMETER > to 0. No, wait. I get what you say. The optimzation settings of the test case should have no influence on the code inside the DLL. That doesn't make sense for sure. However, I ran the testcase under GDB, I could reproduce the issue, and I could fix it by setting mmap_ext.Reserved = 0; Go figure! Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On Sep 6 13:38, Ken Brown via Cygwin wrote: > On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: > > On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: > > > On Sep 5 09:24, Ken Brown via Cygwin wrote: > > > > On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: > > > > > On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: > > > > > > Here are the correct commits: > > > > > > > > > > > > 8169e39ab Cygwin: C++17: register keyword is deprecated > > > > > > 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 > > > > > > bdb7991db Cygwin: workaround a g++ 11.2 initialization bug > > > > > > 801120c1f Cygwin: loader script: add DWARF 5 sections > > > > > > d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning > > > > > > c2fe205b5 strstr: avoid warnings > > > > > > 76c2c7a89 ldexp/ldexpf: avoid assembler warning > > > > > > eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString > > > > > > > > > > > > > > > > > > > > > So there appears to be something wrong with cygwin1.dll > > > > > > > > built with the current build tools (gcc 11.2.0, binutils > > > > > > > > 2.37, not sure what else is relevant). > > > > > > > > > > Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER > > > > > initialization problem that was dealt with in commit bdb7991db. > > > > > > > > More data: When I run the test case under gdb, it succeeds. When I run > > > > it > > > > under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing > > > > with > > > > windows error 87. > > > > > > Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, > > > perhaps? > > > > I tried removing them, and I got the same error. I also tried removing > > static, and I tried removing both static and const. > > BTW, when I reported that the test case succeeds under gdb, that only > happens when I build the test case without optimization. If I build with > -O2, it fails under gdb also. [In all my tests, I built cygwin1.dll without > optimization.] This makes no sense to me at all. Good hint. I found the culprit. With optimization, the code doesn't set the "Reserved" bits in the first struct of MEM_EXTENDED_PARAMETER to 0. This is at least required by the VirtualAlloc2 function, though. Needless to say that this behaviour isn't documented... I'll push a patch shortly. Thanks, Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/6/2021 1:38 PM, Ken Brown via Cygwin wrote: On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: On Sep 5 09:24, Ken Brown via Cygwin wrote: On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: Here are the correct commits: 8169e39ab Cygwin: C++17: register keyword is deprecated 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 bdb7991db Cygwin: workaround a g++ 11.2 initialization bug 801120c1f Cygwin: loader script: add DWARF 5 sections d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning c2fe205b5 strstr: avoid warnings 76c2c7a89 ldexp/ldexpf: avoid assembler warning eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString So there appears to be something wrong with cygwin1.dll built with the current build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant). Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER initialization problem that was dealt with in commit bdb7991db. More data: When I run the test case under gdb, it succeeds. When I run it under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with windows error 87. Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, perhaps? I tried removing them, and I got the same error. I also tried removing static, and I tried removing both static and const. BTW, when I reported that the test case succeeds under gdb, that only happens when I build the test case without optimization. If I build with -O2, it fails under gdb also. [In all my tests, I built cygwin1.dll without optimization.] This makes no sense to me at all. There can be a number of possibilities, but I wonder about a variable uninitialized along some path. By accident, the contents may have a non-failing value with -O0 by the situation can be different for -O2. If you're dealing with concurrency and such, then -O0 and -O2 can act differently with respect to races. Just some thoughts ... Best - Eliot Moss -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: On Sep 5 09:24, Ken Brown via Cygwin wrote: On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: Here are the correct commits: 8169e39ab Cygwin: C++17: register keyword is deprecated 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 bdb7991db Cygwin: workaround a g++ 11.2 initialization bug 801120c1f Cygwin: loader script: add DWARF 5 sections d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning c2fe205b5 strstr: avoid warnings 76c2c7a89 ldexp/ldexpf: avoid assembler warning eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString So there appears to be something wrong with cygwin1.dll built with the current build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant). Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER initialization problem that was dealt with in commit bdb7991db. More data: When I run the test case under gdb, it succeeds. When I run it under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with windows error 87. Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, perhaps? I tried removing them, and I got the same error. I also tried removing static, and I tried removing both static and const. BTW, when I reported that the test case succeeds under gdb, that only happens when I build the test case without optimization. If I build with -O2, it fails under gdb also. [In all my tests, I built cygwin1.dll without optimization.] This makes no sense to me at all. Ken -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: On Sep 5 09:24, Ken Brown via Cygwin wrote: On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: Here are the correct commits: 8169e39ab Cygwin: C++17: register keyword is deprecated 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 bdb7991db Cygwin: workaround a g++ 11.2 initialization bug 801120c1f Cygwin: loader script: add DWARF 5 sections d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning c2fe205b5 strstr: avoid warnings 76c2c7a89 ldexp/ldexpf: avoid assembler warning eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString So there appears to be something wrong with cygwin1.dll built with the current build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant). Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER initialization problem that was dealt with in commit bdb7991db. More data: When I run the test case under gdb, it succeeds. When I run it under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with windows error 87. Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, perhaps? I tried removing them, and I got the same error. I also tried removing static, and I tried removing both static and const. Ken -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On Sep 5 09:24, Ken Brown via Cygwin wrote: > On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: > > On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: > > > Here are the correct commits: > > > > > > 8169e39ab Cygwin: C++17: register keyword is deprecated > > > 3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2 > > > bdb7991db Cygwin: workaround a g++ 11.2 initialization bug > > > 801120c1f Cygwin: loader script: add DWARF 5 sections > > > d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning > > > c2fe205b5 strstr: avoid warnings > > > 76c2c7a89 ldexp/ldexpf: avoid assembler warning > > > eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString > > > > > > > > > > > > So there appears to be something wrong with cygwin1.dll > > > > > built with the current build tools (gcc 11.2.0, binutils > > > > > 2.37, not sure what else is relevant). > > > > Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER > > initialization problem that was dealt with in commit bdb7991db. > > More data: When I run the test case under gdb, it succeeds. When I run it > under strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with > windows error 87. Are the const's I added to the MEM_EXTENDED_PARAMETER data invalid, perhaps? Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote:
On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote:
On 9/4/2021 6:54 PM, Ken Brown via Cygwin wrote:
On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote:
I've reduced the procps failure to the following test case:
$ cat mmap_test.c
#include
#include
#include
int
main ()
{
void *addr;
int page_size = getpagesize ();
addr = mmap (0, page_size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (addr == MAP_FAILED)
perror ("mmap");
}
$ gcc mmap_test.c
$ ./a
mmap: Invalid argument
This happens if I use cygwin1.dll built from the current git master. But it
also happens if I build from cygwin-3_2_0-release with the recent patches
applied that make Cygwin build without error:
0e12b4939 Cygwin: C++17: register keyword is deprecated
a7340e0c0 Cygwin: dumper: fix up GCC pragma for g++ 11.2
2a212c086 Cygwin: workaround a g++ 11.2 initialization bug
9e3f1737e Cygwin: loader script: add DWARF 5 sections
bdfd2b004 Cygwin: testsuite: avoid "conflicting types" gcc warning
6fc498e2e strstr: avoid warnings
26da270b2 ldexp/ldexpf: avoid assembler warning
edce2a557 Cygwin: fix declaration of RtlInitEmptyUnicodeString
Sorry, those commit ids are wrong. They're what I got after rebasing the
master branch so that those commits come first.
[Pressed Send too soon.]
Here are the correct commits:
8169e39ab Cygwin: C++17: register keyword is deprecated
3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2
bdb7991db Cygwin: workaround a g++ 11.2 initialization bug
801120c1f Cygwin: loader script: add DWARF 5 sections
d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning
c2fe205b5 strstr: avoid warnings
76c2c7a89 ldexp/ldexpf: avoid assembler warning
eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString
So there appears to be something wrong with cygwin1.dll built with the
current build tools (gcc 11.2.0, binutils 2.37, not sure what else is
relevant).
Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER
initialization problem that was dealt with in commit bdb7991db.
More data: When I run the test case under gdb, it succeeds. When I run it under
strace, I see VirtualAlloc2 in fhandler_dev_zero::mmap failing with windows
error 87.
Ken
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote:
On 9/4/2021 6:54 PM, Ken Brown via Cygwin wrote:
On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote:
I've reduced the procps failure to the following test case:
$ cat mmap_test.c
#include
#include
#include
int
main ()
{
void *addr;
int page_size = getpagesize ();
addr = mmap (0, page_size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (addr == MAP_FAILED)
perror ("mmap");
}
$ gcc mmap_test.c
$ ./a
mmap: Invalid argument
This happens if I use cygwin1.dll built from the current git master. But it
also happens if I build from cygwin-3_2_0-release with the recent patches
applied that make Cygwin build without error:
0e12b4939 Cygwin: C++17: register keyword is deprecated
a7340e0c0 Cygwin: dumper: fix up GCC pragma for g++ 11.2
2a212c086 Cygwin: workaround a g++ 11.2 initialization bug
9e3f1737e Cygwin: loader script: add DWARF 5 sections
bdfd2b004 Cygwin: testsuite: avoid "conflicting types" gcc warning
6fc498e2e strstr: avoid warnings
26da270b2 ldexp/ldexpf: avoid assembler warning
edce2a557 Cygwin: fix declaration of RtlInitEmptyUnicodeString
Sorry, those commit ids are wrong. They're what I got after rebasing the
master branch so that those commits come first.
[Pressed Send too soon.]
Here are the correct commits:
8169e39ab Cygwin: C++17: register keyword is deprecated
3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2
bdb7991db Cygwin: workaround a g++ 11.2 initialization bug
801120c1f Cygwin: loader script: add DWARF 5 sections
d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning
c2fe205b5 strstr: avoid warnings
76c2c7a89 ldexp/ldexpf: avoid assembler warning
eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString
So there appears to be something wrong with cygwin1.dll built with the
current build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant).
Wait a minute...I'll bet this is related to the MEM_EXTENDED_PARAMETER
initialization problem that was dealt with in commit bdb7991db.
Ken
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 2021-09-04 16:37, Ken Brown via Cygwin wrote:
I've reduced the procps failure to the following test case:
$ cat mmap_test.c
#include
#include
#include
int
main ()
{
void *addr;
int page_size = getpagesize ();
addr = mmap (0, page_size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (addr == MAP_FAILED)
perror ("mmap");
}
$ gcc mmap_test.c
$ ./a
mmap: Invalid argument
Check on Linux and retry on Cygwin without MAP_PRIVATE?
MAP_PRIVATE implies an fd but MAP_ANON does not, and MAP_ANON is in
Linux(2)/Unix? not POSIX(3p).
--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada
This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/4/2021 6:54 PM, Ken Brown via Cygwin wrote:
On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote:
I've reduced the procps failure to the following test case:
$ cat mmap_test.c
#include
#include
#include
int
main ()
{
void *addr;
int page_size = getpagesize ();
addr = mmap (0, page_size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (addr == MAP_FAILED)
perror ("mmap");
}
$ gcc mmap_test.c
$ ./a
mmap: Invalid argument
This happens if I use cygwin1.dll built from the current git master. But it
also happens if I build from cygwin-3_2_0-release with the recent patches
applied that make Cygwin build without error:
0e12b4939 Cygwin: C++17: register keyword is deprecated
a7340e0c0 Cygwin: dumper: fix up GCC pragma for g++ 11.2
2a212c086 Cygwin: workaround a g++ 11.2 initialization bug
9e3f1737e Cygwin: loader script: add DWARF 5 sections
bdfd2b004 Cygwin: testsuite: avoid "conflicting types" gcc warning
6fc498e2e strstr: avoid warnings
26da270b2 ldexp/ldexpf: avoid assembler warning
edce2a557 Cygwin: fix declaration of RtlInitEmptyUnicodeString
Sorry, those commit ids are wrong. They're what I got after rebasing the master
branch so that those commits come first.
[Pressed Send too soon.]
Here are the correct commits:
8169e39ab Cygwin: C++17: register keyword is deprecated
3ca80b360 Cygwin: dumper: fix up GCC pragma for g++ 11.2
bdb7991db Cygwin: workaround a g++ 11.2 initialization bug
801120c1f Cygwin: loader script: add DWARF 5 sections
d5cc66426 Cygwin: testsuite: avoid "conflicting types" gcc warning
c2fe205b5 strstr: avoid warnings
76c2c7a89 ldexp/ldexpf: avoid assembler warning
eeeb5650c Cygwin: fix declaration of RtlInitEmptyUnicodeString
So there appears to be something wrong with cygwin1.dll built with the current
build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant).
Ken
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]
On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote:
I've reduced the procps failure to the following test case:
$ cat mmap_test.c
#include
#include
#include
int
main ()
{
void *addr;
int page_size = getpagesize ();
addr = mmap (0, page_size, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (addr == MAP_FAILED)
perror ("mmap");
}
$ gcc mmap_test.c
$ ./a
mmap: Invalid argument
This happens if I use cygwin1.dll built from the current git master. But it
also happens if I build from cygwin-3_2_0-release with the recent patches
applied that make Cygwin build without error:
0e12b4939 Cygwin: C++17: register keyword is deprecated
a7340e0c0 Cygwin: dumper: fix up GCC pragma for g++ 11.2
2a212c086 Cygwin: workaround a g++ 11.2 initialization bug
9e3f1737e Cygwin: loader script: add DWARF 5 sections
bdfd2b004 Cygwin: testsuite: avoid "conflicting types" gcc warning
6fc498e2e strstr: avoid warnings
26da270b2 ldexp/ldexpf: avoid assembler warning
edce2a557 Cygwin: fix declaration of RtlInitEmptyUnicodeString
Sorry, those commit ids are wrong. They're what I got after rebasing the master
branch so that those commits come first.
So there appears to be something wrong with cygwin1.dll built with the current
build tools (gcc 11.2.0, binutils 2.37, not sure what else is relevant).
Ken
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
