Re: Windows 8 Release Preview. fork problems with rsync

[Larry Hall (Cygwin)]

On 6/21/2012 9:30 AM, Assaf wrote:

Hi,

I'm also experiencing the following error on a Windows 2012

I can't get my Python paramiko SSHClient to connect and execute remote commands

Sometimes its ok, sometimes it requires me to restart the connection

 From ssh log:


   10349 [main] sshd 732 open_stackdumpfile: Dumping stack trace to
sshd.exe.stackdump
   2 [main] sshd 2320 fork: child -1 - forked process died
unexpectedly, retry 0, exit code 256, errno 11
   4 [main] sshd 2604 fhandler_dev_zero::fixup_mmap_after_fork:
requested 0xFE13 != 0x0 mem alloc base 0x0, state 0x1, size
851968, Win32 error 487
3798 [main] sshd 2604 C:\cygwin\usr\sbin\sshd.exe: *** fatal error
in forked process - recreate_mmaps_after_fork_failed
7578 [main] sshd 2604 open_stackdumpfile: Dumping stack trace to
sshd.exe.stackdump
   2 [main] sshd 2956 fork: child -1 - forked process died
unexpectedly, retry 0, exit code 256, errno 11


We are running Windows 2012 RC1

Any advice is appreciated


?


--
Larry

_

A: Yes.
> Q: Are you sure?
>> A: Because it reverses the logical flow of conversation.
>>> Q: Why is top posting annoying in email?

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Assaf]

Hi,

I'm also experiencing the following error on a Windows 2012

I can't get my Python paramiko SSHClient to connect and execute remote commands

Sometimes its ok, sometimes it requires me to restart the connection

From ssh log:


  10349 [main] sshd 732 open_stackdumpfile: Dumping stack trace to
sshd.exe.stackdump
      2 [main] sshd 2320 fork: child -1 - forked process died
unexpectedly, retry 0, exit code 256, errno 11
      4 [main] sshd 2604 fhandler_dev_zero::fixup_mmap_after_fork:
requested 0xFE13 != 0x0 mem alloc base 0x0, state 0x1, size
851968, Win32 error 487
   3798 [main] sshd 2604 C:\cygwin\usr\sbin\sshd.exe: *** fatal error
in forked process - recreate_mmaps_after_fork_failed
   7578 [main] sshd 2604 open_stackdumpfile: Dumping stack trace to
sshd.exe.stackdump
      2 [main] sshd 2956 fork: child -1 - forked process died
unexpectedly, retry 0, exit code 256, errno 11


We are running Windows 2012 RC1

Any advice is appreciated

Thanks

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Corinna Vinschen]

On Jun  7 10:35, Ryan Johnson wrote:
> On 06/06/2012 12:12 PM, Corinna Vinschen wrote:
> >On Jun  6 16:41, Bertrand Latinville wrote:
> >>I'm using
> >>
> >>  rsync --chmod=ug=rwX -arvz --prune-empty-dirs  --include="*/"
> >>--include-from=include-file.txt  --exclude="*" ${source_dir}/
> >>${dest_dir}
> >Thanks.  I can confirm the effect.  For no apparent reason, the OS
> >reserves a 1 Megs shared memory region, top-down allocated, of which it
> >uses about 20K.  It's not the PEB or one of the TEBs, though.  Nor is
> >it a thread stack.  I checked, and it turns out that it's allocated
> >in every process, on 32 and 64 bit systems.  That's kind of worrying
> >since that's bound to collide with mmaped regions and pthread stacks a
> >lot.  I don't know what to do at this point.
> Given that the OS always gets there first, why not just adjust
> Cygwin's definition of "top" for win8? Or does heap randomization
> move that mystery chunk around?

Yes, the chunk is moved around.  Thousands of tests show a lowest
memory slot, but the problem is that top-down allocation is not a
manual functionality done by Cygwin.  It's just a flag in calls
to VirtualAlloc or MapViewOfFileEx.  Therefore, if we have to define
our own highest memory slot, it requires Cygwin to do some trickery
on its own, which was just not necessary so far.

I'm going to ask Microsoft about this issue first.  Maybe it's 
something only in the release preview which will go away in the
gold release.  If so, I won't start to workaround it.


Corinna


-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader  cygwin AT cygwin DOT com
Red Hat

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Corinna Vinschen]

On Jun  7 10:12, Noel Grandin wrote:
> 
> 
> On 2012-06-06 20:29, Corinna Vinschen wrote:
> 
> >On Jun  6 17:59, Nick Lowe wrote:
> >>>Thanks.  I can confirm the effect.  For no apparent reason, the OS
> >>>reserves a 1 Megs shared memory region, top-down allocated, of which it
> >>>uses about 20K.  It's not the PEB or one of the TEBs, though.  Nor is
> >>>it a thread stack.  I checked, and it turns out that it's allocated
> >>>in every process, on 32 and 64 bit systems.  That's kind of worrying
> >>>since that's bound to collide with mmaped regions and pthread stacks a
> >>>lot.  I don't know what to do at this point.
> >
> 
> 
> 
> I assume you've seem the VMMap tool?
> http://technet.microsoft.com/en-us/sysinternals/dd535533.aspx

Yes, and I also know how to use /proc/$PID/maps to get a memory map
of a process.  It's not a problem to see the used memory slot.  The
problem is that it exists at all.  There's no such thing on pre-W8
systems.  Only the PEB and the TEBs are allocated top-down and they
usually don't collide.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader  cygwin AT cygwin DOT com
Red Hat

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Ryan Johnson]

On 06/06/2012 12:12 PM, Corinna Vinschen wrote:

On Jun  6 16:41, Bertrand Latinville wrote:

I'm using

  rsync --chmod=ug=rwX -arvz --prune-empty-dirs  --include="*/"
--include-from=include-file.txt  --exclude="*" ${source_dir}/
${dest_dir}

Thanks.  I can confirm the effect.  For no apparent reason, the OS
reserves a 1 Megs shared memory region, top-down allocated, of which it
uses about 20K.  It's not the PEB or one of the TEBs, though.  Nor is
it a thread stack.  I checked, and it turns out that it's allocated
in every process, on 32 and 64 bit systems.  That's kind of worrying
since that's bound to collide with mmaped regions and pthread stacks a
lot.  I don't know what to do at this point.
Given that the OS always gets there first, why not just adjust Cygwin's 
definition of "top" for win8? Or does heap randomization move that 
mystery chunk around?


Ryan


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Noel Grandin]

On 2012-06-06 20:29, Corinna Vinschen wrote:


On Jun  6 17:59, Nick Lowe wrote:

Thanks.  I can confirm the effect.  For no apparent reason, the OS
reserves a 1 Megs shared memory region, top-down allocated, of which it
uses about 20K.  It's not the PEB or one of the TEBs, though.  Nor is
it a thread stack.  I checked, and it turns out that it's allocated
in every process, on 32 and 64 bit systems.  That's kind of worrying
since that's bound to collide with mmaped regions and pthread stacks a
lot.  I don't know what to do at this point.






I assume you've seem the VMMap tool?
http://technet.microsoft.com/en-us/sysinternals/dd535533.aspx



--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Corinna Vinschen]

Please, don't http://cygwin.com/acronyms/#TOFU 
Thank you.

On Jun  6 17:59, Nick Lowe wrote:
> > Thanks.  I can confirm the effect.  For no apparent reason, the OS
> > reserves a 1 Megs shared memory region, top-down allocated, of which it
> > uses about 20K.  It's not the PEB or one of the TEBs, though.  Nor is
> > it a thread stack.  I checked, and it turns out that it's allocated
> > in every process, on 32 and 64 bit systems.  That's kind of worrying
> > since that's bound to collide with mmaped regions and pthread stacks a
> > lot.  I don't know what to do at this point.
> 
> Urgh! Hmm.. Do you see the same effect when running the process in
> question under the Windows 8 operating system context?
> 
> If you manifest and include:
> 
> 
>   
>  
> 
> 
> 
> 
> 
>   
>   
> http://msdn.microsoft.com/en-us/library/windows/desktop/hh848036(v=vs.85).aspx
> 
> Does the odd behaviour still persist?

Yes.

> Could it be possibly be something to do with the fault tolerant heap
> (FTH) and changes they've made to it? I would try ensuring that is
> switched off too...

That's not the problem either.  This problem also occurs on the 2012
server which doesn't run FTH.

But thanks for any idea.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader  cygwin AT cygwin DOT com
Red Hat

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Nick Lowe]

Urgh! Hmm.. Do you see the same effect when running the process in
question under the Windows 8 operating system context?

If you manifest and include:


  
 





  
  
http://msdn.microsoft.com/en-us/library/windows/desktop/hh848036(v=vs.85).aspx

Does the odd behaviour still persist?

Could it be possibly be something to do with the fault tolerant heap
(FTH) and changes they've made to it? I would try ensuring that is
switched off too...

(Process Hacker will show the context of a running process.)

Nick

> Thanks.  I can confirm the effect.  For no apparent reason, the OS
> reserves a 1 Megs shared memory region, top-down allocated, of which it
> uses about 20K.  It's not the PEB or one of the TEBs, though.  Nor is
> it a thread stack.  I checked, and it turns out that it's allocated
> in every process, on 32 and 64 bit systems.  That's kind of worrying
> since that's bound to collide with mmaped regions and pthread stacks a
> lot.  I don't know what to do at this point.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Corinna Vinschen]

On Jun  6 16:41, Bertrand Latinville wrote:
> I'm using
> 
>  rsync --chmod=ug=rwX -arvz --prune-empty-dirs  --include="*/"
> --include-from=include-file.txt  --exclude="*" ${source_dir}/
> ${dest_dir}

Thanks.  I can confirm the effect.  For no apparent reason, the OS
reserves a 1 Megs shared memory region, top-down allocated, of which it
uses about 20K.  It's not the PEB or one of the TEBs, though.  Nor is
it a thread stack.  I checked, and it turns out that it's allocated
in every process, on 32 and 64 bit systems.  That's kind of worrying
since that's bound to collide with mmaped regions and pthread stacks a
lot.  I don't know what to do at this point.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader  cygwin AT cygwin DOT com
Red Hat

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Bertrand Latinville]

I'm using

 rsync --chmod=ug=rwX -arvz --prune-empty-dirs  --include="*/"
--include-from=include-file.txt  --exclude="*" ${source_dir}/
${dest_dir}

The include list from include-file.txt is up to four lines with some
wildcard like
folder1/folder2/*.dll
folder3/**


2012/6/6 Corinna Vinschen
> On Jun  6 12:57, Bertrand Latinville wrote:
>> Hello,
>>
>> I'm using cygwin 1.7.75 on Windows 8 Release Preview 64 bits.
>> Bash scripts are launched via jenkins.
>>
>> I'm getting some problems with rsync giving fork errors.
>> This is not systematic,this happens once over 6 times.
>>
>> building file list ... done
>>      1 [main] rsync 1728 fhandler_dev_zero::fixup_mmap_after_fork:
>> requested 0xFED0 != 0x0 mem alloc base 0x0, state 0x1, size
>> 262144, Win32 error 487
>
> Apparently rsync calls mmap to create an anonymous mapping.  mmap
> memory areas are allocated top down.  Also top-down allocated are
> the OS areas which correspond to the PEB and the TEBs.  The above
> memory collision is probably one where the mmap area collides with
> a TEB or PEB.  I'm testing on W* as well and I'm running a W2012
> server all the time since last Friday, but I didn't observe this
> problem yet.  I also can't reproduce it with some arbitrary rsync -a
> command.  What rsync command are you using?
>
>

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Windows 8 Release Preview. fork problems with rsync

[Corinna Vinschen]

On Jun  6 12:57, Bertrand Latinville wrote:
> Hello,
> 
> I'm using cygwin 1.7.75 on Windows 8 Release Preview 64 bits.
> Bash scripts are launched via jenkins.
> 
> I'm getting some problems with rsync giving fork errors.
> This is not systematic,this happens once over 6 times.
> 
> building file list ... done
>      1 [main] rsync 1728 fhandler_dev_zero::fixup_mmap_after_fork:
> requested 0xFED0 != 0x0 mem alloc base 0x0, state 0x1, size
> 262144, Win32 error 487

Apparently rsync calls mmap to create an anonymous mapping.  mmap
memory areas are allocated top down.  Also top-down allocated are
the OS areas which correspond to the PEB and the TEBs.  The above
memory collision is probably one where the mmap area collides with
a TEB or PEB.  I'm testing on W* as well and I'm running a W2012
server all the time since last Friday, but I didn't observe this
problem yet.  I also can't reproduce it with some arbitrary rsync -a
command.  What rsync command are you using?


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader  cygwin AT cygwin DOT com
Red Hat

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple