Efficient posix_spawn implementation for Cygwin
posix_spawn [1] is an optional POSIX facility that allows programs to start other programs without using fork or vfork. I've created an efficient implementation of posix_spawn for Cygwin. The code is available at https://github.com/dcolascione/cygspawn. This library uses Cygwin's nonstandard spawn* family calls in process.h to provide the full suite of posix_spawn* functionality. Implementing posix_spawn in terms of spawn turns out to be non-trivial. Nevertheless, using posix_spawn can improve process startup performance considerably, especially for large programs: + env ITER=1000 JUNKBYTES=209715200 ./testspawn true real0m5.179s user0m1.550s sys 0m2.661s + env ITER=1000 JUNKBYTES=209715200 ./testfork true real1m57.523s user0m3.049s sys 1m52.339s (JUNKBYTES here tells both programs to malloc 200MB of private data before doing anything else; this allocation simulates the working set of a large program.) Hopefully, it will be more feasible to add posix_spawn support to other projects than to add Cygwin-specific cases for direct calls to spawn. posix_spawn is natively supported on OS X, and gnulib provides an implementation in terms of fork or vfork for all other common platforms. One example: Emacs is an example of a large program that routinely runs subprocesses. In my case, fork latency was so great that I had to disable Emacs' support for version control integration: I didn't want to wait for two seconds after opening a new file while Emacs queried the . I'm working on adding support to posix_spawn to Emacs [2]; when it works, I'll be able to use the built-in version control integration under Cygwin just like I do on other Unix systems. [1] http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html [2] https://github.com/dcolascione/emacs/tree/cygw32 -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Efficient posix_spawn implementation for Cygwin
On 03/01/2012 5:25 PM, dan...@dancol.org wrote: posix_spawn [1] is an optional POSIX facility that allows programs to start other programs without using fork or vfork. I've created an efficient implementation of posix_spawn for Cygwin. The code is available at https://github.com/dcolascione/cygspawn. This library uses Cygwin's nonstandard spawn* family calls in process.h to provide the full suite of posix_spawn* functionality. Implementing posix_spawn in terms of spawn turns out to be non-trivial. Nevertheless, using posix_spawn can improve process startup performance considerably, especially for large programs: This probably belongs at cygwin-dev, but it sounds interesting (especially if it allows less-frequent invocation of the rebaseall ritual). However, you should check out the copyright assignment requirements [2] if you want the code to make it upstream. [2] http://cygwin.com/contrib.html Regards, Ryan -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Efficient posix_spawn implementation for Cygwin
On 1/3/2012 2:43 PM, Ryan Johnson wrote: On 03/01/2012 5:25 PM, dan...@dancol.org wrote: posix_spawn [1] is an optional POSIX facility that allows programs to start other programs without using fork or vfork. I've created an efficient implementation of posix_spawn for Cygwin. The code is available at https://github.com/dcolascione/cygspawn. This library uses Cygwin's nonstandard spawn* family calls in process.h to provide the full suite of posix_spawn* functionality. Implementing posix_spawn in terms of spawn turns out to be non-trivial. Nevertheless, using posix_spawn can improve process startup performance considerably, especially for large programs: This probably belongs at cygwin-dev, but it sounds interesting (especially if it allows less-frequent invocation of the rebaseall ritual). This code is a user library. It doesn't require anything not already in cygwin1.dll. While it'd be _better_ to integrate this functionality into the core, of course, doing so isn't required to make the feature work: that's why I posted to this list. However, you should check out the copyright assignment requirements [2] if you want the code to make it upstream. Of course I'll assign copyright if the code makes it upstream in some form. Thanks, Daniel Colascione -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Efficient posix_spawn implementation for Cygwin
On Tue, Jan 03, 2012 at 05:43:30PM -0500, Ryan Johnson wrote: On 03/01/2012 5:25 PM, dan...@dancol.org wrote: posix_spawn [1] is an optional POSIX facility that allows programs to start other programs without using fork or vfork. I've created an efficient implementation of posix_spawn for Cygwin. The code is available at https://github.com/dcolascione/cygspawn. This library uses Cygwin's nonstandard spawn* family calls in process.h to provide the full suite of posix_spawn* functionality. Implementing posix_spawn in terms of spawn turns out to be non-trivial. Nevertheless, using posix_spawn can improve process startup performance considerably, especially for large programs: This probably belongs at cygwin-dev, No, it really doesn't. It is just a library that uses cygwin functions. but it sounds interesting (especially if it allows less-frequent invocation of the rebaseall ritual). Since the VAST majority of UNIX/linux programs use fork/exec I don't see how this would really have much of an effect. cgf -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: Efficient posix_spawn implementation for Cygwin
On 1/3/2012 4:20 PM, Christopher Faylor wrote: but it sounds interesting (especially if it allows less-frequent invocation of the rebaseall ritual). Since the VAST majority of UNIX/linux programs use fork/exec I don't see how this would really have much of an effect. The idea is to add upstream support for posix_spawn when it would be useful. Because posix_spawn is a standardized facility, because other systems usefully implement it too, and because posix_spawn can be implemented in terms of fork for systems that lack posix_spawn (with gnulib providing such an implementation), adding this support might be easier than adding explicit support for Cygwin's spawn. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
