Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-28 Thread Christopher Faylor
It's time to move this discussion out of cygwin-patches and into cygwin-developers. I'd appreciate it if people continued over there. cgf

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-28 Thread Warren Young
On 9/28/2010 9:10 AM, Christopher Faylor wrote: It isn't extremely surprising that Linux access speed for a filesystem in a simulated environment, which presumably does not go through multiple layers of DLLs, would be faster than Cygwin. I think it more likely that the HGFS driver doesn't try t

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-28 Thread Christopher Faylor
On Tue, Sep 28, 2010 at 08:56:18AM -0600, Warren Young wrote: >On 9/22/2010 7:44 AM, Christopher Faylor wrote: >> >> What is /mnt/hgfs/C in this case? How is it mounted? > >HGFS is the Host-Guest File System, a VMware technology that lets it >export host volumes to the guest in a high-speed way.

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-28 Thread Warren Young
On 9/22/2010 7:44 AM, Christopher Faylor wrote: What is /mnt/hgfs/C in this case? How is it mounted? HGFS is the Host-Guest File System, a VMware technology that lets it export host volumes to the guest in a high-speed way. If you used old versions of VMware Workstation for Linux, you may

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-27 Thread Yoni Londner
Hi, > What is /mnt/hgfs/C in this case? How is it mounted? .host:/ on /mnt/hgfs type vmhgfs (rw,ttl=5) I am now preparing an easy to use scandir/stat performance testing program that will perform it in various methods and printout comparison results in an easy to use manner. It will compare c

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-22 Thread Christopher Faylor
On Wed, Sep 22, 2010 at 07:50:14AM +0200, Yoni Londner wrote: >Hi, > > > I'm not exactly concerned about Linux being way faster accessing an NTFS > > drive. After all it's the OS itself and comes with it's own NTFS driver > > which obviously is streamlined for typical POSIX operations. > >I did no

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-22 Thread Corinna Vinschen
On Sep 22 11:32, Corinna Vinschen wrote: > On Sep 22 07:45, Yoni Londner wrote: > > I checked out why, and found out that #1 and #2 don't modify the > > access time of the file, whereas #3 does. This already immediately > > I just checked this and I can't see that it does. If it would do > so, sh

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-22 Thread Corinna Vinschen
On Sep 22 07:45, Yoni Londner wrote: > Hi, > > > There's also the problem of handling NFS shares. However, I just had an > > idea how to speed up symlink_info::check without neglecting NFS shares. > > This will take some time, though since it turns a lot of code upside > > down. Stay tuned. > >

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-21 Thread Yoni Londner
Hi, > I'm not exactly concerned about Linux being way faster accessing an NTFS > drive. After all it's the OS itself and comes with it's own NTFS driver > which obviously is streamlined for typical POSIX operations. I did not test & compare to using the Linux NTFS, rather I compared with Linux

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-21 Thread Yoni Londner
Hi, > There's also the problem of handling NFS shares. However, I just had an > idea how to speed up symlink_info::check without neglecting NFS shares. > This will take some time, though since it turns a lot of code upside > down. Stay tuned. This sounds great! Cygwin filesystem performance is

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-14 Thread Corinna Vinschen
On Sep 13 13:28, Yoni Londner wrote: > Hi, > > > However, isn't that kind of a chicken/egg situation? If you want to > > reuse the content of the FILE_BOTH{_ID}_DIRECTORY_INFORMATION structure > > from a previous call to readdir, you would have to call the > > I am not talking about reusing info

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-14 Thread Corinna Vinschen
On Sep 13 13:45, Yoni Londner wrote: > Hi, > > >> Abstract: I prepared a patch that improves Cygwin Filesystem > >> performance by x4 on Cygwin 1.7.5 (1.7.5 vanilla 530ms --> 1.7.5 > >> patched 120ms). I ported the patch to 1.7.7, did tests, and found > >> out that 1.7.7 had a very serious 9x (!)

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-13 Thread Yoni Londner
Hi, > However, isn't that kind of a chicken/egg situation? If you want to > reuse the content of the FILE_BOTH{_ID}_DIRECTORY_INFORMATION structure > from a previous call to readdir, you would have to call the I am not talking about reusing info from a previous readdir. Every single file cygwi

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-12 Thread Corinna Vinschen
On Sep 12 13:41, Corinna Vinschen wrote: > On Sep 12 10:49, Yoni Londner wrote: > > Hi, > > > > The caching-speed up is trivial: > > We store the the FileFullDirectoryInformation fields, and if any of > > them change - we re-read the file. > > > > Its not (in practical life) possible to change a

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-12 Thread Corinna Vinschen
On Sep 12 10:49, Yoni Londner wrote: > Hi, > > The caching-speed up is trivial: > We store the the FileFullDirectoryInformation fields, and if any of > them change - we re-read the file. > > Its not (in practical life) possible to change a file without > causing a modification on > FileIndex/Cre

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-12 Thread Yoni Londner
Hi, The caching-speed up is trivial: We store the the FileFullDirectoryInformation fields, and if any of them change - we re-read the file. Its not (in practical life) possible to change a file without causing a modification on FileIndex/CreationTime/LastWriteTime/ChangeTime/EndOfFile/Alloca

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-10 Thread Corinna Vinschen
On Sep 10 13:23, Christopher Faylor wrote: > On Fri, Sep 10, 2010 at 05:08:40PM +0200, Corinna Vinschen wrote: > >What I'm still mulling over is a good idea to re-use the results of a > >former call to readdir in a stat call. One problem here is to make sure > >that a subsequent stat call is *real

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-10 Thread Christopher Faylor
On Fri, Sep 10, 2010 at 05:08:40PM +0200, Corinna Vinschen wrote: >What I'm still mulling over is a good idea to re-use the results of a >former call to readdir in a stat call. One problem here is to make sure >that a subsequent stat call is *really* accessing the same file as the >former readdir

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-10 Thread Corinna Vinschen
On Sep 6 15:24, Corinna Vinschen wrote: > > - GetVolumeInfo: The C:\ drive does not tend to be changed every > > millisecond! Therefore no reason for every filesystem syscall to > > call it. Caching this further increased the performance. > > Does your FS caching take volume mount points into acc

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-06 Thread Christopher Faylor
Thanks for the patch and for all of the work you put into it. On Mon, Sep 06, 2010 at 03:24:09PM +0200, Corinna Vinschen wrote: >The patch is also missing a ChangeLog entry. I only took a quick glance >over the patch itself. The code doesn't look correctly formatted in GNU >style. Also, using t

Re: Cygwin Filesystem Performance degradation 1.7.5 vs 1.7.7, and methods for improving performance

2010-09-06 Thread Corinna Vinschen
Hi Yoni, On Sep 6 12:52, Yoni Londner wrote: > Hi, > > Abstract: I prepared a patch that improves Cygwin Filesystem > performance by x4 on Cygwin 1.7.5 (1.7.5 vanilla 530ms --> 1.7.5 > patched 120ms). I ported the patch to 1.7.7, did tests, and found > out that 1.7.7 had a very serious 9x (!) pe