Re: fstat st_size on open files on Parallels filesystem is wrong

2015-11-04 Thread Corinna Vinschen 
On Nov  4 01:35, Jonathan Lennox wrote:
> On Tuesday, November 3 2015, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
> saying:
> 
> > Btw., there's more than one problem here.  The fact that all drives
> > have the same volume name *and* a serial number of 0 leads to all
> > drives being identified as the same drive.  I have to add some code
> > creating a reproducible serial number from scratch if the original
> > serial number is 0, but for testing, I didn't implement this yet.
> 
> I've confirmed that this is the case for all the prlsf drives I have (drives
> U:-Z:).
> 
> > Talking about testing.  I created a test DLL which provides a lot
> > of output when stracing the calls.  I'll send you a private mail
> > with the URL to this DLL in a minute.  Please install it, and under
> > that DLL, run
> > 
> >   $ strace -o stat.trace ./stat-size-test.exe /cygdrive/y/foo
> > 
> > I'll need to have a look into that strace to (hopefully) see what's
> > going wrong.
> 
> Attached.

Thanks, but...  are you sure you created this with the test DLL I sent
you the URL for?  The attached strace looks like stock Cygwin.  It
doesn't contain any of the additional strace output the DLL is supposed
to generate and some of it is in a code path you can't avoid when
calling functions on files.

I'll upload a new DLL to the URL I mentioned in PM in a minute, just to
be sure I didn't screw this up myself.


Thanks,
Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgp5HItX2jiri.pgp
Description: PGP signature

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-11-03 Thread Jonathan Lennox 
On Tuesday, November 3 2015, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
saying:

> Btw., there's more than one problem here.  The fact that all drives
> have the same volume name *and* a serial number of 0 leads to all
> drives being identified as the same drive.  I have to add some code
> creating a reproducible serial number from scratch if the original
> serial number is 0, but for testing, I didn't implement this yet.

I've confirmed that this is the case for all the prlsf drives I have (drives
U:-Z:).

> Talking about testing.  I created a test DLL which provides a lot
> of output when stracing the calls.  I'll send you a private mail
> with the URL to this DLL in a minute.  Please install it, and under
> that DLL, run
> 
>   $ strace -o stat.trace ./stat-size-test.exe /cygdrive/y/foo
> 
> I'll need to have a look into that strace to (hopefully) see what's
> going wrong.

Attached.

--- Process 3620 created
--- Process 3620 loaded C:\Windows\System32\ntdll.dll at 7FFA60F2
--- Process 3620 loaded C:\Windows\System32\kernel32.dll at 7FFA60CC
--- Process 3620 loaded C:\Windows\System32\KernelBase.dll at 7FFA5E41
--- Process 3620 thread 6728 created
--- Process 3620 loaded C:\cygwin64\bin\cygwin1.dll at 00018004
--- Process 3620 thread 3204 created
8   8 [main] stat-size-test (3620) 
**
  407 415 [main] stat-size-test (3620) Program name: 
Y:\Emacs-Modtime\stat-size-test.exe (windows pid 3620)
  285 700 [main] stat-size-test (3620) OS version:   Windows NT-10.0
  295 995 [main] stat-size-test (3620) 
**
  3471342 [main] stat-size-test (3620) sigprocmask: 0 = sigprocmask (0, 
0x0, 0x180301128)
  5881930 [main] stat-size-test 3620 open_shared: name shared.5, n 5, 
shared 0x18003 (wanted 0x18003), h 0xAC, *m 6
  2872217 [main] stat-size-test 3620 user_heap_info::init: heap base 
0x6, heap top 0x6, heap size 0x2000 (536870912)
  2922509 [main] stat-size-test 3620 open_shared: name 
S-1-5-21-4096061939-177889333-3710151321-1394.1, n 1, shared 0x18002 
(wanted 0x18002), h 0xA8, *m 6
  2672776 [main] stat-size-test 3620 user_info::create: opening user shared 
for 'S-1-5-21-4096061939-177889333-3710151321-1394' at 0x18002
  3083084 [main] stat-size-test 3620 user_info::create: user shared version 
AB1FCCE8
  2933377 [main] stat-size-test 3620 fhandler_pipe::create: name 
\\.\pipe\cygwin-e022582115c10879-3620-sigwait, size 11440, mode 
PIPE_TYPE_MESSAGE
  3093686 [main] stat-size-test 3620 fhandler_pipe::create: pipe read 
handle 0xC4
  2683954 [main] stat-size-test 3620 fhandler_pipe::create: CreateFile: 
name \\.\pipe\cygwin-e022582115c10879-3620-sigwait
  2994253 [main] stat-size-test 3620 fhandler_pipe::create: pipe write 
handle 0xC8
  2824535 [main] stat-size-test 3620 dll_crt0_0: finished dll_crt0_0 
initialization
--- Process 3620 thread 444 created
 10515586 [sig] stat-size-test 3620 wait_sig: entering ReadFile loop, 
my_readsig 0xC4, my_sendsig 0xC8
  3385924 [main] stat-size-test 3620 time: 1446618094 = time(0x0)
  5576481 [main] stat-size-test 3620 mount_info::conv_to_posix_path: 
conv_to_posix_path (Y:\Emacs-Modtime, no-keep-rel, no-add-slash)
  3116792 [main] stat-size-test 3620 normalize_win32_path: Y:\Emacs-Modtime 
= normalize_win32_path (Y:\Emacs-Modtime)
  2277019 [main] stat-size-test 3620 mount_info::conv_to_posix_path: 
/cygdrive/y/Emacs-Modtime = conv_to_posix_path (Y:\Emacs-Modtime)
  2987317 [main] stat-size-test 3620 sigprocmask: 0 = sigprocmask (0, 0x0, 
0x600018128)
  5307847 [main] stat-size-test 3620 _cygwin_istext_for_stdio: fd 0: not 
open
  2358082 [main] stat-size-test 3620 _cygwin_istext_for_stdio: fd 1: not 
open
  2178299 [main] stat-size-test 3620 _cygwin_istext_for_stdio: fd 2: not 
open
  3828681 [main] stat-size-test (3620) open_shared: name cygpid.3620, n 
3620, shared 0x18001 (wanted 0x18001), h 0xF0, *m 2
  2328913 [main] stat-size-test (3620) time: 1446618094 = time(0x0)
  2279140 [main] stat-size-test 3620 pinfo::thisproc: myself dwProcessId 
3620
  3769516 [main] stat-size-test 3620 environ_init: GetEnvironmentStrings 
returned 0x3ADB00
  2749790 [main] stat-size-test 3620 environ_init: 0x6000284F0: !::=::\
  296   10086 [main] stat-size-test 3620 environ_init: 0x600028510: 
ALLUSERSPROFILE=C:\ProgramData
  306   10392 [main] stat-size-test 3620 environ_init: 0x600028540: 
APPDATA=C:\Users\jonathan.VIDYOCRM\AppData\Roaming
  299   10691 [main] stat-size-test 3620 environ_init: 0x600028580: 
COMMONPROGRAMFILES=C:\Program Files\Common Files
  297   10988 [main] stat-size-test 3620 environ_init: 0x6000285C0: 
COMPUTERNAME=VIDYO-LT0519-10
  278   11266 [main] stat-size-test 3620 environ_init: 0x6000285F0: 
COMSPEC=C:\Windows\system32\cmd.exe
  275   11541 [main] stat-size-test 3620 e

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-11-03 Thread Corinna Vinschen 
On Nov  2 17:05, Jonathan Lennox wrote:
> On Monday, November 2 2015, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
> saying:
> 
> > On Nov  2 08:08, Jonathan Lennox wrote:
> > > On Monday, November 2 2015, "Corinna Vinschen" wrote to 
> > > "cygwin@cygwin.com" saying:
> > > > I added support for this filesystem (called prlfs in mount output) and
> > > > without hardlink support for now.  I uploaded a new developer snapshot
> > > > to https://cygwin.com/snapshots/ Please give it a try.
> > > 
> > > No, still seeing the failure in the snapshot:
> > > 
> > > $ ./stat-size-test.exe /cygdrive/y/foo ~/foo
> > > /cygdrive/y/foo: fstat: st_size=0
> > > /cygdrive/y/foo: stat: st_size=12
> > > /home/jonathan/foo: fstat: st_size=12
> > > /home/jonathan/foo: stat: st_size=12
> > 
> > Weird.  There should be no FileNetworkOpenInformation call anymore for
> > Netapp and the PrlSF filesystem.
> > 
> > Does Cygwin correctly recognize the FS?  What does `mount' print?  It
> > should print `type prlfs'.
> 
> $ mount
> C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
> C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
> C:/cygwin64 on / type ntfs (binary,auto)
> C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> D: on /cygdrive/d type iso9660 (binary,posix=0,user,noumount,auto)
> E: on /cygdrive/e type iso9660 (binary,posix=0,user,noumount,auto)
> U: on /cygdrive/u type prlsf (binary,posix=0,user,noumount,auto)
> V: on /cygdrive/v type prlsf (binary,posix=0,user,noumount,auto)
> W: on /cygdrive/w type prlsf (binary,posix=0,user,noumount,auto)
> X: on /cygdrive/x type prlsf (binary,posix=0,user,noumount,auto)
> Y: on /cygdrive/y type prlsf (binary,posix=0,user,noumount,auto)
> Z: on /cygdrive/z type prlsf (binary,posix=0,user,noumount,auto)

"prlsf".  If Cygwin had recognized the FS, it would have printed
"prlfs".  I did this shuffle "sf" vs. "fs" on purpose.  But...

> > Can you please once again call `/usr/lib/csih/getVolInfo.exe Z:' and
> > `/usr/lib/csih/getVolInfo.exe Y:' and paste the output here?  I'm not
> > quite sure because the original getVolInfo call returned a filesystem
> > type of "PrlSF", not "PrlFS" as I had expected.  Cygwin now checks for
> > "PrlSF".
> > 
> 
> $ /usr/lib/csih/getVolInfo.exe /cygdrive/z
> Device Type: 7
> Characteristics: 10
> Volume Name: 
> Serial Number  : 0
> Max Filenamelength : 255
> Filesystemname : 
> Flags  : 3
> [...]

...I don't understand *why* Cygwin doesn't recognize the FS.  It checks
explicitely for the "PrlSF" filesystem name.  I tried this locally
by faking the above information and it worked for me.

Btw., there's more than one problem here.  The fact that all drives
have the same volume name *and* a serial number of 0 leads to all
drives being identified as the same drive.  I have to add some code
creating a reproducible serial number from scratch if the original
serial number is 0, but for testing, I didn't implement this yet.

Talking about testing.  I created a test DLL which provides a lot
of output when stracing the calls.  I'll send you a private mail
with the URL to this DLL in a minute.  Please install it, and under
that DLL, run

  $ strace -o stat.trace ./stat-size-test.exe /cygdrive/y/foo

I'll need to have a look into that strace to (hopefully) see what's
going wrong.

>   FILE_READ_ONLY_VOLUME   : FALSE
>   FILE_SEQUENTIAL_WRITE_ONCE  : FALSE
>   FILE_SUPPORTS_TRANSACTIONS  : FALSE
> 
> (Note that the literal "/usr/lib/csih/getVolInfo.exe Z:" printed
> "NtOpenFile(\??\C:\cygwin64\home\jonathan\Z:) failed, c033", so I assume
> that's not what you want.)

Indeed. Sorry, I was thinking of one of my local test hacks using DOS
paths.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpfFlveAq4fK.pgp
Description: PGP signature

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-11-02 Thread Jonathan Lennox 
On Monday, November 2 2015, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
saying:

> On Nov  2 08:08, Jonathan Lennox wrote:
> > On Monday, November 2 2015, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
> > saying:
> > 
> > > On Nov  2 04:38, Jonathan Lennox wrote:
> > > > Unfortunately, when I do "Run As Administrator" on MinTTY, the Mac 
> > > > drives
> > > > (/cygdrive/z and /cygdrive/y) don't show up. I don't know why that is.  
> > > > So I
> > > > can't test hard links as administrator.
> > > 
> > > That's a security feature of UAC.  You can change that in the registry.
> > > As administrator:
> > > 
> > >   regtool -d set 
> > > /HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Policies/System/EnableLinkedConnections
> > >  1
> > > 
> > > Then reboot.
> > 
> > Didn't work:
> > 
> > $ ls /cygdrive/
> > c  d  e
> > 
> > $ regtool get 
> > /HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Policies/System/EnableLinkedConnections
> > 
>  ^^^?
> 
> That should print "1"
> 
> EnableLinkedConnections is a DWORD value and should be set to 1.

Yes, sorry, it does print 1. Copy & Paste error.

Given "cyg Simple"'s comment, I guess I need to recreate the mapped drives as
the Administrator user?

> > > I added support for this filesystem (called prlfs in mount output) and
> > > without hardlink support for now.  I uploaded a new developer snapshot
> > > to https://cygwin.com/snapshots/ Please give it a try.
> > 
> > No, still seeing the failure in the snapshot:
> > 
> > $ ./stat-size-test.exe /cygdrive/y/foo ~/foo
> > /cygdrive/y/foo: fstat: st_size=0
> > /cygdrive/y/foo: stat: st_size=12
> > /home/jonathan/foo: fstat: st_size=12
> > /home/jonathan/foo: stat: st_size=12
> 
> Weird.  There should be no FileNetworkOpenInformation call anymore for
> Netapp and the PrlSF filesystem.
> 
> Does Cygwin correctly recognize the FS?  What does `mount' print?  It
> should print `type prlfs'.

$ mount
C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
C:/cygwin64 on / type ntfs (binary,auto)
C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
D: on /cygdrive/d type iso9660 (binary,posix=0,user,noumount,auto)
E: on /cygdrive/e type iso9660 (binary,posix=0,user,noumount,auto)
U: on /cygdrive/u type prlsf (binary,posix=0,user,noumount,auto)
V: on /cygdrive/v type prlsf (binary,posix=0,user,noumount,auto)
W: on /cygdrive/w type prlsf (binary,posix=0,user,noumount,auto)
X: on /cygdrive/x type prlsf (binary,posix=0,user,noumount,auto)
Y: on /cygdrive/y type prlsf (binary,posix=0,user,noumount,auto)
Z: on /cygdrive/z type prlsf (binary,posix=0,user,noumount,auto)


> Can you please once again call `/usr/lib/csih/getVolInfo.exe Z:' and
> `/usr/lib/csih/getVolInfo.exe Y:' and paste the output here?  I'm not
> quite sure because the original getVolInfo call returned a filesystem
> type of "PrlSF", not "PrlFS" as I had expected.  Cygwin now checks for
> "PrlSF".
> 

$ /usr/lib/csih/getVolInfo.exe /cygdrive/z
Device Type: 7
Characteristics: 10
Volume Name: 
Serial Number  : 0
Max Filenamelength : 255
Filesystemname : 
Flags  : 3
  FILE_CASE_SENSITIVE_SEARCH  : TRUE
  FILE_CASE_PRESERVED_NAMES   : TRUE
  FILE_UNICODE_ON_DISK: FALSE
  FILE_PERSISTENT_ACLS: FALSE
  FILE_FILE_COMPRESSION   : FALSE
  FILE_VOLUME_QUOTAS  : FALSE
  FILE_SUPPORTS_SPARSE_FILES  : FALSE
  FILE_SUPPORTS_REPARSE_POINTS: FALSE
  FILE_SUPPORTS_REMOTE_STORAGE: FALSE
  FILE_VOLUME_IS_COMPRESSED   : FALSE
  FILE_SUPPORTS_OBJECT_IDS: FALSE
  FILE_SUPPORTS_ENCRYPTION: FALSE
  FILE_NAMED_STREAMS  : FALSE
  FILE_READ_ONLY_VOLUME   : FALSE
  FILE_SEQUENTIAL_WRITE_ONCE  : FALSE
  FILE_SUPPORTS_TRANSACTIONS  : FALSE

$ /usr/lib/csih/getVolInfo.exe /cygdrive/y
Device Type: 7
Characteristics: 10
Volume Name: 
Serial Number  : 0
Max Filenamelength : 255
Filesystemname : 
Flags  : 3
  FILE_CASE_SENSITIVE_SEARCH  : TRUE
  FILE_CASE_PRESERVED_NAMES   : TRUE
  FILE_UNICODE_ON_DISK: FALSE
  FILE_PERSISTENT_ACLS: FALSE
  FILE_FILE_COMPRESSION   : FALSE
  FILE_VOLUME_QUOTAS  : FALSE
  FILE_SUPPORTS_SPARSE_FILES  : FALSE
  FILE_SUPPORTS_REPARSE_POINTS: FALSE
  FILE_SUPPORTS_REMOTE_STORAGE: FALSE
  FILE_VOLUME_IS_COMPRESSED   : FALSE
  FILE_SUPPORTS_OBJECT_IDS: FALSE
  FILE_SUPPORTS_ENCRYPTION: FALSE
  FILE_NAMED_STREAMS  : FALSE
  FILE_READ_ONLY_VOLUME   : FALSE
  FILE_SEQUENTIAL_WRITE_ONCE  : FALSE
  FILE_SUPPORTS_TRANSACTIONS  : FALSE

(Note that the literal "/usr/lib/csih/getVolInfo.exe Z:" printed
"NtOpenFile(\??\C:\cygwin64\home\jonathan\Z:) failed, c033", so I assume
that's not what you want.)

-- 
Jonathan Lennox
len...@cs.columbia.edu

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-11-02 Thread Corinna Vinschen 
On Nov  2 08:08, Jonathan Lennox wrote:
> On Monday, November 2 2015, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
> saying:
> 
> > On Nov  2 04:38, Jonathan Lennox wrote:
> > > Unfortunately, when I do "Run As Administrator" on MinTTY, the Mac drives
> > > (/cygdrive/z and /cygdrive/y) don't show up. I don't know why that is.  
> > > So I
> > > can't test hard links as administrator.
> > 
> > That's a security feature of UAC.  You can change that in the registry.
> > As administrator:
> > 
> >   regtool -d set 
> > /HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Policies/System/EnableLinkedConnections
> >  1
> > 
> > Then reboot.
> 
> Didn't work:
> 
> $ ls /cygdrive/
> c  d  e
> 
> $ regtool get 
> /HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Policies/System/EnableLinkedConnections
> 
 ^^^?

That should print "1"

EnableLinkedConnections is a DWORD value and should be set to 1.

> $ uname -a
> CYGWIN_NT-10.0 Vidyo-LT0519-10 2.3.0(0.291/5/3) 2015-11-02 11:15 x86_64 Cygwin
> 
> Is Windows 10 the same?

Yes.  Works for me.

> > I added support for this filesystem (called prlfs in mount output) and
> > without hardlink support for now.  I uploaded a new developer snapshot
> > to https://cygwin.com/snapshots/ Please give it a try.
> 
> No, still seeing the failure in the snapshot:
> 
> $ ./stat-size-test.exe /cygdrive/y/foo ~/foo
> /cygdrive/y/foo: fstat: st_size=0
> /cygdrive/y/foo: stat: st_size=12
> /home/jonathan/foo: fstat: st_size=12
> /home/jonathan/foo: stat: st_size=12

Weird.  There should be no FileNetworkOpenInformation call anymore for
Netapp and the PrlSF filesystem.

Does Cygwin correctly recognize the FS?  What does `mount' print?  It
should print `type prlfs'.

Can you please once again call `/usr/lib/csih/getVolInfo.exe Z:' and
`/usr/lib/csih/getVolInfo.exe Y:' and paste the output here?  I'm not
quite sure because the original getVolInfo call returned a filesystem
type of "PrlSF", not "PrlFS" as I had expected.  Cygwin now checks for
"PrlSF".


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgp08kmGbFY5F.pgp
Description: PGP signature

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-11-02 Thread Jonathan Lennox 
On Monday, November 2 2015, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
saying:

> On Nov  2 04:38, Jonathan Lennox wrote:
> > Unfortunately, when I do "Run As Administrator" on MinTTY, the Mac drives
> > (/cygdrive/z and /cygdrive/y) don't show up. I don't know why that is.  So I
> > can't test hard links as administrator.
> 
> That's a security feature of UAC.  You can change that in the registry.
> As administrator:
> 
>   regtool -d set 
> /HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Policies/System/EnableLinkedConnections
>  1
> 
> Then reboot.

Didn't work:

$ ls /cygdrive/
c  d  e

$ regtool get 
/HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Policies/System/EnableLinkedConnections

$ uname -a
CYGWIN_NT-10.0 Vidyo-LT0519-10 2.3.0(0.291/5/3) 2015-11-02 11:15 x86_64 Cygwin

Is Windows 10 the same?

> I added support for this filesystem (called prlfs in mount output) and
> without hardlink support for now.  I uploaded a new developer snapshot
> to https://cygwin.com/snapshots/ Please give it a try.

No, still seeing the failure in the snapshot:

$ ./stat-size-test.exe /cygdrive/y/foo ~/foo
/cygdrive/y/foo: fstat: st_size=0
/cygdrive/y/foo: stat: st_size=12
/home/jonathan/foo: fstat: st_size=12
/home/jonathan/foo: stat: st_size=12

$ uname -a
CYGWIN_NT-10.0 Vidyo-LT0519-10 2.3.0(0.291/5/3) 2015-11-02 11:15 x86_64 Cygwin

(See https://cygwin.com/ml/cygwin/2014-04/msg00488.html for the source code
to stat-size-test.c).

Oddly, though, the original problematic behavior I saw in emacs (with it
falsely detecting files as being modified externally) has gone away. Maybe
emacs added a workaround for this?

-- 
Jonathan Lennox
len...@cs.columbia.edu

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-11-02 Thread cyg Simple 
On 11/2/2015 4:38 AM, Jonathan Lennox wrote:
> 
> Unfortunately, when I do "Run As Administrator" on MinTTY, the Mac drives
> (/cygdrive/z and /cygdrive/y) don't show up. I don't know why that is.  So I
> can't test hard links as administrator.
> 

This is due to "Run As Administrator" is a different user with a
different environment.  You would need to recreate the mapped drives.

-- 
cyg Simple

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-11-02 Thread Corinna Vinschen 
On Nov  2 04:38, Jonathan Lennox wrote:
> On Wednesday, October 21 2015, "Corinna Vinschen" wrote to 
> "cygwin@cygwin.com" saying:
> > On Oct  8 12:16, Jonathan Lennox wrote:
> > > No such luck, despite two major version revisions of Parallels Desktop 
> > > (I'm
> > > now on version 11.0.2) and moving to Windows 10 as the guest OS -- the bug
> > > perists, unchanged.  So it looks like Cygwin will need to add a workaround
> > > for this filesystem to fix the problem.
> > 
> > Ok, we could do that.  Can you compile and run the testcase from
> > https://cygwin.com/ml/cygwin/2014-04/msg00523.html again?  Does it
> > still show 0 vs. 12 bytes?  Dumb extra test: Does the output change
> > if you reorder the calls, requesting FileStandardInformation first,
> > FileNetworkOpenInformation second?
> 
> I re-ran the test, no change. Changing the order gives the same result --
> FileStandardInformation works, FileNetworkOpenInformation doesn't.

Ok, so it's not about some caching.

> > Just create a hardlink on that drive using native means:
> > 
> >   $ touch foo
> >   $ cmd /c mklink /h bar foo
> > 
> > Error at this point?  No hardlinks.  Otherwise:
> 
> "You do not have sufficient privilege to perform this operation."  Is that
> sufficient proof?

No.

> Unfortunately, when I do "Run As Administrator" on MinTTY, the Mac drives
> (/cygdrive/z and /cygdrive/y) don't show up. I don't know why that is.  So I
> can't test hard links as administrator.

That's a security feature of UAC.  You can change that in the registry.
As administrator:

  regtool -d set 
/HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Windows/CurrentVersion/Policies/System/EnableLinkedConnections
 1

Then reboot.

> >   $ ls -li foo bar
> > 
> > Are the inode numbers identical?  Congrats, hardlinks work.  But given
> > the general FAT-iness of the getVolInfo output, I guess it doesn't
> > maintain hardlinks.
> 
> However, when I create a hardlink on the underlying (Mac) file system, the
> inode numbers that Cygwin shows are not identical.  So "no hardlinks" seems
> very likely.

Given the filesystem flags, Cygwin treats your FS as some kind of FAT
anyway, so this isn't convincing.  I doubt they work anyway, but if you
really want to test it, use your orignal testcase and replace the
NtQueryInformationFile like this:

  FILE_INTERNAL_INFORMATION fii;
  status = NtQueryInformationFile (h, &io, &fii, sizeof fii,
   FileInternalInformation);
  if (!NT_SUCCESS (status))
fprintf (stderr, "NtQueryInformationFile: 0x%08x\n", status);
  else
printf ("inode number %llu\n", fii.FileId.QuadPart);

Then call this application multiple times on some well known hardlinks
to the same file.  If they have the same number, all the time, hardlinks
work.

I added support for this filesystem (called prlfs in mount output) and
without hardlink support for now.  I uploaded a new developer snapshot
to https://cygwin.com/snapshots/ Please give it a try.


Thanks,
Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpEsjMhEKvUb.pgp
Description: PGP signature

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-11-02 Thread Jonathan Lennox 
On Wednesday, October 21 2015, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
saying:

> On Oct  8 12:16, Jonathan Lennox wrote:
> > Hi, following up on this issue from last year.  The message I'm replying to
> > is at .
> > 
> > The problem is weird behavior in Parallels Desktop-hosted Windows VMs, when
> > accessing the host's native Mac OS X filesystem.  See the thread for the
> > details.
> > 
> > On Wednesday, April 23 2014, "Corinna Vinschen" wrote to 
> > "cygwin@cygwin.com" saying:
> > 
> > > > At this point this is looking pretty clearly like a Parallels Tools bug.
> > > > I'll report it to them.
> > > 
> > > Yes, that sounds good.  Given that, I'm wondering if we should try to
> > > workaround this problem at all or rather wait to see if the vendor will
> > > fix the issue.
> > 
> > No such luck, despite two major version revisions of Parallels Desktop (I'm
> > now on version 11.0.2) and moving to Windows 10 as the guest OS -- the bug
> > perists, unchanged.  So it looks like Cygwin will need to add a workaround
> > for this filesystem to fix the problem.
> 
> Ok, we could do that.  Can you compile and run the testcase from
> https://cygwin.com/ml/cygwin/2014-04/msg00523.html again?  Does it
> still show 0 vs. 12 bytes?  Dumb extra test: Does the output change
> if you reorder the calls, requesting FileStandardInformation first,
> FileNetworkOpenInformation second?

I re-ran the test, no change. Changing the order gives the same result --
FileStandardInformation works, FileNetworkOpenInformation doesn't.

> Just create a hardlink on that drive using native means:
> 
>   $ touch foo
>   $ cmd /c mklink /h bar foo
> 
> Error at this point?  No hardlinks.  Otherwise:

"You do not have sufficient privilege to perform this operation."  Is that
sufficient proof?

Unfortunately, when I do "Run As Administrator" on MinTTY, the Mac drives
(/cygdrive/z and /cygdrive/y) don't show up. I don't know why that is.  So I
can't test hard links as administrator.

>   $ ls -li foo bar
> 
> Are the inode numbers identical?  Congrats, hardlinks work.  But given
> the general FAT-iness of the getVolInfo output, I guess it doesn't
> maintain hardlinks.

However, when I create a hardlink on the underlying (Mac) file system, the
inode numbers that Cygwin shows are not identical.  So "no hardlinks" seems
very likely.

-- 
Jonathan Lennox
lennox at cs dot columbia dot edu

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-10-21 Thread Corinna Vinschen 
On Oct  8 12:16, Jonathan Lennox wrote:
> Hi, following up on this issue from last year.  The message I'm replying to
> is at .
> 
> The problem is weird behavior in Parallels Desktop-hosted Windows VMs, when
> accessing the host's native Mac OS X filesystem.  See the thread for the
> details.
> 
> On Wednesday, April 23 2014, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
> saying:
> 
> > > At this point this is looking pretty clearly like a Parallels Tools bug.
> > > I'll report it to them.
> > 
> > Yes, that sounds good.  Given that, I'm wondering if we should try to
> > workaround this problem at all or rather wait to see if the vendor will
> > fix the issue.
> 
> No such luck, despite two major version revisions of Parallels Desktop (I'm
> now on version 11.0.2) and moving to Windows 10 as the guest OS -- the bug
> perists, unchanged.  So it looks like Cygwin will need to add a workaround
> for this filesystem to fix the problem.

Ok, we could do that.  Can you compile and run the testcase from
https://cygwin.com/ml/cygwin/2014-04/msg00523.html again?  Does it
still show 0 vs. 12 bytes?  Dumb extra test: Does the output change
if you reorder the calls, requesting FileStandardInformation first,
FileNetworkOpenInformation second?

> The output of /usr/lib/csih/getVolInfo seems to be unchanged from the output
> I reported last year.
> 
> > Thanks.  This looks pretty much like a filesystem pretending to be
> > FAT-like.  There may be another problem lurking, which is, are the inode
> > numbers (called "FileId" or "IndexNumber" in Windows) persistant?  With
> > FAT this is not the case, and given the above, it might be a problem...
> > 
> > ...or not.  I just realize that Cygwin doesn't even try to use the
> > FileId as inode number on filesystems with FILE_PERSISTENT_ACLS==FALSE
> > so, never mind.
> > 
> > OTOH, does it support hardlinks?  If so, two hardlinks to the
> > same file would have different inode numbers on Cygwin.
> 
> How would I figure these points out?

Just create a hardlink on that drive using native means:

  $ touch foo
  $ cmd /c mklink /h bar foo

Error at this point?  No hardlinks.  Otherwise:

  $ ls -li foo bar

Are the inode numbers identical?  Congrats, hardlinks work.  But given
the general FAT-iness of the getVolInfo output, I guess it doesn't
maintain hardlinks.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpWqDZk65Ta9.pgp
Description: PGP signature

Re: fstat st_size on open files on Parallels filesystem is wrong

2015-10-08 Thread Jonathan Lennox 
Hi, following up on this issue from last year.  The message I'm replying to
is at .

The problem is weird behavior in Parallels Desktop-hosted Windows VMs, when
accessing the host's native Mac OS X filesystem.  See the thread for the
details.

On Wednesday, April 23 2014, "Corinna Vinschen" wrote to "cygwin@cygwin.com" 
saying:

> > At this point this is looking pretty clearly like a Parallels Tools bug.
> > I'll report it to them.
> 
> Yes, that sounds good.  Given that, I'm wondering if we should try to
> workaround this problem at all or rather wait to see if the vendor will
> fix the issue.

No such luck, despite two major version revisions of Parallels Desktop (I'm
now on version 11.0.2) and moving to Windows 10 as the guest OS -- the bug
perists, unchanged.  So it looks like Cygwin will need to add a workaround
for this filesystem to fix the problem.

The output of /usr/lib/csih/getVolInfo seems to be unchanged from the output
I reported last year.

> Thanks.  This looks pretty much like a filesystem pretending to be
> FAT-like.  There may be another problem lurking, which is, are the inode
> numbers (called "FileId" or "IndexNumber" in Windows) persistant?  With
> FAT this is not the case, and given the above, it might be a problem...
> 
> ...or not.  I just realize that Cygwin doesn't even try to use the
> FileId as inode number on filesystems with FILE_PERSISTENT_ACLS==FALSE
> so, never mind.
> 
> OTOH, does it support hardlinks?  If so, two hardlinks to the
> same file would have different inode numbers on Cygwin.

How would I figure these points out?

-- 
Jonathan Lennox
len...@cs.columbia.edu

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: fstat st_size on open files on Parallels filesystem is wrong

2014-04-23 Thread Corinna Vinschen 
On Apr 23 12:47, len...@cs.columbia.edu wrote:
> On Wednesday, April 23 2014, "Corinna Vinschen" wrote to "cygwin at 
> cygwin.com" saying:
> 
> > Rather than calling GetFileInformationByHandle, try this
> > [...]
> Okay, looks like FileNetworkOpenInformation is succeeding, but returning a
> bad EndOfFile value.
> 
> $ ./win32-size-test 'z:\foo'
> z:\foo: NtQueryInformationFile(FileNetworkOpenInformation): EndOfFile=0
> z:\foo: NtQueryInformationFile(FileStandardInformation): EndOfFile=12
> 
> $ gdb --quiet --args ./win32-size-test.exe 'z:\foo'
> Reading symbols from /cygdrive/z/Emacs-Modtime/win32-size-test.exe...done.
> (gdb) break 82
> Breakpoint 1 at 0x100401400: file win32-size-test.c, line 82.
> (gdb) run
> Starting program: /cygdrive/z/Emacs-Modtime/win32-size-test.exe z:\\foo
> [New Thread 10540.0x48dc]
> [New Thread 10540.0x39d4]
> z:\\foo: NtQueryInformationFile(FileNetworkOpenInformation): EndOfFile=0
> z:\\foo: NtQueryInformationFile(FileStandardInformation): EndOfFile=12

Wow, strange.

> At this point this is looking pretty clearly like a Parallels Tools bug.
> I'll report it to them.

Yes, that sounds good.  Given that, I'm wondering if we should try to
workaround this problem at all or rather wait to see if the vendor will
fix the issue.

> > Also, to add handling for the Parallels filesystem to Cygwin, I'd
> > need the info printed by the getVolInfo tool from the csih package:
> > 
> >   $ /usr/lib/csih/getVolInfo /cygdrive/z
> 
> $ /usr/lib/csih/getVolInfo.exe /cygdrive/z/
> Device Type: 7
> Characteristics: 10
> Volume Name: 
> Serial Number  : 0
> Max Filenamelength : 255
> Filesystemname : 
> Flags  : 3
>   FILE_CASE_SENSITIVE_SEARCH  : TRUE
>   FILE_CASE_PRESERVED_NAMES   : TRUE
>   FILE_UNICODE_ON_DISK: FALSE
>   FILE_PERSISTENT_ACLS: FALSE
>   FILE_FILE_COMPRESSION   : FALSE
>   FILE_VOLUME_QUOTAS  : FALSE
>   FILE_SUPPORTS_SPARSE_FILES  : FALSE
>   FILE_SUPPORTS_REPARSE_POINTS: FALSE
>   FILE_SUPPORTS_REMOTE_STORAGE: FALSE
>   FILE_VOLUME_IS_COMPRESSED   : FALSE
>   FILE_SUPPORTS_OBJECT_IDS: FALSE
>   FILE_SUPPORTS_ENCRYPTION: FALSE
>   FILE_NAMED_STREAMS  : FALSE
>   FILE_READ_ONLY_VOLUME   : FALSE
>   FILE_SEQUENTIAL_WRITE_ONCE  : FALSE
>   FILE_SUPPORTS_TRANSACTIONS  : FALSE

Thanks.  This looks pretty much like a filesystem pretending to be
FAT-like.  There may be another problem lurking, which is, are the inode
numbers (called "FileId" or "IndexNumber" in Windows) persistant?  With
FAT this is not the case, and given the above, it might be a problem...

...or not.  I just realize that Cygwin doesn't even try to use the
FileId as inode number on filesystems with FILE_PERSISTENT_ACLS==FALSE
so, never mind.

OTOH, does it support hardlinks?  If so, two hardlinks to the
same file would have different inode numbers on Cygwin.


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpu8bqzg5WVs.pgp
Description: PGP signature

Re: fstat st_size on open files on Parallels filesystem is wrong

2014-04-23 Thread lennox 
On Wednesday, April 23 2014, "Corinna Vinschen" wrote to "cygwin at cygwin.com" 
saying:

> Rather than calling GetFileInformationByHandle, try this
> 
>   #include 
> 
>   [...]
> 
>   NTSTATUS status;
>   IO_STATUS_BLOCK io;
>   FILE_NETWORK_OPEN_INFORMATION fnoi;
>   
>   status = NtQueryInformationFile (file, &io, &fnoi, sizeof fnoi,
>FileNetworkOpenInformation);
>   if (!NT_SUCCESS (status))
> {
>   fprintf (stderr, "NtQueryInformationFile: 0x%08x\n", status);
>   [...]
> }
>   printf("%s: NtQueryInformationFile: nFileSize=%" PRIu64 "\n",
>argv[i], fnoi.EndOfFile.QuadPart);
> 
> Maybe that's really the problem.  If so, either the EndOfFile info
> is wrong, or (more likely) the call to NtQueryInformationFile returns
> with a non-0 status code, which would be interesting to know.  It's
> probably not STATUS_INVALID_PARAMETER, otherwise you probably wouldn't
> have seen a problem at all.
> 
> Replacing the above call with
> 
>   FILE_STANDARD_INFORMATION fsi;
>   status = NtQueryInformationFile (file, &io, &fsi, sizeof fsi,
>FileStandardInformation);
>   [...]
>   printf("%s: NtQueryInformationFile: nFileSize=%" PRIu64 "\n",
>argv[i], fsi.EndOfFile.QuadPart);
> 
> should then give the correct result.

Okay, looks like FileNetworkOpenInformation is succeeding, but returning a
bad EndOfFile value.

$ ./win32-size-test 'z:\foo'
z:\foo: NtQueryInformationFile(FileNetworkOpenInformation): EndOfFile=0
z:\foo: NtQueryInformationFile(FileStandardInformation): EndOfFile=12

$ gdb --quiet --args ./win32-size-test.exe 'z:\foo'
Reading symbols from /cygdrive/z/Emacs-Modtime/win32-size-test.exe...done.
(gdb) break 82
Breakpoint 1 at 0x100401400: file win32-size-test.c, line 82.
(gdb) run
Starting program: /cygdrive/z/Emacs-Modtime/win32-size-test.exe z:\\foo
[New Thread 10540.0x48dc]
[New Thread 10540.0x39d4]
z:\\foo: NtQueryInformationFile(FileNetworkOpenInformation): EndOfFile=0
z:\\foo: NtQueryInformationFile(FileStandardInformation): EndOfFile=12

Breakpoint 1, main (argc=2, argv=0x22aaf0) at win32-size-test.c:82
82  if (!CloseHandle(file)) {
(gdb) p fnoi
$1 = {CreationTime = {{LowPart = 1493948928, HighPart = 30367507}, u = {
  LowPart = 1493948928, HighPart = 30367507},
QuadPart = 13042745092000}, LastAccessTime = {{LowPart = 1493948928,
  HighPart = 30367507}, u = {LowPart = 1493948928, HighPart = 30367507},
QuadPart = 13042745092000}, LastWriteTime = {{LowPart = 1493948928,
  HighPart = 30367507}, u = {LowPart = 1493948928, HighPart = 30367507},
QuadPart = 13042745092000}, ChangeTime = {{LowPart = 1493948928,
  HighPart = 30367507}, u = {LowPart = 1493948928, HighPart = 30367507},
QuadPart = 13042745092000}, AllocationSize = {{LowPart = 12,
  HighPart = 0}, u = {LowPart = 12, HighPart = 0}, QuadPart = 12},
  EndOfFile = {{LowPart = 0, HighPart = 0}, u = {LowPart = 0, HighPart = 0},
QuadPart = 0}, FileAttributes = 128}
(gdb) p fsi
$2 = {AllocationSize = {{LowPart = 12, HighPart = 0}, u = {LowPart = 12,
  HighPart = 0}, QuadPart = 12}, EndOfFile = {{LowPart = 12,
  HighPart = 0}, u = {LowPart = 12, HighPart = 0}, QuadPart = 12},
  NumberOfLinks = 1, DeletePending = 0 '\000', Directory = 0 '\000'}

At this point this is looking pretty clearly like a Parallels Tools bug.
I'll report it to them.


> Also, to add handling for the Parallels filesystem to Cygwin, I'd
> need the info printed by the getVolInfo tool from the csih package:
> 
>   $ /usr/lib/csih/getVolInfo /cygdrive/z

$ /usr/lib/csih/getVolInfo.exe /cygdrive/z/
Device Type: 7
Characteristics: 10
Volume Name: 
Serial Number  : 0
Max Filenamelength : 255
Filesystemname : 
Flags  : 3
  FILE_CASE_SENSITIVE_SEARCH  : TRUE
  FILE_CASE_PRESERVED_NAMES   : TRUE
  FILE_UNICODE_ON_DISK: FALSE
  FILE_PERSISTENT_ACLS: FALSE
  FILE_FILE_COMPRESSION   : FALSE
  FILE_VOLUME_QUOTAS  : FALSE
  FILE_SUPPORTS_SPARSE_FILES  : FALSE
  FILE_SUPPORTS_REPARSE_POINTS: FALSE
  FILE_SUPPORTS_REMOTE_STORAGE: FALSE
  FILE_VOLUME_IS_COMPRESSED   : FALSE
  FILE_SUPPORTS_OBJECT_IDS: FALSE
  FILE_SUPPORTS_ENCRYPTION: FALSE
  FILE_NAMED_STREAMS  : FALSE
  FILE_READ_ONLY_VOLUME   : FALSE
  FILE_SEQUENTIAL_WRITE_ONCE  : FALSE
  FILE_SUPPORTS_TRANSACTIONS  : FALSE


Here's my test code.  Compiles with either Cygwin or mingw64 GCC, but not
classic mingw or Visual Studio due to missing headers/declarations.  Link
with -lntdll.

#include 
#include 
#include 
#include 

#if __LP64__
#define PRIuDWORD "u"
#define PRIxDWORD "x"
#else
#define PRIuDWORD "lu"
#define PRIxDWORD "lx"
#endif

static const char* geterr(DWORD err)
{
static char msg[1024];

FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM | 
FORMAT_MESSAGE_IGNORE_INSERTS |
FORMAT_MESSAGE_MAX_WIDTH_MASK, NULL, err,
MAKELANGID(LA

Re: fstat st_size on open files on Parallels filesystem is wrong

2014-04-23 Thread Corinna Vinschen 
On Apr 22 16:57, len...@cs.columbia.edu wrote:
> On Tuesday, April 22 2014, "Corinna Vinschen" wrote to "cygwin at cygwin.com" 
> saying:
> > On Apr 21 14:46, lennox at cs.columbia.edu wrote:
> > > [...]
> > > This is using Posix APIs -- open() / write() -- not C APIs, fopen() /
> > > fwrite(), so there shouldn't be a buffer?  Notice that the test behaves 
> > > as I
> > > expect for a file on NTFS.
> > > 
> > > Adding a call to fsync() prior to the fstat() call doesn't change 
> > > anything.
> > 
> > This is actually a bad sign.  The problem you're describing occurs on
> > NFS, too.  If you write to the file, a subsequent call to fetch stat
> > attributes does not return the actual size of the file, but the size at
> > the time the handle has been opened.
> > 
> > However, on NFS, a call to FlushFileBuffers helps to kick stat back into
> > shape.  That's the Win32 function called from fsync as well.  What is
> > Cygwin supposed to do if that doesn't work?
> 
> Okay, interesting further investigation.
> 
> The Parallels filesystem appears to work correctly when I repeat the test
> case using Windows kernel32 APIs -- specifically, FileInformationByHandle --
> so something's different between the kernel32 APIs and the ntdll APIs that
> Cygwin uses.
> 
> Sample code for Win32 test attached.  Works identically with Cygwin, MinGW,
> or Visual C++.
> 
> Just spitballing here, but -- I see that cygwin's file_get_fnoi function
> (which is where fhandler_base::fstat_by_handle gets its size parameter)
> tries NtQueryInformationFile(FileNetworkOpenInformation) before
> NtQueryInformationFile(FileStandardInformation).  Is it possible that the
> Parallels filesystem isn't filling out all fields of that API properly?  Is
> there a straightforward way I could debug this?

You could use the same functions.  Since you had a look into the sources
anyway, you probably saw the comments:

  /* Some FSes (Netapps) don't implement FileNetworkOpenInformation. */
  [...call NtQueryInformationFile(FileNetworkOpenInformation) unless
  the skip_network_open_inf flag is given...]
  if (status == STATUS_INVALID_PARAMETER)
 /* Apart from accessing Netapps, this also occurs when accessing SMB
share root dirs hosted on NT4. */
 [...call NtQueryInformationFile(FileBasicInformation) and
  NtQueryInformationFile(FileStandardInformation)...]

Rather than calling GetFileInformationByHandle, try this

  #include 

  [...]

  NTSTATUS status;
  IO_STATUS_BLOCK io;
  FILE_NETWORK_OPEN_INFORMATION fnoi;
  
  status = NtQueryInformationFile (file, &io, &fnoi, sizeof fnoi,
   FileNetworkOpenInformation);
  if (!NT_SUCCESS (status))
{
  fprintf (stderr, "NtQueryInformationFile: 0x%08x\n", status);
  [...]
}
  printf("%s: NtQueryInformationFile: nFileSize=%" PRIu64 "\n",
 argv[i], fnoi.EndOfFile.QuadPart);

Maybe that's really the problem.  If so, either the EndOfFile info
is wrong, or (more likely) the call to NtQueryInformationFile returns
with a non-0 status code, which would be interesting to know.  It's
probably not STATUS_INVALID_PARAMETER, otherwise you probably wouldn't
have seen a problem at all.

Replacing the above call with

  FILE_STANDARD_INFORMATION fsi;
  status = NtQueryInformationFile (file, &io, &fsi, sizeof fsi,
   FileStandardInformation);
  [...]
  printf("%s: NtQueryInformationFile: nFileSize=%" PRIu64 "\n",
 argv[i], fsi.EndOfFile.QuadPart);

should then give the correct result.

Also, to add handling for the Parallels filesystem to Cygwin, I'd
need the info printed by the getVolInfo tool from the csih package:

  $ /usr/lib/csih/getVolInfo /cygdrive/z


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgpTdB7vbnRpX.pgp
Description: PGP signature

Re: fstat st_size on open files on Parallels filesystem is wrong

2014-04-22 Thread lennox 
On Tuesday, April 22 2014, "Corinna Vinschen" wrote to "cygwin at cygwin.com" 
saying:

> On Apr 21 14:46, lennox at cs.columbia.edu wrote:
> > On Monday, April 21 2014, "Andrey Repin" wrote to "lennox at
> > cs.columbia.edu, cygwin at cygwin.com" saying:
> > 
> > > Greetings, lennox at cs.columbia.edu!
> > > 
> > > > I’m running cygwin64 1.7.29 in a Windows 8.1 Pro virtual machine, 
> > > > running in
> > > > Parallels Desktop 9.0.24229 on Mac OS X 10.9.2.
> > > 
> > > > Parallels Desktop automatically mounts my Mac OS X home directory as a 
> > > > Z:
> > > > drive in Windows.  Cygwin mount reports this drive as being type 
> > > > "prlsf".
> > > 
> > > > Unfortunately, I've discovered that if I have an open file on this
> > > > filesystem which has been written to, the size returned by Cygwin 
> > > > fstat() on
> > > > the open file is wrong.  A stat() of the file after it's been closed is
> > > > correct.
> > > 
> > > > This has the consequence that emacs always thinks saved files have been
> > > > modified externally, since emacs looks at files' sizes (as well as their
> > > > modification times) to detect external changes.  This makes emacs
> > > > near-unusable.
> > > 
> > > > This problem does not occur for files in my Cygwin home directory, or 
> > > > other
> > > > locations mounted on my Windows C: drive.
> > > 
> > > > I've attached a simple unit test program that illustrates the problem.
> > > > I've also attached my cygcheck -s -v -r output.
> > > 
> > > > Any ideas?  Is this a Cygwin bug, a Parallels bug, or something else?
> > > > Glancing over the Cygwin code, I see that there are a few cases where 
> > > > fstat
> > > > has special cases for certain filesystem types.
> > > 
> > > You never flushing the buffer in your test code, or I'm reading it wrong?
> > 
> > This is using Posix APIs -- open() / write() -- not C APIs, fopen() /
> > fwrite(), so there shouldn't be a buffer?  Notice that the test behaves as I
> > expect for a file on NTFS.
> > 
> > Adding a call to fsync() prior to the fstat() call doesn't change anything.
> 
> This is actually a bad sign.  The problem you're describing occurs on
> NFS, too.  If you write to the file, a subsequent call to fetch stat
> attributes does not return the actual size of the file, but the size at
> the time the handle has been opened.
> 
> However, on NFS, a call to FlushFileBuffers helps to kick stat back into
> shape.  That's the Win32 function called from fsync as well.  What is
> Cygwin supposed to do if that doesn't work?

Okay, interesting further investigation.

The Parallels filesystem appears to work correctly when I repeat the test
case using Windows kernel32 APIs -- specifically, FileInformationByHandle --
so something's different between the kernel32 APIs and the ntdll APIs that
Cygwin uses.

Sample code for Win32 test attached.  Works identically with Cygwin, MinGW,
or Visual C++.

Just spitballing here, but -- I see that cygwin's file_get_fnoi function
(which is where fhandler_base::fstat_by_handle gets its size parameter)
tries NtQueryInformationFile(FileNetworkOpenInformation) before
NtQueryInformationFile(FileStandardInformation).  Is it possible that the
Parallels filesystem isn't filling out all fields of that API properly?  Is
there a straightforward way I could debug this?



$ ./stat-size-test.exe '/cygdrive/z/foo'
/cygdrive/z/foo: fstat: st_size=0
/cygdrive/z/foo: stat: st_size=12

$ ./win32-size-test.exe 'z:\foo'
z:\foo: FileInformationByHandle: nFileSize=12

$ mount
C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
C:/cygwin64 on / type ntfs (binary,auto)
C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
S: on /cygdrive/s type ntfs (binary,posix=0,user,noumount,auto)
U: on /cygdrive/u type ntfs (binary,posix=0,user,noumount,auto)
Z: on /cygdrive/z type prlsf (binary,posix=0,user,noumount,auto)

#include 
#include 
#include 

#if __LP64__
#define PRI_DWORD "u"
#else
#define PRI_DWORD "lu"
#endif

static const char* geterr(DWORD err)
{
static char msg[1024];

FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM | 
FORMAT_MESSAGE_IGNORE_INSERTS |
FORMAT_MESSAGE_MAX_WIDTH_MASK, NULL, err,
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
(LPSTR)msg, sizeof(msg), NULL);

return msg;
}

int main(int argc, char* argv[])
{
int i;
int status = 0;
for (i = 1; i < argc; i++) {
HANDLE file;
DWORD err, bytesWritten;
BY_HANDLE_FILE_INFORMATION fileInfo;

file = CreateFileA(argv[i], GENERIC_WRITE, FILE_SHARE_WRITE,
NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if (file == INVALID_HANDLE_VALUE) {
err = GetLastError();
fprintf(stderr, "%s: CreateFile: %s (err %" 
PRI_DWORD")\n", argv[i],
geterr(err), err);

Re: fstat st_size on open files on Parallels filesystem is wrong

2014-04-22 Thread lennox 
On Tuesday, April 22 2014, "Corinna Vinschen" wrote to "cygwin at cygwin.com" 
saying:

> On Apr 21 14:46, lennox at cs.columbia.edu wrote:
> > On Monday, April 21 2014, "Andrey Repin" wrote to "lennox at
> > cs.columbia.edu, cygwin at cygwin.com" saying:
> > 
> > > Greetings, lennox at cs.columbia.edu!
> > > 
> > > > I’m running cygwin64 1.7.29 in a Windows 8.1 Pro virtual machine, 
> > > > running in
> > > > Parallels Desktop 9.0.24229 on Mac OS X 10.9.2.
> > > 
> > > > Parallels Desktop automatically mounts my Mac OS X home directory as a 
> > > > Z:
> > > > drive in Windows.  Cygwin mount reports this drive as being type 
> > > > "prlsf".
> > > 
> > > > Unfortunately, I've discovered that if I have an open file on this
> > > > filesystem which has been written to, the size returned by Cygwin 
> > > > fstat() on
> > > > the open file is wrong.  A stat() of the file after it's been closed is
> > > > correct.
> > > 
> > > > This has the consequence that emacs always thinks saved files have been
> > > > modified externally, since emacs looks at files' sizes (as well as their
> > > > modification times) to detect external changes.  This makes emacs
> > > > near-unusable.
> > > 
> > > > This problem does not occur for files in my Cygwin home directory, or 
> > > > other
> > > > locations mounted on my Windows C: drive.
> > > 
> > > > I've attached a simple unit test program that illustrates the problem.
> > > > I've also attached my cygcheck -s -v -r output.
> > > 
> > > > Any ideas?  Is this a Cygwin bug, a Parallels bug, or something else?
> > > > Glancing over the Cygwin code, I see that there are a few cases where 
> > > > fstat
> > > > has special cases for certain filesystem types.
> > > 
> > > You never flushing the buffer in your test code, or I'm reading it wrong?
> > 
> > This is using Posix APIs -- open() / write() -- not C APIs, fopen() /
> > fwrite(), so there shouldn't be a buffer?  Notice that the test behaves as I
> > expect for a file on NTFS.
> > 
> > Adding a call to fsync() prior to the fstat() call doesn't change anything.
> 
> This is actually a bad sign.  The problem you're describing occurs on
> NFS, too.  If you write to the file, a subsequent call to fetch stat
> attributes does not return the actual size of the file, but the size at
> the time the handle has been opened.
> 
> However, on NFS, a call to FlushFileBuffers helps to kick stat back into
> shape.  That's the Win32 function called from fsync as well.  What is
> Cygwin supposed to do if that doesn't work?

It's certainly possible that the file system driver in the Parallels Tools
has a bug, and if so I'm happy to report it to Parallels.

I'll see if I can reproduce the test case using just Win32 APIs.  Are there
any particular gotchas I should watch out for, or should just looking at
what Win32 functions are called in fhandler_disk_file.cc be sufficient?


-- 
Jonathan Lennox
lennox at cs.columbia.edu

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: fstat st_size on open files on Parallels filesystem is wrong

2014-04-22 Thread Corinna Vinschen 
On Apr 21 14:46, len...@cs.columbia.edu wrote:
> On Monday, April 21 2014, "Andrey Repin" wrote to "lennox at
> cs.columbia.edu, cygwin at cygwin.com" saying:
> 
> > Greetings, lennox at cs.columbia.edu!
> > 
> > > I’m running cygwin64 1.7.29 in a Windows 8.1 Pro virtual machine, running 
> > > in
> > > Parallels Desktop 9.0.24229 on Mac OS X 10.9.2.
> > 
> > > Parallels Desktop automatically mounts my Mac OS X home directory as a Z:
> > > drive in Windows.  Cygwin mount reports this drive as being type "prlsf".
> > 
> > > Unfortunately, I've discovered that if I have an open file on this
> > > filesystem which has been written to, the size returned by Cygwin fstat() 
> > > on
> > > the open file is wrong.  A stat() of the file after it's been closed is
> > > correct.
> > 
> > > This has the consequence that emacs always thinks saved files have been
> > > modified externally, since emacs looks at files' sizes (as well as their
> > > modification times) to detect external changes.  This makes emacs
> > > near-unusable.
> > 
> > > This problem does not occur for files in my Cygwin home directory, or 
> > > other
> > > locations mounted on my Windows C: drive.
> > 
> > > I've attached a simple unit test program that illustrates the problem.
> > > I've also attached my cygcheck -s -v -r output.
> > 
> > > Any ideas?  Is this a Cygwin bug, a Parallels bug, or something else?
> > > Glancing over the Cygwin code, I see that there are a few cases where 
> > > fstat
> > > has special cases for certain filesystem types.
> > 
> > You never flushing the buffer in your test code, or I'm reading it wrong?
> 
> This is using Posix APIs -- open() / write() -- not C APIs, fopen() /
> fwrite(), so there shouldn't be a buffer?  Notice that the test behaves as I
> expect for a file on NTFS.
> 
> Adding a call to fsync() prior to the fstat() call doesn't change anything.

This is actually a bad sign.  The problem you're describing occurs on
NFS, too.  If you write to the file, a subsequent call to fetch stat
attributes does not return the actual size of the file, but the size at
the time the handle has been opened.

However, on NFS, a call to FlushFileBuffers helps to kick stat back into
shape.  That's the Win32 function called from fsync as well.  What is
Cygwin supposed to do if that doesn't work?


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Maintainer cygwin AT cygwin DOT com
Red Hat


pgplOBvSUdPD8.pgp
Description: PGP signature

Re: fstat st_size on open files on Parallels filesystem is wrong

2014-04-21 Thread lennox 
On Monday, April 21 2014, "Andrey Repin" wrote to "lennox at
cs.columbia.edu, cygwin at cygwin.com" saying:

> Greetings, lennox at cs.columbia.edu!
> 
> > I’m running cygwin64 1.7.29 in a Windows 8.1 Pro virtual machine, running in
> > Parallels Desktop 9.0.24229 on Mac OS X 10.9.2.
> 
> > Parallels Desktop automatically mounts my Mac OS X home directory as a Z:
> > drive in Windows.  Cygwin mount reports this drive as being type "prlsf".
> 
> > Unfortunately, I've discovered that if I have an open file on this
> > filesystem which has been written to, the size returned by Cygwin fstat() on
> > the open file is wrong.  A stat() of the file after it's been closed is
> > correct.
> 
> > This has the consequence that emacs always thinks saved files have been
> > modified externally, since emacs looks at files' sizes (as well as their
> > modification times) to detect external changes.  This makes emacs
> > near-unusable.
> 
> > This problem does not occur for files in my Cygwin home directory, or other
> > locations mounted on my Windows C: drive.
> 
> > I've attached a simple unit test program that illustrates the problem.
> > I've also attached my cygcheck -s -v -r output.
> 
> > Any ideas?  Is this a Cygwin bug, a Parallels bug, or something else?
> > Glancing over the Cygwin code, I see that there are a few cases where fstat
> > has special cases for certain filesystem types.
> 
> You never flushing the buffer in your test code, or I'm reading it wrong?

This is using Posix APIs -- open() / write() -- not C APIs, fopen() /
fwrite(), so there shouldn't be a buffer?  Notice that the test behaves as I
expect for a file on NTFS.

Adding a call to fsync() prior to the fstat() call doesn't change anything.

I've modeled this as closely as I could on what it appears emacs is doing,
as far as I could tell.

-- 
Jonathan Lennox
lennox at cs.columbia.edu

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: fstat st_size on open files on Parallels filesystem is wrong

2014-04-21 Thread Andrey Repin 
Greetings, len...@cs.columbia.edu!

> I’m running cygwin64 1.7.29 in a Windows 8.1 Pro virtual machine, running in
> Parallels Desktop 9.0.24229 on Mac OS X 10.9.2.

> Parallels Desktop automatically mounts my Mac OS X home directory as a Z:
> drive in Windows.  Cygwin mount reports this drive as being type "prlsf".

> Unfortunately, I've discovered that if I have an open file on this
> filesystem which has been written to, the size returned by Cygwin fstat() on
> the open file is wrong.  A stat() of the file after it's been closed is
> correct.

> This has the consequence that emacs always thinks saved files have been
> modified externally, since emacs looks at files' sizes (as well as their
> modification times) to detect external changes.  This makes emacs
> near-unusable.

> This problem does not occur for files in my Cygwin home directory, or other
> locations mounted on my Windows C: drive.

> I've attached a simple unit test program that illustrates the problem.
> I've also attached my cygcheck -s -v -r output.

> Any ideas?  Is this a Cygwin bug, a Parallels bug, or something else?
> Glancing over the Cygwin code, I see that there are a few cases where fstat
> has special cases for certain filesystem types.

You never flushing the buffer in your test code, or I'm reading it wrong?


--
WBR,
Andrey Repin (anrdae...@yandex.ru) 21.04.2014, <22:33>

Sorry for my terrible english...