Re: stat st_birthtim(espec)
On Oct 19 02:06, Phil Budne wrote: > While checking if an OSS project of mine (www.snobol4.org/csnobol4) > compiled cleanly under Cygwin, I was happy to discover that struct > stat contains file birth time as on various BSD based systems. > > BUT, I was unhappy to find out that MacOS 10.15 and Cygwin 3.1.7 have > non-overlapping definitions for birth time information. > > FreeBSD 12, NetBSD 9, and OSX 10.15 all have a "timespec" available as > "st_birthtimespec", but Cygwin does not. > > On FreeBSD, NetBSD, and Cygwin the actual member of struct stat is > st_birthtim, but on OSX, the member is st_birthtimespec, and there is > no define for st_birthtim. > > A st_birthtimespec define in cygwin/stat.h would make it easier to > write portable code. Well, portable code should use the st_atim/st_mtim/st_ctim members directly and not use st_birthtim at all. For the headers we're usually looking for Linux compatibility in the first place and if that's any indication, we should add all the st_XYZtimensec members, but *only* if __USE_XOPEN2K8 is defined (equivalent _XOPEN_SOURCE >= 700 for Cygwin/newlib) GLibc does not provide any st_XYZtimespec definitions, though. I don't think it really makes a lot of sense to add those with portability in mind. Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: stat() lstat() not able to read long filename with cyrillic chars?
On 25/12/2015 01:04, Andrey Repin wrote: > Greetings, Corinna Vinschen! > >>> First, I have read the FAQ and this mailing archive :) >>> [..] > >> NAME_MAX is 255. On Windows this is the number of UTF-16 chars >> unfortunately. On POSIX systems (as on Cygwin) this is the >> number of bytes. Long UTF-16 strings in cyrillic take twice as >> much UTF-8 chars as it has UTF-16 chars, so NAME_MAX in utf-8 >> cyrillics translates into a maximum of 127 UTF-16 chars. Ok, I understand. Thanks for your explanation. > > Aren't POSIX restrictions are a bit different? Namely 128 bytes > per path element and 4096 bytes for file name? Seen the sample file name it seems truncated rather near 256 bytes (~ 128 UTF-16 chars) than 4096 bytes... > >> If you need access to UTF-16 filenames with more characters, you >> can switch to a one-byte charset temporarily, e.g. > >> $ LC_ALL=ru_RU your_app > >> to switch to iso-8859-5 or > >> $ LC_ALL=ru_RU.CP1251 > >> to switch to Windows codepage 1251. See >> https://cygwin.com/cygwin-ug-net/setup-locale.html > > >> HTH, Corinna > > > -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() lstat() not able to read long filename with cyrillic chars?
Greetings, Corinna Vinschen! >> First, I have read the FAQ and this mailing archive :) >> >> Here is the problem I meet: >> >> In a directory are placed three files using windows 8's explorer: >> - a short Cyrillic filename "абваб.txt" >> - a long Cyrillic filename >> "абвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабваб.txt" >> - a long Latin filename >> "ababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababa.txt" >> >> >> >From a C program compiled under Cygwin, I can obtain the corresponding >> filename strings using readdir_r()... >> >> "\320\260\320\261\320\262\320\260\320\261.txt" >> "\320\260\320\261\320\262\320\260\320\261\320\262\320\260\320\261 [snipped]" >> "abababababaababababa [snipped]" >> >> ... but passing these strings in turn to lstat() or stat() returns 0 as >> expected for all except for the long Cyrillic filename. > NAME_MAX is 255. On Windows this is the number of UTF-16 chars > unfortunately. On POSIX systems (as on Cygwin) this is the number of > bytes. Long UTF-16 strings in cyrillic take twice as much UTF-8 chars > as it has UTF-16 chars, so NAME_MAX in utf-8 cyrillics translates into > a maximum of 127 UTF-16 chars. Aren't POSIX restrictions are a bit different? Namely 128 bytes per path element and 4096 bytes for file name? > If you need access to UTF-16 filenames with more characters, you can > switch to a one-byte charset temporarily, e.g. > $ LC_ALL=ru_RU your_app > to switch to iso-8859-5 or > $ LC_ALL=ru_RU.CP1251 > to switch to Windows codepage 1251. See > https://cygwin.com/cygwin-ug-net/setup-locale.html > HTH, > Corinna -- With best regards, Andrey Repin Friday, December 25, 2015 03:03:51 Sorry for my terrible english...
Re: stat() lstat() not able to read long filename with cyrillic chars?
On Dec 23 20:44, Denis Corbin wrote: > Hi, > > First, I have read the FAQ and this mailing archive :) > > Here is the problem I meet: > > In a directory are placed three files using windows 8's explorer: > - a short Cyrillic filename "абваб.txt" > - a long Cyrillic filename > "абвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабвабваб.txt" > - a long Latin filename > "ababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababa.txt" > > > >From a C program compiled under Cygwin, I can obtain the corresponding > filename strings using readdir_r()... > > "\320\260\320\261\320\262\320\260\320\261.txt" > "\320\260\320\261\320\262\320\260\320\261\320\262\320\260\320\261 [snipped]" > "abababababaababababa [snipped]" > > ... but passing these strings in turn to lstat() or stat() returns 0 as > expected for all except for the long Cyrillic filename. NAME_MAX is 255. On Windows this is the number of UTF-16 chars unfortunately. On POSIX systems (as on Cygwin) this is the number of bytes. Long UTF-16 strings in cyrillic take twice as much UTF-8 chars as it has UTF-16 chars, so NAME_MAX in utf-8 cyrillics translates into a maximum of 127 UTF-16 chars. If you need access to UTF-16 filenames with more characters, you can switch to a one-byte charset temporarily, e.g. $ LC_ALL=ru_RU your_app to switch to iso-8859-5 or $ LC_ALL=ru_RU.CP1251 to switch to Windows codepage 1251. See https://cygwin.com/cygwin-ug-net/setup-locale.html HTH, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat signature.asc Description: PGP signature
Re: stat() and tilde prefix (was bad bash tab completion)
On 02/07/2013 12:00 AM, Shaddy Baddah wrote:
Please find the patch discussed attached. It probably needs to be a bit
more generic, maybe with some precompiler directives to limit it to
cygwin? Although if it is just for cygwin, perhaps it can just go in the
cygports patch?
The build of bash has nothing to do with cygports; if I decide to take
this, I will include something like it the next time I build bash for
cygwin.
Do I need to put an attn bash maintainer on the subject line?
Nah, I read the list.
*r++ = '\\';
/* XXX -- check for standalone tildes here and backslash-quote them */
- if (s == text *s == '~' file_exists (text))
+ if (s == text *s == '~' tilde_file_exists (text)) {
No need to add the {} here.
*r++ = '\\';
+ }
*r++ = *s;
}
*r = '\0';
diff --git a/general.c b/general.c
index fdadf1d..b279cbe 100644
--- a/general.c
+++ b/general.c
@@ -544,6 +544,28 @@ file_exists (fn)
}
int
+tilde_file_exists (fn)
This function is misnamed for what it does, which is attempt work around
the fact that cygwin's handling of .. is not POSIX-compliant. I think a
better fix would be to change file_exists() itself instead of adding a
misnamed wrapper function; then bashline.c wouldn't even need patching.
The string 'tilde' need not even be in the patch; what you are really
after is a function that says that if '..' is found within a string
being probed for existence, then add an additional check to see if the
prefix of that string exists as a directory.
But I don't mind experimenting with the idea - it remains to be seen
whether people will complain that bash is noticeably slower because it
takes time to double-check instead of rely on cygwin's non-POSIX
shortcut. And the slowdown would only be on paths containing a '..'; I
would NOT be checking for symlinks (even though symlinks containing ..
are also being interpreted in a non-POSIX manner, it is much more
expensive to second-guess if you have to check every name for being a
symlink than it is to just check for literal ..).
--
Eric Blake eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org
signature.asc
Description: OpenPGP digital signature
Re: stat() and tilde prefix (was bad bash tab completion)
Am 07.02.2013 16:30, schrieb Eric Blake: ... ... the fact that cygwin's handling of .. is not POSIX-compliant. I think a better fix would be to change file_exists() itself instead of adding a misnamed wrapper function; then bashline.c wouldn't even need patching. The string 'tilde' need not even be in the patch; what you are really after is a function that says that if '..' is found within a string being probed for existence, then add an additional check to see if the prefix of that string exists as a directory. But I don't mind experimenting with the idea - it remains to be seen whether people will complain that bash is noticeably slower because it takes time to double-check instead of rely on cygwin's non-POSIX shortcut. And the slowdown would only be on paths containing a '..'; I would NOT be checking for symlinks (even though symlinks containing .. are also being interpreted in a non-POSIX manner, it is much more expensive to second-guess if you have to check every name for being a symlink than it is to just check for literal ..). Do I interpret correctly that you talk about bash filename completion here? Referring to http://cygwin.com/ml/cygwin/2013-01/msg00201.html, I'd like to point out that while this .. thing is a cygwin bug, or known downside as Corinna says, the same issue occurs on Linux precisely with filename completion which isn't consistent there either. So it would be over-fixing to handle that specifically in bash. On the other hand, considering again this downside: If the core cygwin filesystem function would follow this approach, simply checking for an occurrence of .. first before resolving the filename, wouldn't that be an acceptable fix without inappropriate penalty? -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Feb 7 17:10, Thomas Wolff wrote: Am 07.02.2013 16:30, schrieb Eric Blake: ... ... the fact that cygwin's handling of .. is not POSIX-compliant. I think a better fix would be to change file_exists() itself instead of adding a misnamed wrapper function; then bashline.c wouldn't even need patching. The string 'tilde' need not even be in the patch; what you are really after is a function that says that if '..' is found within a string being probed for existence, then add an additional check to see if the prefix of that string exists as a directory. But I don't mind experimenting with the idea - it remains to be seen whether people will complain that bash is noticeably slower because it takes time to double-check instead of rely on cygwin's non-POSIX shortcut. And the slowdown would only be on paths containing a '..'; I would NOT be checking for symlinks (even though symlinks containing .. are also being interpreted in a non-POSIX manner, it is much more expensive to second-guess if you have to check every name for being a symlink than it is to just check for literal ..). Do I interpret correctly that you talk about bash filename completion here? Referring to http://cygwin.com/ml/cygwin/2013-01/msg00201.html, I'd like to point out that while this .. thing is a cygwin bug, or known downside as Corinna says, the same issue occurs on Linux precisely with filename completion which isn't consistent there either. So it would be over-fixing to handle that specifically in bash. On the other hand, considering again this downside: If the core cygwin filesystem function would follow this approach, simply checking for an occurrence of .. first before resolving the filename, wouldn't that be an acceptable fix without inappropriate penalty? You don't know what you're asking for (can of worms, etc.) This is some kind of chicken egg problem. - The path must be normalized, otherwise we can't reliably convert the POSIX path prefix to a Windows pathname. - Only after converting to a Windows path, we can perform file checks. - Rinse and repeat. Also, the path normalization is performed in an entirely distinct function from the mount point conversion, and this in turn is another function than the path function handling symlinks and devices. Changing that requires to implement the path conversion functionality almost from scratch. Given the age of some of these functions, I'd like to have done that for a long time, but I'm constantly shying away since I don't want to break what is working today. There's, of course, still the aforementioned chicken-egg problem. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Thu, Feb 07, 2013 at 05:23:22PM +0100, Corinna Vinschen wrote: On Feb 7 17:10, Thomas Wolff wrote: Am 07.02.2013 16:30, schrieb Eric Blake: ... ... the fact that cygwin's handling of .. is not POSIX-compliant. I think a better fix would be to change file_exists() itself instead of adding a misnamed wrapper function; then bashline.c wouldn't even need patching. The string 'tilde' need not even be in the patch; what you are really after is a function that says that if '..' is found within a string being probed for existence, then add an additional check to see if the prefix of that string exists as a directory. But I don't mind experimenting with the idea - it remains to be seen whether people will complain that bash is noticeably slower because it takes time to double-check instead of rely on cygwin's non-POSIX shortcut. And the slowdown would only be on paths containing a '..'; I would NOT be checking for symlinks (even though symlinks containing .. are also being interpreted in a non-POSIX manner, it is much more expensive to second-guess if you have to check every name for being a symlink than it is to just check for literal ..). Do I interpret correctly that you talk about bash filename completion here? Referring to http://cygwin.com/ml/cygwin/2013-01/msg00201.html, I'd like to point out that while this .. thing is a cygwin bug, or known downside as Corinna says, the same issue occurs on Linux precisely with filename completion which isn't consistent there either. So it would be over-fixing to handle that specifically in bash. On the other hand, considering again this downside: If the core cygwin filesystem function would follow this approach, simply checking for an occurrence of .. first before resolving the filename, wouldn't that be an acceptable fix without inappropriate penalty? You don't know what you're asking for (can of worms, etc.) This is some kind of chicken egg problem. - The path must be normalized, otherwise we can't reliably convert the POSIX path prefix to a Windows pathname. - Only after converting to a Windows path, we can perform file checks. - Rinse and repeat. Also, the path normalization is performed in an entirely distinct function from the mount point conversion, and this in turn is another function than the path function handling symlinks and devices. Changing that requires to implement the path conversion functionality almost from scratch. Given the age of some of these functions, I'd like to have done that for a long time, but I'm constantly shying away since I don't want to break what is working today. There's, of course, still the aforementioned chicken-egg problem. What she said. cgf -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
Hi,
On 15 Jan 2013 23:33, Shaddy Baddah wrote:
snip
From what I make of it, there needs to be a patch that, although can
work generically, adds checks only required for Cygwin. And therefore
is specific to the Cygwin package.
The check would be an extension of the file_exists() function, perhaps
called tilde_file_exists(), which determines if the tilde prefix forms
a directory component of the path (strchr('/')?). If it does not, the
file_exists() check is sufficient. If it does, then the check of if
that directory exists is logically and'ed to the result of
file_exists().
Does that sound about right?
A check like this might be a good idea. Ultimately I would be glad to
be able to come up with more correct code in Cygwin while not getting
slower, of course. But that's wishful thinking for now.
Bash, patched in the way I have described, seems to fix the tab
completion issue.
I will tidy up the work and publish the patch at some point soon. I may
have taken a naive approach, so review comments are welcome.
Please find the patch discussed attached. It probably needs to be a bit
more generic, maybe with some precompiler directives to limit it to
cygwin? Although if it is just for cygwin, perhaps it can just go in the
cygports patch?
Do I need to put an attn bash maintainer on the subject line?
diff --git a/bashline.c b/bashline.c
index 3cbb18f..31841fa 100644
--- a/bashline.c
+++ b/bashline.c
@@ -3478,8 +3478,9 @@ quote_word_break_chars (text)
if (mbschr (rl_completer_word_break_characters, *s))
*r++ = '\\';
/* XXX -- check for standalone tildes here and backslash-quote them */
- if (s == text *s == '~' file_exists (text))
+ if (s == text *s == '~' tilde_file_exists (text)) {
*r++ = '\\';
+ }
*r++ = *s;
}
*r = '\0';
diff --git a/general.c b/general.c
index fdadf1d..b279cbe 100644
--- a/general.c
+++ b/general.c
@@ -544,6 +544,28 @@ file_exists (fn)
}
int
+tilde_file_exists (fn)
+ char *fn;
+{
+ struct stat sb;
+ char *slash, *dirpart;
+ int istildedir;
+
+ istildedir = 0;
+
+ slash = strchr (fn, '/');
+ if ((slash != 0) (slash != fn))
+{
+ dirpart = (char *)xmalloc ((slash - fn) + 1);
+ strncpy (dirpart, fn, (slash - fn) + 1);
+ istildedir = file_isdir (dirpart);
+ free (dirpart);
+}
+
+ return (!((slash != 0) (slash != fn) ! istildedir) (stat (fn, sb) == 0));
+}
+
+int
file_isdir (fn)
char *fn;
{
diff --git a/general.h b/general.h
index 2b31c58..f6b7ab4 100644
--- a/general.h
+++ b/general.h
@@ -302,6 +302,7 @@ extern int sh_openpipe __P((int *));
extern int sh_closepipe __P((int *));
extern int file_exists __P((char *));
+extern int tilde_file_exists __P((char *));
extern int file_isdir __P((char *));
extern int file_iswdir __P((char *));
extern int dot_or_dotdot __P((const char *));
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Jan 14 16:37, Ryan Johnson wrote: On 14/01/2013 3:24 PM, Stephan Mueller wrote: Perhaps (as you may well have already considered): - replace the path prefix by the mount point first? (this may be naïve on my part, but it's not clear to me that .. early in a path should be able to influence which mount point is substituted) If I mount something at /foo/bar/baz, then /foo/bar/baz/../../blah definitely shouldn't end up inside baz. - test directory existence of the component preceding .. before collapsing (in the example above) b/.. to nothing. - trust that for a/b3/b2/b/../../../c, the existence test of a/b3/b2/b before collapsing b/.. to nothing implies existence of b2 and b3 so no further tests are needed for 'runs' of .. components with enough components before them Understood that this is still going to cause a slowdown because paths with .. are not uncommon, but it would reduce the number of additional existence checks from one-per-path-component to one-per-run-of-.., which means none in the case of paths without .. in them. The rest seems totally reasonable to me, FWIW. That's a chicken/egg-like situation. To be able to convert the POSIX path prefix (aka mount point) to the matching DOS path prefix, the path has to be normalized. But without being converted to a DOS path, you can't check path components. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Jan 14 23:14, Thomas Wolff wrote: Am 14.01.2013 11:00, schrieb Corinna Vinschen: ... The first step of converting a POSIX path to a Windows path is to normalize the path. . and .. components are simply dropped: a/b/./c - a\b\c a/b/../c - a\c which isn't correct already (even if everything exists) because if b is a symbolic link, b/.. is *not* . - (I think I came across this bug a few times already without really noticing it as a bug, having taken it as some spurious glitch...) Yes, that's a known downside. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
Hi,
On 15 Jan 2013 03:13, Corinna Vinschen wrote:
It seems to me then that a patch to bash may be in order? I can see how
the bash check is the right thing to do. It doesn't want the special
tilde expansion to mask and disallow referencing of real tilde prefixed
paths. So the stat() check is the quick win to determine that.
From what I make of it, there needs to be a patch that, although can
work generically, adds checks only required for Cygwin. And therefore
is specific to the Cygwin package.
The check would be an extension of the file_exists() function, perhaps
called tilde_file_exists(), which determines if the tilde prefix forms
a directory component of the path (strchr('/')?). If it does not, the
file_exists() check is sufficient. If it does, then the check of if
that directory exists is logically and'ed to the result of
file_exists().
Does that sound about right?
A check like this might be a good idea. Ultimately I would be glad to
be able to come up with more correct code in Cygwin while not getting
slower, of course. But that's wishful thinking for now.
Bash, patched in the way I have described, seems to fix the tab
completion issue.
I will tidy up the work and publish the patch at some point soon. I may
have taken a naive approach, so review comments are welcome.
--
Regards,
Shaddy
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
Greetings, Thomas Wolff! The first step of converting a POSIX path to a Windows path is to normalize the path. . and .. components are simply dropped: a/b/./c - a\b\c a/b/../c - a\c which isn't correct already (even if everything exists) because if b is a symbolic link, b/.. is *not* . - (I think I came across this bug a few times already without really noticing it as a bug, having taken it as some spurious glitch...) (Not sure whether this case is covered by further arguments in this thread) Only if it's a Cygwin symlink. Which I'm avoiding in my daily work, since NTFS now offers the same functionality. -- WBR, Andrey Repin (anrdae...@freemail.ru) 15.01.2013, 23:38 Sorry for my terrible english... -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On 1/15/2013 2:39 PM, Andrey Repin wrote: Greetings, Thomas Wolff! The first step of converting a POSIX path to a Windows path is to normalize the path. . and .. components are simply dropped: a/b/./c - a\b\c a/b/../c - a\c which isn't correct already (even if everything exists) because if b is a symbolic link, b/.. is *not* . - (I think I came across this bug a few times already without really noticing it as a bug, having taken it as some spurious glitch...) (Not sure whether this case is covered by further arguments in this thread) Only if it's a Cygwin symlink. Which I'm avoiding in my daily work, since NTFS now offers the same functionality. Certainly if the native facilities work for you, you should use them. But I think there's been enough discussion in the past on this subject to acknowledge that the native functionality doesn't support all that Cygwin symlinks do. I'm making this (very) brief statement for the benefit of those that come across this in the archives. Anyone seeking clarification or more details should look in the archives for previous discussions on this subject. -- Larry _ A: Yes. Q: Are you sure? A: Because it reverses the logical flow of conversation. Q: Why is top posting annoying in email? -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Jan 14 01:17, Christopher Faylor wrote: On Mon, Jan 14, 2013 at 04:21:25PM +1100, Shaddy Baddah wrote: In investigating this, I believe the issue I am having is due to how stat() handles tilde prefixed paths. On linux we see: linux$ $ python -c 'import os; print os.stat(~/..)' Traceback (most recent call last): File string, line 1, in module OSError: [Errno 2] No such file or directory: '~/..' and on cygwin we see: cygwin$ python -c 'import os; print os.stat(~/..)' posix.stat_result(st_mode=16832, st_ino=562949953496729L, st_dev=4174909669L, st_nlink=1, st_uid=42037, st_gid=10513, st_size=0L, st_atime=1357616166, st_mtime=1357616166, st_ctime=1357616166) It is a bug. It's not just ~. Any nonexistent directory will work, like foo/... And it's a bug which isn't easily fixed. Since about the dawn of time, Cygwin's core path handling evaluates the path in a non-POSIX manner, mainly for performance reasons. POSIX demands to evaluate a path from left to right (thus tripping over the non-existant ~ or foo directory). Windows OTOH skips testing all parent directories of a path, and while this can be changed(*), it's the default setting since the earliest Windows NT versions. So, since Cygwin can't rely on the OS to this job when it has to convert a POSIX path to a Windows path internally, Cygwin would have to check the existence of every single path component from left to right to emulate the POSIX requirements. But that would be a big performance hit, so Cygwin's path handling code tries to be clever to avoid having to call too many OS functions: The first step of converting a POSIX path to a Windows path is to normalize the path. . and .. components are simply dropped: a/b/./c - a\b\c a/b/../c - a\c Then the path prefix is replaced by the matching mount point. Eventually it evaluates the path from right to left. Consider a valid, normalized path with 10 components. Under POSIX rules this requires 10 checks for existence. No problem for the Linux kernel since it has everything under control anyway and the test is blazingly fast. But Cygwin is not the OS so it has to call the necessary OS functions 10 times. By checking from right to left, Cygwin has to call the OS functions only once, if the file exists, two times if the file does not exist, but its parent dir exists, and so on. On top of that, the entire chore has to restart when tripping over a symbolic link. Since the predominant number of file operations are performed on existing paths, or at least paths for which the parent dir exists, Cygwin reduces the number of OS operations to convert a POSIX to a Windows path. The price we're paying is this very deviation from the POSIX standard. Corinna (*) User right Bypass Traverse Checking, by default enabled for everyone. -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
Hi,
On 14/01/13 21:00, Corinna Vinschen wrote:
On Jan 14 01:17, Christopher Faylor wrote:
On Mon, Jan 14, 2013 at 04:21:25PM +1100, Shaddy Baddah wrote:
In investigating this, I believe the issue I am having is due to how
stat() handles tilde prefixed paths. On linux we see:
linux$ $ python -c 'import os; print os.stat(~/..)'
Traceback (most recent call last):
File string, line 1, inmodule
OSError: [Errno 2] No such file or directory: '~/..'
and on cygwin we see:
cygwin$ python -c 'import os; print os.stat(~/..)'
posix.stat_result(st_mode=16832, st_ino=562949953496729L,
st_dev=4174909669L, st_nlink=1, st_uid=42037, st_gid=10513, st_size=0L,
st_atime=1357616166, st_mtime=1357616166, st_ctime=1357616166)
It is a bug. It's not just ~. Any nonexistent directory will
work, like foo/...
And it's a bug which isn't easily fixed. Since about the dawn of time,
Cygwin's core path handling evaluates the path in a non-POSIX manner,
mainly for performance reasons.
Thank you both for your explanations. If my understanding is correct,
stat() will be returning for the current working directory.
It seems to me then that a patch to bash may be in order? I can see how
the bash check is the right thing to do. It doesn't want the special
tilde expansion to mask and disallow referencing of real tilde prefixed
paths. So the stat() check is the quick win to determine that.
From what I make of it, there needs to be a patch that, although can
work generically, adds checks only required for Cygwin. And therefore
is specific to the Cygwin package.
The check would be an extension of the file_exists() function, perhaps
called tilde_file_exists(), which determines if the tilde prefix forms
a directory component of the path (strchr('/')?). If it does not, the
file_exists() check is sufficient. If it does, then the check of if
that directory exists is logically and'ed to the result of
file_exists().
Does that sound about right?
--
Regards,
Shaddy
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Mon, Jan 14, 2013 at 11:00:02AM +0100, Corinna Vinschen wrote: On Jan 14 01:17, Christopher Faylor wrote: On Mon, Jan 14, 2013 at 04:21:25PM +1100, Shaddy Baddah wrote: In investigating this, I believe the issue I am having is due to how stat() handles tilde prefixed paths. On linux we see: linux$ $ python -c 'import os; print os.stat(~/..)' Traceback (most recent call last): File string, line 1, in module OSError: [Errno 2] No such file or directory: '~/..' and on cygwin we see: cygwin$ python -c 'import os; print os.stat(~/..)' posix.stat_result(st_mode=16832, st_ino=562949953496729L, st_dev=4174909669L, st_nlink=1, st_uid=42037, st_gid=10513, st_size=0L, st_atime=1357616166, st_mtime=1357616166, st_ctime=1357616166) It is a bug. It's not just ~. Any nonexistent directory will work, like foo/... And it's a bug which isn't easily fixed. Since about the dawn of time, Cygwin's core path handling evaluates the path in a non-POSIX manner, mainly for performance reasons. POSIX demands to evaluate a path from left to right (thus tripping over the non-existant ~ or foo directory). Windows OTOH skips testing all parent directories of a path, and while this can be changed(*), it's the default setting since the earliest Windows NT versions. So, since Cygwin can't rely on the OS to this job when it has to convert a POSIX path to a Windows path internally, Cygwin would have to check the existence of every single path component from left to right to emulate the POSIX requirements. But that would be a big performance hit, so Cygwin's path handling code tries to be clever to avoid having to call too many OS functions: The first step of converting a POSIX path to a Windows path is to normalize the path. . and .. components are simply dropped: a/b/./c - a\b\c a/b/../c - a\c Then the path prefix is replaced by the matching mount point. Eventually it evaluates the path from right to left. Consider a valid, normalized path with 10 components. Under POSIX rules this requires 10 checks for existence. No problem for the Linux kernel since it has everything under control anyway and the test is blazingly fast. But Cygwin is not the OS so it has to call the necessary OS functions 10 times. By checking from right to left, Cygwin has to call the OS functions only once, if the file exists, two times if the file does not exist, but its parent dir exists, and so on. On top of that, the entire chore has to restart when tripping over a symbolic link. Since the predominant number of file operations are performed on existing paths, or at least paths for which the parent dir exists, Cygwin reduces the number of OS operations to convert a POSIX to a Windows path. The price we're paying is this very deviation from the POSIX standard. Also: c:\dir foo\bar\..\.. Volume in drive S is share Serial number is e620:3c3d Directory of S:\* 1/11/2013 9:58 DIR. 12/26/2012 21:34 DIR.. 1/12/2013 16:27 DIRbin 1/14/2013 10:20 DIRcgf ... I don't have a foo directory but cmd was happy to just ignore that fact and show my the root directory. This is YA place where Windows and Linux differ drastically. cgf -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Jan 14 10:27, Christopher Faylor wrote: On Mon, Jan 14, 2013 at 11:00:02AM +0100, Corinna Vinschen wrote: The first step of converting a POSIX path to a Windows path is to normalize the path. . and .. components are simply dropped: a/b/./c - a\b\c a/b/../c - a\c [...] Also: c:\dir foo\bar\..\.. Volume in drive S is share Serial number is e620:3c3d Directory of S:\* 1/11/2013 9:58 DIR. 12/26/2012 21:34 DIR.. 1/12/2013 16:27 DIRbin 1/14/2013 10:20 DIRcgf ... I don't have a foo directory but cmd was happy to just ignore that fact and show my the root directory. This is YA place where Windows and Linux differ drastically. Indeed. Before writing my mail I tested the GetFullPathName function, and I was not exactly surprised to find that it behaves as you describe for CMD. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Jan 15 01:36, Shaddy Baddah wrote:
Hi,
On 14/01/13 21:00, Corinna Vinschen wrote:
On Jan 14 01:17, Christopher Faylor wrote:
On Mon, Jan 14, 2013 at 04:21:25PM +1100, Shaddy Baddah wrote:
In investigating this, I believe the issue I am having is due to how
stat() handles tilde prefixed paths. On linux we see:
linux$ $ python -c 'import os; print os.stat(~/..)'
Traceback (most recent call last):
File string, line 1, inmodule
OSError: [Errno 2] No such file or directory: '~/..'
and on cygwin we see:
cygwin$ python -c 'import os; print os.stat(~/..)'
posix.stat_result(st_mode=16832, st_ino=562949953496729L,
st_dev=4174909669L, st_nlink=1, st_uid=42037, st_gid=10513, st_size=0L,
st_atime=1357616166, st_mtime=1357616166, st_ctime=1357616166)
It is a bug. It's not just ~. Any nonexistent directory will
work, like foo/...
And it's a bug which isn't easily fixed. Since about the dawn of time,
Cygwin's core path handling evaluates the path in a non-POSIX manner,
mainly for performance reasons.
Thank you both for your explanations. If my understanding is correct,
stat() will be returning for the current working directory.
In the above case, yes.
It seems to me then that a patch to bash may be in order? I can see how
the bash check is the right thing to do. It doesn't want the special
tilde expansion to mask and disallow referencing of real tilde prefixed
paths. So the stat() check is the quick win to determine that.
From what I make of it, there needs to be a patch that, although can
work generically, adds checks only required for Cygwin. And therefore
is specific to the Cygwin package.
The check would be an extension of the file_exists() function, perhaps
called tilde_file_exists(), which determines if the tilde prefix forms
a directory component of the path (strchr('/')?). If it does not, the
file_exists() check is sufficient. If it does, then the check of if
that directory exists is logically and'ed to the result of
file_exists().
Does that sound about right?
A check like this might be a good idea. Ultimately I would be glad to
be able to come up with more correct code in Cygwin while not getting
slower, of course. But that's wishful thinking for now.
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Project Co-Leader cygwin AT cygwin DOT com
Red Hat
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Mon, Jan 14, 2013 at 05:04:17PM +0100, Corinna Vinschen wrote: On Jan 14 10:27, Christopher Faylor wrote: On Mon, Jan 14, 2013 at 11:00:02AM +0100, Corinna Vinschen wrote: The first step of converting a POSIX path to a Windows path is to normalize the path. . and .. components are simply dropped: a/b/./c - a\b\c a/b/../c - a\c [...] Also: c:\dir foo\bar\..\.. Volume in drive S is share Serial number is e620:3c3d Directory of S:\* 1/11/2013 9:58 DIR. 12/26/2012 21:34 DIR.. 1/12/2013 16:27 DIRbin 1/14/2013 10:20 DIRcgf ... I don't have a foo directory but cmd was happy to just ignore that fact and show my the root directory. This is YA place where Windows and Linux differ drastically. Indeed. Before writing my mail I tested the GetFullPathName function, and I was not exactly surprised to find that it behaves as you describe for CMD. Right. It's not just CMD. A standard windows program will behave similarly. cgf -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
RE: stat() and tilde prefix (was bad bash tab completion)
On Jan 14 01:17, Christopher Faylor wrote: It is a bug. It's not just ~. Any nonexistent directory will work, like foo/... Corinna wrote: And it's a bug which isn't easily fixed. Since about the dawn of time, Cygwin's core path handling evaluates the path in a non-POSIX manner, mainly for performance reasons. and: Cygwin would have to check the existence of every single path component from left to right to emulate the POSIX requirements. and: The first step of converting a POSIX path to a Windows path is to normalize the path. . and .. components are simply dropped: a/b/./c - a\b\c a/b/../c - a\c Then the path prefix is replaced by the matching mount point. and: Ultimately I would be glad to be able to come up with more correct code in Cygwin while not getting slower, of course. But that's wishful thinking for now. Perhaps (as you may well have already considered): - replace the path prefix by the mount point first? (this may be naïve on my part, but it's not clear to me that .. early in a path should be able to influence which mount point is substituted) - test directory existence of the component preceding .. before collapsing (in the example above) b/.. to nothing. - trust that for a/b3/b2/b/../../../c, the existence test of a/b3/b2/b before collapsing b/.. to nothing implies existence of b2 and b3 so no further tests are needed for 'runs' of .. components with enough components before them Understood that this is still going to cause a slowdown because paths with .. are not uncommon, but it would reduce the number of additional existence checks from one-per-path-component to one-per-run-of-.., which means none in the case of paths without .. in them. stephan();
Re: stat() and tilde prefix (was bad bash tab completion)
On 14/01/2013 3:24 PM, Stephan Mueller wrote: Perhaps (as you may well have already considered): - replace the path prefix by the mount point first? (this may be naïve on my part, but it's not clear to me that .. early in a path should be able to influence which mount point is substituted) If I mount something at /foo/bar/baz, then /foo/bar/baz/../../blah definitely shouldn't end up inside baz. - test directory existence of the component preceding .. before collapsing (in the example above) b/.. to nothing. - trust that for a/b3/b2/b/../../../c, the existence test of a/b3/b2/b before collapsing b/.. to nothing implies existence of b2 and b3 so no further tests are needed for 'runs' of .. components with enough components before them Understood that this is still going to cause a slowdown because paths with .. are not uncommon, but it would reduce the number of additional existence checks from one-per-path-component to one-per-run-of-.., which means none in the case of paths without .. in them. The rest seems totally reasonable to me, FWIW. I wouldn't put it past a Makefile (esp. one generated by autotools) or a gcc header search path to generate paths like: /a/b/c/d/../e/../f/../../../g/h (which resolves to a/g/h, if I counted dots correctly). Even then, though, it's only three checks to achieve posix-compliant behavior. Ryan -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
Am 14.01.2013 11:00, schrieb Corinna Vinschen: ... The first step of converting a POSIX path to a Windows path is to normalize the path. . and .. components are simply dropped: a/b/./c - a\b\c a/b/../c - a\c which isn't correct already (even if everything exists) because if b is a symbolic link, b/.. is *not* . - (I think I came across this bug a few times already without really noticing it as a bug, having taken it as some spurious glitch...) (Not sure whether this case is covered by further arguments in this thread) -- Thomas -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() and tilde prefix (was bad bash tab completion)
On Mon, Jan 14, 2013 at 04:21:25PM +1100, Shaddy Baddah wrote: In investigating this, I believe the issue I am having is due to how stat() handles tilde prefixed paths. On linux we see: linux$ $ python -c 'import os; print os.stat(~/..)' Traceback (most recent call last): File string, line 1, in module OSError: [Errno 2] No such file or directory: '~/..' and on cygwin we see: cygwin$ python -c 'import os; print os.stat(~/..)' posix.stat_result(st_mode=16832, st_ino=562949953496729L, st_dev=4174909669L, st_nlink=1, st_uid=42037, st_gid=10513, st_size=0L, st_atime=1357616166, st_mtime=1357616166, st_ctime=1357616166) It is a bug. It's not just ~. Any nonexistent directory will work, like foo/... cgf -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: stat() returning EFAULT?
Sjors Gielen, le Tue 24 Feb 2009 16:54:28 +0100, a écrit :
Here's the test case and output:
#include errno.h
int main() {
if(stat(test) != 0) perror(Calling stat() on test);
if(stat(test.exe) != 0) perror(Calling stat() on test.exe);
return 0;
}
Compile with -Wall, you're missing the second argument of stat().
Samuel
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
Re: stat(2) of a directory
Under *nix, I can use stat(2) of a directory to know when its contents have changed (file added or deleted from it). Under cygwin, this doesn't work because a directory's modification time doesn't change when its contents change. Any recommendations as to how to monitor a directory to know when its contents change under cygwin? On Win9x, this is basically impossible, since Windows does not provide that capability (try it - touch . is a noop on cygwin on Win9x). On Windows NT, the capability exists, but since Windows itself is not doing the timestamp update, it would severely slow cygwin down to open the directory and touch the directory's timestamp on every open or unlink, just to implement this requirement of POSIX/SUSv3 properly. It has been suggested in the past that perhaps a 'CYGWIN=posixly_correct' option be added that does all these slower corner cases, but no one has bothered to implement it. -- Eric Blake -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
On 1/15/06, Christopher Faylor [EMAIL PROTECTED] wrote: On Sat, Jan 14, 2006 at 11:37:33PM -0600, Gary R. Van Sickle wrote: [snip] I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We Huh? Has it even been 24 hours since you suggested Cygwin be changed in a non-standardized manner merely to band-aid a broken third-party IRC client? The fact that you have this opinion makes me think that you and other people are not entirely clear on what we're supposed to be doing here even though it only takes one sentence to explain things: We're trying to emulate linux. So, think about this sentence and then explain how it jives with your assertion that I was trying to do something in a non-standardized manner. How is it non-standardized to make cygwin emulate linux more closely so that programs which build on linux are more likely to build on cygwin? It's curious that you'd see a parallel between modifying cygwin to accommodate a specific broken commerical virus scanner and modifying Cygwin in such a way that more programs are likely to build with it. I don't get that at all. The slope you mention has already been visited on more than one occaision. Sorry, but you'll have to come up with more convincing examples than the ones you've used. I'm sure that, through the years, I've probably said sorry that's the way it works to people who've complained that cygwin didn't work like linux, but my opinion has been evolving since the goal of the project was changed to target a specific system (linux). I think I've made the point a few times in the last year that linux is the target that cygwin is shooting for so I don't see any inconsistency here at all. I'm fairly surprised that you're arguing this point at all. When I saw that you'd responded to this thread, I thought I'd be seeing a rant about stupid virus checkers and how Cygwin shouldn't be going out of its way to work with them. I didn't expect that you'd be arguing the issue (or the meta-issue) of precedent for allowing such a change. And doesn't Cygwin still create sparse files for the benefit of one single third-party application? The slope you mention has already been visited on more than one occaision. Cygwin creates sparse files in some situations because that's how linux/unix works. IIRC, we actually made an accommodation to windows to not create sparse files as often as linux because people complained about the default linux behavior. I don't recall doing this for a third-party windows application, however. I thought we just did it because linux works that way but I may be misremembering. If the change was to make a *linux* program work better then I think we're once again on pretty firm ground. We were, once again, changing things to make it easier for linux programs to port without problem to Cygwin. And, once again, this is nowhere near the same thing as modifying cygwin to accommodate a broken virus checker. [snip] However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this is what I would call a special concession to third party software and I'm not interested in littering the code with those. Again, that last sentence is simply not a true statement, unless you want to split hairs about the littering part. And I have to question the veracity of a PTC statement that has as its prerequisites that the patch involve no actual code. If you didn't get what I was saying, I'm wondering why you couldn't just ask for clarification without questioning the truthfulness of what I said. However, let me try to clarify again. If someone found that one function caused a problem but another equivalent function didn't, then using the equivalent function would be ok. If someone found that a 10 line test was necessary to determine if a virus scanner was running then that is not something that I would want to do. If someone found that avoiding the use of pipes and using mailslots instead fixed things, then that would not be ok. I
Re: stat(2) triggers on-demand virus scan
On Mon, Jan 16, 2006 at 10:42:10PM -0600, * * wrote: On 1/15/06, Christopher Faylor [EMAIL PROTECTED] wrote: On Sat, Jan 14, 2006 at 11:37:33PM -0600, Gary R. Van Sickle wrote: [snip] I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We Huh? Has it even been 24 hours since you suggested Cygwin be changed in a non-standardized manner merely to band-aid a broken third-party IRC client? The fact that you have this opinion makes me think that you and other people are not entirely clear on what we're supposed to be doing here even though it only takes one sentence to explain things: We're trying to emulate linux. So, think about this sentence and then explain how it jives with your assertion that I was trying to do something in a non-standardized manner. How is it non-standardized to make cygwin emulate linux more closely so that programs which build on linux are more likely to build on cygwin? It's curious that you'd see a parallel between modifying cygwin to accommodate a specific broken commerical virus scanner and modifying Cygwin in such a way that more programs are likely to build with it. I don't get that at all. The slope you mention has already been visited on more than one occaision. Sorry, but you'll have to come up with more convincing examples than the ones you've used. I'm sure that, through the years, I've probably said sorry that's the way it works to people who've complained that cygwin didn't work like linux, but my opinion has been evolving since the goal of the project was changed to target a specific system (linux). I think I've made the point a few times in the last year that linux is the target that cygwin is shooting for so I don't see any inconsistency here at all. I'm fairly surprised that you're arguing this point at all. When I saw that you'd responded to this thread, I thought I'd be seeing a rant about stupid virus checkers and how Cygwin shouldn't be going out of its way to work with them. I didn't expect that you'd be arguing the issue (or the meta-issue) of precedent for allowing such a change. And doesn't Cygwin still create sparse files for the benefit of one single third-party application? The slope you mention has already been visited on more than one occaision. Cygwin creates sparse files in some situations because that's how linux/unix works. IIRC, we actually made an accommodation to windows to not create sparse files as often as linux because people complained about the default linux behavior. I don't recall doing this for a third-party windows application, however. I thought we just did it because linux works that way but I may be misremembering. If the change was to make a *linux* program work better then I think we're once again on pretty firm ground. We were, once again, changing things to make it easier for linux programs to port without problem to Cygwin. And, once again, this is nowhere near the same thing as modifying cygwin to accommodate a broken virus checker. [snip] However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this is what I would call a special concession to third party software and I'm not interested in littering the code with those. Again, that last sentence is simply not a true statement, unless you want to split hairs about the littering part. And I have to question the veracity of a PTC statement that has as its prerequisites that the patch involve no actual code. If you didn't get what I was saying, I'm wondering why you couldn't just ask for clarification without questioning the truthfulness of what I said. However, let me try to clarify again. If someone found that one function caused a problem but another equivalent function didn't, then using the equivalent function would be ok. If someone found that a 10 line test was necessary to determine if a virus scanner was running then that is not something that I would want to do. If someone found that avoiding the use of pipes and using mailslots
Re: stat(2) triggers on-demand virus scan
Gary R. Van Sickle wrote: [snip] I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We Huh? Has it even been 24 hours since you suggested Cygwin be changed in a non-standardized manner merely to band-aid a broken third-party IRC client? And doesn't Cygwin still create sparse files for the benefit of one single third-party application? The slope you mention has already been visited on more than one occaision. I'm sorry, but IMO this is /not/ the same thing. What CGF suggested was to implement things in the same manner as Linux - which is one of the main goals of this project - allowing gcc to behaving in non-ansi mode by default. This would be a major undertaking, yes, as AFAICT it involves updating, creating, and generally rewriting more than a few header files in order for it to work without adversely affecting how things work now. As for creating sparse files - imo this is better than how windows normally does it, regardless of the app in question. [snip] However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this is what I would call a special concession to third party software and I'm not interested in littering the code with those. Again, that last sentence is simply not a true statement, unless you want to split hairs about the littering part. And I have to question the veracity of a PTC statement that has as its prerequisites that the patch involve no actual code. Matter of interpretation. IMO, CGF is saying he doesn't want large quantities of complex code. A small patch that altered cygwin's behaviour in such a way so as to be more friendly towards ZA or various AV products would be more than sufficient. Perhaps Corinna has a different opinion and will convince me otherwise but, until that time, I just thought I would make the ground rules clear. I thought this was obvious stuff but I guess it wasn't. No, and I guess it still isn't. Is to me. BTW, OP: Update your 1.3.x install. It's the 21st century for God's sake. On this I wholeheartedly agree. -- Spinning complacently in the darkness, covered and blinded by a blanket of little lives, false security has lulled the madness of this world into a slumber. Wake up! An eye is upon you, staring straight down and keenly through, seeing all that you are and everything that you will never be. Yes, an eye is upon you, an eye ready to blink. So face forward, with arms wide open and mind reeling. Your future has arrived... Are you ready to go? -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
pmcferrin wrote: Here is something a little OT When making comparisons between multiple runs to run timing tests before and after a change, it there a way I can guarantee more consistent results? e.g. Condider the following: time find . -print | wc -l I can run the above sequence 3 times in a row and get huge differences due to OS caching and probably application caching (281 secs, 11 secs, 0.8 secs respectively). I ran my tests (earlier in this thread) so many times that I got consistent timings - this gave the figures I presented. All interference from other software eliminated/minimized; the only way I can think of getting consistent run times. Adding anythng at all gives uncertainty and varying figures. Your measurements will most likely differ, even to a great extent - dependeing on how much junk (i.e. SW) you have installed. I consider this to be a best possible state for running the test; one can't get any better (i.e. shorter) run times without removing selected utilities and services - degrading the usability of the machine in the process. This clearly shows one known weak side of general operating systems - most of the time there is so much different things going on that you can't count on any predefined completion time for a given task. This problem is addressed in RTOS'es and the like, and also is the subject for research - e.g. some of the projects here http://www.sics.se/cna/software.html I believe. When it comes to practical cygwin use, this run time inconsistency shows even more - cygwin could be considered an OS (POSIX emulation) on top of another OS (Win = ! Loose = Not Loose = Loose Nuts ;-). Is there any known way within MS XP Pro to flush all caching other than a reboot?? Maybe CacheSet from sysinternals.com ? - http://www.sysinternals.com/Utilities/CacheSet.html I have NOT tried it on XP, and thus can't recommend it in any way. AFAIK it should mimic the functionality of the [vcache] section settings in older windows WIN.INI file (or was it SYSTEM.INI?). I run into similar problems when validating multiple copies of a CD-R by calculating the checksum. The first checksum is valid but I can't trust the remainder due to OS caching. You don't trust the OS to do cache handling correctly on CD-R's with exactly the same contents, well what can I say - how would you distinguish the difference between those CD-R's? Somewhere long ago I've read that when running CMD.EXE - hitting CTRL-C at the prompt should re-read disk info for 'CWD' IIRC (this was a long time ago; in the age of 3.5 diskettes) ... I dunno if this still is true. How about; Insert and briefly access a different CD in between, making the OS/fs aware of disk changes? -Paul SNIP /H -- -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
On Sat, Jan 14, 2006 at 11:37:33PM -0600, Gary R. Van Sickle wrote: [snip] I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We Huh? Has it even been 24 hours since you suggested Cygwin be changed in a non-standardized manner merely to band-aid a broken third-party IRC client? The fact that you have this opinion makes me think that you and other people are not entirely clear on what we're supposed to be doing here even though it only takes one sentence to explain things: We're trying to emulate linux. So, think about this sentence and then explain how it jives with your assertion that I was trying to do something in a non-standardized manner. How is it non-standardized to make cygwin emulate linux more closely so that programs which build on linux are more likely to build on cygwin? It's curious that you'd see a parallel between modifying cygwin to accommodate a specific broken commerical virus scanner and modifying Cygwin in such a way that more programs are likely to build with it. I don't get that at all. The slope you mention has already been visited on more than one occaision. Sorry, but you'll have to come up with more convincing examples than the ones you've used. I'm sure that, through the years, I've probably said sorry that's the way it works to people who've complained that cygwin didn't work like linux, but my opinion has been evolving since the goal of the project was changed to target a specific system (linux). I think I've made the point a few times in the last year that linux is the target that cygwin is shooting for so I don't see any inconsistency here at all. I'm fairly surprised that you're arguing this point at all. When I saw that you'd responded to this thread, I thought I'd be seeing a rant about stupid virus checkers and how Cygwin shouldn't be going out of its way to work with them. I didn't expect that you'd be arguing the issue (or the meta-issue) of precedent for allowing such a change. And doesn't Cygwin still create sparse files for the benefit of one single third-party application? The slope you mention has already been visited on more than one occaision. Cygwin creates sparse files in some situations because that's how linux/unix works. IIRC, we actually made an accommodation to windows to not create sparse files as often as linux because people complained about the default linux behavior. I don't recall doing this for a third-party windows application, however. I thought we just did it because linux works that way but I may be misremembering. If the change was to make a *linux* program work better then I think we're once again on pretty firm ground. We were, once again, changing things to make it easier for linux programs to port without problem to Cygwin. And, once again, this is nowhere near the same thing as modifying cygwin to accommodate a broken virus checker. [snip] However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this is what I would call a special concession to third party software and I'm not interested in littering the code with those. Again, that last sentence is simply not a true statement, unless you want to split hairs about the littering part. And I have to question the veracity of a PTC statement that has as its prerequisites that the patch involve no actual code. If you didn't get what I was saying, I'm wondering why you couldn't just ask for clarification without questioning the truthfulness of what I said. However, let me try to clarify again. If someone found that one function caused a problem but another equivalent function didn't, then using the equivalent function would be ok. If someone found that a 10 line test was necessary to determine if a virus scanner was running then that is not something that I would want to do. If someone found that avoiding the use of pipes and using mailslots instead fixed things, then that would not be ok. Basically, trivial changes which involve minimal perturbation of the code are ok. Changes which specifically test for a broken virus scanner are not
RE: stat(2) triggers on-demand virus scan
pmcferrin wrote: The stat(2) system call runs very slowly because it is constantlt triggering the McAfee on-demand virus scanner to scan the file that is being stat'ed. This may not seem like a big thing but I frequently stat thousands of files at a batch. I find that the stat runs much faster when I temporarily disable the on-demand virus scanner. Judging from previous messages on this list it *seems* that one of the slowest things you can do in cygwin is accessing files; stat(), fopen() and the like. In general... FWIW/IMO; If you have the option to replace M*Af**[1] with a just as good an AV, then do that - I suggest to avoid Sym*ntec[2] products too as they seem to have similar problems. OTOH, I have good experience with what you find at f-secure dot com - I've had this one installed since cygwin 1.3.x was current, and prior to that. I've always considered S. and M. AV's to be CPU hogs in general terms - and have found f-secure to be much lighter in this respect. Now I wonder how M. and S. AV's compare to what I have done in a simple (attached) comparasion with fsecure V5.30 ON/OFF (Use e.g. NOTEPAD, and a monospace font to view it) /H [1] I've got previous experience with having it on my private PC. [2] I'm forced to live with such a thing at work. -- Test object: Windows partition with some additional SW installed Included on this disk is a huge cygwin installation. Test run several times prior taking these samples. Also making sure it ran without interference from other running software - this was to ensure somewhat persistent timings. AV ON AV OFF find =prints= 201195 (files+dirs) real30.089 28.165 27.875 real27.547 28.113 27.988 user 5.576 5.498 5.779 user 5.529 5.732 5.451 sys 23.966 22.123 21.638 sys 21.562 21.842 21.874 find - per file/dir, microseconds calculated from the above file/dir count 150 140 139 137 140 139 du -s =prints= 7431252 real87.608 88.285 87.523 real43.355 41.916 41.815 user 8.155 8.037.89 user 7.358 8.015 7.624 sys 77.156 77.905 77.734 sys 33.531 32.062 32.312 du - per file/dir, microseconds calculated from the above file/dir count 435 439 435 215 208 208 From this it seems that du does something that triggers the f-secure AV in some way (AV doing the same as du?). This has the impact of doubling the scan time per file/dir. It would be interesting to see similar measurements done with McAFee and Symantec antivirus packages. -- actual test session log follows -- $ cd W: # Sempron 2800+, 1G RAM - has Win2K and related files on W: --- AV enabled -- $ for (( i=0; i3 ;i++ )) ;do time find 2/dev/null -printf %f\n | wc -l ;done 201195 real0m30.089s user0m5.576s sys 0m23.966s 201195 real0m28.165s user0m5.498s sys 0m22.123s 201195 real0m27.875s user0m5.779s sys 0m21.638s 201195 -- AV disabled -- $ for (( i=0; i3 ;i++ )) ;do time find 2/dev/null -printf %f\n | wc -l ;done 201195 real0m27.547s user0m5.529s sys 0m21.562s 201195 real0m28.113s user0m5.732s sys 0m21.842s 201195 real0m27.988s user0m5.451s sys 0m21.874s 201195 $ for (( i=0; i3 ;i++ )) ;do time du -s 2/dev/null ;done 7431252 . real0m43.355s user0m7.358s sys 0m33.531s 7431252 . real0m41.916s user0m8.015s sys 0m32.062s 7431252 . real0m41.815s user0m7.624s sys 0m32.312s 7431252 . -- AV enabled -- $ for (( i=0; i3 ;i++ )) ;do time du -s 2/dev/null ;done 7431252 . real1m27.608s user0m8.155s sys 1m17.156s 7431252 . real1m28.285s user0m8.030s sys 1m17.905s 7431252 . real1m27.523s user0m7.890s sys 1m17.734s 7431252 . $ uname -a CYGWIN_NT-5.0 amd 1.5.19s(0.149/4/2) 20051229 16:10:48 i686 unknown unknown Cygwin -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
The stat(2) system call runs very slowly because it is constantlt triggering the McAfee on-demand virus scanner to scan the file that is being stat'ed. This may not seem like a big thing but I frequently stat thousands of files at a batch. I find that the stat runs much faster when I temporarily disable the on-demand virus scanner. Judging from previous messages on this list it *seems* that one of the slowest things you can do in cygwin is accessing files; stat(), fopen() and the like. In general... FWIW/IMO; If you have the option to replace M*Af**[1] with a just as good an AV, then do that - I suggest to avoid Sym*ntec[2] products too as they seem to have similar problems. I've reported similar issues with ZoneAlarm and its on-demand scanning. This is the price that is paid on WinDoze for a generally less sure environment and having to compensate with such tools. That being said, I don't believe its reasonable to just give up or constrain one's self to particular security products as a work around, although their may be other reasons. The choice of a given firewall or security tool may not be an option, should this prohibit one from using cygwin effectively in any given environment? I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I've started to notice that even on one system that I have that is relatively isolated, and not encumbered with security software, bash, ssh, sh and other programs seems to consume an inordinate amount of CPU. This thread points to stat, I've been looking at process creation, perhaps it is a bit of both. WinDoze systems tend to create monolithic programs, where as UNIX/Linux/POSIX tend to create lots of small programs and invoke them often, which inherently causes more file accesses and process creation. I suspect their are multiple issues, I also suspect they can eventually be addressed. Brett Brett C. Serkez, Techie -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that cygwin work around things like this. Maybe this is a given but I thought I'd just make it clear. I've started to notice that even on one system that I have that is relatively isolated, and not encumbered with security software, bash, ssh, sh and other programs seems to consume an inordinate amount of CPU. Try setting CYGWIN=noinordinate . cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. ZoneLabs offical stance is that they don't support emulated environments. Humm... So if neither are willing to change, then what? I don't know Symantec's or McAfee's offical stance. As far as coding being 'perfectly acceptable', that is a matter of point-of- view. If it causes such behavior, is it acceptable? While I respect the purist point of view, one I've held over the years, seems that we need to be practical sometimes. Are you saying that if the problem could be isolated, and reasonable changes proposed, you wouldn't make/allow them? Do we want IT administrators to utilize Cygwin to integrate with the Linux/UNIX environment? If this means not being able to effectively use Cygwin if they are also required to run ZoneAlarm, Norton, McAfee, what choice do you think they'll make, or more precisely, their management will make on their behalf? Since we ultimately don't know the root cause, this discussion is premature, however if the group isn't going to be open to changes, there is no point trying to find them, time would be better spent finding alternates. That would be ashame as I think Cygwin is the most progessive tool available for IT and development work. Certainly for those attempting to bridge the Linux/UNIX - Windows environment. Brett Brett C. Serkez, Techie -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
Brett Serkez wrote: On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. ZoneLabs offical stance is that they don't support emulated environments. Humm... So if neither are willing to change, then what? I don't know Symantec's or McAfee's offical stance. They're likely to be the same. While cgf's statement could be interpreted in the same vein, you have to look at it from other points of view as well.. See comments below. As far as coding being 'perfectly acceptable', that is a matter of point-of- view. If it causes such behavior, is it acceptable? Cygwin is not what is at fault here - it is the way many on-demand virus scanners interact with the OS, the OS itself, and how ZA hooks in to the net-code, that cause these issues. Cygwin code is correct according to ms sdk and available documentation - it uses the correct and most accurate methods to implement POSIX functionality and *nix integration that are available and known. (Note that this statement is not entirely accurate, or the next would not be at all) Obviously, as time goes on, improvements can be made, but that is the nature of software development. While I respect the purist point of view, one I've held over the years, seems that we need to be practical sometimes. Are you saying that if the problem could be isolated, and reasonable changes proposed, you wouldn't make/allow them? Do we want IT administrators to utilize Cygwin to integrate with the Linux/UNIX environment? If this means not being able to effectively use Cygwin if they are also required to run ZoneAlarm, Norton, McAfee, what choice do you think they'll make, or more precisely, their management will make on their behalf? If a change could be made to correct the issues that cygwin has on systems that have these products 'inflicted' upon them [1], without causing any reduction in performance in other circumstances, making the code vastly more complex, or requiring additional resources or user intervention, then I suspect it would become a PTC issue. [1] - I choose the word inflicted deliberately - in my experience, norton and mcafee are very bad AV products, and only getting worse as they forcibly integrate other products into them. They frequently miss genuine threats and generate false positives, and fail thorough tests on a regular basis. As a rule, IT admins have sufficient say that they can change company policy in regards to AV and firewall software. Indeed, in a corporate environment, it is *extremely* unusual to come across software firewalls being in use, beyond possibly an implementation of IPTables filtering traffic between the network and the internet, and between remote sites. Usually, there will be hardware firewalls, such as FW1, Cisco PIX, or Nokia's firewall product, whose name I forget. If you name a circumstance where office-based machines require a software firewall, a strong argument can be made as to how and why that network has not been properly secured. In my opinion, this also applies to on-access virus scanners. Feel free to ask me why in a direct email, that would be OT for this list. Since we ultimately don't know the root cause, this discussion is premature, however if the group isn't going to be open to changes, there is no point trying to find them, time would be better spent finding alternates. That would be ashame as I think Cygwin is the most progessive tool available for IT and development work. Certainly for those attempting to bridge the Linux/UNIX - Windows environment. In all honest, you have three options that can be used within windows. Cygwin, MinGW, and SFU. Of those, MinGW is not really all that well suited to these circumstances, and SFU does not have nearly the range of capabilities that Cygwin has. While cgf is on record as saying he does not want to work around issues with other software, if evidence were to emerge to show cygwin could use another method without any detriment, I imagine it would be considered. However, specifically with regards to on-access AV, I don't believe there will be another way to deal with
Re: stat(2) triggers on-demand virus scan
On Sat, Jan 14, 2006 at 04:18:43PM -0500, Brett Serkez wrote: On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We already have more than enough to worry about with different windows versions. Adding the combinatorial nightmare of worrying about different windows versions * different virus scanners is not something I'm really interested in. Even if we did want to do this, how do we decide what software we should make concessions for? You're using ZoneAlarm. Is that a really popular package among windows users? If so, do we need a dedicated ZoneAlarm specialist on staff to make sure that every change that Corinna or I make to Cygwin does not cause problems for ZoneAlarm? How about McAfee? Do we need someone to worry about them, too? How about sysinternals? I think some of their software causes cygwin to behave strangely. Do we need a Sysinternals expert? Or do we just pay attention to the loudest voice and then pull the concession code out of Cygwin when the voice goes away because Corinna and I can no longer test and support it? ZoneLabs offical stance is that they don't support emulated environments. Humm... So if neither are willing to change, then what? I don't know Symantec's or McAfee's offical stance. Cygwin is a program which uses standard the win32 api. The fact that the win32 api is used to present a bash prompt is no different than using the win32 api to present a word processor screen. Assuming that the emulated environment above actually refers to Cygwin then failure on Zonealarm's part to fix bugs that cause Cygwin's use of the windows API to misbehave is an arbitrary distinction and a cop-out. As far as coding being 'perfectly acceptable', that is a matter of point-of- view. If it causes such behavior, is it acceptable? It is not a matter of a point of view if code works as documented in a virus-scanner-free environment and fails to work when a virus scanner is installed. However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this is what I would call a special concession to third party software and I'm not interested in littering the code with those. Perhaps Corinna has a different opinion and will convince me otherwise but, until that time, I just thought I would make the ground rules clear. I thought this was obvious stuff but I guess it wasn't. cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
[snip] We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. I'm hoping and assuming it is going to be more a matter of making minor changes, if it requires a major change, then it is more likely Microsoft or some other vendor is at fault. [snip] ZoneLabs offical stance is that they don't support emulated environments. Humm... So if neither are willing to change, then what? I don't know Symantec's or McAfee's offical stance. Cygwin is a program which uses standard the win32 api. The fact that the win32 api is used to present a bash prompt is no different than using the win32 api to present a word processor screen. Assuming that the emulated environment above actually refers to Cygwin then failure on Zonealarm's part to fix bugs that cause Cygwin's use of the windows API to misbehave is an arbitrary distinction and a cop-out. Strongly agreed. I've already pointed this out to them to no avail. As far as coding being 'perfectly acceptable', that is a matter of point-of- view. If it causes such behavior, is it acceptable? It is not a matter of a point of view if code works as documented in a virus-scanner-free environment and fails to work when a virus scanner is installed. From what I've been seeing, I'm starting to suspect that the problem(s) is there in both cases, the scanner simply makes it much more noticable. I do see more CPU consumption that I woud have expected even without the virus scanner and the original poster's calling out stat was most interesting. [snip] Brett Brett C. Serkez, Techie -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
Boy did I open a can of worms! When I looked at the source of Cygwin1.dll a few years ago at the time, the stat(2) basically called a MS API function to retreive the information and then did a simpe return. I think it the faulty design of MS not to privide a function to get status information without triggering all sorts of other OS's overhead (e.g. on-demand scanning). If the stat(2) is following the MS SDK guidelines for POSTIX compatability, then I don't see much other attractive recourse but live with MS supid design. After it *is* MS. Unless there is some obsolete non-POSTIX MS API that retrieves this information that does not have this side effect, then I'll live with it. It does seem silly to me that you can't perform a stat(2) without triggering side effects. Maybe I'm too much of a Unix Geru. Here is something a little OT When making comparisons between multiple runs to run timing tests before and after a change, it there a way I can guarantee more consistent results? e.g. Condider the following: time find . -print | wc -l I can run the above sequence 3 times in a row and get huge differences due to OS caching and probably application caching (281 secs, 11 secs, 0.8 secs respectively). Is there any known way within MS XP Pro to flush all caching other than a reboot?? I run into similar problems when validating multiple copies of a CD-R by calculating the checksum. The first checksum is valid but I can't trust the remainder due to OS caching. -Paul Christopher Faylor wrote: On Sat, Jan 14, 2006 at 04:18:43PM -0500, Brett Serkez wrote: On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We already have more than enough to worry about with different windows versions. Adding the combinatorial nightmare of worrying about different windows versions * different virus scanners is not something I'm really interested in. Even if we did want to do this, how do we decide what software we should make concessions for? You're using ZoneAlarm. Is that a really popular package among windows users? If so, do we need a dedicated ZoneAlarm specialist on staff to make sure that every change that Corinna or I make to Cygwin does not cause problems for ZoneAlarm? How about McAfee? Do we need someone to worry about them, too? How about sysinternals? I think some of their software causes cygwin to behave strangely. Do we need a Sysinternals expert? Or do we just pay attention to the loudest voice and then pull the concession code out of Cygwin when the voice goes away because Corinna and I can no longer test and support it? ZoneLabs offical stance is that they don't support emulated environments. Humm... So if neither are willing to change, then what? I don't know Symantec's or McAfee's offical stance. Cygwin is a program which uses standard the win32 api. The fact that the win32 api is used to present a bash prompt is no different than using the win32 api to present a word processor screen. Assuming that the emulated environment above actually refers to Cygwin then failure on Zonealarm's part to fix bugs that cause Cygwin's use of the windows API to misbehave is an arbitrary distinction and a cop-out. As far as coding being 'perfectly acceptable', that is a matter of point-of- view. If it causes such behavior, is it acceptable? It is not a matter of a point of view if code works as documented in a virus-scanner-free environment and fails to work when a virus scanner is installed. However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this
Re: Stat(2) trigger on-demand virus scan
Brett Serkez wrote: [snip] From what I've been seeing, I'm starting to suspect that the problem(s) is there in both cases, the scanner simply makes it much more noticable. I do see more CPU consumption that I woud have expected even without the virus scanner and the original poster's calling out stat was most interesting. Interesting observation.. I just assummed it was the stat(2) call since the only thing I was doing was a find ... file. I know that find (or at least the SysV version) does a readdir(3) syscall and I just totally ignored it. Even it it was readdir(3), MS should not be triggering on-demand scanner hooks for directory-only operations. -paul -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Stat(2) trigger on-demand virus scan
On Sat, Jan 14, 2006 at 09:12:11PM -0500, Paul McFerrin wrote: Brett Serkez wrote: From what I've been seeing, I'm starting to suspect that the problem(s) is there in both cases, the scanner simply makes it much more noticable. I do see more CPU consumption that I woud have expected even without the virus scanner and the original poster's calling out stat was most interesting. Interesting observation.. I just assummed it was the stat(2) call since the only thing I was doing was a find ... file. I know that find (or at least the SysV version) does a readdir(3) syscall and I just totally ignored it. Even it it was readdir(3), MS should not be triggering on-demand scanner hooks for directory-only operations. I'd suggest taking a look at the source code before drawing any conclusions about what should or shouldn't be happening. stat() is, and always has been, a complicated function. readdir() isn't quite as complicated but it is not just a trivial wrapper around a windows function. Very little in cygwin is that trivial and assuming that things are that simple will certainly lead you to draw faulty conclusions. cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
[snip] I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We Huh? Has it even been 24 hours since you suggested Cygwin be changed in a non-standardized manner merely to band-aid a broken third-party IRC client? And doesn't Cygwin still create sparse files for the benefit of one single third-party application? The slope you mention has already been visited on more than one occaision. [snip] However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this is what I would call a special concession to third party software and I'm not interested in littering the code with those. Again, that last sentence is simply not a true statement, unless you want to split hairs about the littering part. And I have to question the veracity of a PTC statement that has as its prerequisites that the patch involve no actual code. Perhaps Corinna has a different opinion and will convince me otherwise but, until that time, I just thought I would make the ground rules clear. I thought this was obvious stuff but I guess it wasn't. No, and I guess it still isn't. BTW, OP: Update your 1.3.x install. It's the 21st century for God's sake. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
From: Paul McFerrin Sent: Saturday, January 14, 2006 5:19 PM To: Cygwin@Cygwin.com Subject: Re: stat(2) triggers on-demand virus scan Boy did I open a can of worms! No, it's like this on a regular, periodic basis. When I looked at the source of Cygwin1.dll a few years ago at the time, the stat(2) basically called a MS API function to retreive the information and then did a simpe return. I think it the faulty design of MS not to privide a function to get status information without triggering all sorts of other OS's overhead (e.g. on-demand scanning). If the stat(2) is following the MS SDK guidelines for POSTIX ...um, POSIX. It's POSIX. compatability, then I don't see much other attractive recourse but live with MS supid design. What Chris F. said. Plus, while I refuse to defend the ability of MS's Operating Systems to properly work with Disks (admittedly it's a new thing for them), stat's definition and/or the way many programs use stat is the real culprit here. After it *is* MS. Unless there is some obsolete non-POSTIX MS API that retrieves this information that does not have this side effect, then I'll live with it. It does seem silly to me that you can't perform a stat(2) without triggering side effects. Maybe I'm too much of a Unix Geru. Here is something a little OT When making comparisons between multiple runs to run timing tests before and after a change, it there a way I can guarantee more consistent results? e.g. Condider the following: time find . -print | wc -l I can run the above sequence 3 times in a row and get huge differences due to OS caching and probably application caching (281 secs, 11 secs, 0.8 secs respectively). Is there any known way within MS XP Pro to flush all caching other than a reboot?? The short non-rant answer to that is no, there isn't. The long non-rant answer would require a logical explanation from Microsoft as to why their filesystem infrastructure is completely incapable of properly handling removable media, and that is highly unlikely to be forthcoming. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
From: Brett Serkez Sent: Saturday, January 14, 2006 3:19 PM To: cygwin@cygwin.com; cygwin@cygwin.com Subject: Re: stat(2) triggers on-demand virus scan On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. ZoneLabs offical stance is that they don't support emulated environments. I have to assume whoever said or wrote that was either thinking Wine, or not thinking at all, since Cygwin is ultimately no different than any other Windows application from their software's perspective. Humm... So if neither are willing to change, then what? I don't know Symantec's or McAfee's offical stance. Last I checked it was cause more problems than the viruses we purportedly protect you from would. Look guys, the bottom line here is that on-access virus scanners cause trouble. Not just for Cygwin, and not just particular ones. Scan your incoming email, scan your downloads, do your backups, cross your fingers, and hope for a horrible death for the virus-writing idiots of the world. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
On Sat, Jan 14, 2006 at 12:56:11AM -0500, Paul McFerrin wrote: I have an antique question. I'm currently running cygwin.dll version: 1.3.6 ! (don't laugh). I use cygwin daily in my work and I'm happy not to disturb things that are not broken. The stat(2) system call runs very slowly because it is constantlt triggering the McAfee on-demand virus scanner to scan the file that is being stat'ed. This may not seem like a big thing but I frequently stat thousands of files at a batch. I find that the stat runs much faster when I temporarily disable the on-demand virus scanner. When I examined the code within cygwin1.dll, I really didn't see anything cygwin was doing to trigger this behavior so I assummed it was a MS feature. I'm using XP Pro with NO SP2. Has the behavior of the stat changed over the years that might solve this problem? It *might* prompt me to upgrade if this behavior has been improved. It is a novel concept, but why not just try it? Back up your cygwin directory, install a new version, and see if there is any difference. If you don't like what you see, then restore the directory. It's either that or trust some random mailing list voice to say yes or no and then find out for yourself just how authoritative they really are. Given that every installation is different, no one is going to be able to give you a definitive answer anyway. cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat file -- cygwin vs. Windows size?
At 01:06 PM 6/24/2005, you wrote: This buffer is being built for SpamAssassin which later gives an error saying (to the effect) Content-Length mismatch: Expected 818 bytes, got 798 bytes My suspicion is that stat is counting cr-lf as two characters but the input routines are treating these as one. If the file has about 20 lines, then that's 20 missing characters??? Yes, this is right. And yes, this could be the cause of the situation you're noticing. -- Larry Hall http://www.rfk.com RFK Partners, Inc. (508) 893-9779 - RFK Office 838 Washington Street (508) 893-9889 - FAX Holliston, MA 01746 -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
My suspicion is that stat is counting cr-lf as two characters but the input routines are treating these as one. If the file has about 20 lines, then that's 20 missing characters??? Yes, this is right. And yes, this could be the cause of the situation you're noticing. Is there a standard Cygwin 'idiom' or function for dealing with this mismatch, or should I just re-invent the wheel. Seems like I read (skimmed) something related to this in the Cygwin manual, probably near the back in the programming introduction I know I picked up the concept somewhere (somewhere recent that is, as I have dealt with this across at least five different OS conventions but not recently and specifically on Cygwin.) -- Thanks, Herb -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
On Fri, 24 Jun 2005, Herb Martin wrote: My suspicion is that stat is counting cr-lf as two characters but the input routines are treating these as one. If the file has about 20 lines, then that's 20 missing characters??? Yes, this is right. And yes, this could be the cause of the situation you're noticing. Is there a standard Cygwin 'idiom' or function for dealing with this mismatch, or should I just re-invent the wheel. Sure -- just force binary mode on the file (i.e., open it using O_BINARY for the open() call, or the rb mode for the fopen() call). The mount type only applies if the mode is unspecified. Seems like I read (skimmed) something related to this in the Cygwin manual, probably near the back in the programming introduction I know I picked up the concept somewhere (somewhere recent that is, as I have dealt with this across at least five different OS conventions but not recently and specifically on Cygwin.) HTH, Igor -- http://cs.nyu.edu/~pechtcha/ |\ _,,,---,,_[EMAIL PROTECTED] ZZZzz /,`.-'`'-. ;-;;,_[EMAIL PROTECTED] |,4- ) )-,_. ,\ ( `'-' Igor Pechtchanski, Ph.D. '---''(_/--' `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-. Meow! The Sun will pass between the Earth and the Moon tonight for a total Lunar eclipse... -- WCBS Radio Newsbrief, Oct 27 2004, 12:01 pm EDT -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
On Fri, 24 Jun 2005, Igor Pechtchanski wrote:
On Fri, 24 Jun 2005, Herb Martin wrote:
My suspicion is that stat is counting cr-lf as two characters but the
input routines are treating these as one.
If the file has about 20 lines, then that's 20 missing characters???
Yes, this is right. And yes, this could be the cause of the
situation you're noticing.
Is there a standard Cygwin 'idiom' or function for dealing with this
mismatch, or should I just re-invent the wheel.
Sure -- just force binary mode on the file (i.e., open it using O_BINARY
for the open() call, or the rb mode for the fopen() call). The mount
type only applies if the mode is unspecified.
A clarification: force binary mode on the opened file in your program, not
the actual on-disk data.
Note that if you do that, you'd also need to handle the CR ('\r', or 0x0d)
characters explicitly in your program.
HTH,
Igor
--
http://cs.nyu.edu/~pechtcha/
|\ _,,,---,,_[EMAIL PROTECTED]
ZZZzz /,`.-'`'-. ;-;;,_[EMAIL PROTECTED]
|,4- ) )-,_. ,\ ( `'-' Igor Pechtchanski, Ph.D.
'---''(_/--' `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-. Meow!
The Sun will pass between the Earth and the Moon tonight for a total
Lunar eclipse... -- WCBS Radio Newsbrief, Oct 27 2004, 12:01 pm EDT
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
Re: stat file -- cygwin vs. Windows size?
Herb Martin wrote: Is there a standard Cygwin 'idiom' or function for dealing with this mismatch, or should I just re-invent the wheel. Sure. Don't use text mode. Open the file in binary mode (O_BINARY with open(), b with fopen()), or call setmode(fd, O_BINARY) once open, or link against binmode.o. Or just don't use text mode mounts. Brian -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
At 02:08 PM 6/24/2005, you wrote: My suspicion is that stat is counting cr-lf as two characters but the input routines are treating these as one. If the file has about 20 lines, then that's 20 missing characters??? Yes, this is right. And yes, this could be the cause of the situation you're noticing. Is there a standard Cygwin 'idiom' or function for dealing with this mismatch, or should I just re-invent the wheel. If you actually believe that you want the file without cr/nl conversion during a read, then you want to open it in binary mode (fopen() with rb instead of r or open() with '| O_BINARY' appended). This *may* be the solution in this case. Since the default mode for opening files is always text but there is no difference in format/behavior between text and binary on UNIX/Linux, you wouldn't see an issue there. -- Larry Hall http://www.rfk.com RFK Partners, Inc. (508) 893-9779 - RFK Office 838 Washington Street (508) 893-9889 - FAX Holliston, MA 01746 -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Herb Martin Sent: Friday, June 24, 2005 1:08 PM To: 'Cygwin List' Subject: RE: stat file -- cygwin vs. Windows size? My suspicion is that stat is counting cr-lf as two characters but the input routines are treating these as one. If the file has about 20 lines, then that's 20 missing characters??? Yes, this is right. And yes, this could be the cause of the situation you're noticing. Is there a standard Cygwin 'idiom' or function for dealing with this mismatch, or should I just re-invent the wheel. As to the former, no, not Cygwin specifically. The problem appears to be that SpamAssassin is making the incorrect but all-too-common assumption that text file == file of 8-bit ASCII characters with '\n' EOL characters. This is as incorrect as thinking picture file == JPEG file. Cygwin does have a number of fetures to bandaid many such broken Unix codes, primarily the text mode mount feature, but these are just that, a band-aid, not a fix of the root problem (and in your case (and in fact in a similar case in mutt), it can't solve the problem). As others have indicated, the real and true solution here is to open the file in binary mode and handle the various EOL chachter combinations in the SpamAssasin code. Which, yeah, is unfortunately reinventing a wheel which should have been permanently reinvented in the last century. But hey, it's only the first few years of the 21st century, maybe by the 22nd we'll have this whole CRLF/LF/CR/LFCR thing sorted out. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
At 02:46 PM 6/24/2005, Gary R. Van Sickle wrote: But hey, it's only the first few years of the 21st century, maybe by the 22nd we'll have this whole CRLF/LF/CR/LFCR thing sorted out. Yeah, I'm guessing this will be solved just after the advent of practical fusion reactors and the development of warp drive. So we have a ways to go yet. ;-) -- Larry Hall http://www.rfk.com RFK Partners, Inc. (508) 893-9779 - RFK Office 838 Washington Street (508) 893-9889 - FAX Holliston, MA 01746 -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
Is there a standard Cygwin 'idiom' or function for dealing with this mismatch, or should I just re-invent the wheel. If you actually believe that you want the file without cr/nl conversion during a read, then you want to open it in binary mode (fopen() with rb instead of r or open() with '| O_BINARY' appended). This *may* be the solution in this case. Since the default mode for opening files is always text but there is no difference in format/behavior between text and binary on UNIX/Linux, you wouldn't see an issue there. Actually I am between a rock and hard place -- email server on one side and SpamD on the other. Apparently the SpamD 'protocol' requires passing the size to SpamD. I don't want to start re-writing code all over either program -- I just want to talk the source email system into telling spamd whatever it needs to know to be happy. Currently, I am accumulating bytes, and will use that, but I am missing something and not getting the write count (YET.) -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
The binary size is accurate, text, by its nature may never be correct on any operating system, since it is buffered, parsed, etc by the OS in an OS dependent way. If you use a binary mode then you will be fine. On Fri, 24 Jun 2005, Herb Martin wrote: Is there a standard Cygwin 'idiom' or function for dealing with this mismatch, or should I just re-invent the wheel. If you actually believe that you want the file without cr/nl conversion during a read, then you want to open it in binary mode (fopen() with rb instead of r or open() with '| O_BINARY' appended). This *may* be the solution in this case. Since the default mode for opening files is always text but there is no difference in format/behavior between text and binary on UNIX/Linux, you wouldn't see an issue there. Actually I am between a rock and hard place -- email server on one side and SpamD on the other. Apparently the SpamD 'protocol' requires passing the size to SpamD. I don't want to start re-writing code all over either program -- I just want to talk the source email system into telling spamd whatever it needs to know to be happy. Currently, I am accumulating bytes, and will use that, but I am missing something and not getting the write count (YET.) -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - - - Jason Pyeron PD Inc. http://www.pdinc.us - - Partner Sr. Manager 7 West 24th Street #100 - - +1 (410) 808-6646 (c) Baltimore, Maryland 21218 - - - -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, purge the message from your system and notify the sender immediately. Any other use of the email by you is prohibited. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
Thanks folks -- the confirmation that I was on the right path was a big help. The suggestions to do it right were well intentioned but impractical since I didn't want to take over support for TWO major software packages (or either one for that matter.) A small patch seems to work. (Keep the bytes spooled and send that number rather than whatever stat was showing.) Since the bytes spooled to the file are what gets sent to spamd that seems to be an accurate number. I had a little trouble at first since it seemed the file was cached (if it was written more than once it really was only written to disk ONE TIME -- so zero'ing the counter in the wrong place was hosing my first naive attempt. -- Herb -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat file -- cygwin vs. Windows size?
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jason Pyeron Sent: Friday, June 24, 2005 2:40 PM To: cygwin@cygwin.com Subject: RE: stat file -- cygwin vs. Windows size? The binary size is accurate, text, by its nature may never be correct on any operating system, since it is buffered, parsed, etc by the OS in an OS dependent way. Actually I am not sure that's correct. I am unaware of any *OS* that does anything like that (maybe the DOS INT13 stuff did, but we're talking ancient history there). The ones I can think of are sane enough to treat files as what they are, i.e. a string of bytes, at the system call level, and do no inspection of any kind on the contents (none that you're supposed to have to care about anyway). The culprit in this confusion is not the OSes but the C runtimes, and the fact that on different OSes, some text file formats are more common than others. The C runtime essentially assumes that all files are text files, when of course this is not and has never been the case. What really should be done is the deprecation of all texty features of the FILE object (e.g. stuff like fprintf()), and create a new FILE_TEXT object which inherits from FILE and adds all the texty operations such as fprintf(), fscanf(), etc, in addition to being able to handle any of the myriad text file formats in existence. But that would make too much sense, so I for one shall not hold my breath. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Stat
Reid Thompson wrote: download coreutils from the gnu ftp site ./configure -- at first glance appears to configure with no errors make -- at first glance appears to build with no errors cp stat.exe to /bin or make install ( will attempt to install the just built versions of all coreutils executables -- this would likely NOT be recommended) $ /C/Downloads/coreutils-5.2.1/src/stat /usr/bin/su File: `/usr/bin/su' Size: 29184 Blocks: 29 IO Block: 1024 regular file Device: 58326116h/1479696662d Inode: 45850 Links: 1 Access: (0700/-rwx--) Uid: (11192/Reid.Thompson) Gid: (10513/Domain Users) Access: 2004-07-21 01:29:33.738351300 -0400 Modify: 2003-07-24 01:14:03.00100 -0400 Change: 2003-08-01 08:10:27.122386500 -0400 reid Thanks for that bit of hand holding. No hiccups to report. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Stat
On Jul 20 22:06, George wrote: The 'stat' utility doesn't seem to be part of my Cygwin distribution. Is this an error I've made during the installation or is it simply not included? It's not included since nobody volunteered to maintain coreutils so far. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Co-Project Leader mailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Stat
Corinna Vinschen wrote: On Jul 20 22:06, George wrote: The 'stat' utility doesn't seem to be part of my Cygwin distribution. Is this an error I've made during the installation or is it simply not included? It's not included since nobody volunteered to maintain coreutils so far. Corinna Thanks for the info. Guess there's always sed, awk and cut. ;-) -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Stat
download coreutils from the gnu ftp site ./configure -- at first glance appears to configure with no errors make -- at first glance appears to build with no errors cp stat.exe to /bin or make install ( will attempt to install the just built versions of all coreutils executables -- this would likely NOT be recommended) $ /C/Downloads/coreutils-5.2.1/src/stat /usr/bin/su File: `/usr/bin/su' Size: 29184 Blocks: 29 IO Block: 1024 regular file Device: 58326116h/1479696662d Inode: 45850 Links: 1 Access: (0700/-rwx--) Uid: (11192/Reid.Thompson) Gid: (10513/Domain Users) Access: 2004-07-21 01:29:33.738351300 -0400 Modify: 2003-07-24 01:14:03.00100 -0400 Change: 2003-08-01 08:10:27.122386500 -0400 reid George wrote: Corinna Vinschen wrote: On Jul 20 22:06, George wrote: The 'stat' utility doesn't seem to be part of my Cygwin distribution. Is this an error I've made during the installation or is it simply not included? It's not included since nobody volunteered to maintain coreutils so far. Corinna Thanks for the info. Guess there's always sed, awk and cut. ;-) -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat()/lstat() problem (?)
On Apr 28 11:27, bertrand marquis wrote: ZXPLESPAC001, Ext a ?crit: The question is: - is the behavior on Linux/Solaris normal ? I fact there ain't a '//bin' only a '/bin', but even all the shells treat them as representing the same path (BTW 'cd //bin' on cygwin/bash doesn't work ...) this is a normal error under cygwin, you should look at the FAQ: // refers to network places - is it an error in cygwin ? Did all pathes (oops is my english very clear ?) have to treat dupplicated '/' as single '/' ? Is the notion of a pathname normalized somewhere (maybe posix ?) ? you should remove all // under cygwin to avoid those errors see: http://cygwin.com/faq/faq_toc.html#TOC34 This is backed by SUSv3, btw. Two leading slashes are allowed to have a implementation defined meaning (SMB network paths in case of Windows). Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Co-Project Leader mailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat()/lstat() problem (?)
ZXPLESPAC001, Ext a écrit:
Hi,
here is my question/problem (see the example program below):
-//
#include stdio.h
#include sys/types.h
#include sys/stat.h
#include errno.h
static int is_dir(char * dr)
{
struct stat st;
if(stat(dr, st) == -1)
{
perror(stat);
return -1;
}
if(lstat(dr, st) == -1)
{
perror(lstat);
return -1;
}
}
int main(int argc,char **argv)
{
int rc;
rc=is_dir(//bin);
rc=is_dir(/bin);
}
-//
With my version of cygwin(Windows NT Ver 4.0 Build 1381 Service Pack 6 -
cygwin 1.5.9-1)
the first call to is_dir() produces an error(stat: No such file or
directory)
BUT the same code was compiled and run on Linux (RH9) and Sun (Solaris2.8)
and
produces no errors !!
The question is:
- is the behavior on Linux/Solaris normal ? I fact there ain't a '//bin'
only a '/bin',
but even all the shells treat them as representing the same path (BTW 'cd
//bin' on
cygwin/bash doesn't work ...)
this is a normal error under cygwin, you should look at the FAQ: //
refers to network places
- is it an error in cygwin ? Did all pathes (oops is my english very clear
?) have to
treat dupplicated '/' as single '/' ? Is the notion of a pathname normalized
somewhere
(maybe posix ?) ?
you should remove all // under cygwin to avoid those errors
see:
http://cygwin.com/faq/faq_toc.html#TOC34
-
CE COURRIER ELECTRONIQUE EST A USAGE STRICTEMENT INFORMATIF ET NE SAURAIT ENGAGER DE QUELQUE MANIERE QUE CE SOIT EADS ASTRIUM SAS, NI SES FILIALES.
SI UNE ERREUR DE TRANSMISSION OU UNE ADRESSE ERRONEE A MAL DIRIGE CE COURRIER, MERCI D'EN INFORMER L'EXPEDITEUR EN LUI FAISANT UNE REPONSE PAR COURRIER ELECTRONIQUE DES RECEPTION. SI VOUS N'ETES PAS LE DESTINATAIRE DE CE COURRIER, VOUS NE DEVEZ PAS L'UTILISER, LE CONSERVER, EN FAIRE ETAT, LE DISTRIBUER, LE COPIER, L'IMPRIMER OU EN REVELER LE CONTENU A UNE TIERCE PARTIE.
This email is for information only and will not bind EADS Astrium SAS in any contract or obligation, nor its subsidiaries.
If you have received it in error, please notify the sender by return email. If you are not the addressee of this email, you must not use, keep, disseminate, copy, print or otherwise deal with it.
-
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
Re: stat matters
On Thu, May 29, 2003 at 10:32:39PM -0400, Pierre A. Humblet wrote: At 11:33 PM 5/28/2003 -0400, Christopher Faylor wrote: On Tue, May 27, 2003 at 07:48:43PM -0400, Pierre A. Humblet wrote: So I suggest a more radical approach: do not check for root dir at all but whenever FindFirstFile fails with winerror 2 (although we know the file did exist a few ms ago and we have its attributes), call fstat_helper with zero dates and lengths. I guess this is the best approach. Want to work up a patch? Done, but it's not that simple. The error is not 2 for remote drives. Also I don't know what it might be on all other systems. So I check for directory but not for specific errors. The worst that can occur is that a directory that was being deleted while the stat was in progress will show up with a wrong date. 2003-05-29 Pierre Humblet [EMAIL PROTECTED] * fhandler_disk_file.cc (fhandler_disk_file::fstat_by_name): Assume an existing directory is a root if FindFirstFile fails. Please apply. cgf
Re: stat matters
At 11:33 PM 5/28/2003 -0400, Christopher Faylor wrote:
On Tue, May 27, 2003 at 07:48:43PM -0400, Pierre A. Humblet wrote:
So I suggest a more radical approach: do not check for root dir at all
but
whenever FindFirstFile fails with winerror 2 (although we know the
file did exist a few ms ago and we have its attributes), call
fstat_helper
with zero dates and lengths.
I guess this is the best approach. Want to work up a patch?
Done, but it's not that simple. The error is not 2 for remote drives. Also
I don't know what it might be on all other systems. So I check for directory
but not for specific errors. The worst that can occur is that a directory
that was being deleted while the stat was in progress will show up with a
wrong date.
2003-05-29 Pierre Humblet [EMAIL PROTECTED]
* fhandler_disk_file.cc (fhandler_disk_file::fstat_by_name): Assume
an existing directory is a root if FindFirstFile fails.
Index: fhandler_disk_file.cc
===
RCS file: /cvs/src/src/winsup/cygwin/fhandler_disk_file.cc,v
retrieving revision 1.53
diff -u -p -r1.53 fhandler_disk_file.cc
--- fhandler_disk_file.cc 27 May 2003 07:44:26 - 1.53
+++ fhandler_disk_file.cc 30 May 2003 02:11:36 -
@@ -108,18 +108,7 @@ fhandler_disk_file::fstat_by_name (struc
set_errno (ENOENT);
res = -1;
}
- else if (pc-isdir () strlen (*pc) = strlen (pc-root_dir ()))
-{
- FILETIME ft = {};
- res = fstat_helper (buf, pc, ft, ft, ft, 0, 0);
-}
- else if ((handle = FindFirstFile (*pc, local)) == INVALID_HANDLE_VALUE)
-{
- debug_printf (FindFirstFile failed for '%s', %E, (char *) *pc);
- __seterrno ();
- res = -1;
-}
- else
+ else if ((handle = FindFirstFile (*pc, local)) != INVALID_HANDLE_VALUE)
{
FindClose (handle);
res = fstat_helper (buf, pc,
@@ -128,6 +117,17 @@ fhandler_disk_file::fstat_by_name (struc
local.ftLastWriteTime,
local.nFileSizeHigh,
local.nFileSizeLow);
+}
+ else if (pc-isdir ())
+{
+ FILETIME ft = {};
+ res = fstat_helper (buf, pc, ft, ft, ft, 0, 0);
+}
+ else
+{
+ debug_printf (FindFirstFile failed for '%s', %E, (char *) *pc);
+ __seterrno ();
+ res = -1;
}
return res;
}
Re: stat/fstat incompatibility w/ unix sockets
On Thu, Feb 20, 2003 at 06:47:39PM -0500, Paul Swartz wrote:
On 20 Feb 2003 at 22:40, Corinna Vinschen wrote:
As you can see, there's also nothing which would help you in using the
result to identify the sockets as being the same. Even the timestamps
aren't identical.
No one field can say it's the same file. All of them together, or a
sufficiently large number of them, does.
Nope. How should they? The really interesting fields are not identical
as e. g. st_dev, st_ino, st_mode, st_ctime. Anyway, that's not a Cygwin
issue anymore.
Even if it isn't completely portable, at the very least, things like the
uid/gid definatly shouldn't change. The ctime/mtime probably shouldn't
either, I'm less sure about atime. I don't have a copy of POSIX, does that
say anything about the stat/fstat semantics?
SUSv3: http://www.opengroup.org/
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Developermailto:[EMAIL PROTECTED]
Red Hat, Inc.
#include sys/types.h
#include sys/socket.h
#include sys/stat.h
#include errno.h
#include stdio.h
#include string.h
#include sys/un.h
#include unistd.h
int
main (int argc, char **argv)
{
int fd;
struct sockaddr_un addr;
socklen_t addrlen;
char buf[100];
struct stat st;
/* creating test unix domain socket pipe */
fd = socket(AF_UNIX, SOCK_DGRAM, 0);
/* SOCK_DGRAM require this to bind */
bzero((char *)addr, sizeof(addr));
/* fill in addr structire */
addr.sun_family = AF_UNIX;
strcpy(addr.sun_path, pipe.101);
addrlen = sizeof(addr.sun_family) + strlen(addr.sun_path) + 1;
/* unlink old socket file */
unlink(addr.sun_path);
/* bind socket to special file named pipe.100 in current directory */
bind(fd, (struct sockaddr *)addr, addrlen);
printf (STAT : \%s\\n, addr.sun_path);
if (!stat (addr.sun_path, st))
{
printf (st_dev: 0x%x\n, st.st_dev);
printf (st_ino: %lu\n, st.st_ino);
printf (st_mode : 0%o\n, st.st_mode);
printf (st_nlink : %d\n, st.st_nlink);
printf (st_uid: %d\n, st.st_uid);
printf (st_gid: %d\n, st.st_gid);
printf (st_rdev : 0x%x\n, st.st_rdev);
printf (st_size : %ld\n, st.st_size);
printf (st_blksize: %ld\n, st.st_blksize);
printf (st_blocks : %ld\n, st.st_blocks);
printf (st_ctime : %ld\n, st.st_ctime);
printf (st_mtime : %ld\n, st.st_mtime);
printf (st_atime : %ld\n, st.st_atime);
}
printf (FSTAT : \%s\\n, addr.sun_path);
if (!fstat (fd, st))
{
printf (st_dev: 0x%x\n, st.st_dev);
printf (st_ino: %lu\n, st.st_ino);
printf (st_mode : 0%o\n, st.st_mode);
printf (st_nlink : %d\n, st.st_nlink);
printf (st_uid: %d\n, st.st_uid);
printf (st_gid: %d\n, st.st_gid);
printf (st_rdev : 0x%x\n, st.st_rdev);
printf (st_size : %ld\n, st.st_size);
printf (st_blksize: %ld\n, st.st_blksize);
printf (st_blocks : %ld\n, st.st_blocks);
printf (st_ctime : %ld\n, st.st_ctime);
printf (st_mtime : %ld\n, st.st_mtime);
printf (st_atime : %ld\n, st.st_atime);
}
return 0;
}
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Bug reporting: http://cygwin.com/bugs.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
Re: stat/fstat incompatibility w/ unix sockets
On Thu, Feb 20, 2003 at 02:02:40AM -0500, Paul Swartz wrote: Content-Description: Mail message body The man page for stat/fstat says that the results returned should be the same. However, when asking for the fstat on a unix socket, the result is not the same. The attached python code demonstrates this problem. Is this an already known issue with a solution already? Are there plans to resolve it? *What* is the difference? Some details would be helpful. Especially I'm not using python at all. A short test case in C would be appreciated or at least a description of the difference you see. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developermailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat/fstat incompatibility w/ unix sockets
OK, here's what the script looks like on Cygwin: - stat from filename (49536, 1374655, 27579L, 1, 1005, 513, 51, 1045764368, 1045764368, 1045764368) stat from fileno (49590, 1672, 2816L, 1, 0, 0, 0, 1045764368, 1045764368, 1045764368) - The lists correspond to (st_mode, st_ino, st_dev, st_nlink, st_uid, st_gid, st_size, st_atime, st_mtime, st_ctime) As you can see, most of the numbers are not the same. In fact, the only ones that /are/ the same at st_nlink and st_atime. On *nix, the results look like this: - stat from filename (49536, 3228941L, 769L, 1, 1037, 1037, 0L, 1045736044, 1045736044, 1045736044) stat from fileno (49663, 5351966L, 0L, 1, 1037, 1037, 0L, 1045736044, 1045736044, 1045736044) - The only ones on *nix that are different are the first three (st_mode, st_ino, and, st_dev). The difference in the fstat on the opened fileno and the stat on the filename make it difficult, and probably impossible in some cases, to tell if the socket opened is the same as the file. This makes writing secure software that uses UNIX sockets difficult, if not impossible. -p -- Paul Swartz (o_ http://twistedmatrix.com/users/z3p.twistd/ //\ [EMAIL PROTECTED] V_/_ AIM: Z3Penguin -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat/fstat incompatibility w/ unix sockets
On Thu, Feb 20, 2003 at 01:23:43PM -0500, Paul Swartz wrote: OK, here's what the script looks like on Cygwin: - stat from filename (49536, 1374655, 27579L, 1, 1005, 513, 51, 1045764368, 1045764368, 1045764368) stat from fileno (49590, 1672, 2816L, 1, 0, 0, 0, 1045764368, 1045764368, 1045764368) - The lists correspond to (st_mode, st_ino, st_dev, st_nlink, st_uid, st_gid, st_size, st_atime, st_mtime, st_ctime) As you can see, most of the numbers are not the same. In fact, the only ones that /are/ the same at st_nlink and st_atime. On *nix, the results look like this: - stat from filename (49536, 3228941L, 769L, 1, 1037, 1037, 0L, 1045736044, 1045736044, 1045736044) stat from fileno (49663, 5351966L, 0L, 1, 1037, 1037, 0L, 1045736044, 1045736044, 1045736044) - The only ones on *nix that are different are the first three (st_mode, st_ino, and, st_dev). The difference in the fstat on the opened fileno and the stat on the filename make it difficult, and probably impossible in some cases, to tell if the socket opened is the same as the file. This makes writing secure software that uses UNIX sockets difficult, if not impossible. I created a testcase which allows me to reproduce your observation. First of all let me say that thanks to your report I could find the problem in Cygwin which explains the differences between stat() and fstat(). However, I'm not quite sure if that will help you. I've created a AF_UNIX socket called pipe.101 and this is the output of my testcase: STAT : pipe.101 st_dev: b00 st_ino: -857905661 st_mode : 49590 st_nlink : 1 st_uid: 0 st_gid: 0 st_rdev : b00 st_size : 0 st_blksize: 1024 st_blocks : 0 st_ctime : 1045775480 st_mtime : 1045775480 st_atime : 1045775480 FSTAT : pipe.101 st_dev: b00 st_ino: 1872 st_mode : 49590 st_nlink : 1 st_uid: 0 st_gid: 0 st_rdev : b00 st_size : 0 st_blksize: 1024 st_blocks : 0 st_ctime : 1045775480 st_mtime : 1045775480 st_atime : 1045775480 Note that except for st_ino all other fields are identical. But other than that, I don't see *any* field which you could use to identify the results being the same file. I've checked the same testcase on Linux and I'm getting the following results: STAT : pipe.101 st_dev: 821 st_ino: 33442 st_mode : 49645 st_nlink : 1 st_uid: 500 st_gid: 100 st_rdev : 0 st_size : 0 st_blksize: 4096 st_blocks : 0 st_ctime : 1045776973 st_mtime : 1045776973 st_atime : 1045776973 FSTAT : pipe.101 st_dev: 0 st_ino: 378241 st_mode : 49663 st_nlink : 1 st_uid: 500 st_gid: 100 st_rdev : 0 st_size : 0 st_blksize: 4096 st_blocks : 0 st_ctime : 1045776957 st_mtime : 1045776957 st_atime : 1045776804 As you can see, there's also nothing which would help you in using the result to identify the sockets as being the same. Even the timestamps aren't identical. Bottom line: Trying to use the result of fstat/stat to identify a socket is definitely non-portable and will give you more headaches than useful results. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developermailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat/fstat incompatibility w/ unix sockets
On 20 Feb 2003 at 22:40, Corinna Vinschen wrote: I created a testcase which allows me to reproduce your observation. First of all let me say that thanks to your report I could find the problem in Cygwin which explains the differences between stat() and fstat(). However, I'm not quite sure if that will help you. I've created a AF_UNIX socket called pipe.101 and this is the output of my testcase: (result snipped) Could you attach this test case? Also, are you just creating the socket and then checking it, or listening on the socket too? Note that except for st_ino all other fields are identical. But other than that, I don't see *any* field which you could use to identify the results being the same file. I've checked the same testcase on Linux and I'm getting the following results: (more results snipped) As you can see, there's also nothing which would help you in using the result to identify the sockets as being the same. Even the timestamps aren't identical. No one field can say it's the same file. All of them together, or a sufficiently large number of them, does. Bottom line: Trying to use the result of fstat/stat to identify a socket is definitely non-portable and will give you more headaches than useful results. Even if it isn't completely portable, at the very least, things like the uid/gid definatly shouldn't change. The ctime/mtime probably shouldn't either, I'm less sure about atime. I don't have a copy of POSIX, does that say anything about the stat/fstat semantics? -p -- Paul Swartz (o_ http://twistedmatrix.com/users/z3p.twistd/ //\ [EMAIL PROTECTED] V_/_ AIM: Z3Penguin -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Bug reporting: http://cygwin.com/bugs.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
