Re: stat(2) triggers on-demand virus scan
On Mon, Jan 16, 2006 at 10:42:10PM -0600, * * wrote: >On 1/15/06, Christopher Faylor <[EMAIL PROTECTED]> wrote: >> On Sat, Jan 14, 2006 at 11:37:33PM -0600, Gary R. Van Sickle wrote: >> >[snip] >> I just wanted to make it clear that we aren't going to be >> >>making any >> special concessions to a product like a virus scanner which cause >> perfectly acceptable code to misbehave. If that is the >> >>case then it >> is a situation for the virus scanner to work out. It's not a >> requirement that Cygwin work around things like this. >> >>> >> >>>Well, that is a pretty strong statement, I'd expect from a >> >>for-profit >> >>>company run by corporate management. >> >> >> >>This is a practical decision. >> >> >> >>We are not going to visit the slippery slope of adding code to Cygwin >> >>to work around other third party software. We >> > >> >Huh? Has it even been 24 hours since you suggested Cygwin be changed >> >in a non-standardized manner merely to band-aid a broken third-party >> >IRC client? >> >> The fact that you have this opinion makes me think that you and other >> people are not entirely clear on what we're supposed to be doing here >> even though it only takes one sentence to explain things: >> >> We're trying to emulate linux. >> >> So, think about this sentence and then explain how it jives with your >> assertion that I was trying to do something in a "non-standardized >> manner". How is it "non-standardized" to make cygwin emulate linux >> more closely so that programs which build on linux are more likely to >> build on cygwin? >> >> It's curious that you'd see a parallel between modifying cygwin to >> accommodate a specific broken commerical virus scanner and modifying >> Cygwin in such a way that more programs are likely to build with it. I >> don't get that at all. >> >> >The slope you mention has already been visited on more than one >> >occaision. >> >> Sorry, but you'll have to come up with more convincing examples than the >> ones you've used. I'm sure that, through the years, I've probably said >> "sorry that's the way it works" to people who've complained that cygwin >> didn't work like linux, but my opinion has been evolving since the goal >> of the project was changed to target a specific system (linux). I think >> I've made the point a few times in the last year that linux is the >> target that cygwin is shooting for so I don't see any inconsistency here >> at all. >> >> I'm fairly surprised that you're arguing this point at all. When I saw >> that you'd responded to this thread, I thought I'd be seeing a rant >> about stupid virus checkers and how Cygwin shouldn't be going out of its >> way to work with them. I didn't expect that you'd be arguing the issue >> (or the meta-issue) of precedent for allowing such a change. >> >> >And doesn't Cygwin still create sparse files for the benefit of one >> >single third-party application? The slope you mention has already been >> >visited on more than one occaision. >> >> Cygwin creates sparse files in some situations because that's how >> linux/unix works. IIRC, we actually made an accommodation to windows to >> not create sparse files as often as linux because people complained >> about the default linux behavior. I don't recall doing this for a >> third-party windows application, however. I thought we just did it >> because linux works that way but I may be misremembering. If the change >> was to make a *linux* program work better then I think we're once again >> on pretty firm ground. We were, once again, changing things to make it >> easier for linux programs to port without problem to Cygwin. And, once >> again, this is nowhere near the same thing as modifying cygwin to >> accommodate a broken virus checker. >> >> >[snip] >> >>However, this is a free software project so people have the ability to >> >>inspect the source code and offer patches. If someone offers a patch >> >>to fix problems with a virus scanner which doesn't involve any special >> >>tests for the virus scanner, doesn't involve extra code to work around >> >>the virus scanner, and doesn't involve doing something like, say, using >> >>sockets instead of pipes because the virus scanner doesn't like pipes, >> >>then, sure, we'll consider the code. Otherwise, this is what I would >> >>call a "special concession to third party software" and I'm not >> >>interested in littering the code with those. >> > >> >Again, that last sentence is simply not a true statement, unless you >> >want to split hairs about the "littering" part. And I have to question >> >the veracity of a "PTC" statement that has as its prerequisites that >> >the patch involve no actual code. >> >> If you didn't get what I was saying, I'm wondering why you couldn't just >> ask for clarification without questioning the truthfulness of what I >> said. >> >> However, let me try to clarify again. If someone found that one >> function caused a problem but another equivalent funct
Re: stat(2) triggers on-demand virus scan
On 1/15/06, Christopher Faylor <[EMAIL PROTECTED]> wrote: > On Sat, Jan 14, 2006 at 11:37:33PM -0600, Gary R. Van Sickle wrote: > >[snip] > I just wanted to make it clear that we aren't going to be > >>making any > special concessions to a product like a virus scanner which cause > perfectly acceptable code to misbehave. If that is the > >>case then it > is a situation for the virus scanner to work out. It's not a > requirement that Cygwin work around things like this. > >>> > >>>Well, that is a pretty strong statement, I'd expect from a > >>for-profit > >>>company run by corporate management. > >> > >>This is a practical decision. > >> > >>We are not going to visit the slippery slope of adding code to Cygwin > >>to work around other third party software. We > > > >Huh? Has it even been 24 hours since you suggested Cygwin be changed > >in a non-standardized manner merely to band-aid a broken third-party > >IRC client? > > The fact that you have this opinion makes me think that you and other > people are not entirely clear on what we're supposed to be doing here > even though it only takes one sentence to explain things: > > We're trying to emulate linux. > > So, think about this sentence and then explain how it jives with your > assertion that I was trying to do something in a "non-standardized > manner". How is it "non-standardized" to make cygwin emulate linux > more closely so that programs which build on linux are more likely to > build on cygwin? > > It's curious that you'd see a parallel between modifying cygwin to > accommodate a specific broken commerical virus scanner and modifying > Cygwin in such a way that more programs are likely to build with it. I > don't get that at all. > > >The slope you mention has already been visited on more than one > >occaision. > > Sorry, but you'll have to come up with more convincing examples than the > ones you've used. I'm sure that, through the years, I've probably said > "sorry that's the way it works" to people who've complained that cygwin > didn't work like linux, but my opinion has been evolving since the goal > of the project was changed to target a specific system (linux). I think > I've made the point a few times in the last year that linux is the > target that cygwin is shooting for so I don't see any inconsistency here > at all. > > I'm fairly surprised that you're arguing this point at all. When I saw > that you'd responded to this thread, I thought I'd be seeing a rant > about stupid virus checkers and how Cygwin shouldn't be going out of its > way to work with them. I didn't expect that you'd be arguing the issue > (or the meta-issue) of precedent for allowing such a change. > > >And doesn't Cygwin still create sparse files for the benefit of one > >single third-party application? The slope you mention has already been > >visited on more than one occaision. > > Cygwin creates sparse files in some situations because that's how > linux/unix works. IIRC, we actually made an accommodation to windows to > not create sparse files as often as linux because people complained > about the default linux behavior. I don't recall doing this for a > third-party windows application, however. I thought we just did it > because linux works that way but I may be misremembering. If the change > was to make a *linux* program work better then I think we're once again > on pretty firm ground. We were, once again, changing things to make it > easier for linux programs to port without problem to Cygwin. And, once > again, this is nowhere near the same thing as modifying cygwin to > accommodate a broken virus checker. > > >[snip] > >>However, this is a free software project so people have the ability to > >>inspect the source code and offer patches. If someone offers a patch > >>to fix problems with a virus scanner which doesn't involve any special > >>tests for the virus scanner, doesn't involve extra code to work around > >>the virus scanner, and doesn't involve doing something like, say, using > >>sockets instead of pipes because the virus scanner doesn't like pipes, > >>then, sure, we'll consider the code. Otherwise, this is what I would > >>call a "special concession to third party software" and I'm not > >>interested in littering the code with those. > > > >Again, that last sentence is simply not a true statement, unless you > >want to split hairs about the "littering" part. And I have to question > >the veracity of a "PTC" statement that has as its prerequisites that > >the patch involve no actual code. > > If you didn't get what I was saying, I'm wondering why you couldn't just > ask for clarification without questioning the truthfulness of what I > said. > > However, let me try to clarify again. If someone found that one > function caused a problem but another equivalent function didn't, then > using the equivalent function would be ok. If someone found that a 10 > line test was necessary to determine if a virus scanner wa
Re: stat(2) triggers on-demand virus scan
On Sat, Jan 14, 2006 at 11:37:33PM -0600, Gary R. Van Sickle wrote: >[snip] I just wanted to make it clear that we aren't going to be >>making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the >>case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. >>> >>>Well, that is a pretty strong statement, I'd expect from a >>for-profit >>>company run by corporate management. >> >>This is a practical decision. >> >>We are not going to visit the slippery slope of adding code to Cygwin >>to work around other third party software. We > >Huh? Has it even been 24 hours since you suggested Cygwin be changed >in a non-standardized manner merely to band-aid a broken third-party >IRC client? The fact that you have this opinion makes me think that you and other people are not entirely clear on what we're supposed to be doing here even though it only takes one sentence to explain things: We're trying to emulate linux. So, think about this sentence and then explain how it jives with your assertion that I was trying to do something in a "non-standardized manner". How is it "non-standardized" to make cygwin emulate linux more closely so that programs which build on linux are more likely to build on cygwin? It's curious that you'd see a parallel between modifying cygwin to accommodate a specific broken commerical virus scanner and modifying Cygwin in such a way that more programs are likely to build with it. I don't get that at all. >The slope you mention has already been visited on more than one >occaision. Sorry, but you'll have to come up with more convincing examples than the ones you've used. I'm sure that, through the years, I've probably said "sorry that's the way it works" to people who've complained that cygwin didn't work like linux, but my opinion has been evolving since the goal of the project was changed to target a specific system (linux). I think I've made the point a few times in the last year that linux is the target that cygwin is shooting for so I don't see any inconsistency here at all. I'm fairly surprised that you're arguing this point at all. When I saw that you'd responded to this thread, I thought I'd be seeing a rant about stupid virus checkers and how Cygwin shouldn't be going out of its way to work with them. I didn't expect that you'd be arguing the issue (or the meta-issue) of precedent for allowing such a change. >And doesn't Cygwin still create sparse files for the benefit of one >single third-party application? The slope you mention has already been >visited on more than one occaision. Cygwin creates sparse files in some situations because that's how linux/unix works. IIRC, we actually made an accommodation to windows to not create sparse files as often as linux because people complained about the default linux behavior. I don't recall doing this for a third-party windows application, however. I thought we just did it because linux works that way but I may be misremembering. If the change was to make a *linux* program work better then I think we're once again on pretty firm ground. We were, once again, changing things to make it easier for linux programs to port without problem to Cygwin. And, once again, this is nowhere near the same thing as modifying cygwin to accommodate a broken virus checker. >[snip] >>However, this is a free software project so people have the ability to >>inspect the source code and offer patches. If someone offers a patch >>to fix problems with a virus scanner which doesn't involve any special >>tests for the virus scanner, doesn't involve extra code to work around >>the virus scanner, and doesn't involve doing something like, say, using >>sockets instead of pipes because the virus scanner doesn't like pipes, >>then, sure, we'll consider the code. Otherwise, this is what I would >>call a "special concession to third party software" and I'm not >>interested in littering the code with those. > >Again, that last sentence is simply not a true statement, unless you >want to split hairs about the "littering" part. And I have to question >the veracity of a "PTC" statement that has as its prerequisites that >the patch involve no actual code. If you didn't get what I was saying, I'm wondering why you couldn't just ask for clarification without questioning the truthfulness of what I said. However, let me try to clarify again. If someone found that one function caused a problem but another equivalent function didn't, then using the equivalent function would be ok. If someone found that a 10 line test was necessary to determine if a virus scanner was running then that is not something that I would want to do. If someone found that avoiding the use of pipes and using mailslots instead "fixed" things, then that would not be ok. Basically, trivial changes which involve minimal perturbati
RE: stat(2) triggers on-demand virus scan
pmcferrin wrote: > Here is something a little OT > > When making comparisons between multiple runs to run timing tests > before and after a change, it there a way I can guarantee more > consistent results? e.g. Condider the following: > > time find . -print | wc -l > > I can run the above sequence 3 times in a row and get huge > differences due to OS caching and > probably application caching (281 secs, 11 secs, 0.8 secs > respectively). I ran my tests (earlier in this thread) so many times that I got consistent timings - this gave the figures I presented. All interference from other software eliminated/minimized; the only way I can think of getting consistent run times. Adding anythng at all gives uncertainty and varying figures. Your measurements will most likely differ, even to a great extent - dependeing on how much "junk" (i.e. SW) you have installed. I consider this to be a "best possible" state for running the test; one can't get any better (i.e. shorter) run times without removing selected utilities and services - degrading the usability of the machine in the process. This clearly shows one known weak side of general operating systems - most of the time there is so much different things going on that you can't count on any predefined completion time for a given task. This problem is addressed in RTOS'es and the like, and also is the subject for research - e.g. some of the projects here http://www.sics.se/cna/software.html I believe. When it comes to practical cygwin use, this run time inconsistency shows even more - cygwin could be considered an OS (POSIX emulation) on top of another OS (Win => ! Loose => Not Loose => Loose Nuts ;-). > Is there any known way within MS XP Pro to flush > all caching other than a reboot?? Maybe CacheSet from sysinternals.com ? - http://www.sysinternals.com/Utilities/CacheSet.html I have NOT tried it on XP, and thus can't recommend it in any way. AFAIK it should mimic the functionality of the [vcache] section settings in older windows WIN.INI file (or was it SYSTEM.INI?). > I run > into similar problems when validating multiple copies of a CD-R by > calculating the checksum. The first checksum is valid but I can't > trust the remainder due to OS caching. You don't trust the OS to do cache handling correctly on CD-R's with exactly the same contents, well what can I say - how would you distinguish the difference between those CD-R's? Somewhere long ago I've read that when running CMD.EXE - hitting CTRL-C at the prompt should "re-read disk info" for 'CWD' IIRC (this was a long time ago; in the age of 3.5" diskettes) ... I dunno if this still is true. How about; Insert and briefly access a different CD in between, making the OS/fs aware of disk changes? > -Paul /H -- -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
Gary R. Van Sickle wrote: [snip] I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We Huh? Has it even been 24 hours since you suggested Cygwin be changed in a non-standardized manner merely to band-aid a broken third-party IRC client? And doesn't Cygwin still create sparse files for the benefit of one single third-party application? The slope you mention has already been visited on more than one occaision. I'm sorry, but IMO this is /not/ the same thing. What CGF suggested was to implement things in the same manner as Linux - which is one of the main goals of this project - allowing gcc to behaving in non-ansi mode by default. This would be a major undertaking, yes, as AFAICT it involves updating, creating, and generally rewriting more than a few header files in order for it to work without adversely affecting how things work now. As for creating sparse files - imo this is better than how windows normally does it, regardless of the app in question. [snip] However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this is what I would call a "special concession to third party software" and I'm not interested in littering the code with those. Again, that last sentence is simply not a true statement, unless you want to split hairs about the "littering" part. And I have to question the veracity of a "PTC" statement that has as its prerequisites that the patch involve no actual code. Matter of interpretation. IMO, CGF is saying he doesn't want large quantities of complex code. A small patch that altered cygwin's behaviour in such a way so as to be more friendly towards ZA or various AV products would be more than sufficient. Perhaps Corinna has a different opinion and will convince me otherwise but, until that time, I just thought I would make the ground rules clear. I thought this was obvious stuff but I guess it wasn't. No, and I guess it still isn't. Is to me. BTW, OP: Update your 1.3.x install. It's the 21st century for God's sake. On this I wholeheartedly agree. -- Spinning complacently in the darkness, covered and blinded by a blanket of little lives, false security has lulled the madness of this world into a slumber. Wake up! An eye is upon you, staring straight down and keenly through, seeing all that you are and everything that you will never be. Yes, an eye is upon you, an eye ready to blink. So face forward, with arms wide open and mind reeling. Your future has arrived... Are you ready to go? -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
> From: Brett Serkez > Sent: Saturday, January 14, 2006 3:19 PM > To: cygwin@cygwin.com; cygwin@cygwin.com > Subject: Re: stat(2) triggers on-demand virus scan > > > On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: > > >I'm still researching, I was going to respond this is posting at a > > >later time with more insight, but before things get out-of-hand, I > > >wanted to jump in. I suppose I'm still hopeful that we > can zero in > > >on what precisely is causing the on-demand scanners to consume so > > >much CPU. Since Windows programs don't trigger the same level of > > >response (or atleast they don't appear to) their must be > some change > > >that can be made. > > > > I just wanted to make it clear that we aren't going to be > making any > > special concessions to a product like a virus scanner which cause > > perfectly acceptable code to misbehave. If that is the > case then it > > is a situation for the virus scanner to work out. It's not a > > requirement that cygwin work around things like this. > > Well, that is a pretty strong statement, I'd expect from a > for-profit company run by corporate management. ZoneLabs > offical stance is that they don't support emulated > environments. I have to assume whoever said or wrote that was either thinking "Wine", or not thinking at all, since Cygwin is ultimately no different than any other Windows application from their software's perspective. > Humm... So if neither are willing to change, > then what? I don't know Symantec's or McAfee's offical stance. > Last I checked it was "cause more problems than the viruses we purportedly protect you from would". Look guys, the bottom line here is that on-access virus scanners cause trouble. Not just for Cygwin, and not just particular ones. Scan your incoming email, scan your downloads, do your backups, cross your fingers, and hope for a horrible death for the virus-writing idiots of the world. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
> From: Paul McFerrin > Sent: Saturday, January 14, 2006 5:19 PM > To: Cygwin@Cygwin.com > Subject: Re: stat(2) triggers on-demand virus scan > > Boy did I open a can of worms! > No, it's like this on a regular, periodic basis. > When I looked at the source of Cygwin1.dll a few years ago at > the time, the stat(2) basically called a MS API function to > retreive the information and then did a simpe return. > I think it the faulty design of MS not to privide a > function to get status information without triggering all > sorts of other OS's overhead (e.g. on-demand scanning). > > If the stat(2) is following the MS SDK guidelines for POSTIX ...um, POSIX. It's POSIX. > compatability, then I don't see much other attractive > recourse but live with MS supid design. What Chris F. said. Plus, while I refuse to defend the ability of MS's Operating Systems to properly work with Disks (admittedly it's a new thing for them), stat's definition and/or the way many programs use stat is the real culprit here. > After it *is* MS. > Unless there is some obsolete non-POSTIX MS API that > retrieves this information that does not have this side > effect, then I'll live with it. It does seem silly to me > that you can't perform a > stat(2) without triggering side effects. Maybe I'm too much > of a Unix Geru. > > Here is something a little OT > > When making comparisons between multiple runs to run timing > tests before and after a change, it there a way I can > guarantee more consistent results? e.g. Condider the following: > > time find . -print | wc -l > > I can run the above sequence 3 times in a row and get huge > differences due to OS caching and probably application > caching (281 secs, 11 secs, 0.8 secs respectively). Is there > any known way within MS XP Pro to flush all caching other > than a reboot?? The short non-rant answer to that is no, there isn't. The long non-rant answer would require a logical explanation from Microsoft as to why their filesystem infrastructure is completely incapable of properly handling removable media, and that is highly unlikely to be forthcoming. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
[snip] > >>I just wanted to make it clear that we aren't going to be > making any > >>special concessions to a product like a virus scanner which cause > >>perfectly acceptable code to misbehave. If that is the > case then it > >>is a situation for the virus scanner to work out. It's not a > >>requirement that Cygwin work around things like this. > > > >Well, that is a pretty strong statement, I'd expect from a > for-profit > >company run by corporate management. > > This is a practical decision. > > We are not going to visit the slippery slope of adding code > to Cygwin to work around other third party software. We Huh? Has it even been 24 hours since you suggested Cygwin be changed in a non-standardized manner merely to band-aid a broken third-party IRC client? And doesn't Cygwin still create sparse files for the benefit of one single third-party application? The slope you mention has already been visited on more than one occaision. [snip] > However, this is a free software project so people have the > ability to inspect the source code and offer patches. If > someone offers a patch to fix problems with a virus scanner > which doesn't involve any special tests for the virus > scanner, doesn't involve extra code to work around the virus > scanner, and doesn't involve doing something like, say, using > sockets instead of pipes because the virus scanner doesn't > like pipes, then, sure, we'll consider the code. Otherwise, > this is what I would call a "special concession to third > party software" and I'm not interested in littering the code > with those. > Again, that last sentence is simply not a true statement, unless you want to split hairs about the "littering" part. And I have to question the veracity of a "PTC" statement that has as its prerequisites that the patch involve no actual code. > Perhaps Corinna has a different opinion and will convince me > otherwise but, until that time, I just thought I would make > the ground rules clear. I thought this was obvious stuff but > I guess it wasn't. > No, and I guess it still isn't. BTW, OP: Update your 1.3.x install. It's the 21st century for God's sake. -- Gary R. Van Sickle -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
Boy did I open a can of worms! When I looked at the source of Cygwin1.dll a few years ago at the time, the stat(2) basically called a MS API function to retreive the information and then did a simpe return. I think it the faulty design of MS not to privide a function to get status information without triggering all sorts of other OS's overhead (e.g. on-demand scanning). If the stat(2) is following the MS SDK guidelines for POSTIX compatability, then I don't see much other attractive recourse but live with MS supid design. After it *is* MS. Unless there is some obsolete non-POSTIX MS API that retrieves this information that does not have this side effect, then I'll live with it. It does seem silly to me that you can't perform a stat(2) without triggering side effects. Maybe I'm too much of a Unix Geru. Here is something a little OT When making comparisons between multiple runs to run timing tests before and after a change, it there a way I can guarantee more consistent results? e.g. Condider the following: time find . -print | wc -l I can run the above sequence 3 times in a row and get huge differences due to OS caching and probably application caching (281 secs, 11 secs, 0.8 secs respectively). Is there any known way within MS XP Pro to flush all caching other than a reboot?? I run into similar problems when validating multiple copies of a CD-R by calculating the checksum. The first checksum is valid but I can't trust the remainder due to OS caching. -Paul Christopher Faylor wrote: On Sat, Jan 14, 2006 at 04:18:43PM -0500, Brett Serkez wrote: On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that Cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We already have more than enough to worry about with different windows versions. Adding the combinatorial nightmare of worrying about different windows versions * different virus scanners is not something I'm really interested in. Even if we did want to do this, how do we decide what software we should make concessions for? You're using ZoneAlarm. Is that a really popular package among windows users? If so, do we need a dedicated ZoneAlarm specialist on staff to make sure that every change that Corinna or I make to Cygwin does not cause problems for ZoneAlarm? How about McAfee? Do we need someone to worry about them, too? How about sysinternals? I think some of their software causes cygwin to behave strangely. Do we need a Sysinternals expert? Or do we just pay attention to the loudest voice and then pull the concession code out of Cygwin when the voice goes away because Corinna and I can no longer test and support it? ZoneLabs offical stance is that they don't support emulated environments. Humm... So if neither are willing to change, then what? I don't know Symantec's or McAfee's offical stance. Cygwin is a program which uses standard the win32 api. The fact that the win32 api is used to present a bash prompt is no different than using the win32 api to present a word processor screen. Assuming that the "emulated environment" above actually refers to Cygwin then failure on Zonealarm's part to fix bugs that cause Cygwin's use of the windows API to misbehave is an arbitrary distinction and a cop-out. As far as coding being 'perfectly acceptable', that is a matter of point-of- view. If it causes such behavior, is it acceptable? It is not a matter of a point of view if code works as documented in a virus-scanner-free environment and fails to work when a virus scanner is installed. However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this
Re: stat(2) triggers on-demand virus scan
[snip] > We are not going to visit the slippery slope of adding code to Cygwin > to work around other third party software. I'm hoping and assuming it is going to be more a matter of making minor changes, if it requires a major change, then it is more likely Microsoft or some other vendor is at fault. [snip] > >ZoneLabs offical stance is that they don't support emulated > >environments. Humm... So if neither are willing to change, then > >what? I don't know Symantec's or McAfee's offical stance. > > Cygwin is a program which uses standard the win32 api. The fact that > the win32 api is used to present a bash prompt is no different than > using the win32 api to present a word processor screen. Assuming that > the "emulated environment" above actually refers to Cygwin then > failure on Zonealarm's part to fix bugs that cause Cygwin's use of the > windows API to misbehave is an arbitrary distinction and a cop-out. Strongly agreed. I've already pointed this out to them to no avail. > >As far as coding being 'perfectly acceptable', that is a matter of > >point-of- view. If it causes such behavior, is it acceptable? > > It is not a matter of a point of view if code works as documented in a > virus-scanner-free environment and fails to work when a virus scanner > is installed. >From what I've been seeing, I'm starting to suspect that the problem(s) is there in both cases, the scanner simply makes it much more noticable. I do see more CPU consumption that I woud have expected even without the virus scanner and the original poster's calling out stat was most interesting. [snip] Brett Brett C. Serkez, Techie -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
On Sat, Jan 14, 2006 at 04:18:43PM -0500, Brett Serkez wrote: >>On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: >>>I'm still researching, I was going to respond this is posting at a >>>later time with more insight, but before things get out-of-hand, I >>>wanted to jump in. I suppose I'm still hopeful that we can zero in on >>>what precisely is causing the on-demand scanners to consume so much >>>CPU. Since Windows programs don't trigger the same level of response >>>(or atleast they don't appear to) their must be some change that can be >>>made. >> >>I just wanted to make it clear that we aren't going to be making any >>special concessions to a product like a virus scanner which cause >>perfectly acceptable code to misbehave. If that is the case then it is >>a situation for the virus scanner to work out. It's not a requirement >>that Cygwin work around things like this. > >Well, that is a pretty strong statement, I'd expect from a for-profit >company run by corporate management. This is a practical decision. We are not going to visit the slippery slope of adding code to Cygwin to work around other third party software. We already have more than enough to worry about with different windows versions. Adding the combinatorial nightmare of worrying about different windows versions * different virus scanners is not something I'm really interested in. Even if we did want to do this, how do we decide what software we should make concessions for? You're using ZoneAlarm. Is that a really popular package among windows users? If so, do we need a dedicated ZoneAlarm specialist on staff to make sure that every change that Corinna or I make to Cygwin does not cause problems for ZoneAlarm? How about McAfee? Do we need someone to worry about them, too? How about sysinternals? I think some of their software causes cygwin to behave strangely. Do we need a Sysinternals expert? Or do we just pay attention to the loudest voice and then pull the concession code out of Cygwin when the voice goes away because Corinna and I can no longer test and support it? >ZoneLabs offical stance is that they don't support emulated >environments. Humm... So if neither are willing to change, then what? >I don't know Symantec's or McAfee's offical stance. Cygwin is a program which uses standard the win32 api. The fact that the win32 api is used to present a bash prompt is no different than using the win32 api to present a word processor screen. Assuming that the "emulated environment" above actually refers to Cygwin then failure on Zonealarm's part to fix bugs that cause Cygwin's use of the windows API to misbehave is an arbitrary distinction and a cop-out. >As far as coding being 'perfectly acceptable', that is a matter of >point-of- view. If it causes such behavior, is it acceptable? It is not a matter of a point of view if code works as documented in a virus-scanner-free environment and fails to work when a virus scanner is installed. However, this is a free software project so people have the ability to inspect the source code and offer patches. If someone offers a patch to fix problems with a virus scanner which doesn't involve any special tests for the virus scanner, doesn't involve extra code to work around the virus scanner, and doesn't involve doing something like, say, using sockets instead of pipes because the virus scanner doesn't like pipes, then, sure, we'll consider the code. Otherwise, this is what I would call a "special concession to third party software" and I'm not interested in littering the code with those. Perhaps Corinna has a different opinion and will convince me otherwise but, until that time, I just thought I would make the ground rules clear. I thought this was obvious stuff but I guess it wasn't. cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
Brett Serkez wrote: On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. ZoneLabs offical stance is that they don't support emulated environments. Humm... So if neither are willing to change, then what? I don't know Symantec's or McAfee's offical stance. They're likely to be the same. While cgf's statement could be interpreted in the same vein, you have to look at it from other points of view as well.. See comments below. As far as coding being 'perfectly acceptable', that is a matter of point-of- view. If it causes such behavior, is it acceptable? Cygwin is not what is at fault here - it is the way many on-demand virus scanners interact with the OS, the OS itself, and how ZA hooks in to the net-code, that cause these issues. Cygwin code is correct according to ms sdk and available documentation - it uses the correct and most accurate methods to implement POSIX functionality and *nix integration that are available and known. (Note that this statement is not entirely accurate, or the next would not be at all) Obviously, as time goes on, improvements can be made, but that is the nature of software development. While I respect the purist point of view, one I've held over the years, seems that we need to be practical sometimes. Are you saying that if the problem could be isolated, and reasonable changes proposed, you wouldn't make/allow them? Do we want IT administrators to utilize Cygwin to integrate with the Linux/UNIX environment? If this means not being able to effectively use Cygwin if they are also required to run ZoneAlarm, Norton, McAfee, what choice do you think they'll make, or more precisely, their management will make on their behalf? If a change could be made to correct the issues that cygwin has on systems that have these products 'inflicted' upon them [1], without causing any reduction in performance in other circumstances, making the code vastly more complex, or requiring additional resources or user intervention, then I suspect it would become a PTC issue. [1] - I choose the word inflicted deliberately - in my experience, norton and mcafee are very bad AV products, and only getting worse as they forcibly integrate other products into them. They frequently miss genuine threats and generate false positives, and fail thorough tests on a regular basis. As a rule, IT admins have sufficient say that they can change company policy in regards to AV and firewall software. Indeed, in a corporate environment, it is *extremely* unusual to come across software firewalls being in use, beyond possibly an implementation of IPTables filtering traffic between the network and the internet, and between remote sites. Usually, there will be hardware firewalls, such as FW1, Cisco PIX, or Nokia's firewall product, whose name I forget. If you name a circumstance where office-based machines require a software firewall, a strong argument can be made as to how and why that network has not been properly secured. In my opinion, this also applies to on-access virus scanners. Feel free to ask me why in a direct email, that would be OT for this list. Since we ultimately don't know the root cause, this discussion is premature, however if the group isn't going to be open to changes, there is no point trying to find them, time would be better spent finding alternates. That would be ashame as I think Cygwin is the most progessive tool available for IT and development work. Certainly for those attempting to bridge the Linux/UNIX - Windows environment. In all honest, you have three options that can be used within windows. Cygwin, MinGW, and SFU. Of those, MinGW is not really all that well suited to these circumstances, and SFU does not have nearly the range of capabilities that Cygwin has. While cgf is on record as saying he does not want to work around issues with other software, if evidence were to emerge to show cygwin could use another method without any detriment, I imagine it would be considered. However, specifically with regards to on-access AV, I don't believe there will be another way to deal with this,
Re: stat(2) triggers on-demand virus scan
> On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: > >I'm still researching, I was going to respond this is posting at a > >later time with more insight, but before things get out-of-hand, I > >wanted to jump in. I suppose I'm still hopeful that we can zero in > >on what precisely is causing the on-demand scanners to consume so > >much CPU. Since Windows programs don't trigger the same level of > >response (or atleast they don't appear to) their must be some change > >that can be made. > > I just wanted to make it clear that we aren't going to be making any > special concessions to a product like a virus scanner which cause > perfectly acceptable code to misbehave. If that is the case then it > is a situation for the virus scanner to work out. It's not a > requirement that cygwin work around things like this. Well, that is a pretty strong statement, I'd expect from a for-profit company run by corporate management. ZoneLabs offical stance is that they don't support emulated environments. Humm... So if neither are willing to change, then what? I don't know Symantec's or McAfee's offical stance. As far as coding being 'perfectly acceptable', that is a matter of point-of- view. If it causes such behavior, is it acceptable? While I respect the purist point of view, one I've held over the years, seems that we need to be practical sometimes. Are you saying that if the problem could be isolated, and reasonable changes proposed, you wouldn't make/allow them? Do we want IT administrators to utilize Cygwin to integrate with the Linux/UNIX environment? If this means not being able to effectively use Cygwin if they are also required to run ZoneAlarm, Norton, McAfee, what choice do you think they'll make, or more precisely, their management will make on their behalf? Since we ultimately don't know the root cause, this discussion is premature, however if the group isn't going to be open to changes, there is no point trying to find them, time would be better spent finding alternates. That would be ashame as I think Cygwin is the most progessive tool available for IT and development work. Certainly for those attempting to bridge the Linux/UNIX - Windows environment. Brett Brett C. Serkez, Techie -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
On Sat, Jan 14, 2006 at 03:35:17PM -0500, Brett Serkez wrote: >I'm still researching, I was going to respond this is posting at a >later time with more insight, but before things get out-of-hand, I >wanted to jump in. I suppose I'm still hopeful that we can zero in >on what precisely is causing the on-demand scanners to consume so >much CPU. Since Windows programs don't trigger the same level of >response (or atleast they don't appear to) their must be some change >that can be made. I just wanted to make it clear that we aren't going to be making any special concessions to a product like a virus scanner which cause perfectly acceptable code to misbehave. If that is the case then it is a situation for the virus scanner to work out. It's not a requirement that cygwin work around things like this. Maybe this is a given but I thought I'd just make it clear. >I've started to notice that even on one system that I have that is >relatively isolated, and not encumbered with security software, bash, >ssh, sh and other programs seems to consume an inordinate amount of CPU. Try setting CYGWIN=noinordinate . cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
> > The stat(2) system call runs very slowly because it is constantlt > > triggering the McAfee on-demand virus scanner to scan the file that > > is being stat'ed. This may not seem like a big thing but I > > frequently stat thousands of files at a batch. I find that the stat > > runs much faster when I temporarily disable the on-demand virus > > scanner. > > Judging from previous messages on this list it *seems* that one of the > slowest things you can do in cygwin is accessing files; stat(), > fopen() and the like. > > > In general... FWIW/IMO; If you have the option to replace M*Af**[1] > with a just as good an AV, then do that - I suggest to avoid > Sym*ntec[2] products too as they seem to have similar problems. I've reported similar issues with ZoneAlarm and its on-demand scanning. This is the price that is paid on WinDoze for a generally less sure environment and having to compensate with such tools. That being said, I don't believe its reasonable to just give up or constrain one's self to particular security products as a work around, although their may be other reasons. The choice of a given firewall or security tool may not be an option, should this prohibit one from using cygwin effectively in any given environment? I'm still researching, I was going to respond this is posting at a later time with more insight, but before things get out-of-hand, I wanted to jump in. I suppose I'm still hopeful that we can zero in on what precisely is causing the on-demand scanners to consume so much CPU. Since Windows programs don't trigger the same level of response (or atleast they don't appear to) their must be some change that can be made. I've started to notice that even on one system that I have that is relatively isolated, and not encumbered with security software, bash, ssh, sh and other programs seems to consume an inordinate amount of CPU. This thread points to stat, I've been looking at process creation, perhaps it is a bit of both. WinDoze systems tend to create monolithic programs, where as UNIX/Linux/POSIX tend to create lots of small programs and invoke them often, which inherently causes more file accesses and process creation. I suspect their are multiple issues, I also suspect they can eventually be addressed. Brett Brett C. Serkez, Techie -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: stat(2) triggers on-demand virus scan
pmcferrin wrote: > The stat(2) system call runs very slowly because it is constantlt > triggering the McAfee on-demand virus scanner to scan the file that > is being stat'ed. This may not seem like a big thing but I frequently > stat thousands of files at a batch. I find that the stat runs much > faster when I temporarily disable the on-demand virus scanner. Judging from previous messages on this list it *seems* that one of the slowest things you can do in cygwin is accessing files; stat(), fopen() and the like. In general... FWIW/IMO; If you have the option to replace M*Af**[1] with a just as good an AV, then do that - I suggest to avoid Sym*ntec[2] products too as they seem to have similar problems. OTOH, I have good experience with what you find at f-secure dot com - I've had this one installed since cygwin 1.3.x was current, and prior to that. I've always considered S. and M. AV's to be CPU hogs in general terms - and have found f-secure to be much lighter in this respect. Now I wonder how M. and S. AV's compare to what I have done in a simple (attached) comparasion with fsecure V5.30 ON/OFF (Use e.g. NOTEPAD, and a monospace font to view it) /H [1] I've got previous experience with having it on my private PC. [2] I'm forced to live with such a thing at work. -- Test object: Windows partition with some additional SW installed Included on this disk is a huge cygwin installation. Test run several times prior taking these samples. Also making sure it ran without interference from other running software - this was to ensure somewhat persistent timings. AV ON AV OFF find =prints=> 201195 (files+dirs) real30.089 28.165 27.875 real27.547 28.113 27.988 user 5.576 5.498 5.779 user 5.529 5.732 5.451 sys 23.966 22.123 21.638 sys 21.562 21.842 21.874 find - per file/dir, microseconds calculated from the above file/dir count 150 140 139 137 140 139 du -s =prints=> 7431252 real87.608 88.285 87.523 real43.355 41.916 41.815 user 8.155 8.037.89 user 7.358 8.015 7.624 sys 77.156 77.905 77.734 sys 33.531 32.062 32.312 du - per file/dir, microseconds calculated from the above file/dir count 435 439 435 215 208 208 >From this it seems that "du" does something that triggers the f-secure AV in some way (AV doing the same as "du"?). This has the impact of doubling the scan time per file/dir. It would be interesting to see similar measurements done with McAFee and Symantec antivirus packages. -- actual test session log follows -- $ cd W: # Sempron 2800+, 1G RAM - has Win2K and related files on W: --- AV enabled -- $ for (( i=0; i<3 ;i++ )) ;do time find 2>/dev/null -printf "%f\n" | wc -l ;done 201195 real0m30.089s user0m5.576s sys 0m23.966s 201195 real0m28.165s user0m5.498s sys 0m22.123s 201195 real0m27.875s user0m5.779s sys 0m21.638s 201195 -- AV disabled -- $ for (( i=0; i<3 ;i++ )) ;do time find 2>/dev/null -printf "%f\n" | wc -l ;done 201195 real0m27.547s user0m5.529s sys 0m21.562s 201195 real0m28.113s user0m5.732s sys 0m21.842s 201195 real0m27.988s user0m5.451s sys 0m21.874s 201195 $ for (( i=0; i<3 ;i++ )) ;do time du -s 2>/dev/null ;done 7431252 . real0m43.355s user0m7.358s sys 0m33.531s 7431252 . real0m41.916s user0m8.015s sys 0m32.062s 7431252 . real0m41.815s user0m7.624s sys 0m32.312s 7431252 . -- AV enabled -- $ for (( i=0; i<3 ;i++ )) ;do time du -s 2>/dev/null ;done 7431252 . real1m27.608s user0m8.155s sys 1m17.156s 7431252 . real1m28.285s user0m8.030s sys 1m17.905s 7431252 . real1m27.523s user0m7.890s sys 1m17.734s 7431252 . $ uname -a CYGWIN_NT-5.0 amd 1.5.19s(0.149/4/2) 20051229 16:10:48 i686 unknown unknown Cygwin -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: stat(2) triggers on-demand virus scan
On Sat, Jan 14, 2006 at 12:56:11AM -0500, Paul McFerrin wrote: >I have an "antique" question. > >I'm currently running cygwin.dll version: 1.3.6 ! (don't laugh). I use >cygwin daily in my work and I'm happy not to disturb things that are not >broken. > >The stat(2) system call runs very slowly because it is constantlt >triggering the McAfee on-demand virus scanner to scan the file that is >being stat'ed. This may not seem like a big thing but I frequently stat >thousands of files at a batch. I find that the stat runs much faster when >I temporarily disable the on-demand virus scanner. > >When I examined the code within cygwin1.dll, I really didn't see anything >cygwin was doing to trigger this behavior so I assummed it was a MS >feature. I'm using XP Pro with NO SP2. > >Has the behavior of the stat changed over the years that might solve this >problem? It *might* prompt me to upgrade if this behavior has been >improved. It is a novel concept, but why not just try it? Back up your cygwin directory, install a new version, and see if there is any difference. If you don't like what you see, then restore the directory. It's either that or trust some random mailing list voice to say yes or no and then find out for yourself just how authoritative they really are. Given that every installation is different, no one is going to be able to give you a definitive answer anyway. cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
stat(2) triggers on-demand virus scan
I have an "antique" question. I'm currently running cygwin.dll version: 1.3.6 ! (don't laugh). I use cygwin daily in my work and I'm happy not to disturb things that are not broken. The stat(2) system call runs very slowly because it is constantlt triggering the McAfee on-demand virus scanner to scan the file that is being stat'ed. This may not seem like a big thing but I frequently stat thousands of files at a batch. I find that the stat runs much faster when I temporarily disable the on-demand virus scanner. When I examined the code within cygwin1.dll, I really didn't see anything cygwin was doing to trigger this behavior so I assummed it was a MS feature. I'm using XP Pro with NO SP2. Has the behavior of the stat changed over the years that might solve this problem? It *might* prompt me to upgrade if this behavior has been improved. -Paul McFerrin -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
