Thanks Ken, Your thoughtful answer clarified what I have been experiencing. On recent machines readInBackground is definitely a small toll compared to rsync - it is on older slower Macs that the toll got closer to rsync's write and really crunched it. I am trying to accommodate older OS and ppc's so there are issues. I am just learning about inter process communication, pipes etc. and don't have a good overview of it yet. Now I understand that the pipe is the consideration it all makes sense.
My app for years had been using an external log for rsync to write to and a timer for polling and it does work nicely, though with rsync the file can get enormous now with TB's of data transfer and there is overhead with streaming calls to NSTask 'tail' and 'ps' to see if rsync is still running. I was hoping to keep the stream in the app and get the pieces directly - I may actually be able to spec the machine in use and use a simpler, slightly less informative, parsing scheme in that case. If I remove the parsing code and let readInBackground run clean, there is no blocking. Thanks for your wisdom on this. best, Rob On Jan 29, 2011, at 12:14 AM, Ken Thomases wrote: > On Jan 28, 2011, at 4:53 PM, Robert DuToit wrote: > >> Basically I am launching rsync via a helper tool and launchd and need to >> parse the output which is returned to the main app via the BAS response as a >> file descriptor. I set up a notification using readInBackgroundAndNotify . >> >> This works but parsing the data slows down rsync. If the app freezes for any >> reason rsycn stops. [...] I don't quite understand the connection here with >> the main app and rsync since the parent process of rsync seems to be launchd >> but parsing the file descriptor output in the main app slows rsync. > > The parent process is irrelevant. It is the pipe that rsync is writing to > and your app is reading from. > >> Or maybe this is just the two way nature of file descriptors and output, >> that you can't put the brakes on the output. > > Well, you _can_ put the brakes on the output and that's the problem. You > _should not_ put the brakes on the output if you don't want to slow down the > writer. > > The OS kernel is only willing to buffer so much data for a pipe. After the > buffer is full, any attempt to write to the pipe will block until the buffer > is drained by the reader. > > >> Originally I had the BAS request to launch rsync in a separate thread but >> couldn't find a way to use that with a readInBackgroundAndNotify >> Notification so I put the parsing into a loop - however it caused a big >> spike in cpu and sometimes froze up the UI after a long haul. I have found >> using readInBackgroundAndNotify doesn't do this but also won't work in a >> separate thread... > > If you're processing lots of data that's coming in quickly, you might expect > high CPU usage. You may be able to improve that by optimizing your parsing > algorithm. Certainly, rsync is doing a lot of work and producing relatively > less output, so you should be able to keep up. That is, you should have a > lot less work parsing rsync's output than rsync is doing to produce that > output. > > If you're doing any long-running operation on the main thread, that will > freeze up the GUI. It doesn't matter if that is CPU-intensive work or just > blocking waiting for data to arrive. The important thing is you're not > allowing for processing of GUI events because you're not returning control > back to the framework. > > readInBackgroundAndNotify should work on a background thread so long as that > thread is running its run loop. Whether that solves your problem, I don't > know. > > >> So I guess my question is if anyone has any ideas about either running rsync >> in a detached way from the helper tool (I thought that is what was >> happening!) or in the main app being able to read the output independently >> of rsync so it doesn't affect the performance. > > Rsync is running in a separate process, it's true. However, it can't be > completely independent and yet also be communicating its output to the main > app. That's an interaction. And, if the main app is failing to read from > the pipe quickly enough, then the pipe's buffer will fill, and rsync will > block when it tries to write more. This is an inherent fact of interprocess > communication. > > One possible alternative approach is to use the file system as an > intermediary to buffer rsync's output. That is, have rsync direct its output > to a file rather than a pipe. Have your app read from that file. If your > app falls behind, then the file simply grows and rsync isn't slowed down. > You have to be careful when reading from the file to distinguish between > catching up to the _current_ end of the file (as much as rsync has written so > far) vs. reaching the true end of the file (when rsync is done and won't be > writing any more). > > However, you really shouldn't need to do this. As I say, it should be > possible for your app to keep up with rsync. > > Regards, > Ken > _______________________________________________ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
