Re: NSTask oddity with getting stdout
On Jul 26, 2011, at 11:22 PM, Scott Ribe wrote: On Jul 26, 2011, at 5:52 PM, Shane Stanley wrote: In Snow Leopard that worked fine; a notification would be sent when new data was written to the file. In Lion, as soon as it's called it goes into a loop; each time readInBackgroundAndNotify is sent, a new notification comes straight back. An actual file? I'm reading from a pipe, so that could be one difference. Yes. For a file (vnode), there's no such thing as non-blocking or asynchronous I/O through the typical BSD/POSIX APIs (ignoring aio). All attempts to read or write will complete synchronously, blocking even if the file descriptor has been configured as non-blocking and the file system is mounted over a slow or broken network connection. Attempts to test a vnode file descriptor's readability or writability using select() or poll() will always report that the file is readable or writable. kqueue() is a bit weird for vnodes. EVFILT_READ fires whenever the file pointer is not at the end of the file. That means you'll be notified repeatedly until you read to the end of the file, but won't be notified when you're actually at the end of the file. That is, you won't be told to read once more to receive the EOF indication (e.g. zero-length read). After that, you won't be notified again unless and until the file is extended by something writing more data to it, past the current end, or if you seek to some other position. Although this seems to be what Shane wants, for most programmers this is a surprising result and differs quite a lot from other types of file descriptors. It also doesn't correspond to NSFileHandle's documented behavior, since that doesn't differ by descriptor type. Kqueue's EVFILT_WRITE isn't supported at all for vnodes, as it doesn't make sense, since it would always fire immediately. The caller should simply not use kqueue() and should just go ahead and write unconditionally, instead. None of this applies for I/O through a pipe or socket. Regards, Ken ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: NSTask oddity with getting stdout
On 26/7/11 1:14 AM, Scott Ribe scott_r...@elevated-dev.com wrote: On Jul 25, 2011, at 3:40 AM, Shane Stanley wrote: I wonder if they've changed how the readInBackgroundAndNotify works. Yes, they have. I logged a bug on it, and was told that the way it worked in 10.6 was wrong -- whereas it used not to send a notification until there was something to read, it now returns one immediately regardless. Whether that's the problem here, I don't know. That's absolutely demented. If it sends a notification immediately, with or without data, then you're never really using it to read in the background, you're almost just spinning in a busy loop, getting a probably empty NSData, and calling readInBackgroundAndNotify again, as fast as the runloop can deliver the notifications. Exactly. I built a sample app that did exactly that, and logged radar #9120065. I was told that was correct behavior, and that Snow Leopard's NSFileHandle never correctly notified with a zero length data at EOF. The suggested fix was do a blocking -readDataOfLength: call on a background thread. -- Shane Stanley sstan...@myriad-com.com.au 'AppleScriptObjC Explored' www.macosxautomation.com/applescript/apps/ ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: NSTask oddity with getting stdout
On Jul 26, 2011, at 12:18 AM, Shane Stanley wrote: On Jul 25, 2011, at 3:40 AM, Shane Stanley wrote: I wonder if they've changed how the readInBackgroundAndNotify works. Yes, they have. I logged a bug on it, and was told that the way it worked in 10.6 was wrong -- whereas it used not to send a notification until there was something to read, it now returns one immediately regardless. Whether that's the problem here, I don't know. That's absolutely demented. If it sends a notification immediately, with or without data, then you're never really using it to read in the background, you're almost just spinning in a busy loop, getting a probably empty NSData, and calling readInBackgroundAndNotify again, as fast as the runloop can deliver the notifications. Exactly. I built a sample app that did exactly that, and logged radar #9120065. I was told that was correct behavior, and that Snow Leopard's NSFileHandle never correctly notified with a zero length data at EOF. The suggested fix was do a blocking -readDataOfLength: call on a background thread. - Snow Leopard most certainly did correctly notify with a zero length data at EOF for me and did so sufficiently consistently that I am not aware of it ever failing to do so, in thousands of runs. Therefore never is clearly incorrect. - I don't see how what you said ...it now returns one immediately regardless follows from Apple's response that you quoted; maybe I'm missing some context. - What good will a blocking readDataOfLength: call on a background thread do for the case where you don't know what's coming, need to process it as it comes, then need to know when it's closed? Are they seriously suggesting that we need to read a single character at a time using blocking reads on a background thread? That's such a lame suggestion that it would make me ask: why bother with NSTask and NSFileHandle and notifications at all? In that case POSIX APIs would be no harder to use, and would actually work. - I set up some tests in Lion and ran them last night, and never had an instance of getting a notification with zero-length data before the pipe was closed. Only a few dozen runs, but still, 0-length data always indicated pipe close. (I modified the command-line process with a couple of obnoxious pauses, before sending anything and before sending the final lines, so that if there's some timeout it would be likely to be hit.) Maybe you should try to re-open your bug ;-) -- Scott Ribe scott_r...@elevated-dev.com http://www.elevated-dev.com/ (303) 722-0567 voice ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: NSTask oddity with getting stdout
On 27/7/11 3:18 AM, Scott Ribe scott_r...@elevated-dev.com wrote:
Maybe you should try to re-open your bug ;-)
I did, but no reply.
Here's some code that's reading a file that's being written periodically:
-(void)dataFromFile:(NSNotification *)notif {
NSData *data = [[notif userInfo]
objectForKey:NSFileHandleNotificationDataItem];
if ([data length]) {
// do stuff
}
[[notif object] readInBackgroundAndNotify]; // call again
}
In Snow Leopard that worked fine; a notification would be sent when new data
was written to the file. In Lion, as soon as it's called it goes into a
loop; each time readInBackgroundAndNotify is sent, a new notification comes
straight back.
--
Shane Stanley sstan...@myriad-com.com.au
'AppleScriptObjC Explored' www.macosxautomation.com/applescript/apps/
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: NSTask oddity with getting stdout
On Jul 26, 2011, at 5:52 PM, Shane Stanley wrote: Maybe you should try to re-open your bug ;-) I did, but no reply. Too bad, since it seems to me the person who responded didn't know what they were talking about. In Snow Leopard that worked fine; a notification would be sent when new data was written to the file. In Lion, as soon as it's called it goes into a loop; each time readInBackgroundAndNotify is sent, a new notification comes straight back. An actual file? I'm reading from a pipe, so that could be one difference. I would expect that to go into a loop once a data of length 0 comes in, but not before. -- Scott Ribe scott_r...@elevated-dev.com http://www.elevated-dev.com/ (303) 722-0567 voice ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: NSTask oddity with getting stdout
On 25/7/11 3:01 PM, Stephen J. Butler stephen.but...@gmail.com wrote: I wonder if they've changed how the readInBackgroundAndNotify works. Yes, they have. I logged a bug on it, and was told that the way it worked in 10.6 was wrong -- whereas it used not to send a notification until there was something to read, it now returns one immediately regardless. Whether that's the problem here, I don't know. -- Shane Stanley sstan...@myriad-com.com.au 'AppleScriptObjC Explored' www.macosxautomation.com/applescript/apps/ ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: NSTask oddity with getting stdout
On Jul 24, 2011, at 11:01 PM, Stephen J. Butler wrote: That is, abandon processPipeClose and let each pipe handle its own close in the notification when you're sure all of its read notifications have actually been delivered. As you've presented it, processPipeClose isn't needed. Your theory makes sense to me, and I think for now I'll assume it's the explanation for the behavior. In the real code there's things I need to do after receipt of complete output from both pipes; I just stripped that out of processPipeClose to make this easier to read. However, I can still incorporate your suggestion, and just do the after everything processing when the second of curStdOut curStdErr go to nil. If I weren't feeling lazy this morning, I would dig back through version control. I think the code some time ago was more similar to your suggestion, but loaded up with some unnecessary complications, and this race condition is a result of a simplification attempt which I did not get quite right. Thanks a lot. This was, IMO, not a trivial question ;-) -- Scott Ribe scott_r...@elevated-dev.com http://www.elevated-dev.com/ (303) 722-0567 voice ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: NSTask oddity with getting stdout
On Jul 25, 2011, at 3:40 AM, Shane Stanley wrote: I wonder if they've changed how the readInBackgroundAndNotify works. Yes, they have. I logged a bug on it, and was told that the way it worked in 10.6 was wrong -- whereas it used not to send a notification until there was something to read, it now returns one immediately regardless. Whether that's the problem here, I don't know. That's absolutely demented. If it sends a notification immediately, with or without data, then you're never really using it to read in the background, you're almost just spinning in a busy loop, getting a probably empty NSData, and calling readInBackgroundAndNotify again, as fast as the runloop can deliver the notifications. -- Scott Ribe scott_r...@elevated-dev.com http://www.elevated-dev.com/ (303) 722-0567 voice ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: NSTask oddity with getting stdout
On Sun, Jul 24, 2011 at 10:17 PM, Scott Ribe
scott_r...@elevated-dev.com wrote:
I'm using NSTask to run a background process, monitor its output, and
ultimately present a message to the user. All has been fine with this 10.2
through 10.6, as far as I know. Now under 10.7 I saw a case where my code did
not receive quite all the stdout output from the background process. I saw
this once; I haven't tried running it a huge number of times, but it is
definitely not the common case. (And I've never seen it in dev mode running
under the debugger, only on a machine without dev tools. Also BTW, the output
of the bg process is plain ASCII, no worries about multi-byte sequences.)
Does anybody see anything wrong with this code (severely stripped for
legibility):
Interesting. THIS IS JUST ME SPECULATING OUT LOUD. I wonder if they've
changed how the readInBackgroundAndNotify works. In particular, my
guess here would be that:
1) Two threads are created to handle the stdout and stderr (may be
dispatch queues, but let's say threads for this thought experiment).
2) Your task generates output in stdout, and most of it gets consumed.
3) Your task generates its last data and closes its stderr then stdout.
4) The stdout thread reads the last data and readies a notification,
but doesn't get a chance to post it before being swapped.
5) The stderr thread sees a closed socket and posts a notification
before being swapped.
6) The stdout thread wakes up, and posts its notification.
Now you are in a situation where there are two notifications being
delivered, and their order is wrong for your code. Namely, when the
stderr notification arrives first you will do this:
a) see that stderr is closed.
b) try to read all the remaining data from stdout. There is none,
because it has been queued already.
c) remove yourself as a listener for stdout.
d) try to read all the remaining data from stderr.
e) remove yourself as a listener for stderr.
The problem is step (c)! There might be data in the notification queue
that you will never receive.
Here's how I'd write your methods:
- (void) doSomeTask: (NSTask *) task
Same.
- (void) processStdOut: (NSNotification *) ntc
{
NSFileHandle *f = [ntc object];
NSData *d = [[ntc userInfo] objectForKey:
NSFileHandleNotificationDataItem];
if( [d length] == 0 )
{
[[NSNotificationCenter defaultCenter]
removeObserver: self
name: NSFileHandleReadCompletionNotification
object: curStdOut];
curStdOut = nil;
}
else
{
[f readInBackgroundAndNotify];
[curStdOutStr appendString:
[[[NSString alloc] initWithData: d encoding:
NSASCIIStringEncoding] autorelease]];
}
}
- (void) processStdErr: (NSNotification *) ntc
{
NSFileHandle *f = [ntc object];
NSData *d = [[ntc userInfo] objectForKey:
NSFileHandleNotificationDataItem];
if( [d length] == 0 )
{
[[NSNotificationCenter defaultCenter]
removeObserver: self
name: NSFileHandleReadCompletionNotification
object: curStdErr];
curStdErr = nil;
}
else
{
[f readInBackgroundAndNotify];
[curStdErrStr appendString:
[[[NSString alloc] initWithData: d encoding:
NSASCIIStringEncoding] autorelease]];
}
}
That is, abandon processPipeClose and let each pipe handle its own
close in the notification when you're sure all of its read
notifications have actually been delivered. As you've presented it,
processPipeClose isn't needed.
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
