Re: Asynchronous downloading and parsing of XML
On 8 Aug 2011, at 19:20, Thomas Davie wrote: > All of this is asynchronous, so everything works happily :) Sure, "we" have established that your solution works, so far so good. What I am trying to get at is whether there is some design-pattern implied in the docs for the class which should alert me to the behaviour that it will not block the main-thread. An alternative solution to yours, might be batching NSData-objects as they are delivered by NSURLConnection and sending NSXMLParser off on a dispatch queue, so the main thread is not blocked. Mikkel ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
On 8 Aug 2011, at 19:44, Jens Alfke wrote: > It’s clearly a SAX parser. SAX parsers are incremental, that’s their whole > point, otherwise it would be easier just to use a DOM API like NSXMLDocument. The docs implies that it is (related to it): http://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/XMLParsing/Articles/ParserArchitecture.html#//apple_ref/doc/uid/20002263-BCIHAHJE Does that imply anything, inherently, about whether the class blocks the main thread? Mikkel___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
On Aug 8, 2011, at 10:16 AM, Mikkel Islay wrote: > No, that states something about the way NSXMLParser is able to parse from a > stream. It doesn't say anything about its state with respect to the main > thread. It’s clearly a SAX parser. SAX parsers are incremental, that’s their whole point, otherwise it would be easier just to use a DOM API like NSXMLDocument. —Jens___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
On 8 Aug 2011, at 18:16, Mikkel Islay wrote: > On 8 Aug 2011, at 19:04, Thomas Davie wrote: >> >> Parameters >> stream >> The input stream. The content is incrementally loaded from the specified >> stream and parsed. > > No, that states something about the way NSXMLParser is able to parse from a > stream. It doesn't say anything about its state with respect to the main > thread. I'm not sure what you're getting at… The key facts are… NSURLConnection provides data incrementally. The URLConnection delegate takes that data incrementally and provides it to the stream incrementally. The Stream takes the data incrementally and provides it to the parser as it gets it. The XML parser takes the data incrementally and calls the delegate when it parses something successfully. All of this is asynchronous, so everything works happily :) Tom Davie___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
On 8 Aug 2011, at 19:04, Thomas Davie wrote: > > Parameters > stream > The input stream. The content is incrementally loaded from the specified > stream and parsed. No, that states something about the way NSXMLParser is able to parse from a stream. It doesn't say anything about its state with respect to the main thread. Mikkel___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
The hint is that the NSXMLParser docs say:
Parameters
stream
The input stream. The content is incrementally loaded from the specified stream
and parsed.
if (*ra4 != 0xffc78948) { return false; }
On 8 Aug 2011, at 17:59, Mikkel Islay wrote:
> Thanks for the explanation, Jens too.
> The hint that NSXMLParser has that behaviour is that it accepts an
> NSInputStream, or that it relies on a delegate for communicating
> "parse-events"?
>
> Mikkel
>
>
> On 8 Aug 2011, at 10:34, Andreas Grosam wrote:
>
>> No. Asynchronous routines can be implemented above a blocking interface.
>> Blocking and asynchronous interfaces work well together, and you may not
>> even notice the existence of a blocking interface since it is working under
>> the hood as an implementation detail.
>>
>> For instance the parser *could* (we don't know) be implemented as a
>> "recursive decent" parser. This kind of parser will effectively require that
>> the parsing routine blocks, when there is no further input available, since
>> the state of the parser will be in its call stack. So you can't just stop
>> and exit the parse function in the middle of a stream, and then continue
>> with a newly invoked method when more data is available.
>>
>> The parser would just dispatch its events on the main thread where the
>> delegate can handle them. So, from the view point of the delegate, it looks
>> like an asynchronous interface.
>>
> ___
>
> Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
>
> Please do not post admin requests or moderator comments to the list.
> Contact the moderators at cocoa-dev-admins(at)lists.apple.com
>
> Help/Unsubscribe/Update your Subscription:
> http://lists.apple.com/mailman/options/cocoa-dev/tom.davie%40gmail.com
>
> This email sent to tom.da...@gmail.com
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
Thanks for the explanation, Jens too. The hint that NSXMLParser has that behaviour is that it accepts an NSInputStream, or that it relies on a delegate for communicating "parse-events"? Mikkel On 8 Aug 2011, at 10:34, Andreas Grosam wrote: > No. Asynchronous routines can be implemented above a blocking interface. > Blocking and asynchronous interfaces work well together, and you may not even > notice the existence of a blocking interface since it is working under the > hood as an implementation detail. > > For instance the parser *could* (we don't know) be implemented as a > "recursive decent" parser. This kind of parser will effectively require that > the parsing routine blocks, when there is no further input available, since > the state of the parser will be in its call stack. So you can't just stop and > exit the parse function in the middle of a stream, and then continue with a > newly invoked method when more data is available. > > The parser would just dispatch its events on the main thread where the > delegate can handle them. So, from the view point of the delegate, it looks > like an asynchronous interface. > ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
On Aug 7, 2011, at 1:47 PM, Mikkel Islay wrote: > Thanks for sharing with the list. > ... Aren't you in effect throwing away the benefits of asynchronous loading? No. Asynchronous routines can be implemented above a blocking interface. Blocking and asynchronous interfaces work well together, and you may not even notice the existence of a blocking interface since it is working under the hood as an implementation detail. For instance the parser *could* (we don't know) be implemented as a "recursive decent" parser. This kind of parser will effectively require that the parsing routine blocks, when there is no further input available, since the state of the parser will be in its call stack. So you can't just stop and exit the parse function in the middle of a stream, and then continue with a newly invoked method when more data is available. The parser would just dispatch its events on the main thread where the delegate can handle them. So, from the view point of the delegate, it looks like an asynchronous interface. Andreas > The NSInputstream can block, and NSXMLParser presumably will. You will also > have to deal with events in several places, and make sure errors are > propagated up and down the stream. > > Mikkel > ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
On Aug 7, 2011, at 4:47 AM, Mikkel Islay wrote: > I wonder what the purpose of NSURLConnection is in this solution? To do the HTTP fetching. This is a PITA to get right if you implement it yourself, if that’s what you’re thinking of. > Aren't you in effect throwing away the benefits of asynchronous loading? The > NSInputstream can block, and NSXMLParser presumably will. NSXMLParser reads from the input stream without blocking, and calls its delegate when it parses new data. —Jens ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
The purpose of the NSURLConnection is to deal with HTTP… How else would you
propose dealing with it?
I'm not sure why you think this throws away benefits of async loading, can you
clarify that?
Thanks
Tom Davie
On 7 Aug 2011, at 12:47, Mikkel Islay wrote:
> Thanks for sharing with the list.
> I wonder what the purpose of NSURLConnection is in this solution? Aren't you
> in effect throwing away the benefits of asynchronous loading? The
> NSInputstream can block, and NSXMLParser presumably will. You will also have
> to deal with events in several places, and make sure errors are propagated up
> and down the stream.
>
> Mikkel
>
>
> On 6 Aug 2011, at 22:08, Thomas Davie wrote:
>
>> Just for reference, With the help of Mike Ash I figured out how to do this
>> sensibly. Note, code typed straight into Mail.app, so it may not be dead
>> perfect, but it should show the idea:
>>
>> //When creating the NSURLConnection and parser:
>> - (void)doMagicParsingStuff
>> {
>> NSInputStream *iStream;
>> NSOutputStream *oStream;
>> CFStreamCreateBoundPair(NULL, &iStream, &oStream, 4096);
>> [oStream setDelegate:self];
>> NSRunLoop *runLoop = [NSRunLoop currentRunLoop];
>> [iStream scheduleInRunLoop:runLoop forMode:NSDefaultRunLoopMode];
>> [oStream scheduleInRunLoop:runLoop forMode:NSDefaultRunLoopMode];
>> finishedLoading = NO;
>> [iStream open];
>> [oStream open];
>> [self setParserStream:oStream];
>> NSXMLParser *parser = [[[NSXMLParser alloc] initWithStream:iStream]
>> autorelease];
>> }
>>
>> - (void)attemptToWriteToStream
>> {
>> NSUInteger written = [[self parserStream] writeBytes:[[self buffer] bytes]
>> maxLength:[[self buffer] length]];
>> [[self buffer] replaceBytesInRange:NSMakeRange(0,written) withBytes:""
>> length:0];
>> }
>>
>> // In the output stream delegate:
>> - (void)stream:(NSStream *)s handleEvent:(NSStreamEvent)event
>> {
>> if (NSStreamEventHasSpaceAvailable == event)
>> {
>> if (finishedLoading && [[self buffer] length] == 0)
>> {
>> [[self parserStream] close];
>> }
>> else
>> {
>> [self attepmtToWriteToStream];
>> }
>> }
>> }
>>
>> // In the NSURLConnection Delegate:
>> - (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)d
>> {
>> [[self buffer] appendData:d];
>> [self attemptToWriteToStream];
>> }
>>
>> - (void)connectionDidFinishLoading:(NSURLConnection *)connection
>> {
>>finishedLoading = YES;
>> }
>>
>>
>> On 6 Aug 2011, at 12:14, Mikkel Islay wrote:
>>
>>> You can't hook up NSURLConnection to NSXMLParser like that because it loads
>>> its data asynchronously. Rather, feed it (a copy of) the NSData object
>>> generated by NSURLConnection, or bypass NSURLConnection completely and load
>>> your data via NSInputStream. In either case you will have to verify that
>>> the data you feed to NSXMLParser is complete.
>>>
>>> Mikkel
>>>
>>> On 6 Aug 2011, at 12:02, Thomas Davie wrote:
>>>
I'm trying to figure out how to download and parse XML at the same time.
NSXMLParser seems to have support for this by providing it with an
NSInputStream. However, I see no way to get an NSInputStream that
contains the data that an NSURLConnection is downloading.
>>>
>>
>
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
Thanks for sharing with the list.
I wonder what the purpose of NSURLConnection is in this solution? Aren't you in
effect throwing away the benefits of asynchronous loading? The NSInputstream
can block, and NSXMLParser presumably will. You will also have to deal with
events in several places, and make sure errors are propagated up and down the
stream.
Mikkel
On 6 Aug 2011, at 22:08, Thomas Davie wrote:
> Just for reference, With the help of Mike Ash I figured out how to do this
> sensibly. Note, code typed straight into Mail.app, so it may not be dead
> perfect, but it should show the idea:
>
> //When creating the NSURLConnection and parser:
> - (void)doMagicParsingStuff
> {
>NSInputStream *iStream;
>NSOutputStream *oStream;
>CFStreamCreateBoundPair(NULL, &iStream, &oStream, 4096);
>[oStream setDelegate:self];
>NSRunLoop *runLoop = [NSRunLoop currentRunLoop];
>[iStream scheduleInRunLoop:runLoop forMode:NSDefaultRunLoopMode];
>[oStream scheduleInRunLoop:runLoop forMode:NSDefaultRunLoopMode];
>finishedLoading = NO;
>[iStream open];
>[oStream open];
>[self setParserStream:oStream];
>NSXMLParser *parser = [[[NSXMLParser alloc] initWithStream:iStream]
> autorelease];
> }
>
> - (void)attemptToWriteToStream
> {
>NSUInteger written = [[self parserStream] writeBytes:[[self buffer] bytes]
> maxLength:[[self buffer] length]];
>[[self buffer] replaceBytesInRange:NSMakeRange(0,written) withBytes:""
> length:0];
> }
>
> // In the output stream delegate:
> - (void)stream:(NSStream *)s handleEvent:(NSStreamEvent)event
> {
>if (NSStreamEventHasSpaceAvailable == event)
>{
>if (finishedLoading && [[self buffer] length] == 0)
>{
>[[self parserStream] close];
>}
>else
>{
>[self attepmtToWriteToStream];
>}
>}
> }
>
> // In the NSURLConnection Delegate:
> - (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)d
> {
>[[self buffer] appendData:d];
>[self attemptToWriteToStream];
> }
>
> - (void)connectionDidFinishLoading:(NSURLConnection *)connection
> {
> finishedLoading = YES;
> }
>
>
> On 6 Aug 2011, at 12:14, Mikkel Islay wrote:
>
>> You can't hook up NSURLConnection to NSXMLParser like that because it loads
>> its data asynchronously. Rather, feed it (a copy of) the NSData object
>> generated by NSURLConnection, or bypass NSURLConnection completely and load
>> your data via NSInputStream. In either case you will have to verify that the
>> data you feed to NSXMLParser is complete.
>>
>> Mikkel
>>
>> On 6 Aug 2011, at 12:02, Thomas Davie wrote:
>>
>>>
>>> I'm trying to figure out how to download and parse XML at the same time.
>>> NSXMLParser seems to have support for this by providing it with an
>>> NSInputStream. However, I see no way to get an NSInputStream that contains
>>> the data that an NSURLConnection is downloading.
>>
>
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Re: Asynchronous downloading and parsing of XML
Just for reference, With the help of Mike Ash I figured out how to do this
sensibly. Note, code typed straight into Mail.app, so it may not be dead
perfect, but it should show the idea:
//When creating the NSURLConnection and parser:
- (void)doMagicParsingStuff
{
NSInputStream *iStream;
NSOutputStream *oStream;
CFStreamCreateBoundPair(NULL, &iStream, &oStream, 4096);
[oStream setDelegate:self];
NSRunLoop *runLoop = [NSRunLoop currentRunLoop];
[iStream scheduleInRunLoop:runLoop forMode:NSDefaultRunLoopMode];
[oStream scheduleInRunLoop:runLoop forMode:NSDefaultRunLoopMode];
finishedLoading = NO;
[iStream open];
[oStream open];
[self setParserStream:oStream];
NSXMLParser *parser = [[[NSXMLParser alloc] initWithStream:iStream]
autorelease];
}
- (void)attemptToWriteToStream
{
NSUInteger written = [[self parserStream] writeBytes:[[self buffer] bytes]
maxLength:[[self buffer] length]];
[[self buffer] replaceBytesInRange:NSMakeRange(0,written) withBytes:""
length:0];
}
// In the output stream delegate:
- (void)stream:(NSStream *)s handleEvent:(NSStreamEvent)event
{
if (NSStreamEventHasSpaceAvailable == event)
{
if (finishedLoading && [[self buffer] length] == 0)
{
[[self parserStream] close];
}
else
{
[self attepmtToWriteToStream];
}
}
}
// In the NSURLConnection Delegate:
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)d
{
[[self buffer] appendData:d];
[self attemptToWriteToStream];
}
- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
finishedLoading = YES;
}
On 6 Aug 2011, at 12:14, Mikkel Islay wrote:
> You can't hook up NSURLConnection to NSXMLParser like that because it loads
> its data asynchronously. Rather, feed it (a copy of) the NSData object
> generated by NSURLConnection, or bypass NSURLConnection completely and load
> your data via NSInputStream. In either case you will have to verify that the
> data you feed to NSXMLParser is complete.
>
> Mikkel
>
> On 6 Aug 2011, at 12:02, Thomas Davie wrote:
>
>>
>> I'm trying to figure out how to download and parse XML at the same time.
>> NSXMLParser seems to have support for this by providing it with an
>> NSInputStream. However, I see no way to get an NSInputStream that contains
>> the data that an NSURLConnection is downloading.
>
___
Cocoa-dev mailing list (Cocoa-dev@lists.apple.com)
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
Asynchronous downloading and parsing of XML
Hi everyone, I'm trying to figure out how to download and parse XML at the same time. NSXMLParser seems to have support for this by providing it with an NSInputStream. However, I see no way to get an NSInputStream that contains the data that an NSURLConnection is downloading. Is this possible in some sane way? Thanks Tom Davie___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
