Sorry if I'm entering the discussion late, but we solved this problem by extending FilterInputStream and FilterOutputStream. We use a special character (ASCII EOM for our stuff) to mark the end of an XML document (and precede it with an ESC character if it occurs in the document, and of course the ESC is also ESCed). We then override the read and write methods to check/insert the characters in the streams. We also override the close methods so that they don't actually close the underlying streams. We then pass this stream to the parser.

I've attached the java code for these classes.

The transmitting side that uses this class looks something like this:

Socket s = ...;
OutputStream out = s.getOutputStream ();
BufferedOutputStream bos = new BufferedOutputStream (out);
while (...more...) {
    XMLMessage.XMLOutputStream xos = new XMLMessage.XMLOutputStream (bos);
    writeDocument (xos);
    // assuming that writeDocument closes the xos when it's done.
}

And, the reading/parsing side that uses this class looks something like this:

Socket s = ...;
InputStream in = s.getInputStream ();
BufferedInputStream bis = new BufferedInputStream (in);
while (...more...) {
    XMLMessage.XMLInputStream xis = new XMLMessage.XMLInputStream (bis);
    ... possibly wrap xis in a InputReader and hand it off to the parser
}

Of course you'll want to catch the appropriate exceptions, etc.

We've found that this technique is efficient, handles ill-formed xml documents correctly as they pertain to subsequent documents, and is easy to use.

Hope this helps,
Don Bate

At 10:20 AM +0000 2/23/06, Mike Skells wrote:
Hi,
I believe that you can have PI, comments, whitespace etc after the root
element, is that significant for you ?

----
I have the same problem in one of out applications. We looked to the format
the HTTP uses, and 'borrowed' the ideas from there. Due to the volume of the
data that we handled then we could not afford the overhaed ov scanning each
byte /char for a specific marker.

What we do is insert a Content-Length:<xx><cr> marker that details the
length of the content( which is usually in our case a document> in bytes,
after encoding.

This has a drawback inthat the document needs to be prepared before being
sent, and therefore buffered, but for this application it is not an issue
and the documents that we handle are ver very small (a few hundred bytes)
but we have to manage 100-500 per second.

You could insert a marker to indicate a continuation, if your documents are
large, and you cannot afford the buffering

E.g.
More:4000
<4000 bytes>
More:4000
<4000 bytes>
More:4000
<4000 bytes>
Complete:407
<407 bytes that make up the rest of the document>

It works well for us. May not suit you

--------------

If you truly cannot change the protocol, then if the documents that are sent
have a <?xml ... Header that you could use that as the marker, but his does
mean that you would not know that a documen is complete until the next
document is started, so you would always be one behind.

Just my 2c

Mike


 -----Original Message-----
 From: Joseph Kesselman [mailto:[EMAIL PROTECTED]
 Sent: 21 February 2006 18:21
 To: [email protected]
 Cc: [email protected]; Polk, John R.
 Subject: Re: How to read multiple XML from socket: cannot
 change the protocol (Re: How to handle continuous stream of XML)

 Note too that a well-formed XML document can only have one
 top-level element -- everything after that is normally
 discarded -- so that too could be used as a clue for diviing
 a multiple-document stream.

 Or you could invent some new marker between documents, and
 have your input-stream filter use that to break up the docs.

 Or you could just pack all the XML files into a zipfile, send
 that, and have your recieving tool unpack that into separate
 > files. This would have the advantage of not having to
 (slightly) break people's expectations about whether what
 they're getting back form the server is one document or
 several... and might actually improve performance, especially
 on larger documents; XML compresses wonderfully.

 Whichever approach you use, note that this isn't really an
 XML problem; it's a stream management problem. The XML parser
 expects to see a stream that presents only a single XML
 document, so breaking up the stream into multiple docs has to
 happend before it reaches the parser.


 "Ooof! There's a wasp in the room!"
 "Get out! Quick! Before it gets to the tiger...!" -- Monty
 Python, _Matching_Tie_And_Handkerchief_

 ______________________________________
 Joe Kesselman -- Beware of Blueshift!
 "The world changed profoundly and unpredictably the day Tim
 Berners Lee got bitten by a radioactive spider." -- Rafe
 Culpin, in r.m.filk


 ---------------------------------------------------------------------
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]





---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


--
Don Bate               | Specializing in Consulting and Mentoring in
Bate Consulting, Inc   | Object-Oriented Technologies,
                       | Software Architecture, and Software Process
(972) 618-0208 voice
(972) 618-0216 fax
[EMAIL PROTECTED]

Attachment: %XMLMessage.java
Description: application/applefile

Attachment: XMLMessage.java
Description: Binary data

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to