Sax (or stax) is an example of streaming with a higher level format, but there are plenty of other ways as well. The *critical* performance element is to *never* have to accumulate an entire intermediate document on either side (eg json object or xml Dom) if you can avoid it. You end up requiring 4x the resources (or more), extra latency, more parsing, and more garbage collection.
I'll get with Jim webber and propose a prototype of alternatives. Note also that the lack of binary I/O in the browser without flash/java/silverlight is a challenge, but we can work around it. ----- Reply message ----- From: "Michael DeHaan" <michael.deh...@gmail.com> Date: Fri, Apr 22, 2011 12:18 pm Subject: [Neo4j] REST results pagination To: "Neo4j user discussions" <user@lists.neo4j.org> On Thu, Apr 21, 2011 at 5:00 PM, Michael Hunger <michael.hun...@neotechnology.com> wrote: > Really cool discussion so far, > > I would also prefer streaming over paging as with that approach we can give > both ends more of the control they need. Just in case we're not talking about the same kind of streaming -- when I think streaming, I think "streaming uploads", "streaming downloads", etc. If the REST format is JSON (or XML, whatever), that's a /document/ so you can't just say "read the next (up to) 512 bytes" and work on it. It becomes a more low-level endeavor because if you're in the middle of reading a record, or don't even have the "end of list" terminator, what you have isn't parseable yet. I'm sure a lot of hacking could be done to make the client figure out if he had enough other than the closing array element, but it's a lot to ask of a JSON client. So I'm interested in how, in that proposal, the REST API might stream results to a client, because for the streaming to be meaningful, you need to be able to parse what you get back and know where the boundaries are (or build a buffer until you fill in a datastructure enough to operate on it). I don't see that working with JSON/REST so much. It seems to imply a message bus. --Michael _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
