Here at 49 responses, I'd like to reiterate Craigs point earlier that we are really talking about several separate issues, and I'm wondering if we should split this discussion up, because it is getting very hard to follow it.
As I see it, we are looking at* *three things: * Paging** **Use case: *UI code that presents a paged, infinite scrolled or similar interface to the user. Peeking at results for debugging or other purposes. * Streaming** **Use case: *Returning huge data sets without killing anyone. * Sorting **Use case: *Presenting lists of things to users, applications that care about the order of results for some other reason. I think we've come to agree that streaming and paging serve similar but different purposes and are not quite able to replace each others functionality. I also took the liberty to elevate sorting to it's own topic, because I believe it should be a generic thing you can do on a result set, whereas paging and streaming are different means of returning a result set. If we want to continue this discussion, would anyone object to splitting it into these three parts? /Jake On Sun, Apr 24, 2011 at 2:18 AM, Rick Bullotta <rick.bullo...@thingworx.com>wrote: > Let's discuss sometime soon. Creating resources that need to be cached or > saved in session state bring with them a whole bunch of negative aspects... > > > > ----- Reply message ----- > From: "Michael Hunger" <michael.hun...@neotechnology.com> > Date: Fri, Apr 22, 2011 10:57 pm > Subject: [Neo4j] REST results pagination > To: "Neo4j user discussions" <user@lists.neo4j.org> > > I spent some time looking at what others are doing for inspiration. > > I kind of like the Riak/Basho approach with multipart-chunks and the > approach of explictely creating a resource for the query that can be > navigated (either via pages or first,next,[prev,last] links) and expires > (and could be reconstructed). > > Cheers > > Michael > > Good discussion: > http://stackoverflow.com/questions/924472/paging-in-a-rest-collection > > CouchDB: > http://wiki.apache.org/couchdb/HTTP_Document_API > startKey + limit, endKey + limit, sorting, insert/update order > > Mongooese: [cursor-id]+batch_size > > > OrientDB: .../[limit] > > Sones: no real rest API, but a SQL on top of the graph: > http://developers.sones.de/documentation/graph-query-language/select/ > with limit, offset, but also depth (for graph) > > HBase explcitly creates scanners, which can be then access with next > operations, and expire after no activity for a certain timeout > > > riak: > http://wiki.basho.com/REST-API.html > client-id header for client identification -> sticky? > optional query parameters for including properties, and if to stream the > data keys=[true,false,stream] > > If “keys=stream”, the response will be transferred using chunked-encoding, > where each chunk is a JSON object. The first chunk will contain the “props” > entry (if props was not set to false). Subsequent chunks will contain > individual JSON objects with the “keys” entry containing a sublist of the > total keyset (some sublists may be empty). > riak seems to support partial json, non closed elements: -d > '{"props":{"n_val":5' > > returns multiple responses in one go, Content-Type: multipart/mixed; > boundary=YinLMzyUR9feB17okMytgKsylvh > > --YinLMzyUR9feB17okMytgKsylvh > Content-Type: application/x-www-form-urlencoded > Link: </riak/test>; rel="up" > Etag: 16vic4eU9ny46o4KPiDz1f > Last-Modified: Wed, 10 Mar 2010 18:01:06 GMT > > {"bar":"baz"} > (this block can be repeated n times) > --YinLMzyUR9feB17okMytgKsylvh-- > * Connection #0 to host 127.0.0.1 left intact > * Closing connection #0 > > Query results: > Content-Type – always multipart/mixed, with a boundary specified > Understanding the response body > > The response body will always be multipart/mixed, with each chunk > representing a single phase of the link-walking query. Each phase will also > be encoded in multipart/mixed, with each chunk representing a single object > that was found. If no objects were found or “keep” was not set on the phase, > no chunks will be present in that phase. Objects inside phase results will > include Location headers that can be used to determine bucket and key. In > fact, you can treat each object-chunk similarly to a complete response from > read object, without the status code. > < HTTP/1.1 200 OK > < Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger) > < Expires: Wed, 10 Mar 2010 20:24:49 GMT > < Date: Wed, 10 Mar 2010 20:14:49 GMT > < Content-Type: multipart/mixed; boundary=JZi8W8pB0Z3nO3odw11GUB4LQCN > < Content-Length: 970 > < > > --JZi8W8pB0Z3nO3odw11GUB4LQCN > Content-Type: multipart/mixed; boundary=OjZ8Km9J5vbsmxtcn1p48J91cJP > > --OjZ8Km9J5vbsmxtcn1p48J91cJP > Content-Type: application/json > Etag: 3pvmY35coyWPxh8mh4uBQC > Last-Modified: Wed, 10 Mar 2010 20:14:13 GMT > > {"riak":"CAP"} > --OjZ8Km9J5vbsmxtcn1p48J91cJP-- > > --JZi8W8pB0Z3nO3odw11GUB4LQCN > Content-Type: multipart/mixed; boundary=RJKFlAs9PrdBNfd74HANycvbA8C > > --RJKFlAs9PrdBNfd74HANycvbA8C > Location: /riak/test/doc2 > Content-Type: application/json > Etag: 6dQBm9oYA1mxRSH0e96l5W > Last-Modified: Wed, 10 Mar 2010 18:11:41 GMT > > {"foo":"bar"} > --RJKFlAs9PrdBNfd74HANycvbA8C-- > > --JZi8W8pB0Z3nO3odw11GUB4LQCN-- > * Connection #0 to host 127.0.0.1 left intact > * Closing connection #0 > > Riak - MapReduce: > Optional query parameters: > > * chunked – when set to true, results will be returned one at a time in > multipart/mixed format using chunked-encoding. > Important headers: > > * Content-Type – application/json when chunked is not true, > otherwise multipart/mixed with application/json parts > > Other interesting endpoints: /ping, /stats > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
