jorgebay commented on a change in pull request #1539: URL: https://github.com/apache/tinkerpop/pull/1539#discussion_r787809380
########## File path: docs/src/reference/gremlin-variants.asciidoc ########## @@ -1721,6 +1721,32 @@ IMPORTANT: The preferred method for setting a per-request timeout for scripts is with bytecode may try `g.with(EVALUATION_TIMEOUT, 500)` within a script. Scripts with multiple traversals and multiple timeouts will be interpreted as a sum of all timeouts identified in the script for that request. + +==== Processing results as they are returned from the Gremlin server + + +The Gremlin JavaScript driver maintains a WebSocket connection to the Gremlin server and receives messages according to the `batchSize` parameter on the per request settings or the `resultIterationBatchSize` value configured for the Gremlin server. When submitting scripts the default behavior is to wait for the entire result set to be returned from a query before allowing any processing on the result set. + +The following examples assume that you have 100 vertices in your graph. + +[source,javascript] +---- +const result = await client.submit("g.V()"); +console.log(result.toArray()); // 100 - all the vertices in your graph +---- + +When working with larger result sets it may be beneficial for memory management to process each chunk of data as it is returned from the gremlin server. The Gremlin JavaScript driver can accept an optional callback to run on each chunk of data returned. + +[source,javascript] +---- + +await client.submit("g.V()", {}, { batchSize: 25 }, (data) => { Review comment: I think from the user POV, having a thing (`ResultSet`, `Stream`, ...) that represents the result of the execution is more familiar than having 1 per "batch". The batch is a low level thing. There are several db client libraries that use this approach (blocking iterators in other languages, async iterators in node.js, streams or callbacks). I [worked on the paging api for Cassandra](https://github.com/datastax/nodejs-driver/tree/master/doc/features/paging) that uses several of these approaches (see [here for example](https://github.com/datastax/nodejs-driver#row-streaming-and-pipes)), but these are popular patterns across node.js db client libraries (see [mysql](https://github.com/mysqljs/mysql#streaming-query-rows)) I think that we should return an object that fires events or a stream of result items, we should clearly signal when all the data has been received or when there was an error. If we use callbacks, I think we shouldn't mix it with promises to get the error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tinkerpop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org