[ https://issues.apache.org/jira/browse/RYA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113665#comment-16113665 ]
ASF GitHub Bot commented on RYA-313: ------------------------------------ GitHub user isper3at opened a pull request: https://github.com/apache/incubator-rya/pull/196 RYA-313 Aggregation now is performed over batches of 1000. ## Description >What Changed? Changed the iterator to use the non-deprecated Mongo api, enabled the aggregation framework to work properly. This could be a very good reason to address: [Jira mongo api ticket](https://issues.apache.org/jira/browse/RYA-302) ### Tests >Coverage? No tests, just ran locally and assured that no exception for document size was thrown. ### Links [Jira](https://issues.apache.org/jira/browse/RYA-313) ### Checklist - [ ] Code Review - [ ] Squash Commits #### People To Reivew @amihalik You can merge this pull request into a Git repository by running: $ git pull https://github.com/isper3at/incubator-rya RYA-313 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/196.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #196 ---- commit 3b279972198d34a2bcda775bc6ec4f9a87ade8ba Author: isper3at <smith...@gmail.com> Date: 2017-08-03T23:03:11Z RYA-313 Aggregation now is performed over batches of 1000. ---- > Rya Mongo Blows up on Large result sets > --------------------------------------- > > Key: RYA-313 > URL: https://issues.apache.org/jira/browse/RYA-313 > Project: Rya > Issue Type: Bug > Components: dao > Affects Versions: 3.2.10 > Environment: Mongo DB with Rya 3.2.11-SNAPSHOT with a lot of data in > Rya > Reporter: Aaron Mihalik > Assignee: Andrew Smith > > Simple queries that return a lot of results fail because mongo is trying to > send all of the results back at once. For instance, if I have a lot of data > and run something like: > {noformat} > SELECT * WHERE > { > ?s a ?t. > } > {noformat} > I will get this exception. > {noformat} > Caused by: com.mongodb.MongoCommandException: Command failed with error > 16389: 'aggregation result exceeds maximum document size (16MB)' on server > localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "aggregation > result exceeds maximum document size (16MB)", "code" : 16389 } > {noformat} > I think we need to toss in a "AggregationOptions with Batch = 1000", but I > couldn't get that to work immediately. Somebody with more mongo experience > needs to look at this. > [Here is the line of > code|https://github.com/apache/incubator-rya/blob/master/dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java#L114] -- This message was sent by Atlassian JIRA (v6.4.14#64029)