Thompsonbry.systap added a comment. Concerning thread.
See [1] for a general background on query optimization for blazegraph. The QueryEngine class in blazegraph supports parallelism across queries (concurrent non-blocking queries), within queries (different operators in the same query will execute concurrently), and within operators (most operators can be instantiated multiple times if there is enough work in their input queue and those instances will run concurrently when this happens). At a high level, blazegraph is using threads to schedule IOs. A typical query will have a bunch of PipelineJoins. Those joins will each have their own thread and those threads will be scheduling IOs. (We have looked at async IO quite a bit and concluded that it did not appear to offer a clear win in performance while making the code patterns more complex.) BlazeGraph actually does very well with heavy concurrent client workloads. For example, the commercial OwlIM (now GraphDB) platform is close to blazegraph with 4 clients on a concurrent workload mixture but we completely dominate them with 64 clients each running the same mixture. A lot of the GC overhead is the intermediate solutions awaiting in queues in front of each operator in the query engine. One way to improve the single client performance is to transparently migrate those intermediate solution queues onto native memory, just as we do with the analytic query mode. This would also allow us to increase the vector size and that would improve the performance when there are only a few clients. Thanks, Bryan [1] http://wiki.bigdata.com/wiki/index.php/QueryOptimization TASK DETAIL https://phabricator.wikimedia.org/T90116 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev, Thompsonbry.systap Cc: Thompsonbry.systap, Beebs.systap, Haasepeter, Aklapper, Manybubbles, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, aude, GWicke, daniel, JanZerebecki _______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
