Traditionally this hasn't been a big deal - DataContext is thread-safe (by synchronizing on internal ObjectStore on selects and commits), but otherwise is not expected to serve many threads in parallel. On the other hand my experience with scaling content delivery engines shows that the best performing configuration is a shared DataContext + queries using local cache. So multi-threaded access to a DataContext is an emerging important scenario we need to deal with.
It actually works reasonably well, and depending on your cache settings, you may not notice any problems. However if a significant number of queries still hit the DB (even 5% can be "significant" in an active app), concurrency of selects increases to a point when threads start actively competing for ObjectStore locks. Since the locking affects both read and write API, this competition spreads to simple object by id lookup operations, and may quickly turn into a bottleneck for the entire app. (on a side note, recently I wanted to compare this with shared cache / multi context setup, that is presumably lock-free, but immediately hit this silly bug [1], that we are going to fix soon I hope). Anyways, I am trying to devise a lock-free algorithm for concurrent selects via a single DataContext. To simplify this task we can assume (a) a read-only scenario (i.e. users never modify objects, but the framework does as select results are merged into objects), and (b) thread-safe DataObjects (see my other email today). So sounds simple, right? I wish it was… I am afraid if we just remove ObjectStore synchronization and allow multiple threads to change state of the same objects, chaos will ensue pretty quickly. I can see two more reasonable algorithms. 1. Parallel updates with object-level atomicity - each thread would swap the entire Object[] of each DataObject it updates (see my other email from today on the new DO structure). Of course we have prefetches and to-many relationships (which I haven't thought through) to deal with. Not sure how much chaos this approach will cause. 2. Queue based approach… Place each query result merge operation in an operation queue for a given DataContext. Polling end of the queue will categorize the operations by "affinity", and assign each op to a worker thread, selected from a thread pool based on the above "affinity". Ops that may potentially update the same objects are assigned to the same worker and are processed serially. Ops that have no chance of creating conflict between each other are assigned to separate workers and are processed in parallel. I really like #2 - it is very "modern" in that it can really scale on multi-processor hardware. And it doesn't require a rewrite of prefetch processing code (#1 does and this is a big deal). There's the overhead of the queue append/take of course. So it remains to be seen how well it performs. Its success depends on whether we can devise an efficient affinity algorithm. A simple version can inspect the entities participating in each query, taking prefetches into account and send them to the different workers if there's no overlap.. E.g. (Painting) and (Artist) queries can be processed in parallel; (Painting + Artist prefetch) and (Artist) queries can not. If we can also make it adaptive based on the actual volume and nature of the queries coming through the system, I think we are on to something here. Even a single large query result conversion into objects can be optimized by splitting list processing between multiple workers. So there's some potential to explore. Anyways, just thinking out loud. Sorry if this sounds confusing :) Andrus [1] https://issues.apache.org/jira/browse/CAY-1868
