Once you've scaled up the export from collection4 you can test the
performance of the join by moving the NullStream around the join.
parallel(null(innerJoin(collection 3, collection4)))
Again you'll want to test with different numbers of workers and replicas to
see where you max out performance
innerJoin(intersect(innerJoin(collection1, collection2),
innerJoin(collection 3, collection4)),
collection5)
Let's focus on:
innerJoin(collection 3, collection4))
The first thing to focus on is how fast is the export from collection4. You
can test
Hi,
Which version of solr are you on?
Increasing memory may not be useful as streaming API does not keep stuff in
memory (except may be hash joins).
Increasing replicas (not sharding) and pushing the join computation on
worker solr cluster with #workers > 1 would definitely make things faster.
We are working on a proposal and feeling streaming API along with export
handler will best fit for our usecases. We are already of having a structure in
solr in which we are using graph queries to produce hierarchical structure. Now
from the structure we need to join couple of more collections.
