> I was actually thinking of some kind of custom Lucene/Solr component that > would for example take a query parameter such as &lookitUp=123 and the > component might do a JDBC query against a database or kv store and return > results in some form that would be efficient for Solr/Lucene to process.
If you do this, I'd definitely be interested in using it too. I have the same sort of use cases as you. It would be interesting to figure out if such a component could be used for "join-like behavior" too, as you included in your original use cases too. I'm not entirely sure what that would look like, but I have some problems which trying to solve with Solr often runs up against the lack of ability to do that (and there are lots of questions on the listserv asking "how do I do a join in solr" -- often the questioners can and should really solve their problem without needing to do this, but sometimes there really isn't a good solution without it), and when I try to spec out some workarounds in my head, it often comes up against the need to do what you're describing, efficiently do a query in solr limited by a known list of Solr ideas -- or in my case, sometimes a known (but lengthy, and "OR"d list of facet values for an fq limit -- really the same pattern there, it doesn't matter if the field is the id field or something else) I'm not really sure what the API for a component like this meant to support 'join' kind of behavior would look like, but it would be interesting to think about. Maybe it needs to be able to generate the values against an alternate solr core, with a specified query against that core, in addition to being able to generate with a specified query from a JDBC or kv-store lookup? Or in some cases, even the same solr core -- do a query against the same solr core, take one stored field from the results, and use it to filter the result set of a subsequent query.
