[
https://issues.apache.org/jira/browse/SOLR-1028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13552268#comment-13552268
]
Erick Erickson commented on SOLR-1028:
--------------------------------------
You're right, thanks! I'm a bit reluctant to extend the synchronized block
around the entire following try block just because loading the core can be
quite a lengthy process.
I _have_ to go away until early evening, but I think I can do something about
this tonight. Do you think the risk/reward ratio is OK for 4.1? This will only
manifest itself with transient cores, which is really fully supported in 4.2
not 4.1 (yes, that _is_ a cop-out). But deadlocks are _so_ easy to introduce
when in a rush....
I've raised another JIRA, SOLR-4300 and called it a "blocker" for now, feel
free to change that depending...
This seems really trappy, I'm wondering if for 4.2 I should encapsulate this
whole "lazy" and "transient" and "permanent" distinction into a class that
handles all the messy details, it's too easy to miss one of these places in the
current code. There are several other places in CoreContainer that synchronize
on "cores" and it's not obvious whether all of them should worry about
transient cores or not. E.g. renaming cores.
But what I'm thinking of is a "pending" set of cores, and the code would look
something like this: (just above the "try"). Note this is just a sketch, I
haven't checked it out much at all just wondering if it makes sense.
// there'd be a new static Map of cores declared, call it "pending"
boolean isPending = false;
synchronized(pending) {
if (pending.get(name) == null) {
pending.put(name);
isPending = true;
}
}
while (isPending) {
synchronized (pending) {
if (pending.get(name) == null) { // other thread has completed or we
wouldn't be here
look in the normal and lazy core maps and return the core,
synchronize your lookups. Bump the refcount (open)
can refactor out looking into both lists as above in this method into
a common method, it's used in several places.
throw an error if it's not in one of the cores? return null? Fall
through and try again? Throwing an error seems best
}
}
sleep(100);
Not sure how to bail out here though, this could spin forever. 1 minute?
5 minutes? Never?
}
try {
core = create(desc); // This should throw an error if it fails.
core.open();
if (desc.isTransient()) {
registerLazyCore(name, core, false); // This is a transient core
} else {
register(name, core, false); // This is a "permanent", although
deferred-load core
}
} catch (Exception ex) {
throw recordAndThrow(name, "Unable to create core" + name, ex);
} finally {
synchronized(pending) {
pending.remove(name);
}
}
return core;
> Automatic core loading unloading for multicore
> ----------------------------------------------
>
> Key: SOLR-1028
> URL: https://issues.apache.org/jira/browse/SOLR-1028
> Project: Solr
> Issue Type: New Feature
> Components: multicore
> Affects Versions: 4.0, 5.0
> Reporter: Noble Paul
> Assignee: Erick Erickson
> Fix For: 4.1, 5.0
>
> Attachments: jenkins.jpg, SOLR-1028.patch, SOLR-1028.patch,
> SOLR-1028_testnoise.patch
>
>
> usecase: I have many small cores (say one per user) on a single Solr box .
> All the cores are not be always needed . But when I need it I should be able
> to directly issue a search request and the core must be STARTED automatically
> and the request must be served.
> This also requires that I must have an upper limit on the no:of cores that
> should be loaded at any given point in time. If the limit is crossed the
> CoreContainer must unload a core (preferably the least recently used core)
> There must be a choice of specifying some cores as fixed. These cores must
> never be unloaded
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
