[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-647: -- Attachment: solr-647.patch Here's a slight update: - removes the SolrCore finalizer to prevent too many closes exception (we could do a refcount check too) - calls SolrCore.setName() when a core is added or renamed in the CoreContainer (it was just too wierd to do a core swap and not have the names change). This does not affect JMX. I was going to remove the SolrCore name altogether but things like JMX and logging use it. bq. Can we remove it and always access the name via the CoreDescriptor? Having a name on the core made sense before we added the descriptor... If we did want to change JMX or other things on a name change, it seems like the call needs to be on the SolrCore. Currently this does change the logging string too (probably good for avoiding confusion on a core swap) Unless there are objections, I think this is close enough to commit soon and then propose smaller patches off of trunk. > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul >Assignee: Yonik Seeley > Fix For: 1.3 > > Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-647: -- Attachment: solr-647.patch > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul >Assignee: Yonik Seeley > Fix For: 1.3 > > Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-647: -- Attachment: solr-647.patch Here's an updated patch: - cores is a Map - no prohibition of adding a SolrCore to the CoreContainer multiple times (to be able to access it from multiple URLs, for back compatible migration for example) I was going to remove the SolrCore name altogether but things like JMX and logging use it. That's a weakness in both the way multicore worked (core swap wouldn't swap JMX) and the current patch (core name is independent of how it's mapped via CoreContainer). One resolution would be to have a callback on SolrCore whenever it's name is changed, so JMX and logging strings could be appropriately adjusted. > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul >Assignee: Yonik Seeley > Fix For: 1.3 > > Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, > SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[
https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-647:
---
Attachment: solr-647.patch
bq. Yes, if cores obtained through other methods need to be open for some
reason, then open() would need to be called.
bq. I'm not sure if we have any cases like that.
No, we don't have any cases today. It could have been useful to someone
implementing another filter or servlet.
Since the same effect can be obtained with:
{code}
CoreDescriptor dcore;
CoreContainer container;
...
SolrCore opened = container.getCore(dcore.getName());
{code}
open() can just be incrementAndGet(). (finally!:-))
bq.LOG.severe("XYZ happened. Please report this problem on [EMAIL PROTECTED]")
Seems to be the consensus; updated.
bq.The CoreContainer#create(CoreDescriptor dcore) must close the old core after
creating the new one
Correct, fixed.
> Do SolrCore.close() in a refcounted way
> ---
>
> Key: SOLR-647
> URL: https://issues.apache.org/jira/browse/SOLR-647
> Project: Solr
> Issue Type: Bug
>Affects Versions: 1.3
>Reporter: Noble Paul
>Assignee: Grant Ingersoll
> Fix For: 1.3
>
> Attachments: refcount_example.patch, solr-647.patch, solr-647.patch,
> solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch,
> solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch
>
>
> The method _SolrCore.close()_ directly closes the core . It can cause
> Exceptions for in-flight requests. The _close()_ method should just do a
> decrement on refcount and the actual close must happen when the last request
> being processed by that core instance is completed
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch updated to reflect Yonik's last suggestion. fixed test attempting to close closed core. (throwing an exception was useful for quick detection). > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul >Assignee: Grant Ingersoll > Fix For: 1.3 > > Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch Based on Ryan's comment, simplified the patch to strictly focus on the matter at hand. > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul >Assignee: Grant Ingersoll > Fix For: 1.3 > > Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[
https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-647:
---
Attachment: solr-647.patch
My apologies to all for cluttering the issue.
New simplified version based on Yonik's example for trunk 685913 (post solr-695
commit), dont let the patch size fool you:
CoreContainer.getCore() & CoreContainer.getAdminCore() now return an incref-ed
("opened") core;
core.close() must be called when these 2 have been used.
SolrCore.Reference is gone.
SolrCore.open() & SolrCore.close() method are kept because we can retrieve
cores in 3 "close-unprotected" ways:
1 - a call to CoreDescriptor.getCore(), descriptors that can be retrieved
through CoreContainer.getCoreDescriptors())
2 - a list of close-unprotected cores through CoreContainer.getCores().
3 - SolrCore.getCore() - which is deprecated
The first 2 can be used in a user-defined filter/servlet after the
SolrDispatchFilter falls through the filter-chain, the CoreContainer being
set as an attribute of the request ("org.apache.solr.CoreContainer").
Because of this, we are not always synchronized by the CoreContainer#cores when
we incref/decref.
We can thus try to open() a core which is closed and cant use a simple
refCount.incrementAndGet().
Thus the refCount.get()/refCount.compareAndSet() pattern in both open & close.
The TestHarness is modified to always create a CoreContainer that contains the
"unnamed" core so testCoreMT uses
CoreContainer.getCore("").
Also touched some tests that were using SolrCore.getCore() when they can use
the TestHarness core.
> Do SolrCore.close() in a refcounted way
> ---
>
> Key: SOLR-647
> URL: https://issues.apache.org/jira/browse/SOLR-647
> Project: Solr
> Issue Type: Bug
>Affects Versions: 1.3
>Reporter: Noble Paul
>Assignee: Grant Ingersoll
> Fix For: 1.3
>
> Attachments: refcount_example.patch, solr-647.patch, solr-647.patch,
> solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch,
> SOLR-647.patch, SOLR-647.patch
>
>
> The method _SolrCore.close()_ directly closes the core . It can cause
> Exceptions for in-flight requests. The _close()_ method should just do a
> decrement on refcount and the actual close must happen when the last request
> being processed by that core instance is completed
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Comment: was deleted > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul >Assignee: Grant Ingersoll > Fix For: 1.3 > > Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-647: -- Attachment: refcount_example.patch It seems like some simplifications could be made... see attached refcount_example.patch. I think we need to be able to describe in simple terms how the mechanism works: Cores are created with a reference count of 1 and put in multicore. As long as a core can be obtained from the map, it will have at least a reference count of 1. Thus, if we increment the core's reference during a time when we know that no other core is modifying multicore, we know we have a core that is safe from being asynchronously closed. To destroy a core, simply remove it from the map and call close on it. There are other issues that are still a bit muddled IMO... like the whole role of CoreDescriptor. > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul >Assignee: Grant Ingersoll > Fix For: 1.3 > > Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch fixed 1.6 dependency (sorry & thanks Grant); updated to trunk; fixed a bug in SolrDispatchFilter (Multicore.getAdminCore can return a null core); introduced calls to acquire/release instead and modified EmbeddedSolrServer (for completeness). > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul > Fix For: 1.3 > > Attachments: solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Comment: was deleted > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul > Fix For: 1.3 > > Attachments: solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch new version based on Yonik's comment in solr-545. > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul > Fix For: 1.3 > > Attachments: solr-647.patch, solr-647.patch, solr-647.patch, > solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch Hopefully last version of it. I goofed the MT test on the previous one. Re-added CoreDescriptor.reloadCore(). Note that the same code is included in solr-545. > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul > Fix For: 1.3 > > Attachments: solr-647.patch, solr-647.patch, solr-647.patch, > SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch A few modifications to make things (hopefully) a little clearer & tests (single & multi threaded). This new patch version solely deals with reference counting implementation (not its usage). SolrDispatchFilter/EmbeddedSolrServer should be patched through solr-545. CoreDescriptor (the reloadCore() method) might be better patched through solr-561 (this is easy to reintroduce if needed though). > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul > Fix For: 1.3 > > Attachments: solr-647.patch, solr-647.patch, SOLR-647.patch, > SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch Being hit by the same core issue swapping/closing cores, here is another take at it. > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul > Fix For: 1.3 > > Attachments: solr-647.patch, SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated SOLR-647: -- Priority: Major (was: Minor) Fix Version/s: 1.3 Issue Type: Bug (was: Improvement) > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Bug >Affects Versions: 1.3 >Reporter: Noble Paul > Fix For: 1.3 > > Attachments: SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-647: Attachment: SOLR-647.patch I hope this fixes the race condition > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Improvement >Affects Versions: 1.3 >Reporter: Noble Paul >Priority: Minor > Attachments: SOLR-647.patch, SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-647: Attachment: SOLR-647.patch > Do SolrCore.close() in a refcounted way > --- > > Key: SOLR-647 > URL: https://issues.apache.org/jira/browse/SOLR-647 > Project: Solr > Issue Type: Improvement >Affects Versions: 1.3 >Reporter: Noble Paul >Priority: Minor > Attachments: SOLR-647.patch > > > The method _SolrCore.close()_ directly closes the core . It can cause > Exceptions for in-flight requests. The _close()_ method should just do a > decrement on refcount and the actual close must happen when the last request > being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
