[
https://issues.apache.org/jira/browse/SOLR-11653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated SOLR-11653:
--------------------------------
Attachment: SOLR-11653.patch
Updated patch, significantly moving this issue to being done. I tested it
which uncovered various bugs which I fixed. I tweaked some but not all
metadata param names to align with SOLR-11722; we can fully align them here or
there. Unfortunately there's a bit of redundant stuff in that issue and this
one (and internally in this one between the URP & Cmd) that we can resolve in a
separate issue to tidy up duplication. Some reflection on changes in this
patch:
Overseer & OverseerTaskProcessor: added checks for "operation" being null to
pass a clear error. Of course this should never happen in bug free code, but
my code wasn't bug free at first :-). In OTP I left a nocommit because I think
the order of the nearby code should be reordered a little (a very minor
refactoring). The patch won't easily show this as there isn't enough contextual
lines of code. I think operation should be gotten & verified (not null), then
check for the asyncId logic, and then acquire the messageHandler (thus
immediately before the MH is used).
CollectionsHandler.waitForActiveCollection: I made this method public because
it's important to check for this level of collection readiness before we
actually use it. I had hoped waitForFinalState would be sufficient but tests
proved it wasn't. Having to call this feels bad; I have to wonder if
Collection creation should internally wait for being "active"; what's the harm
in that?
IMO CollectionsHandler.handleResponse is a bad name... it's name suggest it
only handles the response when it performs the intended action too! I have
TODOs to rename to submitToOverseerRPC since it submits to the Overseer queue
and it does so in an RPC way (expects a response; it's not just a message on a
queue). Input on the name is welcome.
Added TZ timezone handling for working with the interval.
What I want to do now is improve testing for race conditions by having the test
submit documents in parallel. I may also get to a minor TODO involving the
addition of a Lock to reduce wasted concurrent calls by the URP to the Overseer
to create a new collection. This is minor because the
RoutedAliasCreateCollectionCmd doesn't do anything if ifHeadCollName doesn't
jive with the actual head collection name.
[~gus_heck] can you please review this now?
> create next time collection based on a fixed time gap
> -----------------------------------------------------
>
> Key: SOLR-11653
> URL: https://issues.apache.org/jira/browse/SOLR-11653
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Reporter: David Smiley
> Assignee: David Smiley
> Attachments: SOLR-11653.patch, SOLR-11653.patch
>
>
> For time series collections (as part of a collection Alias with certain
> metadata), we want to automatically add new collections. In this issue, this
> is about creating the next collection based on a configurable fixed time gap.
> And we will also add this collection synchronously once a document flowing
> through the URP chain exceeds the gap, as opposed to asynchronously in
> advance. There will be some Alias metadata to define in this issue. The
> preponderance of the implementation will be in TimePartitionedUpdateProcessor
> or perhaps a helper to this URP.
> note: other issues will implement pre-emptive creation and capping
> collections by size.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
