[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125295#comment-17125295 ] Erick Erickson commented on SOLR-12823: --- [~murblanc] I'll try to look at this Real Soon Now unless someone beats me to it. > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17119446#comment-17119446 ] Ilan Ginzburg commented on SOLR-12823: -- I rebased the PR due to a minor doc conflict. Is there anything specific that needs to happen before this gets merged or feedback given that the PR doesn't cut it? Anything I can do to speed things up? > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > Time Spent: 3h 40m > Remaining Estimate: 0h > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17111605#comment-17111605 ] Ilan Ginzburg commented on SOLR-12823: -- I changed the status of [https://github.com/apache/lucene-solr/pull/1528] from Draft to Ready for review (and hopefully merge). Note the comment added in the PR, copying here: _Rebased on master. The recent fix to SOLR-14486 solved the failures in the autoscaling sim tests._ _I'm submitting this officially for review now (moving it out of draft PR)._ _Note there are two tests that are failing (that I marked [@ignore|https://github.com/ignore]) and that IMO should either be removed or deserve their own separate Jira for fixing (that I will create once this is merged). I suspect fixes are non trivial:_ * _TestZkChroot: I didn't find ways to reasonably start a test server with a different chroot than /solr to test if it's writing data to the right place in ZooKeeper. Played a while with MiniSolrCloudCluster but chroot seems to be quite hard coded deep in the test framework (passing an already chrooted ZK server defeats the purpose)._ * _BasicZkTest (I didn't dig further)._ > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110952#comment-17110952 ] Jan Høydahl commented on SOLR-12823: SOLR-14303 is for deprecating /admin/zookeeper > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110696#comment-17110696
]
Ilan Ginzburg commented on SOLR-12823:
--
[~noble.paul] is there a Jira for porting the Admin UI to use
{{ZookeeperReadAPI}}?
> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110695#comment-17110695
]
Ilan Ginzburg commented on SOLR-12823:
--
I’m sharing a work in progress (draft) Pull Request, but almost done:
[https://github.com/apache/lucene-solr/pull/1528]
I believe “production” code is complete, but:
* {{org.apache.solr.cloud.autoscaling.sim}} assumes stateFormat 1 and the
presence of {{/clusterstate.json}}. I didn’t look into what changes are
required there. Some code got commented out in order to compile, tests are
failing.
* {{TestZkChroot}} breaks. That test needs to be written differently, possibly
by starting a local SolrCloud with a non hard coded chroot (rather than
{{/solr}}).
* {{BasicZkTest}} fails as well. I’d like to remove it unless somebody thinks
it has value.
This PR removes all code dealing with {{/clusterstate.json}} in Zookeeper. It
also removes {{MIGRATESTATEVERSION}} Collection API action and the notion of
“legacyCloud” (legacyCloud mode no longer supported).
Fun fact: all OverseerTest collections were created in stateFormat 1.
I’d like to start getting feedback on these changes. Will look in the meantime
at changes required in the autoscaling sim code (SOLR-14486), unless you want
to do that [~ab] ?
Any comments or guidance on how to continue from here are most welcome. This is
one of my first PR’s on lucene-solr (actually the second given SOLR-14473 was
merged three hours ago, thanks [~mdrob]!).
> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110694#comment-17110694
]
Noble Paul commented on SOLR-12823:
---
[~murblanc] It's reverted but put back as \{{ZookeeperReadAPI}}. I would
recommend using that API instead of the unsupported {{/admin/zookeeper}}
> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110592#comment-17110592 ] Ilan Ginzburg commented on SOLR-12823: -- [~dsmiley] SOLR-13942 might have been reverted since, I don't see ZkRead.java on master. Also, it did not provide the same service as /admin/zookeeper that's used by the Admin UI. It implemented a recursive dump of the ZK tree below a path. /admin/zookeeper returns what's needed to display the collection list + config and supports filtering and paging. > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110542#comment-17110542 ] Ilan Ginzburg commented on SOLR-12823: -- Anybody feels strongly *against* deleting BasicZkTest? This test seems to be a mix of unrelated concerns, does ZK but not really SolrCloud and I believe most/all of what it's trying to test (which isn't clear to me TBH) is tested elsewhere. > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110537#comment-17110537
]
David Smiley commented on SOLR-12823:
-
{quote}After implementing a cleaner Json return that looked good in response to
a GET request but broke the UI
{quote}
Consider /admin/zookeeper to be newly deprecate in lieu of a superior
replacement [~noble.paul] recently pushed – SOLR-13942
> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17109682#comment-17109682
]
Ilan Ginzburg commented on SOLR-12823:
--
When {{/clusterstate.json}} is not empty Solr should not start.
In that an ERROR log is issued, but the system doesn't nicely stop the startup
process. From the {{bin/solr start}} command line things even look ok.
The check got added as soon as there's a connection to ZK, in the
{{ZkController}} constructor after {{zkClient}} is built. It throws a
{{SolrException}} (or {{ZooKeeperException}}, same behavior) if it's unhappy.
Is there a nicer (yet more drastic) way to give up at this stage rather than
leaving the system in a broken state? (UI Admin returns "HTTP ERROR 404
javax.servlet.UnavailableException: Error processing the request. CoreContainer
is either not initialized or shutting down.")
I know this will not happen very often and if it does by looking at the logs
the user will quickly figure it out, but if things can be made more friendly...
> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17109678#comment-17109678
]
Erick Erickson commented on SOLR-12823:
---
Sounds like you're on top of it. Yeah, updating all the places
clusterstate.json is referenced is certainly a big enough bite for one JIRA
Many thanks for tackling this.
Responding to your note from two days ago, I don't think an explicit command to
remove clusterstate.json is necessary. If it hangs around forever, so be it. If
anyone really wants to remove it,
{code}
bin/solr zk rm /clusterstate.json
{code}
should do it. And it seems safe to remove it in Solr 9 on startup if it's
empty, in which case there it'd be very convoluted for it to be around and
using bin/solr to remove it doesn't seem onerous.
> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17109625#comment-17109625 ] Ilan Ginzburg commented on SOLR-12823: -- Thanks [~erickerickson]. After implementing a cleaner Json return that looked good in response to a GET request but broke the UI, I reimplemented something that made the UI happy... I can clean it up later if the UI wants something more reasonable. It's still a bit cleaner than the original data returned by the endpoint (that wasn't used by the web page anyway). There are many moving parts (a.k.a. technical debt) in getting rid of /clusterstate.json that I'm trying to limit the changes at this stage, so would rather not change the UI at the same time. PR is already over 60 files and I'm not totally done yet. > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17109611#comment-17109611
]
Erick Erickson commented on SOLR-12823:
---
[~murblanc]
Responding to your dev-list question "...{color:#00}There's a choice to be
made regarding /admin/zookeeper endpoint{color}
{color:#00}(ZookeeperInfoHandler).{color}...":
{color:#00}I’m not sure that’s true. The admin UI is being reworked even
now,{color}
there’s been a long discussion in various places about it. So before
jumping through hoops to mimic old behavior it’s probably worthwhile
seeing if it’s worth it. [~marcussorealheis] would you care to comment? For
all I know you're jumping through hoops to _use_ that format
> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108403#comment-17108403 ] Ilan Ginzburg commented on SOLR-12823: -- How about the following behavior in Solr 9: * If /clusterstate.json exists in ZK, fail startup and log a message including some/all content from clusterstate.json + instructions about the new command line option, * Add a command line option to Solr 9 (with a long very explicit name) that when set would lead to the removal of clusterestate.json from Zookeepeer. Solr 9 would only have to be started once with this command. > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17107591#comment-17107591 ] Andrzej Bialecki commented on SOLR-12823: - [~murblanc] I created SOLR-14486 to fix the /clusterstate.json issue in the simulation API. > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106262#comment-17106262 ] Erick Erickson commented on SOLR-12823: --- [~murblanc] Conceptually, legacyCloud should be removed too. I haven't looked to see how difficult that would be. Theoretically it shouldn't be hard. re: Perhaps add a check in Overseer and if the old znode exists and is non-empty, print an ERROR and abort? First, I like the "fail fast" nature of this, we either have to do something like this or deal with questions about "where did replicas go?" if we just ignored clusterstate.json. There are certainly situations I've seen where users have some of their data in state.json for the collection and some in clusterstate.json. Admittedly they had to work to get there ;). If a user was in this situation however, what could they do to get going with Solr 9? If they nuked clusterstate.json, the information would be lost. I suppose they could hand-edit the individual state.json files, but that's difficult/error prone. I'm not necessarily saying we have to do anything special here to handle this case, just making sure we've considered the possibility. Personally I'd be fine with an upgrade note saying "before upgrading to Solr 9, insure you have no data in clusterstate.json. If you do, use the MIGRATESTATEFORMAT command on your pre Solr 9 installation before upgrading". > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106212#comment-17106212
]
Noble Paul commented on SOLR-12823:
---
The {{stateFormat}} attribute was introduced to split the cluster state . The
assumption was that we will use that as a generic version number to make
changes to state. It's almost 6 yrs and there was no need for that. probably,
we should just get rid of it altogether
> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[
https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106151#comment-17106151
]
Jan Høydahl commented on SOLR-12823:
{quote}quotes because I'm removing the stateFormat concept altogether
{quote}
There are other changes planned for state.json as well, such as SOLR-14341 and
SOLR-13951, and I guess we need some versioning in the future too? But if that
is stateFormat or simply a version attribute in state.json iteself I don't
know. [~noble.paul], [~dsmiley]?
> remove clusterstate.json in Lucene/Solr 8.0
> ---
>
> Key: SOLR-12823
> URL: https://issues.apache.org/jira/browse/SOLR-12823
> Project: Solr
> Issue Type: Task
>Reporter: Varun Thacker
>Priority: Major
>
> clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove
> that in 8.0
> It stays empty unless you explicitly ask to create the collection with the
> old "stateFormat" and there is no reason for one to create a collection with
> the old stateFormat.
> We should also remove the "stateFormat" argument in create collection
> We should also remove MIGRATESTATEVERSION as well
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105826#comment-17105826 ] Ilan Ginzburg commented on SOLR-12823: -- [~janhoy], [~ab], the autoscaling simulation framework (org.apache.solr.cloud.autoscaling.sim) blocks clean removal of stateFormat=1 collections and associated management of /clusterstate.json in the Overseer... It currently only supports the old state format. Having it create "stateFormat=2" collections (quotes because I'm removing the stateFormat concept altogether) having each its own state.json shouldn't be a big deal, but I'm not familiar with how it's used to understand the consequences of the absence of a single state ZK file. Also, the concept of a global config "version" (mapping to a znode version) no longer exists with per collection state.json and unfortunately the sim code seems to rely on it quite a bit. I need some guidance on the best way to deal with this here, thanks. > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105764#comment-17105764 ] Ilan Ginzburg commented on SOLR-12823: -- Need to remove the "legacyCloud" option when removing clusterstate.json and have the cluster behave as if it were false. [~erickerickson] do you agree? > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105565#comment-17105565 ] Ilan Ginzburg commented on SOLR-12823: -- Will give it a try, starting with the 9.0 (master) changes. If successful, will then implement the 1st option for 8_x (2nd one feels risky and too intrusive). > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105542#comment-17105542 ] Jan Høydahl commented on SOLR-12823: [~murblanc] volunteered on mailing list to try a PR. I have not looked into this in detail. You should consult [https://lucene.apache.org/solr/guide/8_5/cluster-node-management.html#migratestateformat] and the code of course to understand this move. But this could be an approach: * In 8.6 do at least this ## print a big WARNING if clusterstate.json is used for a collection, telling user to migrate ## OR auto migrate all collections on overseer startup? Is it safe? * In 9.0 (master), do at least this: ## Add migration note to RefGuide, telling users to run MIGRATESTATEFORMAT for all collections before upgrading ## remove parsing of and support for clusterstate.json in Overseer ## Perhaps add a check in Overseer and if the old znode exists and is non-empty, print an ERROR and abort? ## If clusterstatus.json exists and is empty, remove it and print a log > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-12823) remove clusterstate.json in Lucene/Solr 8.0
[ https://issues.apache.org/jira/browse/SOLR-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105402#comment-17105402 ] Jan Høydahl commented on SOLR-12823: Any plan to remove clusterstate.json for Solr 9? And perhaps let Overseer fail if it is non-empty, i.e. if not all collections are migrated to format 2? > remove clusterstate.json in Lucene/Solr 8.0 > --- > > Key: SOLR-12823 > URL: https://issues.apache.org/jira/browse/SOLR-12823 > Project: Solr > Issue Type: Task >Reporter: Varun Thacker >Priority: Major > > clusterstate.json is an artifact of a pre 5.0 Solr release. We should remove > that in 8.0 > It stays empty unless you explicitly ask to create the collection with the > old "stateFormat" and there is no reason for one to create a collection with > the old stateFormat. > We should also remove the "stateFormat" argument in create collection > We should also remove MIGRATESTATEVERSION as well > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
