[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401398#comment-16401398 ] Shalin Shekhar Mangar commented on SOLR-11960: -- bq. Shalin Shekhar Mangar, I believe your comments were addressed Looks good, thanks! > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3, master (8.0) > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400998#comment-16400998 ] ASF subversion and git services commented on SOLR-11960: Commit 3b8eb6cd3e53727812f82711b9aa96d6f511e184 in lucene-solr's branch refs/heads/branch_7_3 from [~tomasflobbe] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3b8eb6c ] SOLR-11960: Don't add property listeners on core registration > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3 > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400993#comment-16400993 ] ASF subversion and git services commented on SOLR-11960: Commit 328587b993f76e5bc8cc72d1fc6262f883a8ab66 in lucene-solr's branch refs/heads/branch_7x from [~tomasflobbe] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=328587b ] SOLR-11960: Don't add property listeners on core registration > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3 > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400989#comment-16400989 ] ASF subversion and git services commented on SOLR-11960: Commit 67dab22f295c8a9966c3c35c722f2f28626d7ec8 in lucene-solr's branch refs/heads/master from [~tomasflobbe] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=67dab22 ] SOLR-11960: Don't add property listeners on core registration > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3 > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400660#comment-16400660 ] Tomás Fernández Löbbe commented on SOLR-11960: -- Thanks Peter. The patch looks good to me. [~shalinmangar], I believe your comments were addressed. I'll commit shortly > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3 > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400304#comment-16400304 ] Tomás Fernández Löbbe commented on SOLR-11960: -- We still need to address Shalin's comments before releasing this. I'll resolve later today > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3 > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400136#comment-16400136
]
Alan Woodward commented on SOLR-11960:
--
{quote}+1 We needn't consider this issue a blocker nor open anymore.
{quote}
Can this be closed and the Blocker tag removed?
> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Blocker
> Fix For: 7.3
>
> Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch,
> SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining
> properties that affect a single collection. Collection properties could be
> stored in a single zookeeper node per collection, making it possible to
> trigger zookeeper watchers for only those Solr nodes that have cores of that
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399125#comment-16399125 ] Peter Rusko commented on SOLR-11960: I've updated the patch. Removed the watcher creation/deletion when registering and unregistering the core and I'm also re-registering watchers in createClusterStateWatchersAndUpdate now. > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3 > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch, SOLR-11960_2.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397538#comment-16397538 ] Tomás Fernández Löbbe commented on SOLR-11960: -- V2 is already verbose which is fine since it's JSON? Yes, sorry, that's what I meant. In V2 it's just a map, not an odd parameter name format > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3 > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397514#comment-16397514 ] David Smiley commented on SOLR-11960: - bq. The multiple at a time is more "expert" (in V2 it doesn't matter) What do you mean by "in V2 it doesn't matter"? Maybe verbosity... as V2 is already verbose which is fine since it's JSON? bq. So, I'd suggest we keep the current format and add the bulk one for collection and cluster properties as a separate Jira +1 We needn't consider this issue a blocker nor open anymore. > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3 > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397478#comment-16397478
]
Tomás Fernández Löbbe commented on SOLR-11960:
--
[~dsmiley] I think both would be nice. For v1 APIs, the single property at a
time format is simpler and more verbose. The multiple at a time is more
"expert" (in V2 it doesn't matter). Also, for collection properties we should
add support for adding properties at collection creation time. So, I'd suggest
we keep the current format and add the bulk one for collection and cluster
properties as a separate Jira. Could be something like:
{code}
=VALUE_2=VALUE_2...
{code}
The same parameters could work for {{action=CLUSTERPROP}} and for
{{action=CREATE}}.
Similarly, we could have
{code}
=VALUE_2=VALUE_2...
{code}
WDYT?
> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Blocker
> Fix For: 7.3
>
> Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch,
> SOLR-11960.patch, SOLR-11960.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining
> properties that affect a single collection. Collection properties could be
> stored in a single zookeeper node per collection, making it possible to
> trigger zookeeper watchers for only those Solr nodes that have cores of that
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395225#comment-16395225 ] David Smiley commented on SOLR-11960: - Admittedly the cluster properties API is similarly handicapped... so if there is no last minute change here it's at least consistent with cluster properties :-/ I suppose both cluster properties & collection properties & alias properties could be improved to support both single value name=value invocation, and a bulk style (using a prefix with wildcard for v1 or parent object for v2)... although it's a bit redundant to have both instead of the more capable bulk style. Perhaps history/legacy must tie our hands here and we eventually arrive at both. > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Blocker > Fix For: 7.3 > > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394237#comment-16394237 ] David Smiley commented on SOLR-11960: - Copying an excerpt of my comment from SOLR-11617: bq. The paint is still wet on SOLR-11960, so to speak. Before it's API gets set in stone (by a 7.3 release), perhaps now is the last moment to give the API more of a bulk parameter feel (like here for aliases) instead of limited to one at a time? Even if the code can only handle one pair right now, at least the API would be what we want it to be. You can see how this is done with alias metadata (soon to be renamed properties). Lets make such a change ASAP; ehh? > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Major > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388178#comment-16388178 ] Shalin Shekhar Mangar commented on SOLR-11960: -- bq. I think you are right, there is no need to register the watch on registerCore. Users of this feature can add a watch themselves by calling registerCollectionPropsWatcher. That's what you mean, right? Yes, but that is not sufficient. We need to take care of ZK reconnects as well. So we should re-create the collection prop watchers in createClusterStateWatchersAndUpdate method. > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Major > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388148#comment-16388148
]
Tomás Fernández Löbbe commented on SOLR-11960:
--
bq. The test failure described at SOLR-12061 started happening at the
c1a44251fe commit on this issue.
Thanks Steve, I'll take a look
bq. It looks like there is no consumer to this API (yet) but watchers are being
created anyway? We should avoid that until we have an actual user for this API.
I think you are right, there is no need to register the watch on
{{registerCore}}. Users of this feature can add a watch themselves by calling
{{registerCollectionPropsWatcher}}. That's what you mean, right?
> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Major
> Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch,
> SOLR-11960.patch, SOLR-11960.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining
> properties that affect a single collection. Collection properties could be
> stored in a single zookeeper node per collection, making it possible to
> trigger zookeeper watchers for only those Solr nodes that have cores of that
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388024#comment-16388024 ] Shalin Shekhar Mangar commented on SOLR-11960: -- It looks like there is no consumer to this API (yet) but watchers are being created anyway? We should avoid that until we have an actual user for this API. > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Major > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387861#comment-16387861
]
Steve Rowe commented on SOLR-11960:
---
The test failure described at SOLR-12061 started happening at the
{{c1a44251fe}} commit on this issue.
> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Major
> Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch,
> SOLR-11960.patch, SOLR-11960.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining
> properties that affect a single collection. Collection properties could be
> stored in a single zookeeper node per collection, making it possible to
> trigger zookeeper watchers for only those Solr nodes that have cores of that
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387129#comment-16387129 ] Tomás Fernández Löbbe commented on SOLR-11960: -- Committed. master: c1a44251fefabb0ed743f1bdaf287ac89ac38758 branch_7x: cfafc47e9c9229fe94b0d367249db66ec6b54132 Thanks Peter! > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Major > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch, > SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384341#comment-16384341
]
Tomás Fernández Löbbe commented on SOLR-11960:
--
Thanks [~prusko]. I moved the property in the V2 API to be per-collection, and
also mapped the parameters propertyName->name and propertyValue->value, so the
V2 API would look like:
{code}
curl -X POST "http://localhost:8983/v2/c/gettingstarted; -d
'{set-collection-property:{name:"foo", value:bar}}'
{code}
> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Major
> Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch,
> SOLR-11960.patch, SOLR-11960.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining
> properties that affect a single collection. Collection properties could be
> stored in a single zookeeper node per collection, making it possible to
> trigger zookeeper watchers for only those Solr nodes that have cores of that
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381323#comment-16381323
]
Peter Rusko commented on SOLR-11960:
{quote}BTW for this issue I personally would have chosen to store collection
properties on the state.json for the collection rather than put this somewhere
else. Consider all the other internal properties which are already in
state.json (e.g. replicationFactor etc.). Was this considered? Why not? Pros
are simplicity of backup and no need to delete with collection deletion, and
using the same watcher mechanism?
{quote}
Yes, I considered it. There are two reasons for choosing a separate json. First
the frequency of the change is different. Collection properties would change
way less frequently than state.json and not parsing out the properties blob on
every state change seemed the right way to go. But more importantly, state.json
changes should go via the overseer, which seemed to be a bit of an overkill
here.
> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Major
> Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch,
> SOLR-11960.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining
> properties that affect a single collection. Collection properties could be
> stored in a single zookeeper node per collection, making it possible to
> trigger zookeeper watchers for only those Solr nodes that have cores of that
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375698#comment-16375698
]
Gus Heck commented on SOLR-11960:
-
{quote} Since aliases operate in the same namespace as collections, I don't
think of it as separate.
{quote}
Interesting point, and kinda points out the fact that "alias" has been co-opted
into 2 roles, alternate names for collections and collection grouping. This
creates a level of syntactic sugar (query a group just like a single
collection) but grouping and re-naming really are separate concepts. To handle
alternate names for groups of collections you would need recursive alias
resolution.
> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Major
> Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining
> properties that affect a single collection. Collection properties could be
> stored in a single zookeeper node per collection, making it possible to
> trigger zookeeper watchers for only those Solr nodes that have cores of that
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373916#comment-16373916 ] David Smiley commented on SOLR-11960: - BTW for this issue I personally would have chosen to store collection properties on the state.json for the collection rather than put this somewhere else. Consider all the other internal properties which are already in state.json (e.g. replicationFactor etc.). Was this considered? Why not? Pros are simplicity of backup and no need to delete with collection deletion, and using the same watcher mechanism? bq. I think it's probably less confusing the way we did it to have alias level metadata/props, since the time routing is at the alias level, and collections come and go over time Ehh; debatable. I think it's worse for maintenance, docs, and our users, to unnecessarily increase the API surface area (plus rather different plumbing too) by having both. Since aliases operate in the same namespace as collections, I don't think of it as separate. > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Major > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371487#comment-16371487 ] Gus Heck commented on SOLR-11960: - Interesting notion. As far as what we did with time routed aliases, I think it's probably less confusing the way we did it to have alias level metadata/props, since the time routing is at the alias level, and collections come and go over time. The ability to keep metadata at multiple levels of abstraction is good, what might be interesting is a way to combine and keep metadata all in one api. One could identify the targets (subjects) via URIs that correspond to the API such as http://server.example.net:8983/solr/meta/aliases/aliasname http://server.example.net:8983/solr/meta/nodes/nodename http://server.example.net:8983/solr/meta/collections/collectionName/shards/shardname etc. basically make the entire system addressable. Also maybe coordinate that with the v2 api? Just brainstorming, I haven't figured out how that would relate to zookeeper. A unified and consistent way of handling metadata sounds potentially helpful now that we are gaining a 3rd metadata type. Might be possible to give metadata addressable java objects a metaURI() method that automatically provides the key for that object in a global metadata store... > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Major > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371432#comment-16371432 ] David Smiley commented on SOLR-11960: - +1 for collection properties. I recently spent time with [~gus_heck] on alias metadata (== "properties"). I'm having the sinking feeling now that we should have done collection properties more generally given that an "alias" is a collection alias, and so arguably should use the same mechanism. With this feature here, SOLR-11960, I'm wondering if we even need alias properties (nor an API for it) at all? :-/ > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Major > Attachments: SOLR-11960.patch, SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370797#comment-16370797
]
Tomás Fernández Löbbe commented on SOLR-11960:
--
Thanks [~prusko], patch looks great. I modified
{{CollectionPropsTest.testReadWrite}} to check immediately, since we are
getting the value directly from ZooKeeper the change should be immediate, there
is no need to wait. I also added a test,
{{CollectionPropsTest.testReadWriteCached}} that adds a watcher, so that we do
read the cached state. For that case we do need to wait until the value is
asynchronously set.
I’m going to upload a patch with my latest changes and commit shortly. Can you
update the docs for this new command?
> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Peter Rusko
>Assignee: Tomás Fernández Löbbe
>Priority: Major
> Attachments: SOLR-11960.patch, SOLR-11960.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining
> properties that affect a single collection. Collection properties could be
> stored in a single zookeeper node per collection, making it possible to
> trigger zookeeper watchers for only those Solr nodes that have cores of that
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369607#comment-16369607 ] Peter Rusko commented on SOLR-11960: Thanks for the review, here's the updated patch: [^SOLR-11960.patch] > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Assignee: Tomás Fernández Löbbe >Priority: Major > Attachments: SOLR-11960.patch, SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[
https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16361679#comment-16361679
]
Tomás Fernández Löbbe commented on SOLR-11960:
--
Thanks [~prusko], this looks very good. Some comments:
{code:java}
public Map getCollectionProperties(final String collection) {
Map properties = watchedCollectionProps.get(collection);
if (properties == null) {
try {
properties = fetchCollectionProperties(collection, null);
} catch (Exception e) {
throw new SolrException(ErrorCode.SERVER_ERROR, "Error reading
collection properties", e);
}
}
return properties;
}
{code}
I believe you are intentionally not adding the fetched properties back to the
{{watchedCollectionProps}} (because there is no watch, they would get stale).
Could you add a comment about it? I spent some time trying to figure out if
that was intentional or no. Also, could you add javadocs to this method?
In {{PropsWatcher.refreshAndWatch()}} you have this code:
{code:java}
} catch (KeeperException e) {
LOG.error("Unwatched collection: [{}]", coll, e);
{code}
What do you mean there with “Unwatched collection”?
If I understand correctly, on the first time someone calls
{{registerCollectionPropsWatcher}} for a particular zkStateReader and
collection, the watcher will be executed, but from then on, that’s not going to
happen again. Is this correct? If so, I believe we should make that consistent.
Components don’t necessarily know in which order they are loaded and this may
generate some strange behavior, also, multiple cores could belong to the same
collection in the same node and only the first one will trigger the watcher on
the registration. Maybe {{PropsWatcher.refreshAndWait()}} should receive a
boolean parameter and use that to know if it has to notify or not the watchers.
Then, when you are doing {{new PropsWatcher(collection).refreshAndWatch();}}
you’d use {{false}}, and {{true}} from {{PropsWatcher.process(…)}}. WDYT?
In {{refreshAndWatch()}} you are synchronizing on {{getUpdateLock()}}, which
sounds like overkill. I’m wondering if we really need any synchronization here
since we are just submitting tasks to a multi-thread Executor.
Should {{removeCollectionPropsWatcher}} also remove the collection from
{{collectionPropsWatches}} in case of {{canBeRemoved()==true}}?
Could you add a test that validates that collection properties are correctly
backed up and restored when using BACKUP and RESTORE APIs?
> Add collection level properties
> ---
>
> Key: SOLR-11960
> URL: https://issues.apache.org/jira/browse/SOLR-11960
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Peter Rusko
>Priority: Major
> Attachments: 0001-SOLR-11960-add-collection-properties-support.patch,
> SOLR-11960.patch
>
>
> Solr has cluster properties, but no easy and extendable way of defining
> properties that affect a single collection. Collection properties could be
> stored in a single zookeeper node per collection, making it possible to
> trigger zookeeper watchers for only those Solr nodes that have cores of that
> collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11960) Add collection level properties
[ https://issues.apache.org/jira/browse/SOLR-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357769#comment-16357769 ] Peter Rusko commented on SOLR-11960: Here's a proposed patch for collection properties and a new collection admin command that allows changing those properties: [^SOLR-11960.patch] > Add collection level properties > --- > > Key: SOLR-11960 > URL: https://issues.apache.org/jira/browse/SOLR-11960 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Peter Rusko >Priority: Major > Attachments: 0001-SOLR-11960-add-collection-properties-support.patch, > SOLR-11960.patch > > > Solr has cluster properties, but no easy and extendable way of defining > properties that affect a single collection. Collection properties could be > stored in a single zookeeper node per collection, making it possible to > trigger zookeeper watchers for only those Solr nodes that have cores of that > collection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
