[jira] [Commented] (SOLR-8054) Add a GET command to ConfigSets API
[ https://issues.apache.org/jira/browse/SOLR-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944033#comment-14944033 ] Gregory Chanan commented on SOLR-8054: -- Ok, from discussing with [~anshumg] in SOLR-6736 it sounds like we are going to use that jira for modification and this JIRA for GET. I do not currently have a patch for GET. As I wrote above, I think there are two separate steps here (that could be separate jiras, but should have a sensible API between them): 1) Config/Schema API equivalents 2) File-based APIs My plan is to start work on 1) first, someone can take up 2) or I can work on it after 1). But let's figure out a sensible API before we commit anything. > Add a GET command to ConfigSets API > --- > > Key: SOLR-8054 > URL: https://issues.apache.org/jira/browse/SOLR-8054 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: Gregory Chanan >Assignee: Gregory Chanan > > It would be useful to have a command that allows you to view a ConfigSet via > the API rather than going to zookeeper directly. Mainly for security > reasons, e.g. > - solr may have different security requirements than the ZNodes e.g. only > solr can view znodes but any authenticated user can call ConfigSet API > - it's nicer than pointing to the web UI and using the zookeeper viewer, > because of the same security concerns as above and that you don't have to > know the internal zookeeper paths. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8054) Add a GET command to ConfigSets API
[ https://issues.apache.org/jira/browse/SOLR-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943950#comment-14943950 ] Anshum Gupta commented on SOLR-8054: SOLR-6736 was heading towards just that. Let's combine the effort there. It had been on my list for a while and I never got a chance to wrap it up, but I plan to spend some time on this now. > Add a GET command to ConfigSets API > --- > > Key: SOLR-8054 > URL: https://issues.apache.org/jira/browse/SOLR-8054 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: Gregory Chanan >Assignee: Gregory Chanan > > It would be useful to have a command that allows you to view a ConfigSet via > the API rather than going to zookeeper directly. Mainly for security > reasons, e.g. > - solr may have different security requirements than the ZNodes e.g. only > solr can view znodes but any authenticated user can call ConfigSet API > - it's nicer than pointing to the web UI and using the zookeeper viewer, > because of the same security concerns as above and that you don't have to > know the internal zookeeper paths. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8054) Add a GET command to ConfigSets API
[
https://issues.apache.org/jira/browse/SOLR-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943875#comment-14943875
]
Gregory Chanan commented on SOLR-8054:
--
Thanks for the responses [~markrmil...@gmail.com] and [~ysee...@gmail.com].
bq. Thinking about a view config API like this, I think I'd want some way to
get an individual file or all the files (as a zip, in one stream, whatever ends
up making sense) depending on param.
I think both individual file and all file APIs make sense. For the all file
APIs, I'm not sure whether as a zip or all in one stream makes more sense. Is
there an existing API where we provide multiple files in one stream? I'd
rather follow that logic than make something up myself.
{quote}
Should uploading (cloning) configsets be taken into consideration here?
{quote}
Yes, but I've tried to stay away from exposing file-modication based APIs
directly in Solr due to the security issues discussed in SOLR-5287 / SOLR-5539.
One approach to thinking about this is to break the operations into tasks a
cluster-level administrator would do vs tasks an individual would do and seeing
if we can do the later without file-based APIs.
bq. Downloading the config for the purposes of making a backup, with the
ability to restore it later after trying some different things
This falls into tasks an individual would do. An individual could do this
today via cloning their copy via the ConfigSet API and "restoring" via
deleting/copying the old one. That's not super easy, but you could provide a
nicer interface by say, keeping track of the changes with version numbers and
letting you restore from version number. So I don't think this strictly needs
a file-based API.
bq. Essentially cloning a config in a different cluster (testing,
troubleshooting, etc)
That's an interesting use case because it's sort of between what a user would
do and what an administrator would do. One possibility is to have some
higher-level cross-cluster replication functionality that lets you replicate
configs to another cluster. You could imagine this happening on all configs or
some subset. Alternatively, if the user is an administrator of the backup
cluster (which seems likely here), there is nothing stopping you from using the
existing ZkCLI commands. That's just not feasible if ZK security is on and the
user doing the operations doesn't have permissions, but that doesn't seem that
likely in this case.
{quote}Both seem useful (If I'm correctly understanding what you mean by
File-based). APIs may manipulate the state, but dealing with the persisted
state as a whole also seems useful. For instance, cloning a config via config
APIs that deal with individual settings seems difficult.{quote}
Yes, that's a good point. Hopefully the above makes sense -- provide Solr APIs
for end users and have administrators use the ZkCLI.
> Add a GET command to ConfigSets API
> ---
>
> Key: SOLR-8054
> URL: https://issues.apache.org/jira/browse/SOLR-8054
> Project: Solr
> Issue Type: New Feature
> Components: SolrCloud
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>
> It would be useful to have a command that allows you to view a ConfigSet via
> the API rather than going to zookeeper directly. Mainly for security
> reasons, e.g.
> - solr may have different security requirements than the ZNodes e.g. only
> solr can view znodes but any authenticated user can call ConfigSet API
> - it's nicer than pointing to the web UI and using the zookeeper viewer,
> because of the same security concerns as above and that you don't have to
> know the internal zookeeper paths.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8054) Add a GET command to ConfigSets API
[ https://issues.apache.org/jira/browse/SOLR-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943533#comment-14943533 ] Yonik Seeley commented on SOLR-8054: Should uploading (cloning) configsets be taken into consideration here? I'm thinking about usecases involving - Downloading the config for the purposes of making a backup, with the ability to restore it later after trying some different things - Essentially cloning a config in a different cluster (testing, troubleshooting, etc) bq. (I hope) the future is API-based like the Config/Schema API, not File-based. Both seem useful (If I'm correctly understanding what you mean by File-based). APIs may manipulate the state, but dealing with the persisted state as a whole also seems useful. For instance, cloning a config via config APIs that deal with individual settings seems difficult. > Add a GET command to ConfigSets API > --- > > Key: SOLR-8054 > URL: https://issues.apache.org/jira/browse/SOLR-8054 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: Gregory Chanan >Assignee: Gregory Chanan > > It would be useful to have a command that allows you to view a ConfigSet via > the API rather than going to zookeeper directly. Mainly for security > reasons, e.g. > - solr may have different security requirements than the ZNodes e.g. only > solr can view znodes but any authenticated user can call ConfigSet API > - it's nicer than pointing to the web UI and using the zookeeper viewer, > because of the same security concerns as above and that you don't have to > know the internal zookeeper paths. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8054) Add a GET command to ConfigSets API
[ https://issues.apache.org/jira/browse/SOLR-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14943487#comment-14943487 ] Mark Miller commented on SOLR-8054: --- Thinking about a view config API like this, I think I'd want some way to get an individual file or all the files (as a zip, in one stream, whatever ends up making sense) depending on param. > Add a GET command to ConfigSets API > --- > > Key: SOLR-8054 > URL: https://issues.apache.org/jira/browse/SOLR-8054 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: Gregory Chanan >Assignee: Gregory Chanan > > It would be useful to have a command that allows you to view a ConfigSet via > the API rather than going to zookeeper directly. Mainly for security > reasons, e.g. > - solr may have different security requirements than the ZNodes e.g. only > solr can view znodes but any authenticated user can call ConfigSet API > - it's nicer than pointing to the web UI and using the zookeeper viewer, > because of the same security concerns as above and that you don't have to > know the internal zookeeper paths. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8054) Add a GET command to ConfigSets API
[ https://issues.apache.org/jira/browse/SOLR-8054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941964#comment-14941964 ] Gregory Chanan commented on SOLR-8054: -- I started prototyping this out. I think there are potentially multiple tasks here: 1) Config API / Schema API equivalents for ConfigSets See SOLR-7570 for the genesis of this; in short, the collection-based Config API and Schema APIs should not modify the underlying ConfigSet. It makes sense, then, to have GET-able Config API/Schema APIs that are specific to the ConfigSet because: a) these may be different from the Collection-Config representation b) you may not have a collection using the ConfigSet currently in a "template" world (SOLR-5955). Having PUT/POST-able ConfigSet-specific APIs is an open question IMO. 2) "Raw" File based view I'm thinking an http equivalent for "ZkCli -cmd DOWNCONFIG". The motivation is a subset of the motivation for a ConfigSet API as a whole, namely: m1) Doesn't require zookeeper access (which may not be available) m2) Can use the same authentication/authorization mechanisms as the rest of Solr m3) Doesn't require a collection to be instantiated (as today via the ShowFile API) There is a limitation due to http as compared to the ZkCli equivalent, namely that you can't "download" multiple files in a single GET request, see http://stackoverflow.com/questions/1041542/how-to-download-multiple-files-with-one-http-request. So we'd have to do one of the following: o1) zip up the ConfigSet o2) require the client to go file-by-file like the ShowFile API today o3) invent/use our own format that has "file breaks" within a single response I'm not sure the effort here is worth it, considering (I hope) the future is API-based like the Config/Schema API, not File-based. But it does at least seem useful until we are in a world where the majority are using the APIs. As for the options, I don't like o1). I don't have a strong preference for o2) vs o3), although I think I'd lean slightly to o3) just to be able to reduce the number of http calls required to download a config. Thoughts? > Add a GET command to ConfigSets API > --- > > Key: SOLR-8054 > URL: https://issues.apache.org/jira/browse/SOLR-8054 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: Gregory Chanan >Assignee: Gregory Chanan > > It would be useful to have a command that allows you to view a ConfigSet via > the API rather than going to zookeeper directly. Mainly for security > reasons, e.g. > - solr may have different security requirements than the ZNodes e.g. only > solr can view znodes but any authenticated user can call ConfigSet API > - it's nicer than pointing to the web UI and using the zookeeper viewer, > because of the same security concerns as above and that you don't have to > know the internal zookeeper paths. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
