Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Jul 27, 2014, at 11:12 PM, Baohua Yang yangbao...@gmail.com wrote: Dear all Thanks for all the responses! First, I threw the question actually wanting to hear other voices of the similar requirements of dynamic parsing. Glad to get all supports and also questions. The scenario is that sometime we might define some groups in the config file with changing names. One not appropriated example is that we add a network element with properties as [xxx_box] property1=aaa property2=bbb In this case, the parser should automatically read in the xxx_box and parses its properties, which would extend the power of the current config lib. Although we can utilize explicit definitions of groups names in our code to ignore uninterested groups in the configuration file, I think it would be more flexible to keep the functionality to read in those undefined groups. Now I am sure it's impossible to do so with existing olso.config lib. Thanks for all the feedbacks, and would like to get more comments on whether we should implement this! oslo.config wants options to be registered with well-known names so they are discoverable by tools that produce documentation for deployers. Are the xxx_box names going to be like that? Are all of the options in xxx_box groups the same, or can they vary? How would you describe to a deployer what the group names can be and which options can be included in the section? Doug On Fri, Jul 25, 2014 at 1:58 AM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. So the only thing that oslo.config lacks in its interface here is some way to allow the first step. The rest can be overcomed with some sugar. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best wishes! Baohua ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
Dear all Thanks for all the responses! First, I threw the question actually wanting to hear other voices of the similar requirements of dynamic parsing. Glad to get all supports and also questions. The scenario is that sometime we might define some groups in the config file with changing names. One not appropriated example is that we add a network element with properties as [xxx_box] property1=aaa property2=bbb In this case, the parser should automatically read in the xxx_box and parses its properties, which would extend the power of the current config lib. Although we can utilize explicit definitions of groups names in our code to ignore uninterested groups in the configuration file, I think it would be more flexible to keep the functionality to read in those undefined groups. Now I am sure it's impossible to do so with existing olso.config lib. Thanks for all the feedbacks, and would like to get more comments on whether we should implement this! On Fri, Jul 25, 2014 at 1:58 AM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. So the only thing that oslo.config lacks in its interface here is some way to allow the first step. The rest can be overcomed with some sugar. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best wishes! Baohua ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Jul 25, 2014, at 1:19 AM, Yuriy Taraday yorik@gmail.com wrote: On Fri, Jul 25, 2014 at 2:35 AM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 5:43 PM, Yuriy Taraday yorik@gmail.com wrote: On Fri, Jul 25, 2014 at 12:05 AM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 3:08 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 10:31 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 1:58 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. If the options are related to the plugin, could the plugin just register them before it tries to use them? Plugin would have to register its options under a fixed group. But what if we want a number of plugin instances? Presumably something would know a name associated with each instance and could pass it to the plugin to use when registering its options. I guess it’s not clear what problem you’re actually trying to solve by proposing this change to the way the config files are parsed. That doesn’t mean your idea is wrong, just that I can’t evaluate it or point out another solution. So what is it that you’re trying to do that has led to this suggestion? I don't exactly know what the original author's intention is but I don't generally like the fact that all libraries and plugins wanting to use config have to influence global CONF instance. That is a common misconception. The use of a global configuration option is an application developer choice. The config library does not require it. Some of the other modules in the oslo incubator expect a global config object because they started life in applications with that pattern, but as we move them to libraries we are updating the APIs to take a ConfigObj as argument (see oslo.messaging and oslo.db for examples). What I mean is that instead of passing ConfigObj and a section name in arguments for some plugin/lib it would be cleaner to receive an object that represents one section of config, not the whole config at once. The new ConfigFilter class lets you do something like what you want [1]. The options are visible only in the filtered view created by the plugin, so the application can’t see them. That provides better data separation, and prevents the options used by the plugin or library from becoming part of its API. Doug [1] http://docs.openstack.org/developer/oslo.config/cfgfilter.html Yes, it looks like it. Didn't know about that, thanks! I wonder who should wrap CONF object into ConfigFilter - core or plugin. If the plugin wraps the object then the plugin can register new options that other parts of the application can’t see (because their values are only known to the wrapper, which the application doesn’t have). Doug -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Thu, Jul 24, 2014 at 6:10 AM, Baohua Yang yangbao...@gmail.com wrote:
Hi, all
The current oslo.cfg module provides an easy way to load name known
options/groups from he configuration files.
I am wondering if there's a possible solution to dynamically load
them?
For example, I do not know the group names (section name in the
configuration file), but we read the configuration file and detect the
definitions inside it.
#Configuration file:
[group1]
key1 = value1
key2 = value2
Then I want to automatically load the group1. key1 and group2.
key2, without knowing the name of group1 first.
That's actually good question. But it seems to be a bit complicated. You
have to tell to option loader about type of given configuration item.
I also was thinking about this type of feature, and that's what came into
my mind.
I found very useful JSON or YAML format for describing options. Here's
simple example how to describe configuration file that would describe
dynamic configuration.
options.yaml
- groups:
- DEFAULT
- NOT_DEFAULT
- ANOTHER_ONE
- list:
- option_a:
- group: DEFAULT
- value: [a, b ,c]
- description: description
- dict:
- option_b:
- group: DEFAULT
- value: { a:b, c:d}
- description: description
and so on ...
Explanation:
` - groups` attribute defines which groups oslo.config should register.
`- list` - option type
`- option_b` - option descriptor, each descriptor is a dict (string: list)
where key is an option name, and attributes inside it describes to which
group it belongs, its value, and description.
oslo.config would need just parse YAML file and register all options and in
the end you'll receive a set of registered options per their groups.
It's the best variant of dynamic option loading, but i'm open for
discussion.
Best regards,
Denis Makogon
Thanks a lot!
--
Best wishes!
Baohua
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? Doug Thanks a lot! -- Best wishes! Baohua ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. So the only thing that oslo.config lacks in its interface here is some way to allow the first step. The rest can be overcomed with some sugar. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Jul 24, 2014, at 1:58 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. If the options are related to the plugin, could the plugin just register them before it tries to use them? I guess it’s not clear what problem you’re actually trying to solve by proposing this change to the way the config files are parsed. That doesn’t mean your idea is wrong, just that I can’t evaluate it or point out another solution. So what is it that you’re trying to do that has led to this suggestion? Doug So the only thing that oslo.config lacks in its interface here is some way to allow the first step. The rest can be overcomed with some sugar. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Thu, Jul 24, 2014 at 10:31 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 1:58 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. If the options are related to the plugin, could the plugin just register them before it tries to use them? Plugin would have to register its options under a fixed group. But what if we want a number of plugin instances? I guess it’s not clear what problem you’re actually trying to solve by proposing this change to the way the config files are parsed. That doesn’t mean your idea is wrong, just that I can’t evaluate it or point out another solution. So what is it that you’re trying to do that has led to this suggestion? I don't exactly know what the original author's intention is but I don't generally like the fact that all libraries and plugins wanting to use config have to influence global CONF instance. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Jul 24, 2014, at 3:08 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 10:31 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 1:58 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. If the options are related to the plugin, could the plugin just register them before it tries to use them? Plugin would have to register its options under a fixed group. But what if we want a number of plugin instances? Presumably something would know a name associated with each instance and could pass it to the plugin to use when registering its options. I guess it’s not clear what problem you’re actually trying to solve by proposing this change to the way the config files are parsed. That doesn’t mean your idea is wrong, just that I can’t evaluate it or point out another solution. So what is it that you’re trying to do that has led to this suggestion? I don't exactly know what the original author's intention is but I don't generally like the fact that all libraries and plugins wanting to use config have to influence global CONF instance. That is a common misconception. The use of a global configuration option is an application developer choice. The config library does not require it. Some of the other modules in the oslo incubator expect a global config object because they started life in applications with that pattern, but as we move them to libraries we are updating the APIs to take a ConfigObj as argument (see oslo.messaging and oslo.db for examples). Doug -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Fri, Jul 25, 2014 at 12:05 AM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 3:08 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 10:31 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 1:58 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. If the options are related to the plugin, could the plugin just register them before it tries to use them? Plugin would have to register its options under a fixed group. But what if we want a number of plugin instances? Presumably something would know a name associated with each instance and could pass it to the plugin to use when registering its options. I guess it’s not clear what problem you’re actually trying to solve by proposing this change to the way the config files are parsed. That doesn’t mean your idea is wrong, just that I can’t evaluate it or point out another solution. So what is it that you’re trying to do that has led to this suggestion? I don't exactly know what the original author's intention is but I don't generally like the fact that all libraries and plugins wanting to use config have to influence global CONF instance. That is a common misconception. The use of a global configuration option is an application developer choice. The config library does not require it. Some of the other modules in the oslo incubator expect a global config object because they started life in applications with that pattern, but as we move them to libraries we are updating the APIs to take a ConfigObj as argument (see oslo.messaging and oslo.db for examples). What I mean is that instead of passing ConfigObj and a section name in arguments for some plugin/lib it would be cleaner to receive an object that represents one section of config, not the whole config at once. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Jul 24, 2014, at 5:43 PM, Yuriy Taraday yorik@gmail.com wrote: On Fri, Jul 25, 2014 at 12:05 AM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 3:08 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 10:31 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 1:58 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. If the options are related to the plugin, could the plugin just register them before it tries to use them? Plugin would have to register its options under a fixed group. But what if we want a number of plugin instances? Presumably something would know a name associated with each instance and could pass it to the plugin to use when registering its options. I guess it’s not clear what problem you’re actually trying to solve by proposing this change to the way the config files are parsed. That doesn’t mean your idea is wrong, just that I can’t evaluate it or point out another solution. So what is it that you’re trying to do that has led to this suggestion? I don't exactly know what the original author's intention is but I don't generally like the fact that all libraries and plugins wanting to use config have to influence global CONF instance. That is a common misconception. The use of a global configuration option is an application developer choice. The config library does not require it. Some of the other modules in the oslo incubator expect a global config object because they started life in applications with that pattern, but as we move them to libraries we are updating the APIs to take a ConfigObj as argument (see oslo.messaging and oslo.db for examples). What I mean is that instead of passing ConfigObj and a section name in arguments for some plugin/lib it would be cleaner to receive an object that represents one section of config, not the whole config at once. The new ConfigFilter class lets you do something like what you want [1]. The options are visible only in the filtered view created by the plugin, so the application can’t see them. That provides better data separation, and prevents the options used by the plugin or library from becoming part of its API. Doug [1] http://docs.openstack.org/developer/oslo.config/cfgfilter.html -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo.cfg] Dynamically load in options/groups values from the configuration files
On Fri, Jul 25, 2014 at 2:35 AM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 5:43 PM, Yuriy Taraday yorik@gmail.com wrote: On Fri, Jul 25, 2014 at 12:05 AM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 3:08 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 10:31 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 24, 2014, at 1:58 PM, Yuriy Taraday yorik@gmail.com wrote: On Thu, Jul 24, 2014 at 4:14 PM, Doug Hellmann d...@doughellmann.com wrote: On Jul 23, 2014, at 11:10 PM, Baohua Yang yangbao...@gmail.com wrote: Hi, all The current oslo.cfg module provides an easy way to load name known options/groups from he configuration files. I am wondering if there's a possible solution to dynamically load them? For example, I do not know the group names (section name in the configuration file), but we read the configuration file and detect the definitions inside it. #Configuration file: [group1] key1 = value1 key2 = value2 Then I want to automatically load the group1. key1 and group2. key2, without knowing the name of group1 first. If you don’t know the group name, how would you know where to look in the parsed configuration for the resulting options? I can imagine something like this: 1. iterate over undefined groups in config; 2. select groups of interest (e.g. by prefix or some regular expression); 3. register options in them; 4. use those options. Registered group can be passed to a plugin/library that would register its options in it. If the options are related to the plugin, could the plugin just register them before it tries to use them? Plugin would have to register its options under a fixed group. But what if we want a number of plugin instances? Presumably something would know a name associated with each instance and could pass it to the plugin to use when registering its options. I guess it’s not clear what problem you’re actually trying to solve by proposing this change to the way the config files are parsed. That doesn’t mean your idea is wrong, just that I can’t evaluate it or point out another solution. So what is it that you’re trying to do that has led to this suggestion? I don't exactly know what the original author's intention is but I don't generally like the fact that all libraries and plugins wanting to use config have to influence global CONF instance. That is a common misconception. The use of a global configuration option is an application developer choice. The config library does not require it. Some of the other modules in the oslo incubator expect a global config object because they started life in applications with that pattern, but as we move them to libraries we are updating the APIs to take a ConfigObj as argument (see oslo.messaging and oslo.db for examples). What I mean is that instead of passing ConfigObj and a section name in arguments for some plugin/lib it would be cleaner to receive an object that represents one section of config, not the whole config at once. The new ConfigFilter class lets you do something like what you want [1]. The options are visible only in the filtered view created by the plugin, so the application can’t see them. That provides better data separation, and prevents the options used by the plugin or library from becoming part of its API. Doug [1] http://docs.openstack.org/developer/oslo.config/cfgfilter.html Yes, it looks like it. Didn't know about that, thanks! I wonder who should wrap CONF object into ConfigFilter - core or plugin. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
