> I see a chance that we are aligning now our ideas more and more ;) Yes, I see a lot or progress as well (on both sides).
> Basically you have to register PropertySourceProviderSpi's Do you really need to? In DeltaSpike we just use the ServiceLoader mechanism (has good support in SE, EE and even OSGi). In DeltaSpike the ConfigSourceProvider is actually optional. It is there for cases where you need to register multiple ConfigSources (e.g. because you have multiple property or ini files with the same name in various JARs). Usually you don't use this but instead just natively register a ConfigSource itself via the ServiceLoader mechanism. In that case there is not a single line of code at all to register it. We don't need CDI nor anything else beyond pure Java. Even when you have CDI you usually like to initialize the configuration system _before_ you even start the CDI container. Otherwise we would not be able to utilize the config system for @Exclude-ing beans, deactivating Extensions, etc. Thus you _always_ need to rely on the ServiceLoader mechanism. > So for individual property sources (the ones != default) there > is no issue I think, because prioritization is not an issue Oh I think there is. I like to e.g. provide a default configuration for my document archive connector. For easier maintenance I like to put them into an own documentarchive.properties file which is packaged into my documentarchive-connector.jar. This ConfigSource needs to get a priority which is higher than the very default ini and property files but lower than the environment and system properties. Another use case would be to register a cluster wide ConfigSource which reads from one of my application tables from the database. We've seen this quite a few times in bigger projects. >Level 1: globally the hierarchy of property sources, e.g. > > >1. Boot Config, >2. System config, >3. Ear Config, >4. App Config >5. SaaS Config, This is a possible hierarchy dimension. But it is just ONE out of many. Actually it's mixture between 2 different dimensions a.) the time dimension:boot vs system (init) vs app (runtime) b.) the modularity dimension: cluster vs container vs ear vs app Whether a documentarchive.endpoint.url is needed at boot time or at runtime is an implementation detail of my documentarchive-connector (to stay at this example). Back when doing CODI (and then later again at DeltaSpike) we did discuss this many times and found no good solution other than make a step back and ignore _why_ people want the one over the other. The only thing we can do is to make sure that people (programmers, admin folks, etc) have a way to define the order in exactly the way they need. They don't even need a way to redefine the priority of the 'standard' ConfigSources (env, jndi, etc). Instead they can just add their own ConfigSource with a higher ordinal and be done. > Looking at this level, priorities are IMO enough a feasible > mechanism, because it allows to easily add additional > layers transparently as needed ;) While we still have a slightly different understanding about the 'why', we DO agree about the result ;) But we really have to define which default ConfigSources we like to add and which priority they should get. And we imo should not use 1, 2, 3,.. but like with @Priority for Interceptors we should start e.g. with 1000 and keep big spaces in between them. Learned that with Basic in the early 80s... Oh, for custom ConfigSources I actually don't care if the ConfigSource itself knows it's ordinal or if the ordinal is set during registration of the ConfigSource. It is really very similar to the ELResolver chain imo. > I would not define priorities on single file level, but only on property > source level Actually in DeltaSpike each property file is an own ConfigSource. Consider you have myframework.properties in the framework jar. Now I gonna use this jar in my application but like to slightly 'tweak' it's configuration. I just add a myframework.properties to my own projects jar and give it a higher ordinal (just add deltaspike_ordinal=1200 into the file). Voila. > Tamaya should be flexible enough to accomodate most of them, > so application programmers still have their guides, where > and how to add configuration files, resources etc. By adding > additional levels of configuration, e.g. on the deployment nodes, > it is possible to override anything as needed. With 'user' I was talking about the API. What you refer to is part of the SPI. And of course we will have this to add ConfigSources. > Summarizing I think we need both mechanisms. I still don't quite get this approach it seems. You have a default configuration for the whole system which you get via Configuration.current(), right? And you have a mechanism (builder like) to tinker a different Configuration at runtime? Where do you store that? Where do you use that? How to e.g. add graphical UI to allow admins to tweak those settings? I still need a bit help to get the use case. What do you use it for usually? LieGrue, strub On Saturday, 27 December 2014, 18:47, Anatole Tresch <[email protected]> wrote: > >Hi Mark > > >I will try to answer as precise as possible. I see a chance that we are >aligning now our ideas more and more ;) >... (inline) > > >2014-12-27 16:26 GMT+01:00 Mark Struberg <[email protected]>: > >Hi Anatole! >> >>First of all thanks for sharing this sample. You might know your use cases >>but others don't. >>I assume that all this PropertySource construction is done at a single point, >>e.g. a producer metho >> >Yes, but that is only the access point, read ahead... > >Let me ask a few questions: >> >>1.) How do you know which ConfigSources exist in the project? You might be >>one jar in a dozen ones, having no control over them? >> >Basically you have to register PropertySourceProviderSpi's (or whatever they >are named) with your code. How these are loaded depends on the ServiceContext, >so in SE you will probably use ServiceLoader only, whereas in EE you can >additionally rely on CDI to resolve corresponding sources. And as you said >there could be multiple ones, which are visible in a certain context depends >on the ServiceContext. So for individual property sources (the ones != >default) there is no issue I think, because prioritization is not an issue, >additionally individual configs can also use other existing ones in their >metamodel, if needed (in the definition code of the provider itself). > > >Level 1: globally the hierarchy of property sources, e.g. > > >1. Boot Config, >2. System config, >3. Ear Config, >4. App Config >5. SaaS Config, > > >I assumed that this hierarchy is well defined in an environment, and typically >not dynamic during runtime and therefore yes, hardcoded in the metamodel >loaded on boot level. I even think from a security perspective it is not a >good idea to have it dynamically. A provider may not be available (based on >the environment), so when executing a MDB in the ear it might be only the >following sources are available/actively providing data to the current >configuration: > > >1. Boot Config, >2. System config, >3. Ear Config, > > >Looking at this level, priorities are IMO enough a feasible mechanism, because >it allows to easily add additional layers transparently as needed ;) > > >Now, what a Level 1 PropertySource is doing internally is completely another >story. Here in that area I expect more fine grained aggregation operations to >be used in many cases, as suggested in the example. Additionally a >PropertySourceProviderSpi is allowed to provide the effective property sources >contextually, e.g. depending on the current classloader visiblity of items. > > >So given that I adding a priority to a PropertySource is an interesting >aspect. I even would ask, if we have one, if it would make sense to provide >some admin functionality to change the priority during runtime, or adding >additional sources on the fly. When I am using the builder, priorities are >normally ignored, whereas for the overall configuration hierarchy, they are >useful. Additionally we also can use the priorities as well on Level 2 and >beyond, as useful ;) > > >Summarizing I think we need both mechanisms. > > > > >2.) In DeltaSpike each property file could define it's own ordinal with a >simple entry in the property file. That way you can just add a JAR which >overrides a default config from another jar. Would each of them get an own >name? Or do they all share the same name and priority? >> >I would not define priorities on single file level, but only on property >source level. I can still an additional property source and override the keys >I want on source level. But as I said, do we really want application deploy an >additional jar, hereby completely overriding any kind of config. I would say, >no. But we can define (company specific mechanism), how and when config can be >overridden as part of the metamodel. > >3.) How do you deal with modularity? E.g. adding a dependency which needs some >other ConfigSource but knows nothing about your code? >> >>4.) Another modularity question: What about having multiple jars (different >>independent framework parts) and each of them likes to have another >>PropertySouceBuilder configuration? >> >Use different property source names on top level. > > > >>5.) How do you allow containers and framework programmers to add >>PropertySources which affect all the project? >>They define the metamodel and the extension points by the implementation of >>the configuration metamodel. This metamodel as I said should not be dynamic. >> >>6.) >> >>> - you can override along the classloader hierarchy (system, ear, war) >> >>> by adding resources to the classpath at xxx.Usually this does not work well >>> as e.g. all the JARs in WEB-INF/lib have exactly the same 'level'. It's >>> totally up to the ClassLoader mechanism in which order you get them. And >>> that could change on every restart. Been there, done that - doesn't work. >> > >I would never rely on the classloader ordering. Normally your config within >such a level should be consistent. If not you have multiple possibilities: >1) just take one. >2) Just take one and log. >3) throw a Deployment Error >4) take a default >5) Override the conflicting value on the next level of defined config, so the >conflict is not relevant anymore. How this is done, is defined by your >metamodel. > > > >7.) >>> In code the do simply: Configuration config = >>> Configuration.current("Stephen"); >>And what if another jar does Configuration.current("Giovanni")? >>Imo that approach doesn't scale. >> >> >Normally you do Configuration.current(); // the default > > >Configuration.current("name") is for specific isolated/modular configuration >sources (see 3). > > >>I like to align our ideas, so I enumerate my main goals and design principles >>of a config system as I envision it: >> >>A.) have a total separation between the configuration system and it's >>consumers. The 'consumers' API should not know where the result comes from >>and also not what the value is. I think this is the only way we can provide >>an extensible system where e.g. the Operations team could 'overwrite' certain >>values. If we require the userland code to know about the PropertySources >>then we cannot do this. >> > > >Partially agree. Companies have their own way how they configure their >applications. Tamaya should be flexible enough to accomodate most of them, so >application programmers still have their guides, where and how to add >configuration files, resources etc. By adding additional levels of >configuration, e.g. on the deployment nodes, >it is possible to override anything as needed. The ladder, yes, should be >completely transparent. > > >B.) Have a flexible SPI to allow Container Vendors and also application >programmers to add their own PropertySources in a very easy way. >> >Yes and no (see security constraints). I think we have to discuss more >deeply the roles involved and what should be possible. > >C.) The SPI should be extensible via plugin mechanism. If a jar containing a >PropertySource gets added then it should automatically get picked up for this >very application. >> >If the metamodel foresees that to be so, yes. If not, not (see >secuirity...). > >Are we on the same page here? >> >> > > >I hope! > > >LieGrue, >>strub >> >> >> >> >> >> >>> On Saturday, 27 December 2014, 15:35, Anatole Tresch <[email protected]> >>> wrote: >>> > Hi Gerhard >>> >>> all you say is true here, I also will not provide every kind of combination >>> logic, but I want to have a mechanism that is capable enough. >>> >>> @Mark end users don't see this mechanisms here. It used by the power users, >>> which define the "metamodel" how/where configuration is read >>> and assembled, e.g. >>> >>> SPI: PropertySourceProvider { >>> Collection<PropertySource> getPropertySources(); >>> } >>> >>> Lets assume the provider is returning one property source named >>> "Stephen". >>> The power user would then probably write something like that: >>> >>> PropertySource src = >>> PropertySourceBuilder.of("Stephen").addProviders(myProv2, >>> myProv3).withAggregationPolicy(mySpecialPolicy) >>> >>> .addProvider(BranchSpecificOverrideFilter.getPropertySource()).withAggregationPolicy(AggregationPolicy.OVERRIDE_AND_LOG) >>> .addProvider(DefaultJavaEEProviders.getEarProvider()).build(); >>> >>> This src instance then is returned e.g. by the provider (could also be >>> contextually per classloader etc.). So it is not only that the mechanism is >>> much more explicit and powerful, it >>> is also easier to understand. There is exactly one location, where all >>> happens: the PropertySourceProvider. You look at the code and immediately >>> now, what is happening. >>> So the code that is assembling the final property source returned is under >>> full control of the users and nicely explicit. >>> >>> For normal users the mechanism is completely transparent the will typically >>> have some kind of documentation, where its written: >>> >>> - put your config files in the formats .ini or .xml at the location xy >>> - you can override along the classloader hierarchy (system, ear, war) by >>> adding resources to the classpath at xxx. >>> - Branch sepcific overrides are merge automatically into it, see chapter >>> 3.4 for additional details. >>> - Finally it is possible to override entries an system leven, by setting >>> according system properties. >>> >>> In code the do simply: Configuration config = >>> Configuration.current("Stephen"); >>> >>> You get it? >>> >>> >>> >>> 2014-12-27 15:13 GMT+01:00 Gerhard Petracek <[email protected]>: >>> >>>> hi anatole, >>>> >>>> i guess everybody agrees that overriding based on ordinals wouldn't be >>>> enough for every possible use-case. >>>> back then we had a long discussion about it. we ended up with adding a >>>> simple (and optional) ConfigFilter interface. >>>> >>>> @aggregation: >>>> aggregation of key/value pairs is supported implicitly by an ordinal based >>>> approach. >>>> aggregation of values >per key< can be done manually with an own >>>> config-source (which delegates to the others and only aggregates those >>>> results). >>>> yes - that way this use-case is more complicated out-of-the-box. however, >>>> imo if users (think they) need such a complicated config, they should also >>>> need to do a bit more for it. >>>> (i guess in several cases they might question the requirement before they >>>> continue...) >>>> >>>> in general about possible use-cases (and features): >>>> we can never address everything out-of-the-box. >>>> a nice example is bean-validation. i could list a lot of cases which >>> aren't >>>> possible with that api/spi out-of-the-box, however, for the majority >>> it's >>>> good enough. >>>> i saw several users who couldn't replace their existing >>> validation-approach >>>> (as it was). >>>> however, several of them just simplified their validation-logic and were >>>> still happy about the result. >>>> others extended bv e.g. via the spi and only few kept what they used >>>> before >>>> (e.g. because it wasn't possible with bv or the workaround was too >>>> complicated >>>> and they insisted on what they had). >>>> >>>> regards, >>>> gerhard >> >>>> >>>> >>>> >> > > > > >-- > >Anatole Tresch >Java Engineer & Architect, JSR Spec Lead >Glärnischweg 10 >CH - 8620 Wetzikon > > >Switzerland, Europe Zurich, GMT+1 >Twitter: @atsticks >Blogs: http://javaremarkables.blogspot.ch/ >Google: atsticks >Mobile +41-76 344 62 79 > >
