Hi Yep, i will reply later.. ;) stay tuned!
Anatole Mark Struberg <[email protected]> schrieb am Sa., 27. Dez. 2014 um 21:01: > > I see a chance that we are aligning now our ideas more and more ;) > > Yes, I see a lot or progress as well (on both sides). > > > > Basically you have to register PropertySourceProviderSpi's > Do you really need to? In DeltaSpike we just use the ServiceLoader > mechanism (has good support in SE, EE and even OSGi). > > In DeltaSpike the ConfigSourceProvider is actually optional. It is there > for cases where you need to register multiple ConfigSources (e.g. because > you have multiple property or ini files with the same name in various > JARs). Usually you don't use this but instead just natively register a > ConfigSource itself via the ServiceLoader mechanism. In that case there is > not a single line of code at all to register it. > > We don't need CDI nor anything else beyond pure Java. > > Even when you have CDI you usually like to initialize the configuration > system _before_ you even start the CDI container. Otherwise we would not be > able to utilize the config system for @Exclude-ing beans, deactivating > Extensions, etc. > > Thus you _always_ need to rely on the ServiceLoader mechanism. > > > > So for individual property sources (the ones != default) there > > > is no issue I think, because prioritization is not an issue > > Oh I think there is. I like to e.g. provide a default configuration for my > document archive connector. For easier maintenance I like to put them into > an own documentarchive.properties file which is packaged into my > documentarchive-connector.jar. This ConfigSource needs to get a priority > which is higher than the very default ini and property files but lower than > the environment and system properties. > > Another use case would be to register a cluster wide ConfigSource which > reads from one of my application tables from the database. We've seen this > quite a few times in bigger projects. > > > >Level 1: globally the hierarchy of property sources, e.g. > > > > > >1. Boot Config, > >2. System config, > >3. Ear Config, > >4. App Config > >5. SaaS Config, > > This is a possible hierarchy dimension. But it is just ONE out of many. > Actually it's mixture between 2 different dimensions > a.) the time dimension:boot vs system (init) vs app (runtime) > b.) the modularity dimension: cluster vs container vs ear vs app > > > Whether a documentarchive.endpoint.url is needed at boot time or at > runtime is an implementation detail of my documentarchive-connector (to > stay at this example). > > Back when doing CODI (and then later again at DeltaSpike) we did discuss > this many times and found no good solution other than make a step back and > ignore _why_ people want the one over the other. The only thing we can do > is to make sure that people (programmers, admin folks, etc) have a way to > define the order in exactly the way they need. They don't even need a way > to redefine the priority of the 'standard' ConfigSources (env, jndi, etc). > Instead they can just add their own ConfigSource with a higher ordinal and > be done. > > > > > > Looking at this level, priorities are IMO enough a feasible > > > mechanism, because it allows to easily add additional > > > layers transparently as needed ;) > While we still have a slightly different understanding about the 'why', we > DO agree about the result ;) > > But we really have to define which default ConfigSources we like to add > and which priority they should get. And we imo should not use 1, 2, 3,.. > but like with @Priority for Interceptors we should start e.g. with 1000 and > keep big spaces in between them. Learned that with Basic in the early 80s... > > > > Oh, for custom ConfigSources I actually don't care if the ConfigSource > itself knows it's ordinal or if the ordinal is set during registration of > the ConfigSource. It is really very similar to the ELResolver chain imo. > > > > > I would not define priorities on single file level, but only on > property source level > > Actually in DeltaSpike each property file is an own ConfigSource. > > Consider you have myframework.properties in the framework jar. Now I gonna > use this jar in my application but like to slightly 'tweak' it's > configuration. I just add a myframework.properties to my own projects jar > and give it a higher ordinal (just add deltaspike_ordinal=1200 into the > file). Voila. > > > > > > Tamaya should be flexible enough to accomodate most of them, > > so application programmers still have their guides, where > > and how to add configuration files, resources etc. By adding > > > additional levels of configuration, e.g. on the deployment nodes, > > it is possible to override anything as needed. > With 'user' I was talking about the API. What you refer to is part of the > SPI. And of course we will have this to add ConfigSources. > > > > > Summarizing I think we need both mechanisms. > > I still don't quite get this approach it seems. You have a default > configuration for the whole system which you get via > Configuration.current(), right? And you have a mechanism (builder like) to > tinker a different Configuration at runtime? Where do you store that? Where > do you use that? How to e.g. add graphical UI to allow admins to tweak > those settings? I still need a bit help to get the use case. What do you > use it for usually? > > > LieGrue, > strub > > > > On Saturday, 27 December 2014, 18:47, Anatole Tresch <[email protected]> > wrote: > > > >Hi Mark > > > > > >I will try to answer as precise as possible. I see a chance that we are > aligning now our ideas more and more ;) > >... (inline) > > > > > >2014-12-27 16:26 GMT+01:00 Mark Struberg <[email protected]>: > > > >Hi Anatole! > >> > >>First of all thanks for sharing this sample. You might know your use > cases but others don't. > >>I assume that all this PropertySource construction is done at a single > point, e.g. a producer metho > >> > >Yes, but that is only the access point, read ahead... > > > >Let me ask a few questions: > >> > >>1.) How do you know which ConfigSources exist in the project? You might > be one jar in a dozen ones, having no control over them? > >> > >Basically you have to register PropertySourceProviderSpi's (or whatever > they are named) with your code. How these are loaded depends on the > ServiceContext, so in SE you will probably use ServiceLoader only, whereas > in EE you can additionally rely on CDI to resolve corresponding sources. > And as you said there could be multiple ones, which are visible in a > certain context depends on the ServiceContext. So for individual property > sources (the ones != default) there is no issue I think, because > prioritization is not an issue, additionally individual configs can also > use other existing ones in their metamodel, if needed (in the definition > code of the provider itself). > > > > > >Level 1: globally the hierarchy of property sources, e.g. > > > > > >1. Boot Config, > >2. System config, > >3. Ear Config, > >4. App Config > >5. SaaS Config, > > > > > >I assumed that this hierarchy is well defined in an environment, and > typically not dynamic during runtime and therefore yes, hardcoded in the > metamodel loaded on boot level. I even think from a security perspective it > is not a good idea to have it dynamically. A provider may not be > available (based on the environment), so when executing a MDB in the ear > it might be only the following sources are available/actively providing > data to the current configuration: > > > > > >1. Boot Config, > >2. System config, > >3. Ear Config, > > > > > >Looking at this level, priorities are IMO enough a feasible mechanism, > because it allows to easily add additional layers transparently as needed ;) > > > > > >Now, what a Level 1 PropertySource is doing internally is completely > another story. Here in that area I expect more fine grained aggregation > operations to be used in many cases, as suggested in the example. > Additionally a PropertySourceProviderSpi is allowed to provide the > effective property sources contextually, e.g. depending on the current > classloader visiblity of items. > > > > > >So given that I adding a priority to a PropertySource is an interesting > aspect. I even would ask, if we have one, if it would make sense to provide > some admin functionality to change the priority during runtime, or adding > additional sources on the fly. When I am using the builder, priorities are > normally ignored, whereas for the overall configuration hierarchy, they are > useful. Additionally we also can use the priorities as well on Level 2 and > beyond, as useful ;) > > > > > >Summarizing I think we need both mechanisms. > > > > > > > > > >2.) In DeltaSpike each property file could define it's own ordinal with a > simple entry in the property file. That way you can just add a JAR which > overrides a default config from another jar. Would each of them get an own > name? Or do they all share the same name and priority? > >> > >I would not define priorities on single file level, but only on property > source level. I can still an additional property source and override the > keys I want on source level. But as I said, do we really want application > deploy an additional jar, hereby completely overriding any kind of config. > I would say, no. But we can define (company specific mechanism), how and > when config can be overridden as part of the metamodel. > > > >3.) How do you deal with modularity? E.g. adding a dependency which needs > some other ConfigSource but knows nothing about your code? > >> > >>4.) Another modularity question: What about having multiple jars > (different independent framework parts) and each of them likes to have > another PropertySouceBuilder configuration? > >> > >Use different property source names on top level. > > > > > > > >>5.) How do you allow containers and framework programmers to add > PropertySources which affect all the project? > >>They define the metamodel and the extension points by the > implementation of the configuration metamodel. This metamodel as I said > should not be dynamic. > >> > >>6.) > >> > >>> - you can override along the classloader hierarchy (system, ear, war) > >> > >>> by adding resources to the classpath at xxx.Usually this does not work > well as e.g. all the JARs in WEB-INF/lib have exactly the same 'level'. > It's totally up to the ClassLoader mechanism in which order you get them. > And that could change on every restart. Been there, done that - doesn't > work. > >> > > > >I would never rely on the classloader ordering. Normally your config > within such a level should be consistent. If not you have multiple > possibilities: > >1) just take one. > >2) Just take one and log. > >3) throw a Deployment Error > >4) take a default > >5) Override the conflicting value on the next level of defined config, > so the conflict is not relevant anymore. How this is done, is defined by > your metamodel. > > > > > > > >7.) > >>> In code the do simply: Configuration config = > >>> Configuration.current("Stephen"); > >>And what if another jar does Configuration.current("Giovanni")? > >>Imo that approach doesn't scale. > >> > >> > >Normally you do Configuration.current(); // the default > > > > > >Configuration.current("name") is for specific isolated/modular > configuration sources (see 3). > > > > > >>I like to align our ideas, so I enumerate my main goals and design > principles of a config system as I envision it: > >> > >>A.) have a total separation between the configuration system and it's > consumers. The 'consumers' API should not know where the result comes from > and also not what the value is. I think this is the only way we can provide > an extensible system where e.g. the Operations team could 'overwrite' > certain values. If we require the userland code to know about the > PropertySources then we cannot do this. > >> > > > > > >Partially agree. Companies have their own way how they configure their > applications. Tamaya should be flexible enough to accomodate most of them, > so application programmers still have their guides, where and how to add > configuration files, resources etc. By adding additional levels of > configuration, e.g. on the deployment nodes, > >it is possible to override anything as needed. The ladder, yes, should be > completely transparent. > > > > > >B.) Have a flexible SPI to allow Container Vendors and also application > programmers to add their own PropertySources in a very easy way. > >> > >Yes and no (see security constraints). I think we have to discuss more > deeply the roles involved and what should be possible. > > > >C.) The SPI should be extensible via plugin mechanism. If a jar > containing a PropertySource gets added then it should automatically get > picked up for this very application. > >> > >If the metamodel foresees that to be so, yes. If not, not (see > secuirity...). > > > >Are we on the same page here? > >> > >> > > > > > >I hope! > > > > > >LieGrue, > >>strub > >> > >> > >> > >> > >> > >> > >>> On Saturday, 27 December 2014, 15:35, Anatole Tresch < > [email protected]> wrote: > >>> > Hi Gerhard > >>> > >>> all you say is true here, I also will not provide every kind of > combination > >>> logic, but I want to have a mechanism that is capable enough. > >>> > >>> @Mark end users don't see this mechanisms here. It used by the power > users, > >>> which define the "metamodel" how/where configuration is read > >>> and assembled, e.g. > >>> > >>> SPI: PropertySourceProvider { > >>> Collection<PropertySource> getPropertySources(); > >>> } > >>> > >>> Lets assume the provider is returning one property source named > >>> "Stephen". > >>> The power user would then probably write something like that: > >>> > >>> PropertySource src = > >>> PropertySourceBuilder.of("Stephen").addProviders(myProv2, > >>> myProv3).withAggregationPolicy(mySpecialPolicy) > >>> > >>> .addProvider(BranchSpecificOverrideFilter.getPropertySource()). > withAggregationPolicy(AggregationPolicy.OVERRIDE_AND_LOG) > >>> .addProvider(DefaultJavaEEProviders.getEarProvider()).build(); > >>> > >>> This src instance then is returned e.g. by the provider (could also be > >>> contextually per classloader etc.). So it is not only that the > mechanism is > >>> much more explicit and powerful, it > >>> is also easier to understand. There is exactly one location, where all > >>> happens: the PropertySourceProvider. You look at the code and > immediately > >>> now, what is happening. > >>> So the code that is assembling the final property source returned is > under > >>> full control of the users and nicely explicit. > >>> > >>> For normal users the mechanism is completely transparent the will > typically > >>> have some kind of documentation, where its written: > >>> > >>> - put your config files in the formats .ini or .xml at the location xy > >>> - you can override along the classloader hierarchy (system, ear, war) > by > >>> adding resources to the classpath at xxx. > >>> - Branch sepcific overrides are merge automatically into it, see > chapter > >>> 3.4 for additional details. > >>> - Finally it is possible to override entries an system leven, by > setting > >>> according system properties. > >>> > >>> In code the do simply: Configuration config = > >>> Configuration.current("Stephen"); > >>> > >>> You get it? > >>> > >>> > >>> > >>> 2014-12-27 15:13 GMT+01:00 Gerhard Petracek < > [email protected]>: > >>> > >>>> hi anatole, > >>>> > >>>> i guess everybody agrees that overriding based on ordinals wouldn't > be > >>>> enough for every possible use-case. > >>>> back then we had a long discussion about it. we ended up with adding > a > >>>> simple (and optional) ConfigFilter interface. > >>>> > >>>> @aggregation: > >>>> aggregation of key/value pairs is supported implicitly by an ordinal > based > >>>> approach. > >>>> aggregation of values >per key< can be done manually with an own > >>>> config-source (which delegates to the others and only aggregates > those > >>>> results). > >>>> yes - that way this use-case is more complicated out-of-the-box. > however, > >>>> imo if users (think they) need such a complicated config, they > should also > >>>> need to do a bit more for it. > >>>> (i guess in several cases they might question the requirement before > they > >>>> continue...) > >>>> > >>>> in general about possible use-cases (and features): > >>>> we can never address everything out-of-the-box. > >>>> a nice example is bean-validation. i could list a lot of cases which > >>> aren't > >>>> possible with that api/spi out-of-the-box, however, for the majority > >>> it's > >>>> good enough. > >>>> i saw several users who couldn't replace their existing > >>> validation-approach > >>>> (as it was). > >>>> however, several of them just simplified their validation-logic and > were > >>>> still happy about the result. > >>>> others extended bv e.g. via the spi and only few kept what they used > before > >>>> (e.g. because it wasn't possible with bv or the workaround was too > >>>> complicated > >>>> and they insisted on what they had). > >>>> > >>>> regards, > >>>> gerhard > >> > >>>> > >>>> > >>>> > >> > > > > > > > > > >-- > > > >Anatole Tresch > >Java Engineer & Architect, JSR Spec Lead > >Glärnischweg 10 > >CH - 8620 Wetzikon > > > > > >Switzerland, Europe Zurich, GMT+1 > >Twitter: @atsticks > >Blogs: http://javaremarkables.blogspot.ch/ > >Google: atsticks > >Mobile +41-76 344 62 79 > > > > >
