While using XML a lot of it can be found that Multiconf did in Python.
(I am pretty sure most of Spring Config still uses plenty of XML, too, in
theory similar to Multiconf it could use a Groovy DSL, maybe if Pivotal
used Tamaya some day and the Groovy experts here considered such DSL a
worthy effort?;-D)
On Thu, Dec 4, 2014 at 2:53 PM, Tresch, Anatole <
anatole.tre...@credit-suisse.com> wrote:
> Hi Mark
>
> let me do a last try to clarifiy my ideas, I am open to discuss things
> (and do add some ideas at the end, so we can start the discussions right
> away ;) ):
>
> 1) my idea was to define a minimalistic interface for properties first. It
> should as simple as possible to implement. Just using Map<String,String> I
> found not optimal, since a Map requires several methods that are not easy
> to implement in all cases, such as size(). Also Spring provides such a
> similarl abstraction(PropertySource). For this I created PropertyProvider.
> 2) Looking at Configuration there is much more functionality needed to
> have a powerful config API, such as type support (getInt, getBoolean,
> getAdapted), registering of listeners, extension points (with, query). From
> our bank
> additional methods were added for evaluation of the key ranges present,
> assuming hereby a.b.d: a is an area, containing an area b containing a key
> d. So I defined the Configuration interface.
> 3) Now there are very common types of properties sources you want to read
> from. I personally identified: classpath resources, files and URLs/URIs. So
> I added some kind of factory class (PropertyProviders), which allows to
> easily
> construct them descriptively: PropertyProvider proc =
> PropertyProviders.fromPath("classpath:META-INF/cfg/config.ini");
> 4) In many companies you do not have a fixed configuration file name, you
> will have locations, where you consume all files (even recursively). So
> adding resource loader support similar to Spring was compelling:
> PropertyProvider proc =
> PropertyProviders.fromPath("classpath:META-INF/cfg/**/*.ini");
> 5) Looking at what we have in Credit Suisse and what other prople told
> they have in their companies, simple priority based adding of maps is not
> enough. There are cases, where I want to filter out values, prevent
> overriding of certain values etc. Also different locations (classpath,
> files, remote) are mixed and depending on the source different priorities
> may be assigned. Finally configuration pairs themselves can be attributed
> (default entries, global default entries, explicit entries, ..), which also
> lead to different configuration priorities within the same source: or even
> config file, e.g:
>
> <config>
> <defaults>
> <a>aValue</a>
> <b>bValue</b>
> </deault>
>
> <pluginConfig plugin="abc">
> <a>aValue2</a>
> </pluginConfig>
>
> <server ip="1.2.4.5">
> <b>bValue2</b>
> </server>
> </config>
>
> Since I saw definitively the most complex scenarios I wanted to support
> power users doing that kind of modelling more easily. One mechanism to
> achive this is allowing to aggregate and combine maps (=providers), so
> I added the combination methods on PropertyProviders like aggregate,
> intersect, filter, ...
> Now this seems more complicated for you guys than it seems for me ;) I can
> imagine different things that may be simplified:
>
> - Remove PropertyProvider/ PropertyProviders from the API part and just
> add corresponding functionality using a ConfigurationBuilder:
>
> Configuration config =
> ConfigurationBuilder.create("myConfig").addPaths("ab/B/B",
> "../fhgg.xml").with(AggregationPolicy.OVERRIDE).addPaths("jhbjh/kjhkjh/").build();
> Config aggregatedConfig = config.with(Aggregators.from(config);
>
> - We still could put the PropertyProvider to the core implementation
> module, or even add it as a separate module, for the ones that like a more
> leightweight implementation SPI ;)
>
> Configuration conf = Configuration.current(); will still be the accessor
> method of choice. Though I am well aware of the fact that the abstractions
> (SPIs) required to evaluate the right configuration during runtime is not
> yet discussed. Also the current state in the repo in that area is far away
> from finished (I currently try to cleanup a few things, especially
> artifacts that are duplicates...).
>
> Best,
> Anatole
>
>
>
> -----Original Message-----
> From: Mark Struberg [mailto:strub...@yahoo.de]
> Sent: Donnerstag, 4. Dezember 2014 11:08
> To: dev@tamaya.incubator.apache.org
> Subject: Re: UC: Combining PropertyProvider Instances
>
> I think I need a bit time to understand this. I've tried to grasp it for 2
> days now and this might mean that it is too complicated if not even I get
> it. Why are we doing this so complicated? Pulling in all the database logic
> into a tool which main target is that it should be really easy to use?
> What benefit does this add over hiding those details? e.g. in DeltaSpike I
> would just add a DatabaseConfigSource and be done. The user would not even
> see that it magically picks up the values from the db then.
>
>
> LieGrue,
> strub
>
>
>
>
>
> > On Tuesday, 2 December 2014, 14:02, Anatole Tresch <anat...@apache.org>
> wrote:
> > > Hi all
> > find following an additional part of use cases for comment. Note that
> this
> > area IMO definitively may be improved in several ways. Always keep in
> mind
> > that I try to setup the design guide similarly in parallel. It can be
> > easily accessed from the GH mirror:
> >
> >
> https://github.com/apache/incubator-tamaya/blob/master/docs/design/2_CoreConcepts.adoc
> >
> > Cheers,
> > Anatole
> >
> > Combining Property Providers
> >
> > Looking at the structures of configuration system used by large companies
> > we typically encounter some kind of configuration hierarchies that are
> > combined in arbitrary ways. Users of the systems are typically not aware
> of
> > the complexities in this area, since they simply know the possible
> > locations, formats and the overriding policies. Framework providers on
> the
> > other side must face the complexities and it would be very useful if
> Tamaya
> > can support here by providing prebuilt functionality that helps
> > implementing these aspects. All this leads to the feature set of
> combining
> > property providers. Hereby the following strategies are useful:
> >
> > -
> >
> > aggregating providers, hereby later providers added
> > -
> >
> > override any existing entries from earlier providers
> > -
> >
> > combine conflicting entries from earlier providers, e.g. into a
> > comma-separated structure.
> > -
> >
> > may throw a ConfigExcepotion ig entries are conflicting
> > -
> >
> > may only add entries not yet defined by former providers,
> preventing
> > entries that are already present to be overwritte
> > -
> >
> > any custom aggregation strategy, which may be a mix of above
> > -
> >
> > intersecting providers
> > -
> >
> > subtracting providers
> > -
> >
> > filtering providers
> >
> > These common functionality is provided by the PropertyProviders
> > singleton. Additionally to the base strategies above a MetaInfo instance
> > can be passed optionally as well to define the meta information for the
> > newly created provider instances. Let’s assume we have two property
> > providers with the following data:
> > Provider 1
> >
> > a=a
> > b=b
> > c=c
> > g=g
> > h=h
> > i=i
> >
> > Provider 2
> >
> > a=A
> > b=B
> > c=C
> > d=D
> > e=E
> > f=F
> >
> > Looking in detail you see that the entries a,b,c are present in both
> > providers, whereas d,e,f are only present in provider 1, and g,h,i only
> in
> > provider 2.
> > Example Combining PropertyProviders
> >
> > PropertyProvider provider1 = ...
> > PropertyProvider provider2 = ...
> > // aggregate, hereby values from provider 2 override values from
> provider 1
> > PropertyProvider unionOverriding =
> > PropertyProviders.aggregate(AggregationPolicy.OVERRIDE(), provider1,
> > provider2);*System*.out.println("unionOverriding: " +
> > unionOverriding);
> > // ignore duplicates, values present in provider 1 are not overriden
> > by provider 2
> > PropertyProvider unionIgnoringDuplicates =
> > PropertyProviders.aggregate(AggregationPolicy.IGNORE_DUPLICATES(),
> > provider1, provider2);*System*.out.println("unionIgnoringDuplicates: "
> > + unionIgnoringDuplicates);
> > // this variant combines/maps duplicate values into a new value
> > PropertyProvider unionCombined =
> > PropertyProviders.aggregate(AggregationPolicy.COMBINE(), provider1,
> > provider2);*System*.out.println("unionCombined: " + unionCombined);
> > // This variant throws an exception since there are key/value paris in
> > both providers, but with different values*try*{
> > PropertyProviders.aggregate(AggregationPolicy.EXCEPTION(),
> > provider1, provider2);
> > }*catch*(ConfigException e){
> > // expected!
> > }
> >
> > The example above produces the following outpout:
> > Example Combining PropertyProviders
> >
> > AggregatedPropertyProvider{
> > (name = dynamicAggregationTests)
> > a = "[a][A]"
> > b = "[b][B]"
> > c = "[c][C]"
> > d = "[D]"
> > e = "[E]"
> > f = "[F]"
> > g = "[g]"
> > h = "[h]"
> > i = "[i]"
> > }
> > unionOverriding: AggregatedPropertyProvider{
> > (name = <noname>)
> > a = "A"
> > b = "B"
> > c = "C"
> > d = "D"
> > e = "E"
> > f = "F"
> > g = "g"
> > h = "h"
> > i = "i"
> > }
> > unionIgnoringDuplicates: AggregatedPropertyProvider{
> > (name = <noname>)
> > a = "a"
> > b = "b"
> > c = "c"
> > d = "D"
> > e = "E"
> > f = "F"
> > g = "g"
> > h = "h"
> > i = "i"
> > }
> > unionCombined: AggregatedPropertyProvider{
> > (name = <noname>)
> > a = "a,A"
> > b = "b,B"
> > c = "c,C"
> > d = "D"
> > e = "E"
> > f = "F"
> > g = "g"
> > h = "h"
> > i = "i"
> > }
> >
> > No AggregationPolicy is also an interface that can be implemented:
> > AggregationPolicy Interface
> >
> > @FunctionalInterface*public* *interface* *AggregationPolicy* {
> > *String* aggregate(*String* key, *String* value1, *String* value2);
> > }
> >
> > So we can also define our own aggregation strategy using a Lambda
> > expression:
> > Use a Custom AggregationPolicy
> >
> > PropertyProvider provider1 = ...;
> > PropertyProvider provider2 = ...;
> > PropertyProvider props = PropertyProviders.aggregate(
> > (k, v1, v2) -> (v1 != null ? v1 : "") + '[' + v2 +
> > "]",
> > MetaInfo.of("dynamicAggregationTests"),
> > props1, props2);*System*.out.println(props);
> >
> > Additionally we also pass here an instance of MetaInfo. The output of
> > this code snippet is as follows:
> > Listing of dynamic aggregation policy
> >
> > AggregatedPropertyProvider{
> > (name = dynamicAggregationTests)
> > a = "[a][A]"
> > b = "[b][B]"
> > c = "[c][C]"
> > d = "[D]"
> > e = "[E]"
> > f = "[F]"
> > g = "[g]"
> > h = "[h]"
> > i = "[i]"
> > }
> >
> > Summarizing the PropertyProviders singleton allows to combine providers
> > in various forms:
> > Methods provided on PropertyProviders
> >
> > public final class PropertyProviders {
> >
> > private PropertyProviders() {}
> >
> > public static PropertyProvider fromArgs(String... args) {
> > public static PropertyProvider fromArgs(MetaInfo metaInfo, String...
> args)
> > {
> > public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, String... paths) {
> > public static PropertyProvider fromPaths(String... paths) {
> > public static PropertyProvider fromPaths(List<String> paths) {
> > public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, List<String> paths) {
> > public static PropertyProvider fromPaths(MetaInfo metaInfo,
> > List<String> paths) {
> > public static PropertyProvider fromPaths(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, List<String> paths) {
> > public static PropertyProvider fromUris(URI... uris) {
> > public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, URI... uris) {
> > public static PropertyProvider fromUris(List<URI> uris) {
> > public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, List<URI> uris) {
> > public static PropertyProvider fromUris(MetaInfo metaInfo, URI...
> uris)
> > {
> > public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, URI... uris) {
> > public static PropertyProvider fromUris(MetaInfo metaInfo, List<URI>
> > uris) {
> > public static PropertyProvider fromUris(AggregationPolicy
> > aggregationPolicy, MetaInfo metaInfo, List<URI> uris) {
> > public static PropertyProvider fromMap(Map<String, String> map) {
> > public static PropertyProvider fromMap(MetaInfo metaInfo,
> > Map<String, String> map) {
> > public static PropertyProvider empty() {
> > public static PropertyProvider emptyMutable() {
> > public static PropertyProvider empty(MetaInfo metaInfo) {
> > public static PropertyProvider emptyMutable(MetaInfo metaInfo) {
> > public static PropertyProvider fromEnvironmentProperties() {
> > public static PropertyProvider fromSystemProperties() {
> > public static PropertyProvider freezed(PropertyProvider provider) {
> > public static PropertyProvider aggregate(AggregationPolicy
> > mapping, MetaInfo metaInfo, PropertyProvider... providers){
> > public static PropertyProvider aggregate(PropertyProvider...
> providers)
> > {
> > public static PropertyProvider aggregate(List<PropertyProvider>
> > providers) {
> > public static PropertyProvider aggregate(AggregationPolicy
> > mapping, PropertyProvider... propertyMaps) {
> > public static PropertyProvider aggregate(AggregationPolicy
> > mapping, List<PropertyProvider> providers) {
> > public static PropertyProvider mutable(PropertyProvider provider) {
> > public static PropertyProvider intersected(AggregationPolicy
> > aggregationPolicy, PropertyProvider... providers) {
> > public static PropertyProvider intersected(PropertyProvider...
> providers)
> > {
> > public static PropertyProvider subtracted(PropertyProvider target,
> > PropertyProvider... providers) {
> > public static PropertyProvider filtered(Predicate<String> filter,
> > PropertyProvider provider) {
> > public static PropertyProvider
> > contextual(Supplier<PropertyProvider> mapSupplier,
> > Supplier<String>
> > isolationKeySupplier) {
> > public static PropertyProvider delegating(PropertyProvider
> > mainMap, Map<String, String> parentMap) {
> > public static PropertyProvider replacing(PropertyProvider mainMap,
> > Map<String, String> replacementMap) {
> > }
> >
>
