Hi Isuru,
Without duplicating files and adding new files, I would rather prefer to
use the following hierarchy. For each and every different environment we
can have product profiles and include the environment specific values. For
Kubernetes, we can include the clustering section only in manager.yaml file
which resides under environment Kubernetes. And the rest of the other
configurations can be included in the generic file folder, i.e "
wso2/%{::product_name}/%{::product_version}/%{::product_profile}".
:hierarchy:
- "node/%{::clientcert}" - - "
wso2/%{::product_name}/%{::product_version}/%{::environment}
/%{::product_profile}"
- "wso2/%{::product_name}/%{::product_version}/%{::product_profile}"
- "wso2/%{::product_name}/%{::product_version}/default"
- "osfamily/%{::osfamily}"
- "vm_type/%{::vm_type}"
- wso2/common
- common
Thank you!
On Mon, Apr 11, 2016 at 1:07 PM, Isuru Haththotuwa <isu...@wso2.com> wrote:
>
>
> On Mon, Apr 11, 2016 at 12:47 PM, Gayan Gunarathne <gay...@wso2.com>
> wrote:
>
>>
>>
>> On Mon, Apr 11, 2016 at 12:39 PM, Lakmal Warusawithana <lak...@wso2.com>
>> wrote:
>>
>>>
>>>
>>> On Mon, Apr 11, 2016 at 12:22 PM, Imesh Gunaratne <im...@wso2.com>
>>> wrote:
>>>
>>>> Hi Chamila,
>>>>
>>>> On Mon, Apr 11, 2016 at 7:19 AM, Chamila De Alwis <chami...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Isuru, Imesh,
>>>>>
>>>>> IMO we shouldn't have any platform specific restructuring in
>>>>> wso2/puppet-modules. This should be done at the end user's setup. Another
>>>>> point is that we have now decoupled wso2/puppet-modules from
>>>>> wso2/dockerfiles and users are not required to incorporate puppet-modules
>>>>> in their container setup.
>>>>>
>>>>
>>>> A very good concern! The way I see this is little different. Let's
>>>> evaluate the options we have:
>>>>
>>>> 1. Ship generic product/profile Hiera YAML files and let the users
>>>> configure them according to their platform (VM, AWS, Azure, OpenStack,
>>>> K8S,
>>>> Mesos, OpenShift, CF, etc)
>>>> 2. Ship product/profile/platform Hiera YAML files and let users use
>>>> them OOB with very few changes.
>>>>
>>>> +1 for 2nd option. Yes, it has some duplication on configurations, but
>>> customer POV, it is very easy to look at single place to do the minimum
>>> changes. (rather looking for many files in deferent folders to do the
>>> changes)
>>>
>>>
>>>
>>>
>>>> Which one would be the best option? IMO 2nd option would provide a much
>>>> better user experience compared to 1 as it provides platform specific
>>>> values such as clustering configuration & port mappings OOB. User will only
>>>> need to provide values such as database hosts, passwords, identity
>>>> management, etc which are user specific.
>>>>
>>>> The whole idea of this effort is to provide a better user experience.
>>>>
>>>> Thanks
>>>>
>>>>>
>>>>> IMO Docker images will not be able to run OOB on Kubernetes using
>>>>> wso2/puppet-modules and wso2/kubernetes-artifacts. There will anyway be
>>>>> changes related to the Kubernetes Membership Scheme in wso2/puppet-modules
>>>>> and in wso2/kubernetes-artifacts where environment dependent changes such
>>>>> as image names, SecureVault passwords, etc. need to be adjusted.
>>>>>
>>>>>
>>>>> Regards,
>>>>> Chamila de Alwis
>>>>> Committer and PMC Member - Apache Stratos
>>>>> Software Engineer | WSO2 | +94772207163
>>>>> Blog: code.chamiladealwis.com
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Apr 11, 2016 at 1:36 AM, Imesh Gunaratne <im...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Gayan,
>>>>>>
>>>>>> On Sun, Apr 10, 2016 at 5:02 PM, Gayan Gunarathne <gay...@wso2.com>
>>>>>> wrote:
>>>>>>
>>>>>>> IMO this will create maintainability issue. We need to maintain all
>>>>>>> the separate hieradata structure for each scenarios.For the one
>>>>>>> particular
>>>>>>> alternation we need to change whole set of files.
>>>>>>>
>>>>>>
>>>>>> In this scenario user experience is much more important than the
>>>>>> maintainability of few yaml files. If we do not do this, users will not
>>>>>> be
>>>>>> able to use puppet modules OOB until they manually update configuration
>>>>>> values in above files. The whole idea of this effort is to let users do
>>>>>> following:
>>>>>>
>>>>>> - Setup a K8S cluster
>>>>>> - Download puppet modules zip file(s).
>>>>>> - Download docker files
>>>>>> - Build docker images using puppet for different product profiles
>>>>>> - Deploy WSO2 product on K8S using K8S artifacts
>>>>>>
>>>>>> The above process will allow users to deploy any WSO2 product (with
>>>>>> mutlitple deployment patterns) on K8S with zero configurations. This will
>>>>>> be true for any VM based platform or any other container cluster
>>>>>> management
>>>>>> system.
>>>>>>
>>>>>
>> Mainly the target users group of the puppet/hiera files will be system
>> administrators/Dev Ops. So those guys will be consider the fact the
>> maintainability of puppet/hiera files. So if this is a maintainability
>> issue, it will become bad experience for the end user in end of the day.
>>
>>
>>>
>>>>>>> Why can't we do this by using defined types in Hiera and lookup
>>>>>>> parameters for a given instance? Based on the identify keys we set
>>>>>>> for each vm, docker, K8S etc we can select the appropriate data set
>>>>>>> from Hiera file.
>>>>>>>
>>>>>>
>>>>>> Will you be able to provide a sample?
>>>>>>
>>>>>
>>
>> I think we can make this with with Defined Types[1][2] without creating
>> duplicate set of YAML files for each platform. We can do the same as the
>> example given in the document.
>>
> I do not think we need to duplicate the yaml files here. Sorry that the
> sample in the first reply sent by me implied so. What we can do is refactor
> out the hiera data so that data which is changing according to the platform
> (ex.: clustering section) can be moved to a different yaml file(s). For an
> example, clustering for kubernetes scenario can be included in
> *puppet-modules/hieradata/dev/wso2/kubernetes/clustering.yaml*. Here,
> 'kubernetes' is the value of the facter which is used to identify the
> environment. Similarly, other such data can be refactored out.
>
>>
>> [1]
>> https://docs.puppet.com/puppet/latest/reference/lang_defined_types.html
>> [2]http://puppetlunch.com/puppet/hiera.html
>>
>>
>>>
>>>>>> Thanks
>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Gayan
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Apr 9, 2016 at 8:28 AM, Imesh Gunaratne <im...@wso2.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Apr 8, 2016 at 7:48 PM, Isuru Haththotuwa <isu...@wso2.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> hieradata
>>>>>>>>> |--- dev
>>>>>>>>> |--- wso2
>>>>>>>>> |---- <product_name>
>>>>>>>>> |--- <product_version>
>>>>>>>>> |-- *vm*
>>>>>>>>> |--
>>>>>>>>> default.yaml
>>>>>>>>> |--
>>>>>>>>> manager.yaml
>>>>>>>>> |--
>>>>>>>>> worker.yaml
>>>>>>>>> |--*
>>>>>>>>> docker*
>>>>>>>>> |--
>>>>>>>>> default.yaml
>>>>>>>>> |--
>>>>>>>>> manager.yaml
>>>>>>>>> |--
>>>>>>>>> worker.yaml
>>>>>>>>> |--
>>>>>>>>> *kubernetes*
>>>>>>>>> |--
>>>>>>>>> default.yaml
>>>>>>>>> |--
>>>>>>>>> manager.yaml
>>>>>>>>> |--
>>>>>>>>> worker.yaml
>>>>>>>>>
>>>>>>>>>
>>>>>>>> +1 for the suggestion Isuru, will proceed with this. We can add
>>>>>>>> other platforms such as OpenShift, Mesos, Cloud Foundry on the same
>>>>>>>> level.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> *Imesh Gunaratne*
>>>>>>>>>> Senior Technical Lead
>>>>>>>>>> WSO2 Inc: http://wso2.com
>>>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>>>>>>> W: http://imesh.io
>>>>>>>>>> Lean . Enterprise . Middleware
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks and Regards,
>>>>>>>>>
>>>>>>>>> Isuru H.
>>>>>>>>> +94 716 358 048* <http://wso2.com/>*
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> *Imesh Gunaratne*
>>>>>>>> Senior Technical Lead
>>>>>>>> WSO2 Inc: http://wso2.com
>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>>>>> W: http://imesh.io
>>>>>>>> Lean . Enterprise . Middleware
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Dev mailing list
>>>>>>>> Dev@wso2.org
>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Gayan Gunarathne
>>>>>>> Technical Lead, WSO2 Inc. (http://wso2.com)
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>> email : gay...@wso2.com | mobile : +94 775030545
>>>>>>> <%2B94%20766819985>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Dev mailing list
>>>>>>> Dev@wso2.org
>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Imesh Gunaratne*
>>>>>> Senior Technical Lead
>>>>>> WSO2 Inc: http://wso2.com
>>>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>>>> W: http://imesh.io
>>>>>> Lean . Enterprise . Middleware
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Dev mailing list
>>>>>> Dev@wso2.org
>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Imesh Gunaratne*
>>>> Senior Technical Lead
>>>> WSO2 Inc: http://wso2.com
>>>> T: +94 11 214 5345 M: +94 77 374 2057
>>>> W: http://imesh.io
>>>> Lean . Enterprise . Middleware
>>>>
>>>>
>>>> _______________________________________________
>>>> Dev mailing list
>>>> Dev@wso2.org
>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Director - Cloud Architecture; WSO2 Inc.
>>> Mobile : +94714289692
>>> Blog : http://lakmalsview.blogspot.com/
>>>
>>>
>>> _______________________________________________
>>> Dev mailing list
>>> Dev@wso2.org
>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>
>>>
>>
>>
>> --
>>
>> Gayan Gunarathne
>> Technical Lead, WSO2 Inc. (http://wso2.com)
>> Committer & PMC Member, Apache Stratos
>> email : gay...@wso2.com | mobile : +94 775030545 <%2B94%20766819985>
>>
>>
>>
>> _______________________________________________
>> Dev mailing list
>> Dev@wso2.org
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
> Thanks and Regards,
>
> Isuru H.
> +94 716 358 048* <http://wso2.com/>*
>
>
>
> _______________________________________________
> Dev mailing list
> Dev@wso2.org
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>
--
*Pubudu Gunatilaka*
Committer and PMC Member - Apache Stratos
Software Engineer
WSO2, Inc.: http://wso2.com
mobile : +94774078049 <%2B94772207163>
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
