Please note I've pushed the meeting again to Thursday based on availability. Sorry for the constant notifications.
Regards, Chamila de Alwis Committer and PMC Member - Apache Stratos Senior Software Engineer | WSO2 Blog: https://medium.com/@chamilad On Mon, Aug 28, 2017 at 10:39 AM, Chamila De Alwis <chami...@wso2.com> wrote: > I've rescheduled the meeting to 30th August (Wednesday) 1pm-2pm, as Lakmal > is unavailable today. > > > Regards, > Chamila de Alwis > Committer and PMC Member - Apache Stratos > Senior Software Engineer | WSO2 > Blog: https://medium.com/@chamilad > > > > On Wed, Aug 23, 2017 at 5:31 PM, Chamila De Alwis <chami...@wso2.com> > wrote: > >> I agree. We could omit Secure Vaulted or possible fields like "password" >> values from being logged. >> >> I've scheduled a meeting on 28th Monday 2.00PM. Let's consolidate these >> into a plan then. >> >> >> Regards, >> Chamila de Alwis >> Committer and PMC Member - Apache Stratos >> Senior Software Engineer | WSO2 >> Blog: https://medium.com/@chamilad >> >> >> >> On Wed, Aug 23, 2017 at 5:26 PM, Nuwan Dias <nuw...@wso2.com> wrote: >> >>> Logging might be tricky. Some of these configs we're talking about are >>> related to credentials. We shouldn't log credentials in any place. So we >>> probably will have to find a way to omit logging certain variables if we go >>> down the path of logging. >>> >>> On Wed, Aug 23, 2017 at 5:21 PM, Chamila De Alwis <chami...@wso2.com> >>> wrote: >>> >>>> I had an offline chat with IsuruH and Jayanga, during which following >>>> points came into light. >>>> >>>> 1. To address the troubleshooting story, a separate log can be written >>>> at the startup where the ConfigProvider executes that would contain the >>>> configurations overridden for the particular product. The security of this >>>> log should be handled in the same way for other logs. For debugging, this >>>> log file can be requested from a deployment. >>>> 2. There is a possibility that product start-up time might degrade if >>>> this logging is introduced. >>>> 3. Not all configurations need to be set via environment variables. >>>> There can be an attribute in the config annotation which denotes the >>>> environment variable that can be used to override the particular >>>> configuration. This can also help in filling the documentation gap that is >>>> introduced, i.e. the list of environment variables that can be used in a >>>> particular product can be autogenerated when configuration doc generation >>>> story is completed. >>>> 4. Complex types do not need to be supported in environment variable >>>> story. This would vary depending on the likelihood that configuration would >>>> be overridden through an environment variable. >>>> >>>> Let's schedule a meeting to discuss the concerns, including the above, >>>> and come to a conclusion. >>>> >>>> >>>> Regards, >>>> Chamila de Alwis >>>> Committer and PMC Member - Apache Stratos >>>> Senior Software Engineer | WSO2 >>>> Blog: https://medium.com/@chamilad >>>> >>>> >>>> >>>> On Wed, Aug 23, 2017 at 3:01 PM, Chamila De Alwis <chami...@wso2.com> >>>> wrote: >>>> >>>>> IMO logging values would be a security risk. It would be more >>>>> manageable if information on where certain values are retrieved from can >>>>> be >>>>> displayed if that log level is enabled. >>>>> >>>>> >>>>> Regards, >>>>> Chamila de Alwis >>>>> Committer and PMC Member - Apache Stratos >>>>> Senior Software Engineer | WSO2 >>>>> Blog: https://medium.com/@chamilad >>>>> >>>>> >>>>> >>>>> On Wed, Aug 23, 2017 at 3:00 PM, Isuru Haththotuwa <isu...@wso2.com> >>>>> wrote: >>>>> >>>>>> >>>>>> >>>>>> On Wed, Aug 23, 2017 at 2:46 PM, Lakmal Warusawithana < >>>>>> lak...@wso2.com> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Aug 23, 2017 at 2:39 PM, Isuru Haththotuwa <isu...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> While I fully agree that its necessary to be able to inject configs >>>>>>>> via environment variables to be container friendly, I think Jayanga do >>>>>>>> have >>>>>>>> a point. When we are troubleshooting an issue, the first thing to >>>>>>>> check is >>>>>>>> the configurations. When up configuration values are looked up via >>>>>>>> environment variables, how do we know what are the configuration >>>>>>>> parameters >>>>>>>> that are used in the system? One possible option is to write to a log >>>>>>>> as >>>>>>>> Chamila has mentioned, or else write the currently used config params >>>>>>>> to a >>>>>>>> log file at startup. WDYT? However this does not take into account the >>>>>>>> possibility of changing an environment value even after the server has >>>>>>>> started. >>>>>>>> >>>>>>> >>>>>>> @Isuru, it is coming with correct namespace right? and first >>>>>>> priority is given to environment variables. So there should not be an >>>>>>> issue >>>>>>> with identifying the our config from environment variables IMO. >>>>>>> >>>>>> Yes. What I want to highlight is the troubleshooting perspective. We >>>>>> can have a particular set of config values coming from environment >>>>>> variables, another set of config values from deployment.yaml and defaults >>>>>> from the code. If we have log file file with all the currently used >>>>>> configurations values (irrespective of whether they are coming from >>>>>> environment variables, deployment.yaml or defaults in code), which can be >>>>>> written at server startup, it would be useful for the troubleshooting >>>>>> purpose IMHO. >>>>>> >>>>>>> >>>>>>>> On Wed, Aug 23, 2017 at 1:35 PM, Chamila De Alwis < >>>>>>>> chami...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Hi Jayanga, >>>>>>>>> >>>>>>>>> From Support's point of view, requesting all the data that can >>>>>>>>> have an effect is not an overhead IMO. Especially since we already do >>>>>>>>> that >>>>>>>>> now, we request for possible environmental factors, at least after all >>>>>>>>> other efforts are exhausted. When this change is introduced, we would >>>>>>>>> not >>>>>>>>> additionally ask for anything. >>>>>>>>> >>>>>>>>> Furthermore, environment variables would mainly be used in >>>>>>>>> scenarios where Container Cluster Management systems are used, in >>>>>>>>> which >>>>>>>>> case there is a clear cut definition of which environment variables >>>>>>>>> with >>>>>>>>> what values are being applied to Containers. This is how most systems >>>>>>>>> are >>>>>>>>> debugged in CMSs like Kubernetes even now, where commands like >>>>>>>>> `kubectl >>>>>>>>> describe` would give you all the runtime information. >>>>>>>>> >>>>>>>>> Non-Containerized environments like VMs would mostly use the file >>>>>>>>> based approach where it is easy to be coupled with Config Automation >>>>>>>>> tools >>>>>>>>> like Puppet. >>>>>>>>> >>>>>>>>> The concern of troubleshooting can be tackled by enabling a >>>>>>>>> verbose log that would display the environment variable name from >>>>>>>>> which a >>>>>>>>> certain value was retrieved, at the time configs are loaded. This >>>>>>>>> would >>>>>>>>> obviously have to be verbose than INFO, however it would greatly help >>>>>>>>> any >>>>>>>>> debugging process. This is how tools like Puppet handle environment >>>>>>>>> variable injected configs. >>>>>>>>> >>>>>>>>> The argument for file-less environment variables in the lookup >>>>>>>>> hierarchy is how quickly it becomes cumbersome to handle configuration >>>>>>>>> files across different Container images. Having files would soon >>>>>>>>> present us >>>>>>>>> with a situation similar to what we handle now in C4 with >>>>>>>>> configuration >>>>>>>>> repositories for different "patterns" of deployment in Containers, >>>>>>>>> that >>>>>>>>> requires a separate effort to be maintained for different releases. >>>>>>>>> Having >>>>>>>>> low-config convention-based environment variables support is a >>>>>>>>> valuable >>>>>>>>> addition to the C5 configuration lookup model since we've already >>>>>>>>> reduced >>>>>>>>> the number of configuration files. deployment.yaml will anyway be >>>>>>>>> there, >>>>>>>>> and I'm willing to bet that most clients would use that to make >>>>>>>>> configuration changes. However, there are a set of clients who would >>>>>>>>> require the convention-based environment variables for a >>>>>>>>> smoother integration to Containerization than others. >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Chamila de Alwis >>>>>>>>> Committer and PMC Member - Apache Stratos >>>>>>>>> Senior Software Engineer | WSO2 >>>>>>>>> Blog: https://medium.com/@chamilad >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Aug 23, 2017 at 1:03 PM, Jayanga Dissanayake < >>>>>>>>> jaya...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> @Nuwan, configs should be able to inject via >>>>>>>>>> environment properties. What I am saying is all these configs should >>>>>>>>>> be >>>>>>>>>> specified the in the deployment.yaml as {env:placeholder name}. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Jayanga. >>>>>>>>>> >>>>>>>>>> *Jayanga Dissanayake* >>>>>>>>>> Associate Technical Lead >>>>>>>>>> WSO2 Inc. - http://wso2.com/ >>>>>>>>>> lean . enterprise . middleware >>>>>>>>>> email: jaya...@wso2.com >>>>>>>>>> mobile: +94772207259 <+94%2077%20220%207259> >>>>>>>>>> <http://wso2.com/signature> >>>>>>>>>> >>>>>>>>>> On Wed, Aug 23, 2017 at 12:51 PM, Nuwan Dias <nuw...@wso2.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> @Jayanga, how do you make the server run elegantly on containers >>>>>>>>>>> without being able to inject properties via env variables? >>>>>>>>>>> >>>>>>>>>>> On Wed, Aug 23, 2017 at 12:43 PM, Jayanga Dissanayake < >>>>>>>>>>> jaya...@wso2.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> -1, for injecting configs without specifying it in the >>>>>>>>>>>> deployment.yaml >>>>>>>>>>>> >>>>>>>>>>>> Reason: When analyzing most of the issues we request config >>>>>>>>>>>> files (with C5 deployment.yaml) from the users to identify what >>>>>>>>>>>> are the >>>>>>>>>>>> configuration changes. If we have at-least the place holders in >>>>>>>>>>>> the config >>>>>>>>>>>> files, there is a guarantee that we know what are the injected >>>>>>>>>>>> configuration and we could request the actual values of the >>>>>>>>>>>> configs we need. >>>>>>>>>>>> >>>>>>>>>>>> But with the suggested approach, we have to request the >>>>>>>>>>>> 1)config files, 2)all the environment variables and get the union >>>>>>>>>>>> to >>>>>>>>>>>> identify the changed configurations. IMO an additional effort. >>>>>>>>>>>> >>>>>>>>>>>> Hence considering the overhead on the support front I am -1 for >>>>>>>>>>>> the suggested approach. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Jayanga. >>>>>>>>>>>> >>>>>>>>>>>> *Jayanga Dissanayake* >>>>>>>>>>>> Associate Technical Lead >>>>>>>>>>>> WSO2 Inc. - http://wso2.com/ >>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>> email: jaya...@wso2.com >>>>>>>>>>>> mobile: +94772207259 <+94%2077%20220%207259> >>>>>>>>>>>> <http://wso2.com/signature> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Aug 23, 2017 at 9:16 AM, Danesh Kuruppu < >>>>>>>>>>>> dan...@wso2.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Created git issue[1] to track this improvement. >>>>>>>>>>>>> >>>>>>>>>>>>> 1. https://github.com/wso2/carbon-config/issues/48 >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Aug 22, 2017 at 9:50 AM, Danesh Kuruppu < >>>>>>>>>>>>> dan...@wso2.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Chamila, >>>>>>>>>>>>>> >>>>>>>>>>>>>> +1 for the suggested approach. So we can have both ways to >>>>>>>>>>>>>> set environment variables either from placeholder ${env:<key>} >>>>>>>>>>>>>> where we can >>>>>>>>>>>>>> specify any key for the variable or from the new approach. For >>>>>>>>>>>>>> the new >>>>>>>>>>>>>> approach, we need adhere to a certain format as suggested. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> Danesh >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 5:59 PM, Chamila De Alwis < >>>>>>>>>>>>>> chami...@wso2.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In C5, the configuration lookup is done in the environment >>>>>>>>>>>>>>> variables, system properties, deployment.yaml file, and the >>>>>>>>>>>>>>> value provided >>>>>>>>>>>>>>> in the configuration bean class, in that order. However, it >>>>>>>>>>>>>>> looks like >>>>>>>>>>>>>>> environment variable lookup happens only if the particular >>>>>>>>>>>>>>> configuration is >>>>>>>>>>>>>>> noted to do so in the deployment.yaml, specifically if the >>>>>>>>>>>>>>> value of the >>>>>>>>>>>>>>> config is prefixed by a placeholder env:. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> gateWayEndpoint: ${env:gateWayEndpoint} >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> While this pattern may be easy to implement, it is >>>>>>>>>>>>>>> cumbersome to be followed in a Containerization approach and >>>>>>>>>>>>>>> IMO breaks the >>>>>>>>>>>>>>> 12Factor approach to configuration. This is because of the need >>>>>>>>>>>>>>> to edit a >>>>>>>>>>>>>>> physical file in order for the environment variables to be >>>>>>>>>>>>>>> queried. This >>>>>>>>>>>>>>> makes it harder to maintain a single Container Image for >>>>>>>>>>>>>>> different >>>>>>>>>>>>>>> deployments, and manipulate the Container runtimes via >>>>>>>>>>>>>>> environment >>>>>>>>>>>>>>> variables. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> AFAIU, this pattern emerges from the requirements in >>>>>>>>>>>>>>> SecureVault; having a ${sec:} prefix can mark the configuration >>>>>>>>>>>>>>> value to be >>>>>>>>>>>>>>> resolved through Secure Vault resolver. However IMO this, value >>>>>>>>>>>>>>> resolution, >>>>>>>>>>>>>>> is not the same as value lookup. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> IMO, the following should be the lookup order, without >>>>>>>>>>>>>>> having to mark a particular config in the deployment.yaml file. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 1. Environment variables: >>>>>>>>>>>>>>> PREFIX_wso2.NAMESPACE_CONFIGKEY="value", >>>>>>>>>>>>>>> PREFIX_wso2.NAMESPACE_CONFIGPARENT_CONFIG2="value2" >>>>>>>>>>>>>>> 2. deployment.yaml: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> wso2.NAMESPACE: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> CONFIG_KEY: value >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> CONFIG_PARENT: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> CONFIG2: valu2 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 3. Default value in the code >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This approach would be most Container friendly one as there >>>>>>>>>>>>>>> is no need to change files depending on deployment patterns, >>>>>>>>>>>>>>> environments, >>>>>>>>>>>>>>> etc. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> WDYT? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Chamila de Alwis >>>>>>>>>>>>>>> Committer and PMC Member - Apache Stratos >>>>>>>>>>>>>>> Senior Software Engineer | WSO2 >>>>>>>>>>>>>>> Blog: https://medium.com/@chamilad >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> >>>>>>>>>>>>>> *Danesh Kuruppu* >>>>>>>>>>>>>> Senior Software Engineer | WSO2 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Email: dan...@wso2.com >>>>>>>>>>>>>> Mobile: +94 (77) 1690552 <+94%2077%20169%200552> >>>>>>>>>>>>>> Web: WSO2 Inc <https://wso2.com/signature> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> >>>>>>>>>>>>> *Danesh Kuruppu* >>>>>>>>>>>>> Senior Software Engineer | WSO2 >>>>>>>>>>>>> >>>>>>>>>>>>> Email: dan...@wso2.com >>>>>>>>>>>>> Mobile: +94 (77) 1690552 <+94%2077%20169%200552> >>>>>>>>>>>>> Web: WSO2 Inc <https://wso2.com/signature> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>> Architecture@wso2.org >>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>> Architecture@wso2.org >>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Nuwan Dias >>>>>>>>>>> >>>>>>>>>>> Software Architect - WSO2, Inc. http://wso2.com >>>>>>>>>>> email : nuw...@wso2.com >>>>>>>>>>> Phone : +94 777 775 729 <+94%2077%20777%205729> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Architecture mailing list >>>>>>>>>>> Architecture@wso2.org >>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Architecture mailing list >>>>>>>>>> Architecture@wso2.org >>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thanks and Regards, >>>>>>>> >>>>>>>> Isuru H. >>>>>>>> +94 716 358 048 <+94%2071%20635%208048>* <http://wso2.com/>* >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Lakmal Warusawithana >>>>>>> Director - Cloud Architecture; WSO2 Inc. >>>>>>> Mobile : +94714289692 <071%20428%209692> >>>>>>> Blogs : https://medium.com/@lakwarus/ >>>>>>> http://lakmalsview.blogspot.com/ >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thanks and Regards, >>>>>> >>>>>> Isuru H. >>>>>> +94 716 358 048 <+94%2071%20635%208048>* <http://wso2.com/>* >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> >>> -- >>> Nuwan Dias >>> >>> Software Architect - WSO2, Inc. http://wso2.com >>> email : nuw...@wso2.com >>> Phone : +94 777 775 729 <+94%2077%20777%205729> >>> >> >> >
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
