Hi Tharindu, On Mon, Feb 3, 2020 at 4:07 PM Tharindu Dharmarathna <tharin...@wso2.com> wrote:
> Hi Samitha, > > Please find my comments inline below. > > On Mon, Feb 3, 2020 at 3:29 PM Samitha Chathuranga <sami...@wso2.com> > wrote: > >> Hi, >> >> Additionally to this, I could see that there are still the below >> operations executed in the profile optimization script. >> >> * - Config changes in below config files* >> >> - api-manager.xml - enable_policy_deploy, enable_data_publishing, >> enable_blacklist_condition, >> enable_decision_connection (JMSConnectionDetails) >> - axis2.xml - transport.ws.sender.enable, transport.wss.sender.enable >> - registry.xml - indexing.enable >> >> These profile specific configurations are already handled by the >> server_role implementation in infer.json, so no need to do anything in >> profile optimization. >> >> *- Config file replacements* >> >> - axis2_TM.xml >> - axis2_KM.xml >> - registry_TM.xml >> >> This step will need to change the relevant .j2 file according to the > profile. Therefore we have to keep this step in optimizer as itits. > For config file replacement there are the functions replaceAxis2File(), replaceTenantAxis2File() and replaceRegistryXMLFile() in profileSetup.sh [1]. And for the replacement of relavent .jt template files, there are separate functions i.e. replaceAxis2TemplateFile(), replaceTenantAxis2TemplateFile(), replaceRegistryXMLTemplateFile(). So the two operations are independent in the script. So I don't see that the above statement is correct. So we can safely remove the "Config file replacements" AFAIU. Please clarify, if you meant something else. [1] https://github.com/wso2/product-apim/blob/master/modules/distribution/product/src/main/startup-scripts/profileSetup.sh Regards, Samitha > >> - >> >> These operations are not required since, the config file replacement >> won't be effective with the new toml based config model. And the related >> template files are already been replaced via the optimization script. >> >> So I will proceed to remove these redundant operations, within this >> effort itself. >> >> Regards, >> Samitha >> >> >> On Fri, Jan 31, 2020 at 11:31 AM Samitha Chathuranga <sami...@wso2.com> >> wrote: >> >>> And regarding name of the option; >>> >>>> *What is the ideal name for the new option (<new-option>)?* >>>>> Suggestions: --override, --overrideConfig >>>>> >>>> >>>> How about --skipConfigOptimization ? >>>> >>> +1 for this >>> And if we use this, the default behavior should be overriding the >>> configs. >>> >>> >>> On Fri, Jan 31, 2020 at 9:49 AM Samitha Chathuranga <sami...@wso2.com> >>> wrote: >>> >>>> Hi Pubudu, >>>> >>>> IMHO, I prefer to skip the config override when running the profile >>>>> optimization. What if we make this as the default approach? If someone >>>>> wants to override the configs, then with profile optimization, they can >>>>> use >>>>> a new flag called --override or something similar as Samitha mentioned. >>>>> >>>> If we make the default behavior to not-override, then in non >>>> Docker/K8s scenarios the profile optimization will be done incompletely. We >>>> have to think about most generic scenario of profile optimization. Is >>>> Docker/K8s the most generic scenario of customer use cases? I assume not. >>>> >>>> And in the perspective that "the users expected to run the optimization >>>> first and then do the configuration", overriding the configs should be the >>>> default option. >>>> >>>> And thinking about preserving the same old behavior too, as Bhathiya >>>> suggested, overriding configs should be ideal. >>>> >>>> Regards, >>>> Samitha >>>> >>>> On Thu, Jan 30, 2020 at 10:28 AM Pubudu Gunatilaka <pubu...@wso2.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> If we check one of the existing profile toml [1], the values are very >>>>> common and it cannot be used in real production cases. Sample values >>>>> contain commonly used hostnames and admin credentials. If we are to use >>>>> this feature properly with configuration override, users have to change >>>>> these default templates. And then they can use profile optimize along with >>>>> configs override. >>>>> >>>>> IMHO, I prefer to skip the config override when running the profile >>>>> optimization. What if we make this as the default approach? If someone >>>>> wants to override the configs, then with profile optimization, they can >>>>> use >>>>> a new flag called --override or something similar as Samitha mentioned. >>>>> >>>>> In K8s, docker use cases we have a single docker image and we are >>>>> having profile based deployment.toml. We can bring the hostnames to the >>>>> deployment.toml as Nuwan suggested and this would simplify the >>>>> deployment.toml. When running the docker image, we can run the profile >>>>> optimize and it won't affect the deployment.tomls that come in config >>>>> maps. >>>>> At the moment, in docker images, we had to deal with this configuration >>>>> overridden issue. >>>>> >>>>> [1] - >>>>> https://github.com/wso2/product-apim/blob/master/modules/distribution/product/src/main/resources/conf/deployment-templates/api-publisher.toml >>>>> >>>>> Thank you! >>>>> >>>>> On Wed, Jan 29, 2020 at 2:10 PM Harsha Kumara <hars...@wso2.com> >>>>> wrote: >>>>> >>>>>> I'm thinking on real world scenario where customer will keep one base >>>>>> product archive and add the configs. isn't it easy for them to copy those >>>>>> files just to the profile optimizer tool location? >>>>>> >>>>>> On Wed, Jan 29, 2020 at 3:15 PM Bhathiya Jayasekara < >>>>>> bhath...@wso2.com> wrote: >>>>>> >>>>>>> Hi Samitha, >>>>>>> >>>>>>> On Wed, Jan 29, 2020 at 2:29 PM Samitha Chathuranga < >>>>>>> sami...@wso2.com> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Our profile optimization [1] script for 3.0.0 replaces [2] already >>>>>>>> available deployment.toml with a per-profile specific updated/optimized >>>>>>>> toml file. (i.e. api-devportal.toml, gateway-worker.toml, etc.). This >>>>>>>> process affects already configured setups, to lose the configurations. >>>>>>>> >>>>>>>> A user running the profile optimization might follow two approaches. >>>>>>>> >>>>>>>> 1. Run profile optimization on a vanilla pack. >>>>>>>> - So no any configurations done when the profile optimization >>>>>>>> is executed >>>>>>>> 2. Run profile optimization on a configured pack. >>>>>>>> - User/deployment specific configurations are already done in >>>>>>>> the deployment.toml file and user run the profile optimization after >>>>>>>> that. >>>>>>>> >>>>>>> >>>>>>> The user is expected to run the optimization first and then do the >>>>>>> configuration. The real problem here comes with Kubernetes, where we >>>>>>> pass >>>>>>> configs as config maps. These shouldn't be replaced by the optimizer, >>>>>>> because we expect the optimized configs to come via config maps. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> There is no any issue with the Approach-1. But the Approach-2 >>>>>>>> results in the above mentioned issue. A user might have made all his >>>>>>>> configurations in the deployment.toml file. But once the profile >>>>>>>> optimization is executed, he will lose all those config changes. >>>>>>>> >>>>>>>> So the suggested solution is to pass an argument/option when we run >>>>>>>> the optimizing script, which decides whether to replace the >>>>>>>> deployment.toml >>>>>>>> or not. >>>>>>>> >>>>>>>> sh <PRODUCT_HOME>/bin/wso2server.sh --optimize *--<new-option> >>>>>>>> *-Dprofile=api-publisher >>>>>>>> >>>>>>>> In this case, we have to assume(and mandate) that user has manually >>>>>>>> applied the profile optimization related configurations too, into the >>>>>>>> deployment.toml file while doing the other configurations. So if the >>>>>>>> "new >>>>>>>> option" decides not to override the deployment.toml file, the profile >>>>>>>> optimization steps other than deployment.toml changes will be applied. >>>>>>>> >>>>>>>> So we have two concerns on how we use this new option. >>>>>>>> >>>>>>>> *What is the ideal name for the new option (<new-option>)?* >>>>>>>> Suggestions: --override, --overrideConfig >>>>>>>> >>>>>>> >>>>>>> How about --skipConfigOptimization ? >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> *What is the default behavior of the new option?* >>>>>>>> - What is the default behavior if the new option is not passed when >>>>>>>> running the optimization.? >>>>>>>> >>>>>>> >>>>>>> It should be the current behavior. >>>>>>> >>>>>>> Thanks, >>>>>>> Bhathiya >>>>>>> >>>>>>> >>>>>>>> - Should it override the user-made configs with the >>>>>>>> profile-specific deployment toml or not? >>>>>>>> - Concerns: >>>>>>>> - If override the toml, users manual configs will be >>>>>>>> completely removed (may be unknowlingly too). >>>>>>>> - If do not override toml, the profile optimization will be >>>>>>>> done incompletely. Anyway we can echo a message conveying that the >>>>>>>> deployment toml was not overridiiden. >>>>>>>> >>>>>>>> Appreciate your input on this. >>>>>>>> >>>>>>>> [1] >>>>>>>> https://apim.docs.wso2.com/en/next/SetupAndInstall/DeployingWSO2APIManager/DistributedDeployment/product-profiles/ >>>>>>>> [2] >>>>>>>> https://github.com/wso2/product-apim/blob/master/modules/distribution/product/src/main/startup-scripts/profileSetup.sh#L302 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Samitha >>>>>>>> >>>>>>>> -- >>>>>>>> *Samitha Chathuranga* >>>>>>>> *Associate Technical Lead*, *WSO2 Inc.* >>>>>>>> lean.enterprise.middleware >>>>>>>> Mobile: +94715123761 >>>>>>>> >>>>>>>> [image: http://wso2.com/signature] <http://wso2.com/signature> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Bhathiya Jayasekara* | Technical Lead | WSO2 Inc. >>>>>>> (m) +94 71 547 8185 | (e) bhathiya-@t-wso2-d0t-com >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *Harsha Kumara* >>>>>> >>>>>> Technical Lead, WSO2 Inc. >>>>>> Mobile: +94775505618 >>>>>> Email: hars...@wso2.coim >>>>>> Blog: harshcreationz.blogspot.com >>>>>> >>>>>> GET INTEGRATION AGILE >>>>>> Integration Agility for Digitally Driven Business >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Pubudu Gunatilaka* | Technical Lead | WSO2 Inc. >>>>> (m) +94774078049 | (w) +94112145345 | (e) pubu...@wso2.com >>>>> <http://wso2.com/signature> >>>>> >>>>> >>>> >>>> -- >>>> *Samitha Chathuranga* >>>> *Associate Technical Lead*, *WSO2 Inc.* >>>> lean.enterprise.middleware >>>> Mobile: +94715123761 >>>> >>>> [image: http://wso2.com/signature] <http://wso2.com/signature> >>>> >>> >>> >>> -- >>> *Samitha Chathuranga* >>> *Senior Software Engineer*, *WSO2 Inc.* >>> lean.enterprise.middleware >>> Mobile: +94715123761 >>> >>> [image: http://wso2.com/signature] <http://wso2.com/signature> >>> >> >> >> -- >> *Samitha Chathuranga* >> *Associate Technical Lead*, *WSO2 Inc.* >> lean.enterprise.middleware >> Mobile: +94715123761 >> >> [image: http://wso2.com/signature] <http://wso2.com/signature> >> > > > Thanks > > *Tharindu Dharmarathna*Technical Lead > WSO2 Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: *+94779109091* > -- *Samitha Chathuranga* *Associate Technical Lead*, *WSO2 Inc.* lean.enterprise.middleware Mobile: +94715123761 [image: http://wso2.com/signature] <http://wso2.com/signature>
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
