Hi Isuru, That time is fine by me.
thanks, On Thu, Jul 21, 2016 at 3:44 PM, Isuru Haththotuwa <isu...@wso2.com> wrote: > Hi Osura, > > On Thu, Jul 21, 2016 at 11:15 AM, Osura Rathnayake <osura...@gmail.com> > wrote: > >> Hi Mentors, >> >> I will try using Puppet. >> It wasn't a problem with log path/pattern, in fact I used the same log >> path that I used last time. I believe it was a bug from Azure side, please >> check the attached screenshots. >> Shall we please have the meeting on Friday? >> > +1. How about 2.00 - 3.0.0 PM on Friday? > >> >> thank you, >> >> On Wed, Jul 20, 2016 at 6:50 PM, Isuru Haththotuwa <isu...@wso2.com> >> wrote: >> >>> Hi Osura, >>> >>> Shall we have a hangout on Thursday/ Friday to discuss and demonstrate >>> the current progress of the project? Please let us know your preference. >>> >>> On Wed, Jul 20, 2016 at 3:35 PM, Imesh Gunaratne <im...@wso2.com> wrote: >>> >>>> Hi Osura, >>>> >>>> On Mon, Jul 18, 2016 at 4:20 PM, Osura Rathnayake <osura...@gmail.com> >>>> wrote: >>>> >>>>> Hi Mentors, >>>>> >>>>> This is the progress so far. >>>>> >>>>> *Auto scaling* >>>>> >>>>> >>>>> Azure supports two paradigms of auto scaling, vertical and horizontal. >>>>> Vertical scaling, also known as scale up and scale down, means increasing >>>>> or decreasing virtual machine (VM) sizes in response to a workload. As I >>>>> explained in one of my previous emails, vertical auto scaling is achieved >>>>> through monitoring rules. We can set it to trigger an alarm when certain >>>>> conditions are met and also as the action we can set up a runbook to scale >>>>> up or down. I could successfully get VMs to scale up and down using this >>>>> feature. >>>>> >>>> >>>> We would use horizontal scaling in most scenarios. >>>> >>>> >>>>> >>>>> Horizontal scaling, also referred to as scale out and scale in, where >>>>> the number of VMs is altered depending on the workload. Horizontal scaling >>>>> is achieved through virtual machine scale sets (VMSS). One important thing >>>>> about VMSS is that the VMs included should be of the same size and have >>>>> the >>>>> same OS image. All the VMs in the scale set are added to the load >>>>> balancer, >>>>> as a backend pool. Backend pool is a pool of VMs which share the traffic >>>>> coming via the load balancer. We can add auto scale rules, as to when >>>>> additional VMs should be added and removed, based on the conditions. I >>>>> could test some auto scale rules and observed VMs getting added to the >>>>> backend pool. But one problem is that when these VMs getting added, it’s a >>>>> whole new VM. I’m trying to add custom made VMs which has a wso2 product >>>>> installed and configured. >>>>> >>>> >>>> Yes we would need a VM image with required WSO2 product and >>>> pre-requisites to test this. At WSO2 we use Puppet >>>> >>>> for automating the installation process [1]. With Puppet we can use >>>> one base VM image and start a VM for any WSO2 product. Puppet does the WSO2 >>>> product installation at the VM startup. >>>> >>>>> >>>>> Note: only a limited horizontal scaling features are supported in the >>>>> azure portal yet so I’m using REST API to create and manage auto scaling >>>>> features. >>>>> >>>>> >>>>> *Centralized logging* >>>>> >>>>> >>>>> I was able to get log to Log Analytics using custom logs. I have >>>>> collected logs generated from 2 wso2 application servers. You only have to >>>>> add respective VMs to the Log analytics and set the path where logs are >>>>> located. This feature was under public preview, which might have been one >>>>> reason why it didn’t work last time when I tried. >>>>> >>>> >>>> Was the issue with the log file path/pattern? Did it work once you >>>> remove that? >>>> >>>> [1] >>>> https://github.com/wso2/puppet-modules/wiki/Use-WSO2-Puppet-Modules-in-puppet-master-agent-Environment >>>> >>>> Thanks >>>> >>>>> >>>>> Thanks, >>>>> >>>>> On Mon, Jul 18, 2016 at 8:21 AM, Osura Rathnayake <osura...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi Isuru, >>>>>> >>>>>> Please accept my apologies I have messed up names in my last email. >>>>>> I'm not going to be available today due to an unavoidable circumstance so >>>>>> can we please have the meeting on Wednesday? Extremely sorry if it made >>>>>> any >>>>>> inconvenience. I will update you with a detailed email today for sure. >>>>>> >>>>>> Thanks in advance, >>>>>> >>>>>> On Fri, Jul 15, 2016 at 4:04 PM, Isuru Haththotuwa <isu...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Osura, >>>>>>> >>>>>>> Can you send a detailed updated on the current status? Shall we have >>>>>>> a meeting on Monday to review the progress. >>>>>>> >>>>>>> On Tue, Jul 12, 2016 at 2:03 PM, Osura Rathnayake < >>>>>>> osura...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi Imesh, >>>>>>>> >>>>>>>> About dynamically adding members to the load balancer, I will cross >>>>>>>> check it with auto-scaling. I couldn't look into that from >>>>>>>> auto-scaling end >>>>>>>> since I couldn't test it yet. >>>>>>>> >>>>>>>> about monitoring, yes we can do a POC on that. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> On Tue, Jul 12, 2016 at 12:53 PM, Imesh Gunaratne <im...@wso2.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> On Tue, Jul 12, 2016 at 12:09 PM, Osura Rathnayake < >>>>>>>>> osura...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Imesh, >>>>>>>>>> >>>>>>>>>> About centralized logging, I'm trying to get logs to the Log >>>>>>>>>> Analytics using few methods supported in azure. We can either parse >>>>>>>>>> logs in >>>>>>>>>> to syslogs and send to the Log Analytics or create custom logs >>>>>>>>>> specifying >>>>>>>>>> the logs location. As you said, logs shouldn't be in .txt extension, >>>>>>>>>> I got >>>>>>>>>> it clarified from a azure blog. Will update you soon after I could >>>>>>>>>> resolve >>>>>>>>>> it. >>>>>>>>>> >>>>>>>>>> Right, thanks for the update! >>>>>>>>> >>>>>>>>> >>>>>>>>>> No you can't dynamically add VMs to the load balancer. Backend >>>>>>>>>> pool, where all the VMs reside, should be predefined. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Technically that capability should be there. Otherwise we would >>>>>>>>> not be able to autoscale a server cluster dynamically. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> you can auto-scale using scale sets(I'm still researching about >>>>>>>>>> it), that's the equivalent of AWS auto scaling group . also you can >>>>>>>>>> scale >>>>>>>>>> up or down a VM if it exceeds a certain parameter like CPU usage, >>>>>>>>>> using >>>>>>>>>> monitoring rules. >>>>>>>>>> >>>>>>>>>> *Monitoring * >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Azure has a native monitoring tool which involves collecting and >>>>>>>>>> tracking metrics, analyzing log files, defining custom metrics and >>>>>>>>>> logging >>>>>>>>>> generated by specific applications or workloads running in Virtual >>>>>>>>>> Machines. Azure represents monitored data in a graphical manner using >>>>>>>>>> charts. Monitoring also facilitates triggering alarms when certain >>>>>>>>>> conditions are met and also it can be configured to take actions on >>>>>>>>>> the met >>>>>>>>>> conditions. Monitoring is done by the Diagnostic Extension and it has >>>>>>>>>> following capabilities. >>>>>>>>>> >>>>>>>>>> · Collects and uploads the system performance >>>>>>>>>> information from the Linux VM to the user's storage table, including >>>>>>>>>> diagnostic and syslog information. >>>>>>>>>> >>>>>>>>>> · Enables users to customize the data metrics that will >>>>>>>>>> be collected and uploaded. >>>>>>>>>> >>>>>>>>>> · Enables users to upload specified log files to a >>>>>>>>>> designated storage table. >>>>>>>>>> >>>>>>>>>> Note: Azure storage tables are a non-relational, key-value-pair, >>>>>>>>>> storage system suitable for storing massive amounts of unstructured >>>>>>>>>> data. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> We can add monitor rules so that when an alert triggers it >>>>>>>>>> notifies the admins via email. Furthermore we can set to take >>>>>>>>>> automated >>>>>>>>>> actions. Azure automate actions by running runbooks. A runbook is a >>>>>>>>>> set of >>>>>>>>>> tasks that perform some automated process in Azure Automation. We can >>>>>>>>>> create our own runbooks as well. Available runbooks include, >>>>>>>>>> >>>>>>>>>> · Restart VM >>>>>>>>>> >>>>>>>>>> · Stop VM >>>>>>>>>> >>>>>>>>>> · Remove VM >>>>>>>>>> >>>>>>>>>> · Scale up VM >>>>>>>>>> >>>>>>>>>> · Scale down VM >>>>>>>>>> >>>>>>>>>> When scaling up it sets the virtual machine to the next larger >>>>>>>>>> size within the size group and when scaling down it sets the virtual >>>>>>>>>> machine to the next smaller size within the size group. >>>>>>>>>> >>>>>>>>>> More about runbooks and automation [1] >>>>>>>>>> >>>>>>>>> >>>>>>>>> Sounds good, will us be able to do a POC on this? >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Auto scaling * >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Auto-scaling is the process of dynamically allocating the >>>>>>>>>> resources required by an application to match performance >>>>>>>>>> requirements. >>>>>>>>>> Virtual machine scale sets are an Azure Compute resource you can use >>>>>>>>>> to >>>>>>>>>> deploy and manage a set of identical VMs. With all VMs configured >>>>>>>>>> the same, >>>>>>>>>> VM scale sets are designed to support true auto-scale no >>>>>>>>>> pre-provisioning >>>>>>>>>> of VMs is required – and as such makes it easier to build large-scale >>>>>>>>>> services targeting big compute, big data, and containerized >>>>>>>>>> workloads [2]. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Note: I couldn’t practically do this as my azure free account >>>>>>>>>> lets me have only 4 cores and I have used all of them on my current >>>>>>>>>> deployment. I’m getting a new azure account from one of my friends >>>>>>>>>> in a day >>>>>>>>>> so hopefully I will do this on it and update you. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Great! Thanks! >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://azure.microsoft.com/en-us/documentation/articles/automation-intro/ >>>>>>>>>> >>>>>>>>>> [2] >>>>>>>>>> https://azure.microsoft.com/en-us/documentation/articles/virtual-machine-scale-sets-overview/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Jul 11, 2016 at 9:54 AM, Imesh Gunaratne <im...@wso2.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> On Thu, Jul 7, 2016 at 7:11 PM, Osura Rathnayake < >>>>>>>>>>> osura...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Mentors, >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> In addition to refining the membership scheme code, I looked >>>>>>>>>>>> into following features of Azure. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> Good findings Osura, please find few questions inline: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> *Azure dynamic load balancing* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Azure has a native load balancer which is very easy to >>>>>>>>>>>> configure. It’s a layer 4 (TCP, UDP) load balancer which helps to >>>>>>>>>>>> spread >>>>>>>>>>>> traffic among healthy virtual machines. Following are some key >>>>>>>>>>>> terms you >>>>>>>>>>>> need to know. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> Can members be dynamically added and removed to/from a load >>>>>>>>>>> balancer? To check this we may need to explore how autoscaling >>>>>>>>>>> works. On >>>>>>>>>>> AWS this is handled with autoscaling groups [3] >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> *Capturing Virtual Machine Images as templates* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Azure provides the feature of generalizing and capturing >>>>>>>>>>>> virtual machines so that they can be used as templates. This is >>>>>>>>>>>> very useful >>>>>>>>>>>> and time saving when the production environment has many instances >>>>>>>>>>>> of the >>>>>>>>>>>> same kind of virtual machine. When the virtual machine is being >>>>>>>>>>>> generalized >>>>>>>>>>>> all the data in user directories are erased so better to have wso2 >>>>>>>>>>>> product >>>>>>>>>>>> directory not in "/home/*". More about this can be found here [2]. >>>>>>>>>>>> >>>>>>>>>>>> Once the virtual machine is captured, it is stored in the >>>>>>>>>>>> storage account that is associated with the virtual machine. You >>>>>>>>>>>> can either >>>>>>>>>>>> download this or use directly by referring to the URI when you >>>>>>>>>>>> want to make >>>>>>>>>>>> other virtual machines with this template. What would be awesome >>>>>>>>>>>> is if we >>>>>>>>>>>> can fully configure the virtual machine with a given product and >>>>>>>>>>>> make it >>>>>>>>>>>> available to users. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> Yes, this is mandatory. Otherwise we would not be able to >>>>>>>>>>> autoscale a server cluster. >>>>>>>>>>> >>>>>>>>>>> I'm sorry I may have missed, how did it go with centralized >>>>>>>>>>> logging? >>>>>>>>>>> >>>>>>>>>>> [3] >>>>>>>>>>> http://docs.aws.amazon.com/autoscaling/latest/userguide/AutoScalingGroup.html >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> [1] >>>>>>>>>>>> https://azure.microsoft.com/en-us/documentation/articles/load-balancer-overview/ >>>>>>>>>>>> >>>>>>>>>>>> [2] >>>>>>>>>>>> https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-capture-image/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jul 6, 2016 at 12:18 PM, Osura Rathnayake < >>>>>>>>>>>> osura...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Akila, >>>>>>>>>>>>> >>>>>>>>>>>>> Please refer to the screenshots that I have attached. When I >>>>>>>>>>>>> updated localMemberPort to 4200, I can see it being reflected in >>>>>>>>>>>>> logs when >>>>>>>>>>>>> members are joining. So should I still make modifications in the >>>>>>>>>>>>> code? >>>>>>>>>>>>> >>>>>>>>>>>>> .gitignore was added. >>>>>>>>>>>>> >>>>>>>>>>>>> okay I will write test cases in testNG and update >>>>>>>>>>>>> >>>>>>>>>>>>> thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Jul 6, 2016 at 9:06 AM, Akila Ravihansa Perera < >>>>>>>>>>>>> raviha...@wso2.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Jul 5, 2016 at 11:38 AM, Osura Rathnayake < >>>>>>>>>>>>>> osura...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Akila, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please check the modified code. It now takes the value which >>>>>>>>>>>>>>> is specified as localMemberPort in axis2.xml. >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I still don't see any change to the logic of how member >>>>>>>>>>>>>> address is calculated. Can you double check? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please make sure "target/**" directories are ignored from >>>>>>>>>>>>>> .gitignore. These shouldn't be in the repo [1]. You might also >>>>>>>>>>>>>> need to >>>>>>>>>>>>>> ignore any IDE specific files. Have a look at .gitignore in >>>>>>>>>>>>>> Kubernetes >>>>>>>>>>>>>> artifacts [2]. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I see that you have committed some test cases based on JUnit. >>>>>>>>>>>>>> Please note that as a platform we are moving to testng framework >>>>>>>>>>>>>> so better >>>>>>>>>>>>>> to use that. >>>>>>>>>>>>>> @Imesh, Isuru: Please correct me if I'm wrong. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Shall we get a repo created under wso2-incubator for this? >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] >>>>>>>>>>>>>> https://github.com/osuran/azure-membership-scheme/tree/master/target >>>>>>>>>>>>>> [2] >>>>>>>>>>>>>> https://github.com/wso2/kubernetes-artifacts/blob/master/.gitignore >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Akila Ravihansa Perera >>>>>>>>>>>>>> WSO2 Inc.; http://wso2.com/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Blog: http://ravihansa3000.blogspot.com >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Osura Rathnayake >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Regards, >>>>>>>>>>>> Osura Rathnayake >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *Imesh Gunaratne* >>>>>>>>>>> Software Architect >>>>>>>>>>> WSO2 Inc: http://wso2.com >>>>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>>>>>>>> W: https://medium.com/@imesh TW: @imesh >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Regards, >>>>>>>>>> Osura Rathnayake >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Imesh Gunaratne* >>>>>>>>> Software Architect >>>>>>>>> WSO2 Inc: http://wso2.com >>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>>>>>> W: https://medium.com/@imesh TW: @imesh >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Osura Rathnayake >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thanks and Regards, >>>>>>> >>>>>>> Isuru H. >>>>>>> +94 716 358 048* <http://wso2.com/>* >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Osura Rathnayake >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> Osura Rathnayake >>>>> >>>> >>>> >>>> >>>> -- >>>> *Imesh Gunaratne* >>>> Software Architect >>>> WSO2 Inc: http://wso2.com >>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>> W: https://medium.com/@imesh TW: @imesh >>>> >>>> >>> >>> >>> -- >>> Thanks and Regards, >>> >>> Isuru H. >>> +94 716 358 048* <http://wso2.com/>* >>> >>> >>> >> >> >> -- >> Regards, >> Osura Rathnayake >> > > > > -- > Thanks and Regards, > > Isuru H. > +94 716 358 048* <http://wso2.com/>* > > > -- Regards, Osura Rathnayake
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
