+1 for the proposed approach. I will stick to the other thread. Thanks
On Mon, Mar 14, 2016 at 10:00 AM, Imesh Gunaratne <im...@wso2.com> wrote: > > > On Sun, Mar 13, 2016 at 11:37 PM, Nishadi Kirielle <nish...@wso2.com> > wrote: > >> Hi Imesh, >> The reason for choosing SSL termination over SSL pass through is due to >> the complexity of handling separate SSL certificates for each servers >> behind the load balancer in kubernetes cluster. As in App Cloud the >> kubernetes cluster is not direcctly exposed and the communication between >> the load balancer and servers happens internally, we have thought of >> choosing SSL termination approach over SSL pass through. >> > > -1 There is no way we can assume that internal networks are secure. Please > refer the other thread "Configuring load balancing in app cloud with HA > Proxy". > > FYI: I see two threads on the same topic. It might be better to keep them > in one. > > Thanks > >> >> >> Thanks >> >> On Thu, Mar 10, 2016 at 11:21 PM, Imesh Gunaratne <im...@wso2.com> wrote: >> >>> >>> >>> On Thu, Mar 10, 2016 at 1:28 PM, Nishadi Kirielle <nish...@wso2.com> >>> wrote: >>> >>>> Thank you for the suggestion of using the default self signed >>>> certificate. >>>> I have attempted SSL termination approach of terminating the SSL >>>> connection at the load balancer and sending unencrypted connections to the >>>> backend server via the ha proxy configuration of 'ssl verify none'. This >>>> approach allows https traffic to be load balanced and exposed. >>>> >>>> Terminating SSL at the middle of a communication flow would introduce >>> security risks. >>> >>> Thanks >>> >>> >>>> Thanks >>>> >>>> >>>> On Thu, Mar 10, 2016 at 11:20 AM, Imesh Gunaratne <im...@wso2.com> >>>> wrote: >>>> >>>>> >>>>> >>>>> On Thu, Mar 10, 2016 at 10:49 AM, Nishadi Kirielle <nish...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> I have only tested for http traffic earlier. Although the kubernetes >>>>>> service loadbalancer template has support for https, when I have deployed >>>>>> an application ( dell/tomcat ) which has the support for https, the ha >>>>>> proxy load balancer did not identify it as a https service in the haproxy >>>>>> configuration file. It just identified the application as a http >>>>>> application and updated the configuration file accordingly. >>>>>> >>>>> >>>>> Yes, in our K8S services we have defined the protocol as TCP, not as >>>>> HTTPS/SSL. Therefore there is no way for the service load balancer to find >>>>> this information by looking at the services. >>>>> >>>>> >>>>>> Thus I have manually altered the ha proxy configuration file to >>>>>> support for https traffic with a self signed certificate specific for the >>>>>> node ip. But it fails in accessing the application, since the application >>>>>> needs the self signed certificate specific to the application. >>>>>> As a solution for this I'm trying with bind option 'cert' to bind >>>>>> several certificate files[2] of the specific applications. >>>>>> >>>>> >>>>> Shall we try with the default self signed certificate distributed with >>>>> a WSO2 product? >>>>> >>>>> Thanks >>>>> >>>>>> >>>>>> Any suggestions on this are highly appreciated. >>>>>> [1] . >>>>>> https://github.com/kubernetes/contrib/tree/master/service-loadbalancer >>>>>> [2] . >>>>>> https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#5.1-crt >>>>>> >>>>>> Thanks >>>>>> >>>>>> On Wed, Mar 9, 2016 at 10:33 AM, Imesh Gunaratne <im...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Deep, >>>>>>> >>>>>>> On Tue, Mar 8, 2016 at 8:08 PM, Deependra Ariyadewa <d...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> On Mon, Mar 7, 2016 at 10:30 AM, Nishadi Kirielle <nish...@wso2.com >>>>>>>> > wrote: >>>>>>>> >>>>>>>>> Hi All, >>>>>>>>> I have written the blog post on load balancing and session >>>>>>>>> affinity in kubernetes. [1] >>>>>>>>> >>>>>>>> >>>>>>>> I am going test session affinity for HTTPS triffic in Kubernetes >>>>>>>> following your configurations. Did you try to enable session affinity >>>>>>>> for >>>>>>>> HTTPS triffic in Kubernetes. >>>>>>>> >>>>>>>> We would need to configure haproxy with relevant SSL certificates >>>>>>> for HTTPS to work. I do not think we tested it. See [1] for the haproxy >>>>>>> config template used by the service load balancer. This will get >>>>>>> packaged >>>>>>> to the Docker service load balancer Docker image [2]. >>>>>>> >>>>>>> [1] >>>>>>> https://github.com/kubernetes/contrib/blob/master/service-loadbalancer/template.cfg >>>>>>> [2] >>>>>>> https://github.com/kubernetes/contrib/blob/master/service-loadbalancer/Dockerfile >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> >>>>>>>> Thanks, >>>>>>>> Deependra. >>>>>>>> >>>>>>>>> >>>>>>>>> Thank you >>>>>>>>> >>>>>>>>> [1]. >>>>>>>>> http://nishadikirielle.blogspot.com/2016/03/load-balancing-kubernetes-services-and.html >>>>>>>>> >>>>>>>>> On Fri, Mar 4, 2016 at 8:22 PM, Nishadi Kirielle <nish...@wso2.com >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>>> Thanks a lot. I will write a blog post and share it. >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Mar 4, 2016 at 6:07 PM, Sagara Gunathunga < >>>>>>>>>> sag...@wso2.com> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Great, it would be better if Nishadi can write a step by step >>>>>>>>>>> blog post about how to do this. We had to do a 30 hours hackathon >>>>>>>>>>> to >>>>>>>>>>> change MSF4J Pet-store sample due to this issue :) >>>>>>>>>>> >>>>>>>>>>> Thanks ! >>>>>>>>>>> >>>>>>>>>>> On Fri, Mar 4, 2016 at 5:54 PM, Imesh Gunaratne <im...@wso2.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Indeed! Overall great effort!! >>>>>>>>>>>> >>>>>>>>>>>> Thanks >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Mar 4, 2016 at 3:36 PM, Lakmal Warusawithana < >>>>>>>>>>>> lak...@wso2.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Great work Nishadi! >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Mar 4, 2016 at 3:34 PM, Nishadi Kirielle < >>>>>>>>>>>>> nish...@wso2.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> In attempting to configure session affinity in kubernetes >>>>>>>>>>>>>> load balancing, I tried to run nginx alpha ingress controller[1] >>>>>>>>>>>>>> to expose >>>>>>>>>>>>>> the services through ingress. But the generated nginx >>>>>>>>>>>>>> configuration file >>>>>>>>>>>>>> were missing the service ports to access the services. Thus I >>>>>>>>>>>>>> have manually >>>>>>>>>>>>>> updated the configuration file to check the functionality of >>>>>>>>>>>>>> ingress. Since >>>>>>>>>>>>>> session affinity is available in haproxy, I have created a >>>>>>>>>>>>>> haproxy docker >>>>>>>>>>>>>> container and manually updated its configuration file in order >>>>>>>>>>>>>> to check its >>>>>>>>>>>>>> functionality with ingress. >>>>>>>>>>>>>> As per a suggestion of Imesh and Lakmal, I have tried >>>>>>>>>>>>>> kubernetes service loadbalancer repo[2]. There, they have >>>>>>>>>>>>>> developed the >>>>>>>>>>>>>> load balancing directly with pods, bypassing the services. This >>>>>>>>>>>>>> procedure >>>>>>>>>>>>>> corrects the session affinity problem in load balancing in >>>>>>>>>>>>>> kubernetes. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] . >>>>>>>>>>>>>> https://github.com/kubernetes/contrib/tree/master/ingress/controllers/nginx-alpha >>>>>>>>>>>>>> [2] . >>>>>>>>>>>>>> https://github.com/kubernetes/contrib/tree/master/service-loadbalancer >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Feb 29, 2016 at 12:35 PM, Imesh Gunaratne < >>>>>>>>>>>>>> im...@wso2.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Mon, Feb 29, 2016 at 12:12 PM, Lakmal Warusawithana < >>>>>>>>>>>>>>> lak...@wso2.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Feb 29, 2016 at 11:56 AM, Imesh Gunaratne < >>>>>>>>>>>>>>>> im...@wso2.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Lakmal, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Mon, Feb 29, 2016 at 11:37 AM, Lakmal Warusawithana < >>>>>>>>>>>>>>>>> lak...@wso2.com> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> What is we are tying to do here? Are we trying to verify >>>>>>>>>>>>>>>>>> the ClientIP when exposing service via NodePort? IMO its >>>>>>>>>>>>>>>>>> working without >>>>>>>>>>>>>>>>>> issue. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Yes the first step was to verify ClientIP and then try to >>>>>>>>>>>>>>>>> get an Ingress Controller either with nginx or haproxy >>>>>>>>>>>>>>>>> working with session >>>>>>>>>>>>>>>>> affinity. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> If you want to verify ClientIP, Udara has written very >>>>>>>>>>>>>>>> simple code, better to used that. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Guys, we need to resolve this very fast... too much time >>>>>>>>>>>>>>>> taking basic stuff, which we already verified :( >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We are on it Lakmal! Will resolve this ASAP. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Mon, Feb 29, 2016 at 11:37 AM, Lakmal Warusawithana < >>>>>>>>>>>>>>>>> lak...@wso2.com> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> What is we are tying to do here? Are we trying to verify >>>>>>>>>>>>>>>>>> the ClientIP when exposing service via NodePort? IMO its >>>>>>>>>>>>>>>>>> working without >>>>>>>>>>>>>>>>>> issue. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Sun, Feb 28, 2016 at 11:58 PM, Nishadi Kirielle < >>>>>>>>>>>>>>>>>> nish...@wso2.com> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> In order to test the session affinity in Kubernetes, I >>>>>>>>>>>>>>>>>>> have deployed WordPress on a kubernetes cluster with >>>>>>>>>>>>>>>>>>> several replicas and >>>>>>>>>>>>>>>>>>> enabled the session affinity by setting >>>>>>>>>>>>>>>>>>> service.spec.sessionAffinity to >>>>>>>>>>>>>>>>>>> "ClientIP". When the kubernetes service is exposed through >>>>>>>>>>>>>>>>>>> NodePort, I have >>>>>>>>>>>>>>>>>>> tested the accuracy of session affinity using Apache bench >>>>>>>>>>>>>>>>>>> mark for simple >>>>>>>>>>>>>>>>>>> load testing. With a load of 1000 requests and a maximum of >>>>>>>>>>>>>>>>>>> 2 requests >>>>>>>>>>>>>>>>>>> running concurrently, all requests returned successfully >>>>>>>>>>>>>>>>>>> without a failure. >>>>>>>>>>>>>>>>>>> Thus the session affinity is functioning properly when the >>>>>>>>>>>>>>>>>>> services are >>>>>>>>>>>>>>>>>>> exposed via NodePort. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The next attempt is to test the session affinity with >>>>>>>>>>>>>>>>>>> ingress API exposing the services. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Any feedback or suggestions are highly appreciated. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>> *Nishadi Kirielle* >>>>>>>>>>>>>>>>>>> *Software Engineering Intern* >>>>>>>>>>>>>>>>>>> Mobile : +94 (0) 714722148 >>>>>>>>>>>>>>>>>>> Blog : http://nishadikirielle.blogspot.com/ >>>>>>>>>>>>>>>>>>> nish...@wso2.comm >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>>>>>> Dev@wso2.org >>>>>>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>> Lakmal Warusawithana >>>>>>>>>>>>>>>>>> Director - Cloud Architecture; WSO2 Inc. >>>>>>>>>>>>>>>>>> Mobile : +94714289692 >>>>>>>>>>>>>>>>>> Blog : http://lakmalsview.blogspot.com/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>>>>> Dev@wso2.org >>>>>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> *Imesh Gunaratne* >>>>>>>>>>>>>>>>> Senior Technical Lead >>>>>>>>>>>>>>>>> WSO2 Inc: http://wso2.com >>>>>>>>>>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>>>>>>>>>>>>>> W: http://imesh.io >>>>>>>>>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Lakmal Warusawithana >>>>>>>>>>>>>>>> Director - Cloud Architecture; WSO2 Inc. >>>>>>>>>>>>>>>> Mobile : +94714289692 >>>>>>>>>>>>>>>> Blog : http://lakmalsview.blogspot.com/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> *Imesh Gunaratne* >>>>>>>>>>>>>>> Senior Technical Lead >>>>>>>>>>>>>>> WSO2 Inc: http://wso2.com >>>>>>>>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>>>>>>>>>>>> W: http://imesh.io >>>>>>>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> *Nishadi Kirielle* >>>>>>>>>>>>>> *Software Engineering Intern* >>>>>>>>>>>>>> Mobile : +94 (0) 714722148 >>>>>>>>>>>>>> Blog : http://nishadikirielle.blogspot.com/ >>>>>>>>>>>>>> nish...@wso2.com >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Lakmal Warusawithana >>>>>>>>>>>>> Director - Cloud Architecture; WSO2 Inc. >>>>>>>>>>>>> Mobile : +94714289692 >>>>>>>>>>>>> Blog : http://lakmalsview.blogspot.com/ >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Imesh Gunaratne* >>>>>>>>>>>> Senior Technical Lead >>>>>>>>>>>> WSO2 Inc: http://wso2.com >>>>>>>>>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>>>>>>>>> W: http://imesh.io >>>>>>>>>>>> Lean . Enterprise . Middleware >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Dev mailing list >>>>>>>>>>>> Dev@wso2.org >>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Sagara Gunathunga >>>>>>>>>>> >>>>>>>>>>> Architect; WSO2, Inc.; http://wso2.com >>>>>>>>>>> V.P Apache Web Services; http://ws.apache.org/ >>>>>>>>>>> Linkedin; http://www.linkedin.com/in/ssagara >>>>>>>>>>> Blog ; http://ssagara.blogspot.com >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Dev mailing list >>>>>>>>>>> Dev@wso2.org >>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Nishadi Kirielle* >>>>>>>>>> *Software Engineering Intern* >>>>>>>>>> Mobile : +94 (0) 714722148 >>>>>>>>>> Blog : http://nishadikirielle.blogspot.com/ >>>>>>>>>> nish...@wso2.com >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Nishadi Kirielle* >>>>>>>>> *Software Engineering Intern* >>>>>>>>> Mobile : +94 (0) 714722148 >>>>>>>>> Blog : http://nishadikirielle.blogspot.com/ >>>>>>>>> nish...@wso2.com >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> Dev@wso2.org >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Deependra Ariyadewa >>>>>>>> WSO2, Inc. http://wso2.com/ http://wso2.org >>>>>>>> >>>>>>>> email d...@wso2.com; cell +94 71 403 5996 ; >>>>>>>> Blog http://risenfall.wordpress.com/ >>>>>>>> PGP info: KeyID: 'DC627E6F' >>>>>>>> >>>>>>>> *WSO2 - Lean . Enterprise . Middleware* >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Dev mailing list >>>>>>>> Dev@wso2.org >>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Imesh Gunaratne* >>>>>>> Senior Technical Lead >>>>>>> WSO2 Inc: http://wso2.com >>>>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>>>> W: http://imesh.io >>>>>>> Lean . Enterprise . Middleware >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Nishadi Kirielle* >>>>>> *Software Engineering Intern* >>>>>> Mobile : +94 (0) 714722148 >>>>>> Blog : http://nishadikirielle.blogspot.com/ >>>>>> nish...@wso2.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Imesh Gunaratne* >>>>> Senior Technical Lead >>>>> WSO2 Inc: http://wso2.com >>>>> T: +94 11 214 5345 M: +94 77 374 2057 >>>>> W: http://imesh.io >>>>> Lean . Enterprise . Middleware >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Nishadi Kirielle* >>>> *Software Engineering Intern* >>>> Mobile : +94 (0) 714722148 >>>> Blog : http://nishadikirielle.blogspot.com/ >>>> nish...@wso2.com >>>> >>> >>> >>> >>> -- >>> *Imesh Gunaratne* >>> Senior Technical Lead >>> WSO2 Inc: http://wso2.com >>> T: +94 11 214 5345 M: +94 77 374 2057 >>> W: http://imesh.io >>> Lean . Enterprise . Middleware >>> >>> >> >> >> -- >> *Nishadi Kirielle* >> *Software Engineering Intern* >> Mobile : +94 (0) 714722148 >> Blog : http://nishadikirielle.blogspot.com/ >> nish...@wso2.com >> > > > > -- > *Imesh Gunaratne* > Senior Technical Lead > WSO2 Inc: http://wso2.com > T: +94 11 214 5345 M: +94 77 374 2057 > W: http://imesh.io > Lean . Enterprise . Middleware > > -- *Nishadi Kirielle* *Software Engineering Intern* Mobile : +94 (0) 714722148 Blog : http://nishadikirielle.blogspot.com/ nish...@wso2.com
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
