Nice explanation Tom. In addition to that, we will be able to run these containers in multiple pods within a kubernetes cluster when being used for an actual use case (production). This way, it will be easy to deploy, scale and manage. Under those conditions, we may be able to run one component per node as well with Kube-DNS is there to avoid hardcoding IPs and allow service discovery without a problem.
On Mon, 15 Oct 2018 at 23:55, Tom Barber <t...@spicule.co.uk> wrote: > internally as they only 1 process they should run with very little > overhead. > > > I can vouch also though that IO certainly sucks really really badly on my > Mac compared to linux box, and if SK reads this I'm sure he can also relay > some docker horror stories on Mac. > > On Mac it's better since they got rid of the virtualbox bridge, but it's > still pretty bad. > > Tom > > On Mon, 15 Oct 2018, 19:20 Chris Mattmann, <mattm...@apache.org> wrote: > > > Interesting OK. B/c the more containers for me on my local laptop > > typically eat up way more memory and CPU…but maybe that’s just > > me on a Mac lol > > > > > > > > > > > > > > > > > > > > From: Tom Barber <t...@spicule.co.uk> > > Reply-To: "dev@oodt.apache.org" <dev@oodt.apache.org> > > Date: Monday, October 15, 2018 at 9:44 AM > > To: Imesha Sudasingha <imesha...@cse.mrt.ac.lk>, Chris Mattmann < > > mattm...@apache.org> > > Cc: dev <dev@oodt.apache.org> > > Subject: Re: OODT docker builds > > > > > > > > Not at all and if you want to run OODT on Kubernetes for example, that > > > > would be how you’d do it, that way you can upgrade, scale, restart and > fail > > > > components without the entire stack falling over. > > > > > > > > In terms of disk space, don’t forget each image is built on layers, so > for > > > > example, openjdk-8 on alpine is 56mb, that layer would then be used > across > > > > all base images so its only installed once on each host, then you’ve got > > > > your file manager for example which would check in currently at 62MB, so > > > > the entire image size would be 118MB which you could then deploy on 1 > node, > > > > or 100 nodes. > > > > > > > > Then say you’ve got opsui as another dependency, that would be > > > > > > > > tomcat:8 (463mb)+ opsui(73mb)==536mb > > > > > > > > But say you have no interest in workflow etc, thats all you’d deploy. > > > > > > > > In reality it would be much more flexible and much more inline with how > > > > docker containers should be deployed, which is as a single process > > > > container not as a bunch of processes all stuck into 1 unit. > > > > > > > > > > > > > > > > > > > > > > > > On 15 October 2018 at 17:32:32, Chris Mattmann (mattm...@apache.org) > > wrote: > > > > > > > > Isn’t an image per component really heavyweight? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *From: *Tom Barber <t...@spicule.co.uk> > > > > *Date: *Monday, October 15, 2018 at 8:26 AM > > > > *To: *Imesha Sudasingha <imesha...@cse.mrt.ac.lk> > > > > *Cc: *dev <dev@oodt.apache.org>, Chris Mattmann <mattm...@apache.org> > > > > *Subject: *Re: OODT docker builds > > > > > > > > > > > > > > > > Why aren’t we doing so?! :) > > > > > > > > > > > > > > > > Lack of cycles and young kids ;) > > > > > > > > > > > > > > > > I’ll take a stab at it and see where we get to outside of RADIX to get > the > > > > stack in distinct containers and then we’ll look at integrating it into > the > > > > main build then. > > > > > > > > > > > > > > > > > > > > > > > > Tom > > > > > > > > > > > > > > > > On 15 October 2018 at 13:07:56, Imesha Sudasingha ( > imesha...@cse.mrt.ac.lk > > ) > > > > wrote: > > > > > > > > Yes. Agree with you. That is something I have been planning to ask you > for > > > > long; why aren't we doing so. > > > > > > > > I like the idea of having a docker image per component and as you > suggested > > > > we can create docker-compose > > > > > > > > or kubernetes setup for deployments. I like that direction ;-) > > > > > > > > > > > > > > > > As an starting point, we can add an all-in-one docker image to be built > in > > > > the RADIX build, right? > > > > > > > > If you start off, I will be able to join you along the way. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Imesha > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, 15 Oct 2018 at 16:33, Tom Barber <t...@spicule.co.uk> wrote: > > > > > > > > I was thinking about the outputs. Currently everything is 1 docker file > > > > which is fine for some deployments and not for others. > > > > > > > > > > > > > > > > In the RADIX build we should also build individual containers for each > > > > component. For the deployment we could also have a docker-compose file > and > > > > a K8S Helm setup so that you could deploy a distributed OODT setup from > > > > your RADIX output, this could also have the ZK stuff in it so we can > > > > properly utilise the distributed nature we started constructing with the > FM > > > > ZK changes. > > > > > > > > > > > > > > > > > > > > > > > > On 15 October 2018 at 05:08:41, Imesha Sudasingha ( > imesha...@cse.mrt.ac.lk > > ) > > > > wrote: > > > > > > > > Hi Tom, > > > > > > > > I think this will be great since people are adopting docker more and more > > > > and even for a user > > > > once they have built a customized docker image, they can share it among > the > > > > peers > > > > reducing the time spent for configuration by each individual. > > > > > > > > Also we have another option ;-) With distributed configuration management > > > > which I implemented, > > > > users can ask OODT components to download configuration published in > > > > zookeeper. But this will > > > > require zookeeper to be running (either as a container or standalone). As > > > > per my understanding, > > > > configuration is the problem we need to solve when using a pre-built > docker > > > > image? > > > > > > > > In future, if we are able to implement > > > > https://issues.apache.org/jira/browse/OODT-977 > > > > we will be able to run multiple file managers in multiple containers > which > > > > will allow > > > > the load to be distributed and query all at once. So, docker will be the > > > > way to go as I see it. > > > > What do you think? > > > > > > > > Thanks, > > > > Imesha > > > > > > > > > > > > On Sun, 14 Oct 2018 at 15:40, Tom Barber <t...@spicule.co.uk> wrote: > > > > > > > > I’m interested in the Dockerization of OODT but also conscious that most > > > > people use RADIX to build their stuff, which make’s overriding bits of a > > > > prebuilt image tricky. > > > > > > > > I’m wondering if its worth adding an optional Docker profile to RADIX to > > > > add a Docker build step to the backend of people’s RADIX builds if they > so > > > > wanted. > > > > > > > > Thoughts? > > > > > > > > -- > > > > > > > > > > > > Spicule Limited is registered in England & Wales. Company Number: > > > > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > > > > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > > > > > > > > > > > > > > > All engagements > > > > are subject to Spicule Terms and Conditions of Business. This email and > > > > its > > > > contents are intended solely for the individual to whom it is addressed > > > > and > > > > may contain information that is confidential, privileged or otherwise > > > > protected from disclosure, distributing or copying. Any views or opinions > > > > presented in this email are solely those of the author and do not > > > > necessarily represent those of Spicule Limited. The company accepts no > > > > liability for any damage caused by any virus transmitted by this email. > If > > > > you have received this message in error, please notify us immediately by > > > > reply email before deleting it from your system. Service of legal notice > > > > cannot be effected on Spicule Limited by email. > > > > > > > > > > > > > > > > > > > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. > > > > Registered office: First Floor, Telecom House, 125-135 Preston Road, > > > > Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > > > > > > > > > > > All engagements are subject to Spicule Terms and Conditions of Business. > > > > This email and its contents are intended solely for the individual to > whom > > > > it is addressed and may contain information that is confidential, > > > > privileged or otherwise protected from disclosure, distributing or > copying. > > > > Any views or opinions presented in this email are solely those of the > > > > author and do not necessarily represent those of Spicule Limited. The > > > > company accepts no liability for any damage caused by any virus > transmitted > > > > by this email. If you have received this message in error, please notify > us > > > > immediately by reply email before deleting it from your system. Service > of > > > > legal notice cannot be effected on Spicule Limited by email. > > > > > > > > > > > > > > > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. > > > > Registered office: First Floor, Telecom House, 125-135 Preston Road, > > > > Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > > > > > > > > > > > All engagements are subject to Spicule Terms and Conditions of Business. > > > > This email and its contents are intended solely for the individual to > whom > > > > it is addressed and may contain information that is confidential, > > > > privileged or otherwise protected from disclosure, distributing or > copying. > > > > Any views or opinions presented in this email are solely those of the > > > > author and do not necessarily represent those of Spicule Limited. The > > > > company accepts no liability for any damage caused by any virus > transmitted > > > > by this email. If you have received this message in error, please notify > us > > > > immediately by reply email before deleting it from your system. Service > of > > > > legal notice cannot be effected on Spicule Limited by email. > > > > > > > > -- > > > > > > > > > > > > Spicule Limited is registered in England & Wales. Company Number: > > > > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > > > > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > > > > > > > > > > > > > > > All engagements > > > > are subject to Spicule Terms and Conditions of Business. This email and > > its > > > > contents are intended solely for the individual to whom it is addressed > > and > > > > may contain information that is confidential, privileged or otherwise > > > > protected from disclosure, distributing or copying. Any views or opinions > > > > presented in this email are solely those of the author and do not > > > > necessarily represent those of Spicule Limited. The company accepts no > > > > liability for any damage caused by any virus transmitted by this email. > If > > > > you have received this message in error, please notify us immediately by > > > > reply email before deleting it from your system. Service of legal notice > > > > cannot be effected on Spicule Limited by email. > > > > > > > > > > -- > > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > All engagements > are subject to Spicule Terms and Conditions of Business. This email and > its > contents are intended solely for the individual to whom it is addressed > and > may contain information that is confidential, privileged or otherwise > protected from disclosure, distributing or copying. Any views or opinions > presented in this email are solely those of the author and do not > necessarily represent those of Spicule Limited. The company accepts no > liability for any damage caused by any virus transmitted by this email. If > you have received this message in error, please notify us immediately by > reply email before deleting it from your system. Service of legal notice > cannot be effected on Spicule Limited by email. >
