Hi Chamila, Yes, we would need to make our Docker (and any other) artifacts follow the industry accepted standards so that it is easy for users to adopt them. More complexity means user probably will be discouraged to use our stuff.
As you mentioned, the --squash option is a good way to reduce the size of the resulting images, however do we also need to consider if there are any cons in using the --squash approach. According the documentation [1], it says; Note: using this option means the new image will not be able to take advantage of layer sharing with other images and may use significantly more space. Note: using this option you may see significantly more space used due to storing two copies of the image, one for the build cache with all the cache layers in tact, and one for the squashed version. It seems with the --squash option, the resulting images could use significantly more memory, is this a trade off we are willing to make? Given that this is an experimental feature, we can hope that this might be imrpoved in the future to take advantage of layer sharing as well, however better to check on that aspect as well. [1] - https://docs.docker.com/engine/reference/commandline/ build/#squash-an-images-layers-squash-experimental-only On Thu, Nov 9, 2017 at 12:55 PM, Chamila De Alwis <chami...@wso2.com> wrote: > Hi, > > So far, the approach for writing Dockerfiles for WSO2 products has been to > write a generic Dockerfile that can be manipulated to build different > images with different approaches. For example, to generalize the way in > which the product is configured inside the image during build time, a > concept called "Provisioning methods" was introduced. This would allow > different configuration methods ("default" which is copying a plain old > pack into the image, and "puppet" which runs Puppet scripts during the > image build) to configure the image. > > However, the cost of such a generalized approach is the increased > complexity of the script layer around the Dockerfile and the necessity of > the bootstrap tasks done by the scripts. Because of this, > > 1. New users have found it hard to quickly start using the Dockerfiles to > build the images > 2. Users with prior Docker knowledge have struggled to understand the > reasoning behind such a thick layer of bootstrapping > 3. Customization to existing scripts require a steep learning curve > 4. Product teams have found it difficult to commit resources to maintain > the scripts+Dockerfiles > 5. Dockerfiles cannot be used in pure Docker commands without the complex > bootstrapping involved > > One of the main reasons for following such a complex layer of script based > bootstrapping and a somewhat convoluted Dockerfile was to reduce the > resulting Docker image size. At the time, Docker's Copy-On-Write strategy > was not kind on a number of changes we would be doing for a WSO2 Server. > This required "clever" hacks to be performed so that there would be minimal > inter-layer changes. One of these hacks was to copy and modify the WSO2 > server packs and the JDK in the same layer. We were able to reduce the > Docker image sizes for a standard ESB image from somewhere around 2GB to > 800MB through this approach. > > However, Docker has now started to address the image size concerns raised > by the community. One of the solutions Docker has started to put forward is > the --squash flag [1]. This is still an experimental feature but is > available with the recent Docker CE releases. With --squash flag, there is > no need for Dockerfile instruction manipulation to get a smaller Docker > image. It's reasonable to assume that Docker would keep improving this > functionality in the future. > > With this in mind, evolving the current set of Dockerfiles into a simpler > set and reducing the bash script based bootstrapping layer to a minimum is > IMO the best approach to resolve the above-mentioned issues. In such a > simple approach, a Dockerfile would only need the WSO2 Server pack and the > JDK distribution, for it to be built. Pure `docker build` command can be > used to build the image and would be the expected way to do so. A sample > Dockerfile for WSO2 EI/Integrator 6.1.1 can be found at [2]. > > There would be different Dockerfiles for different runtimes of a product, > different versions, and if we plan to use any config automation for image > configuration during the image build, there would be different images for > each approach as well. Yes, this would duplicate a lot of stuff, but it > would reduce the complexity by a few times for the end-user, who would > always be a devops or a developer (wso2 or non-wso2). > > The advantages of such an approach would be > > 1. Easier learning curve required for both types of users who are familiar > with Docker and those who are not. > 2. Increased adaptation and maintenance by the product teams resulting > from the easier learning curve > 3. Can be incorporated into client's existing pipelines easily because of > the lack of bootstrapping needed and the ability to use pure `docker build` > commands > 4. Possibility of adding WSO2 Dockerfiles to DockerHub's official > repositories (Popular Dockerfiles such as Ubuntu [3] and MySQL [4] have > almost zero build time ARGs, and follow a similar approach to what I've > suggested above) > 5. Requires a less complex code structuring (there is no need for a base > image definition) > > Appreciate any input on this. > > > [1] - https://medium.com/@chamilad/docker-image-size- > concerns-out-of-the-window-squash-with-confidence-796f7c48f5c6 > [2] - https://github.com/chamilad/wso2-dockerfiles-new/blob/ > master/wso2ei/6.1.1/Dockerfile > [3] - https://github.com/docker-library/mysql/tree/ > 86431f073b3d2f963d21e33cb8943f0bdcdf143d > [4] - https://github.com/tianon/docker-brew-ubuntu-core/tree/ > 5fce3945d95630c2fc03c21ef8665d92bd824642 > > Regards, > Chamila de Alwis > Committer and PMC Member - Apache Stratos > Senior Software Engineer | WSO2 > +94 77 220 7163 <+94%2077%20220%207163> > Blog: https://medium.com/@chamilad > > > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Thanks, Shariq Associate Technical Lead
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
