IMO image size can be a concern, especially with C4 servers. Users who are used to work with image sizes around 500-600MB would be in for a surprise if we rack up an image to a size about 1.5-2GB. Though what Dilan says is true, that the ease of the process would need to be prioritized, the size should also be kept in mind.
@Shariq, Yes, the squash option has the cons you've mentioned. However, I believe we could depend on it being developed more, as it's still in the experimental feature set. One other thing to remember is that there are two user experiences that we are discussing here. 1. Use of WSO2 Docker images that are pulled from the public registries 2. Use of WSO2 Dockerfiles to build images in user-specific scenarios I think current cons in the squash option may interfere with experience #2 more, as what we might push to a public registry would not need to worry about the build cache or builder space. In the #2 case, IMO prioritizing simplicity over reduced image sizes (that users may or may not need) would be a good bet to make. Regards, Chamila de Alwis Committer and PMC Member - Apache Stratos Senior Software Engineer | WSO2 +94 77 220 7163 Blog: https://medium.com/@chamilad On Fri, Nov 24, 2017 at 10:27 AM, Dilan Udara Ariyaratne <dil...@wso2.com> wrote: > Hi Folks, > > IMO, docker image size is not a major issue nowadays unless you are > extremely worrying about the image pull time and docker registry disk space > of your deployment. > What matters most here is the amount of ease of docker image build and > customization experience that we are hoping to provide to our customers and > giving priority to that aspect, I am +1 with what Chamila is suggesting > here. > > We have already adapted this approach when defining dockerfiles, specific > for K8s/OpenShift deployments and > at the moment, we are in need of adopting the same for purely docker based > container deployments of our products. > > Namely, Dockerfiles for : > [1] - Docker compose > [2] - Docker Swarm > > Thanks, > Dilan. > > *Dilan U. Ariyaratne* > Senior Software Engineer > WSO2 Inc. <http://wso2.com/> > Mobile: +94766405580 <%2B94766405580> > lean . enterprise . middleware > > > On Thu, Nov 23, 2017 at 7:02 PM, Muhammed Shariq <sha...@wso2.com> wrote: > >> Hi Chamila, >> >> Yes, we would need to make our Docker (and any other) artifacts follow >> the industry accepted standards so that it is easy for users to adopt them. >> More complexity means user probably will be discouraged to use our stuff. >> >> As you mentioned, the --squash option is a good way to reduce the size of >> the resulting images, however do we also need to consider if there are any >> cons in using the --squash approach. According the documentation [1], it >> says; >> >> Note: using this option means the new image will not be able to take >> advantage of layer sharing with other images and may use significantly more >> space. >> >> Note: using this option you may see significantly more space used due to >> storing two copies of the image, one for the build cache with all the cache >> layers in tact, and one for the squashed version. >> It seems with the --squash option, the resulting images could use >> significantly more memory, is this a trade off we are willing to make? >> >> Given that this is an experimental feature, we can hope that this might >> be imrpoved in the future to take advantage of layer sharing as well, >> however better to check on that aspect as well. >> >> [1] - https://docs.docker.com/engine/reference/commandline/build >> /#squash-an-images-layers-squash-experimental-only >> >> On Thu, Nov 9, 2017 at 12:55 PM, Chamila De Alwis <chami...@wso2.com> >> wrote: >> >>> Hi, >>> >>> So far, the approach for writing Dockerfiles for WSO2 products has been >>> to write a generic Dockerfile that can be manipulated to build different >>> images with different approaches. For example, to generalize the way in >>> which the product is configured inside the image during build time, a >>> concept called "Provisioning methods" was introduced. This would allow >>> different configuration methods ("default" which is copying a plain old >>> pack into the image, and "puppet" which runs Puppet scripts during the >>> image build) to configure the image. >>> >>> However, the cost of such a generalized approach is the increased >>> complexity of the script layer around the Dockerfile and the necessity of >>> the bootstrap tasks done by the scripts. Because of this, >>> >>> 1. New users have found it hard to quickly start using the Dockerfiles >>> to build the images >>> 2. Users with prior Docker knowledge have struggled to understand the >>> reasoning behind such a thick layer of bootstrapping >>> 3. Customization to existing scripts require a steep learning curve >>> 4. Product teams have found it difficult to commit resources to maintain >>> the scripts+Dockerfiles >>> 5. Dockerfiles cannot be used in pure Docker commands without the >>> complex bootstrapping involved >>> >>> One of the main reasons for following such a complex layer of script >>> based bootstrapping and a somewhat convoluted Dockerfile was to reduce the >>> resulting Docker image size. At the time, Docker's Copy-On-Write strategy >>> was not kind on a number of changes we would be doing for a WSO2 Server. >>> This required "clever" hacks to be performed so that there would be minimal >>> inter-layer changes. One of these hacks was to copy and modify the WSO2 >>> server packs and the JDK in the same layer. We were able to reduce the >>> Docker image sizes for a standard ESB image from somewhere around 2GB to >>> 800MB through this approach. >>> >>> However, Docker has now started to address the image size concerns >>> raised by the community. One of the solutions Docker has started to put >>> forward is the --squash flag [1]. This is still an experimental feature but >>> is available with the recent Docker CE releases. With --squash flag, there >>> is no need for Dockerfile instruction manipulation to get a smaller Docker >>> image. It's reasonable to assume that Docker would keep improving this >>> functionality in the future. >>> >>> With this in mind, evolving the current set of Dockerfiles into a >>> simpler set and reducing the bash script based bootstrapping layer to a >>> minimum is IMO the best approach to resolve the above-mentioned issues. In >>> such a simple approach, a Dockerfile would only need the WSO2 Server pack >>> and the JDK distribution, for it to be built. Pure `docker build` command >>> can be used to build the image and would be the expected way to do so. A >>> sample Dockerfile for WSO2 EI/Integrator 6.1.1 can be found at [2]. >>> >>> There would be different Dockerfiles for different runtimes of a >>> product, different versions, and if we plan to use any config automation >>> for image configuration during the image build, there would be different >>> images for each approach as well. Yes, this would duplicate a lot of stuff, >>> but it would reduce the complexity by a few times for the end-user, who >>> would always be a devops or a developer (wso2 or non-wso2). >>> >>> The advantages of such an approach would be >>> >>> 1. Easier learning curve required for both types of users who are >>> familiar with Docker and those who are not. >>> 2. Increased adaptation and maintenance by the product teams resulting >>> from the easier learning curve >>> 3. Can be incorporated into client's existing pipelines easily because >>> of the lack of bootstrapping needed and the ability to use pure `docker >>> build` commands >>> 4. Possibility of adding WSO2 Dockerfiles to DockerHub's official >>> repositories (Popular Dockerfiles such as Ubuntu [3] and MySQL [4] have >>> almost zero build time ARGs, and follow a similar approach to what I've >>> suggested above) >>> 5. Requires a less complex code structuring (there is no need for a base >>> image definition) >>> >>> Appreciate any input on this. >>> >>> >>> [1] - https://medium.com/@chamilad/docker-image-size-concerns-ou >>> t-of-the-window-squash-with-confidence-796f7c48f5c6 >>> [2] - https://github.com/chamilad/wso2-dockerfiles-new/blob/mast >>> er/wso2ei/6.1.1/Dockerfile >>> [3] - https://github.com/docker-library/mysql/tree/86431f073b3d2 >>> f963d21e33cb8943f0bdcdf143d >>> [4] - https://github.com/tianon/docker-brew-ubuntu-core/tree/5fc >>> e3945d95630c2fc03c21ef8665d92bd824642 >>> >>> Regards, >>> Chamila de Alwis >>> Committer and PMC Member - Apache Stratos >>> Senior Software Engineer | WSO2 >>> +94 77 220 7163 <+94%2077%20220%207163> >>> Blog: https://medium.com/@chamilad >>> >>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Thanks, >> Shariq >> Associate Technical Lead >> > > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > >
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
