I dont think Disk space is really the main issue – its size of the image to download and boot and the memory consumption in the running image.
We arent running into issues with disk space in our environment but rather in memory consumption and to some extent the container init delays. Brian From: onap-tsc@lists.onap.org <onap-tsc@lists.onap.org> On Behalf Of Tal Liron Sent: Monday, February 11, 2019 11:56 AM To: onap-discuss <onap-disc...@lists.onap.org>; DESBUREAUX Sylvain TGI/OLN <sylvain.desbure...@orange.com> Cc: Chaker Al Hakim <chaker.al.ha...@huawei.com>; srinivasa.r.addepa...@intel.com; onap-tsc@lists.onap.org Subject: Re: [onap-discuss] [onap-tsc] Ubuntu and Alpine discussion This conversation has gotten a little bit confusing. Let me try to organize and separate the core issues: 1. Disk space. If we want to reduce the ridiculous storage requirements for ONAP's container images, the solution is not to choose a tiny Linux distribution. Rather, the solution is to choose a single (or small set of) base image(s) for all ONAP container images. Because images are stored as deltas from the derived base image, then the size of that base image is quite negligible. 2. Choice of base distribution. ONAP provides reference images, but in the end production operations will very likely build their own after auditing selections of packages and even the source code. The ONAP project can make a best effort to pre-audit the selection, but in the end cannot provide neither a guarantee nor a warranty. That comes from the operating system vendor and other software vendors of packages like MariaDB, MongoDB, JDK, etc. The ONAP project should make it possible (and better yet: easy) for users to make that choice. I understand this as one of the goals of our CIA project. I think there's also some misunderstanding of the supportability of packages within the containers. Here's an example: if your host operating system is RHEL, but your container image is based on Alpine, then Red Hat cannot support any security vulnerabilities in those Alpine libraries. This includes OpenSSL as well as glib, systemd, etc. In the end, whatever ONAP chooses as base image is arbitrary as long as we allow users to make their own choice. 3. Technical dependencies on specific distributions. Does a certain ONAP image build against a specific version of a library included in a specific version of a specific distribution? This should be considered a bug, because it exactly disallows the freedom of users to choose a base distribution. Not only that, but it's a guarantee for maintenance nightmares in the future. What happens if the next version of that specific distribution changes the library? Who will make sure to refactor and allow for an upgrade of the base? And if you can't upgrade the base, the image will be stuck with an older and possibly broken/insecure library. There are many open source project plagued with this problem, e.g. stuck on Ubuntu 14.04. We need to make sure that we minimize these dependencies. There's also the option of building that specific dependency library from source for that ONAP image, but again we limit the ability for users to get support from a vendor. There are some complications regarding point #3 that should be discussed. For example, many Linux distributions have switched to systemd in recent years, and that causes a major change in how they run (e.g. that's one main reason projects stuck on Ubuntu 14.04 have difficulty moving forward). If systemd is considered a standard for ONAP, then that limits the number of possible choices for base image. Likewise SELinux. We need to examine ONAP as a whole and make a list of these basic technical requirements from a base distribution. On Mon, Feb 11, 2019 at 10:08 AM Sylvain Desbureaux <sylvain.desbure...@orange.com<mailto:sylvain.desbure...@orange.com>> wrote: Hello Chaker, Today, The docker building image process of ONAP is at the very least messy (Morgan did a presentation at TSC meeting last Thursday on this point https://wiki.onap.org/pages/viewpage.action?pageId=6593670&preview=/6593670/54722733/onap_tests.pdf<https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.onap.org_pages_viewpage.action-3FpageId-3D6593670-26preview-3D_6593670_54722733_onap-5Ftests.pdf&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=gZSWqClUIfrznacZC3v84m2JsV0c31x8dhlCzfK61vw&s=ya2lBU-bwwCbzGp-Yy9O1RhXAff5CUPVUr8CLD51vt8&e=>) as we do not know what we test and even if it’s tested. So the base image of an ONAP component doesn’t seems to be the most critical part on improving ONAP. I would prefer to : * Make reliable, consistent tests on any review that come * Know what is tested * Strengthen security by moving from root to another user in the docker images * Slimify ONAP so the tests are faster When we’ll arrive at that, I’ll be happy to see if we can make a consensus on base image. Today, all I want to do is make small images per components. If a component is small on ubuntu/centOs/Debian/openSUSE, fine I won’t touch it. If it’s not, I’ll make a review request with the image that I think is the most suitable, and most of the time it’ll be Alpine. Considered the spread of images today, I think that the choice of image is up to the requester and the “+2” person from the project. Regards, --- Sylvain Desbureaux De : Chaker Al Hakim <chaker.al.ha...@huawei.com<mailto:chaker.al.ha...@huawei.com>> Date : lundi 11 février 2019 à 16:04 À : DESBUREAUX Sylvain TGI/OLN <sylvain.desbure...@orange.com<mailto:sylvain.desbure...@orange.com>>, "srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>" <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>>, "onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>" <onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>>, "onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>" <onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>> Objet : RE: [onap-discuss] [onap-tsc] Ubuntu and Alpine discussion Hello Sylvain, I appreciate your detailed reply.. I am not sure if you were on the call when this topic was discussed. My point was not whether I favor one Linux distribution over other Linux distributions,. It was related to whether a specific distribution is commercially supported. As I mentioned in my previous email, f the goal of the Footprint Optimization project is to support all of the ONAP components, including all of their dependencies, packages, re-certification efforts, etc… on Alpine Linux then let’s make that very clear, ask for another TSC approval and document it in the ONAP wiki so that we are all on board Regards, Chaker From: sylvain.desbure...@orange.com<mailto:sylvain.desbure...@orange.com> [mailto:sylvain.desbure...@orange.com<mailto:sylvain.desbure...@orange.com>] Sent: Monday, February 11, 2019 8:29 AM To: Chaker Al Hakim <chaker.al.ha...@huawei.com<mailto:chaker.al.ha...@huawei.com>>; srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>; onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>; onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> Subject: Re: [onap-discuss] [onap-tsc] Ubuntu and Alpine discussion Hello Chaker and Srini, First of all, I want to thank you to propose yourself to rebuild ONAP docker images in order to shrink their sizes. We’re only 2 guys doing that today (Frank Sandoval and myself) so multiplying by two the workforce is great news. SDNC and AAI have been done, Frank is working on DMaaP and I’m starting Portal as of today. So you can pick any image from https://wiki.onap.org/display/DW/CIA+Dublin+Release+Planning<https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.onap.org_display_DW_CIA-2BDublin-2BRelease-2BPlanning&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=gZSWqClUIfrznacZC3v84m2JsV0c31x8dhlCzfK61vw&s=VnSdGGl71Ellhpb6mGS_-Zj-BN0fb4XZh0fIO3_RZzI&e=> (Robot or SO for example). We want also to make a second pass on already done images to shrink more so you can also review and propose enhancements to the ones already done. Welcome on board! For the base image, you can use Alpine or Ubuntu at your preference. As there are opinions against Alpine, I would like to share my views: * Alpine is “vendor neutral”. If we decide to mandate ubuntu for example, I imagine that Canonical competitors could disagree and push their own images * There have been some concerns (see Ramki email for example) about using musl library and not glibc. Musl libc (https://www.musl-libc.org/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.musl-2Dlibc.org_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=gZSWqClUIfrznacZC3v84m2JsV0c31x8dhlCzfK61vw&s=poPYD6j4o_qe53Bl4dtbbGW4E5u1r0baGpMUieBKPhQ&e=>) is now the default library for OpenWRT and is also used as replacement of uclibc (and glibc) in embedded Real time Linux images. So, we wouldn’t be the only one using it. * Another building block of Alpine is busybox (https://www.busybox.net/<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.busybox.net_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=gZSWqClUIfrznacZC3v84m2JsV0c31x8dhlCzfK61vw&s=z8Z73cBsU572MDe4CF-X3eM_p7rOKeHhKx6fdMSjiDQ&e=>) which is used in a lot of embedded devices (such as Modem router) so again we’re not the only ones (there are billions of device using it). * On the security side, clair scanner (https://github.com/coreos/clair/<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_coreos_clair_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=gZSWqClUIfrznacZC3v84m2JsV0c31x8dhlCzfK61vw&s=MDxY5J6VgFHMwDEloBsPGqcno2-aVPkC65YfGyzkekU&e=>) is a vulnerability tool from CoreOS (now part of RedHat) designerd for container security audit. You can embed it in your CI. Here’s an example of container scan (python script on top of Alpine) and you can see there are no known vulnerabilities : https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci-mqtt-trigger/security/dashboard<https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.com_Orange-2DOpenSource_lfn_ci-5Fcd_chained-2Dci-2Dmqtt-2Dtrigger_security_dashboard&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=gZSWqClUIfrznacZC3v84m2JsV0c31x8dhlCzfK61vw&s=sZnyQnLUNctKjkUucsNpDi3HJ5yzrSkUjmLkY-6bPaE&e=> At the very least, this discussion shows a need for simplifying docker build process in order to tend to have a “simple” way to move from base image XXX to base image YYY. Now, to know what’s the state of OOM as of today (master deployment of ONAP using docker-staging file from integration team, counting 284 containers): ONAP images using : * Debian/Ubuntu: 98 * Alpine: 61 * CentOS/rpm or yum based distro: 21 * Busybox: 3 (actually a lot more as many jobs are using busybox) * Unknown: 1 Please be aware that mileage may vary (+/- 10 range I think) (it’s a “bruteforce” script and there may be / are images counted more than once but it’s what’s on my deployment today) and I counted “only” images from “nexus3.onap.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__nexus3.onap.org&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=gZSWqClUIfrznacZC3v84m2JsV0c31x8dhlCzfK61vw&s=oZknlS9Af-VlxYmTi00kdmw8qDWzVP9JoRLo-q8LPs8&e=>”. So there’s no clear dominant base image and so claiming for “ubuntu only” or “alpine only” is inappropriate. --- Sylvain Desbureaux De : Chaker Al Hakim <chaker.al.ha...@huawei.com<mailto:chaker.al.ha...@huawei.com>> Date : vendredi 8 février 2019 à 15:34 À : "onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>" <onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>>, DESBUREAUX Sylvain TGI/OLN <sylvain.desbure...@orange.com<mailto:sylvain.desbure...@orange.com>>, "srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>" <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>>, "onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>" <onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>> Objet : RE: [onap-discuss] [onap-tsc] Ubuntu and Alpine discussion Hi Team, I was also on the TSC call and was engaged in the discussion as well. I agree with Srini’s position that we should support ONAP on both Ubuntu as well as Alpine Linux. Looking just at the disk image footprint as one driver to support only Alpine Linux may not be the best way to go; In addition, deploying ONAP on a base Linux that is not commercially supported may not be the best approach moving forward. If the end goal of the Footprint Optimization is to ultimately support ONAP only on Alpine Linux then this discussion becomes much broader then just footprint optimization and should be presented and discussed as such at the ArchCom and at the the TSC level to make sure everyone is aware of the potential long term impact. Regards, Chaker From: onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org> [mailto:onap-disc...@lists.onap.org] On Behalf Of Sylvain Desbureaux Sent: Friday, February 08, 2019 4:11 AM To: onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>; srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>; onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> Subject: Re: [onap-discuss] [onap-tsc] Ubuntu and Alpine discussion Hello Srini, Fyi size of images on dockerhub may not be the same once downloaded (I don’t know why): For example, alpine:3.9 is said at 3MB and ubuntu:18.04 is said at 32MB. Once downloaded (on an x86_64 server), here’s what I have: docker images REPOSITORY TAG IMAGE ID CREATED SIZE ubuntu 18.04 47b19964fb50 2 days ago 88.1MB alpine 3.9 caf27325b298 8 days ago 5.53MB Regards, --- Sylvain Desbureaux De : <onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>> au nom de Srini <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>> Répondre à : "onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>" <onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>>, "srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>" <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>> Date : vendredi 8 février 2019 à 00:03 À : "onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>" <onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>> Cc : "onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>" <onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org>> Objet : Re: [onap-discuss] [onap-tsc] Ubuntu and Alpine discussion Hi Adolfo, “Having said that, projects that can clearly demonstrate that they can't migrate to Alpine should be given the opportunity to request an exception and do the work themselves. “ This is the point that raised the discussion in TSC. Above statement seem to indicate that the goal for ONAP is to move to Alpine. So far, my impression was that ONAP would be supported on both Ubuntu and Alpine. Hence the discussion today. Since Alpine is not supported commercially, it is always be a challenge for few to adopt ONAP as is. Ubuntu is open source with commercial support. Now that Ubuntu also has thin version, adopting Ubuntu thin image as base package would address concerns related to size as well as support. Moreover Ubuntu has big eco-system too and many packages get tested against Ubuntu. Since ONAP is also tested so far on Ubuntu based containers, I think work for ONAP teams also would be relatively lesser if we go with Ubuntu minimal image. I have not tried Ubuntu thin image. Again, my 2 cents on studying Ubuntu minimal image and its applicability. Srini From: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> [mailto:onap-tsc@lists.onap.org] On Behalf Of Adolfo Perez-Duran Sent: Thursday, February 7, 2019 12:06 PM To: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> Cc: onap-disc...@lists.onap.org<mailto:onap-disc...@lists.onap.org> Subject: Re: [onap-tsc] Ubuntu and Alpine discussion Hi Srini, Thanks for joining the discussion. On October 8, 2018 PTLs discussed and agree to using Alpine as the base image for footprint minimization. Because images are immutable, once dependencies and libraries are mapped there should not be any perceptible difference introduced by the base image. It may be fair to assume that integration testing occurs (should occur?) at the service (API) level, regardless of what base image was used to create a container. Some projects, such as SO and AAF, had already adopted --on their own-- Alpine as their base image. Also, the CIA team has already engaged a number of teams in the Dublin scope and has contributed patches to migrate to Alpine. Having said that, projects that can clearly demonstrate that they can't migrate to Alpine should be given the opportunity to request an exception and do the work themselves. There is a lot of work to be done (finding common base images that can be used across projects, multi-stage builds, CI/CD parameterization, etc). Migrating to a small base image is only the first step. I encourage you to bring a contribution and help us realize the long term goals. Adolfo On Thu, Feb 7, 2019 at 12:25 PM Srini <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>> wrote: Hi, In today TSC this discussion came up on challenges in integration testing on both Ubuntu based ONAP and Alpine based ONAP. Ubuntu releases minimal Ubuntu for containers in July, 2018. According to https://blog.ubuntu.com/2018/07/09/minimal-ubuntu-released<https://urldefense.proofpoint.com/v2/url?u=https-3A__blog.ubuntu.com_2018_07_09_minimal-2Dubuntu-2Dreleased&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=gZSWqClUIfrznacZC3v84m2JsV0c31x8dhlCzfK61vw&s=rgHgyKk-xqw259kNzkOIHNsQ05aUOzGxpoA9vP3hFa4&e=> - 29Mbytes of docker image size. - It is based 18.04 (LTS ) – Long terms support availability. - Minimal security attack surface - Faster speed to bringup. It seems to be providing best of both worlds - Ubuntu, which ONAP has been basing on so far. Also, supported by Canonical. - Less disk size and faster bootup. - Supports multiple architecture (ARM, x86, PPC etc…) Some more info from dockerhub: https://github.com/docker-library/repo-info/blob/master/repos/ubuntu/remote/18.10.md<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_docker-2Dlibrary_repo-2Dinfo_blob_master_repos_ubuntu_remote_18.10.md&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=e3d1ehx3DI5AoMgDmi2Fzw&m=gZSWqClUIfrznacZC3v84m2JsV0c31x8dhlCzfK61vw&s=oTfTz6iORhCZd6ojFT-3BmNuouVj9NTBBuT302b9x-g&e=> Thanks Srini _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#4603): https://lists.onap.org/g/onap-tsc/message/4603 Mute This Topic: https://lists.onap.org/mt/29701820/21656 Group Owner: onap-tsc+ow...@lists.onap.org Unsubscribe: https://lists.onap.org/g/onap-tsc/leave/2743226/1412191262/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
