it's up to you PTLS, as you prefer
------------------------------------------------------------------------
*De :* TIMONEY, DAN [dt5...@att.com]
*Envoyé :* lundi 27 avril 2020 20:33
*À :* onap-...@lists.onap.org; DESBUREAUX Sylvain TGI/OLN;
onap-tsc@lists.onap.org
*Cc :* RICHOMME Morgan TGI/OLN; Krzysztof Opasiak; Mike Elliott;
CLOSSET, CHRISTOPHE; dmcbr...@linuxfoundation.org; Kenny Paul
*Objet :* Re: [onap-ptl] [OOM] Guilin Release Plan
Sylvain,
Thank you for the excellent summary, which was also explained quite well
at last week’s virtual face to face event.
One process question, though: are there already user stories created for
these items (perhaps SECCOM user stories)? I want to make sure we’re
tracking these requirements when we do our release planning for Guilin.
Thanks!
Dan
*From: *<onap-...@lists.onap.org> on behalf of "Sylvain Desbureaux via
lists.onap.org" <sylvain.desbureaux=orange....@lists.onap.org>
*Reply-To: *"onap-...@lists.onap.org" <onap-...@lists.onap.org>,
"sylvain.desbure...@orange.com" <sylvain.desbure...@orange.com>
*Date: *Monday, April 27, 2020 at 12:22 PM
*To: *"onap-...@lists.onap.org" <onap-...@lists.onap.org>,
"onap-tsc@lists.onap.org" <onap-tsc@lists.onap.org>
*Cc: *RICHOMME Morgan TGI/OLN <morgan.richo...@orange.com>, Krzysztof
Opasiak <k.opas...@samsung.com>, Mike Elliott <mike.elli...@amdocs.com>,
"CLOSSET, CHRISTOPHE" <christophe.clos...@intl.att.com>,
"dmcbr...@linuxfoundation.org" <dmcbr...@linuxfoundation.org>, Kenny
Paul <kp...@linuxfoundation.org>
*Subject: *[onap-ptl] [OOM] Guilin Release Plan
Dear PTLs, dear TSC members,
as RC0 is (almost) passed and Frankfurt branch will be soon created, I
think it's important to share with you all the OOM release plan for Guilin.
As you'll eventually push your changes in OOM, I think it's important
that you know our plans.
# TL;DR;
* If you don't retrieve your certificates automatically, this is the
__first__ thing you must do. See
https://protect2.fireeye.com/url?k=c7d639cc-9a056543-c7d7b283-0cc47a31ce52-21da0bbc9685518d&q=1&u=https%3A%2F%2Fgithub.com%2Fonap%2Foom%2Ftree%2Fmaster%2Fkubernetes%2Fnbi
<https://protect2.fireeye.com/url?k=1cb18522-4162d9ad-1cb00e6d-0cc47a31ce52-e8686001521714c3&q=1&u=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__github.com_onap_oom_tree_master_kubernetes_nbi%26d%3DDwQFAw%26c%3DLFYZ-o9_HUMeMTSQicvjIg%26r%3DqLcfee4a2vOwYSub0bljcQ%26m%3DHceOARp59TvXni8a6PJgJVBW81QZ4IHOTjrxRGuNiU0%26s%3DKJ89qSlgfhIhQT-fXkbJYG8mmP0FgLxUJHMqe1-UUnk%26e%3D>
as an example. `certInit` template (a.k.a `aafInit`) is the **strongly**
preffered way to do that.
* No more than 1 main process per container. If you have more, it'll be
a blocker for updates
* All logs to STDOUT
* No direct commit to Frankfurt but master then cherry-pick
* All upstream components should use an upstream (dockerhub, googlehub)
version
* Common chart version bump
* Mariadb common chart will be upgraded to 10.4.12
* PostgreSQL common chart will be upgraded to 12.2
* Cassandra common chart will be upgraded to 3.11.6
* MongoDB common chart will be upgraded to 4.2.2
* ElasticSearch common chart may be upgraded, waiting for SECCOM
proposed version
* AAF is an optional requirement, meaning your component must work
without AAF (certificates and RBAC), even on degraded mode
* MSB is an optional requirement, meaning your component must work
without MSB
* Your component can use http as **server** and **client**
* Password removal will continue on common charts (postgreSQL at least)
and start on your component, be prepared to receive so call for help
* Commit messages must be meaningful and follow the format shown below.
* Proper crash (if your component fails, it must exit with code > 0, and
not wait or exit with code 0)
* Ingress will be the default deployment option (via Nginx Ingress). No
more access via NodePort per default
* New code will be submitted only if pods + healthchecks + basic tests
are OK
* No root access to any Database from application container
* No configuration generation using sed in the application container
# Certificates
Certificates in Docker are not allowed in Francfurt release per SECCOM
recommendation.
They won't be allowed either in helm chart starting branching of
Frankfurt. This means you must move to automatic retrieval at boot.
You'll either have to:
* use `certInit` template (formerly known as `aafInit` template) (see
[NBI](https://protect2.fireeye.com/url?k=9d275237-c0f40eb8-9d26d978-0cc47a31ce52-b1b7935132d5481d&q=1&u=https%3A%2F%2Fgithub.com%2Fonap%2Foom%2Ftree%2Fmaster%2Fkubernetes%2Fnbi
<https://protect2.fireeye.com/url?k=a23482fd-ffe7de72-a23509b2-0cc47a31ce52-6f70498f5a780bb4&q=1&u=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__github.com_onap_oom_tree_master_kubernetes_nbi%26d%3DDwQFAw%26c%3DLFYZ-o9_HUMeMTSQicvjIg%26r%3DqLcfee4a2vOwYSub0bljcQ%26m%3DHceOARp59TvXni8a6PJgJVBW81QZ4IHOTjrxRGuNiU0%26s%3DKJ89qSlgfhIhQT-fXkbJYG8mmP0FgLxUJHMqe1-UUnk%26e%3D>)
as an example)
* do it by your own + explain why you can't use `certInit` template
(will be subject to acceptance / not acceptance from both SECCOM and OOM
team)
# No More than 1 main process per container
Several containers uses several process (main component + database(s))
in the same docker.
Each container should have only one concern. Decoupling applications
into multiple containers makes it easier to scale horizontally and reuse
containers. For instance, a web application stack might consist of three
separate containers, each with its own unique image, to manage the web
application, database, and an in-memory cache in a decoupled manner.
Limiting each container to one process is a good rule of thumb, but it
is not a hard and fast rule. For example, not only can containers be
spawned with an init process, some programs might spawn additional
processes of their own accord. For instance, Celery can spawn multiple
worker processes, and Apache can create one process per request.
This will not be accetped anymore as it's a very bad pattern
(https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.docker.com_develop_develop-2Dimages_dockerfile-5Fbest-2Dpractices_&d=DwQFAw&c=LFYZ-o9_HUMeMTSQicvjIg&r=qLcfee4a2vOwYSub0bljcQ&m=HceOARp59TvXni8a6PJgJVBW81QZ4IHOTjrxRGuNiU0&s=QuU9n_5-csVHuOz_IadQvZI5yOaJL8hfcttcxWnDek8&e=>).
# All logs to STDOUT
As per SECCOM (and testers ;) ) requests, all the logs **must** go to
STDOUT. It'll easier troubleshooting and future centralized log work.
# Smart Healthchecks
If your healtchecks just verify that `/status` is answering 200, there
are not useful as readiness/liveness probes can do it.
So create meaningful healtchecks or remove them if they can be done via
readiness/liveness probes.
# Relevant commit messages:
the commit message (on OOM) **must** follow this form:
```
[NAME_OF_COMPONENT|DOC|COMMON|GENERIC] Meaningful title (from OOM side)
at least one sentence explaining the change done in this patch, cause
and consequences and possibly more of course
Issue-ID: AS_WE_ARE_FORCED_BUT_MEANINGLESS
Change-ID: xxx
Sign-off: xxx
```
Commit message will be the last stuff that will stay with our code so it
must clearly explain the changes, the "why" and the consequences.
If it change OOM behavior in any way, documentation **must** be also
updated.
Starting Frankfurt branching, merge requests which are not following
this pattern will not be merged.
Please read the following pages and follow the guidelines for writing
commit message contained therein.
*
https://protect2.fireeye.com/url?k=131fc297-4ecc9e18-131e49d8-0cc47a31ce52-e7e61fcdb6382a78&q=1&u=http%3A%2F%2Fbit.ly%2Fgoodcommitmessages
<https://protect2.fireeye.com/url?k=b52967df-e8fa3b50-b528ec90-0cc47a31ce52-78900e0144cd02f9&q=1&u=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__bit.ly_goodcommitmessages%26d%3DDwQFAw%26c%3DLFYZ-o9_HUMeMTSQicvjIg%26r%3DqLcfee4a2vOwYSub0bljcQ%26m%3DHceOARp59TvXni8a6PJgJVBW81QZ4IHOTjrxRGuNiU0%26s%3D8TSHbPBYRXs4hLPF0AyUWy3t67S_NZ4jnv1CybU9jtM%26e%3D>
*
https://protect2.fireeye.com/url?k=55432544-089079cb-5542ae0b-0cc47a31ce52-e2ee1a09c48098f0&q=1&u=http%3A%2F%2Fwho-t.blogspot.com%2F2009%2F12%2Fon-commit-messages.html
<https://protect2.fireeye.com/url?k=e20b02bb-bfd85e34-e20a89f4-0cc47a31ce52-643d3722ecd41661&q=1&u=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__who-2Dt.blogspot.com_2009_12_on-2Dcommit-2Dmessages.html%26d%3DDwQFAw%26c%3DLFYZ-o9_HUMeMTSQicvjIg%26r%3DqLcfee4a2vOwYSub0bljcQ%26m%3DHceOARp59TvXni8a6PJgJVBW81QZ4IHOTjrxRGuNiU0%26s%3Dw857YnB7tIRHodxOgiaNsAaCbJBrKD-8PVK60_9oGXo%26e%3D>
*
https://protect2.fireeye.com/url?k=4d4c82e2-109fde6d-4d4d09ad-0cc47a31ce52-f428cfc7f42f8d7d&q=1&u=http%3A%2F%2Fdep.debian.net%2Fdeps%2Fdep3%2F
<https://protect2.fireeye.com/url?k=1083ab73-4d50f7fc-1082203c-0cc47a31ce52-abe50245697d3ff9&q=1&u=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__dep.debian.net_deps_dep3_%26d%3DDwQFAw%26c%3DLFYZ-o9_HUMeMTSQicvjIg%26r%3DqLcfee4a2vOwYSub0bljcQ%26m%3DHceOARp59TvXni8a6PJgJVBW81QZ4IHOTjrxRGuNiU0%26s%3DJ02E092N_nvYErW7hnEiVcptrpNlqERHHGrOwGA_wLY%26e%3D>
# No root access to any Database from application container
If you need to create users, tables etc do it from init container.
You can either:
* use common.mariadb-init template for MariaDB (will be extended for
PostgreSQL)
* use your init container + explain why you can't use the common chart
# No configuration generation using sed in the application container
It can be delivered as a config map or if it has to be somehow processed
this should happen in the init container.
# Kubernetes upgrade to 1.18
In order to move to Kuebernetes 1.18, significant changes will be done
on helm charts, using templates we've made.
It shouldn't be harmful on your side.
Moving to 1.16+ will allow use to use
[startupProbe](https://v1-16.docs.kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
<https://urldefense.proofpoint.com/v2/url?u=https-3A__v1-2D16.docs.kubernetes.io_docs_tasks_configure-2Dpod-2Dcontainer_configure-2Dliveness-2Dreadiness-2Dstartup-2Dprobes_&d=DwQFAw&c=LFYZ-o9_HUMeMTSQicvjIg&r=qLcfee4a2vOwYSub0bljcQ&m=HceOARp59TvXni8a6PJgJVBW81QZ4IHOTjrxRGuNiU0&s=tM2BQ0LyxoMwxXA2d66SLP2ZQOa0o6cXQNJkW5Yeoy4&e=>),
if you've have slow starting components, please tell us!
# Databases upgrades
Databases will move to follow [SECCOM
proposition](https://protect2.fireeye.com/url?k=1c04c9e2-41d7956d-1c0542ad-0cc47a31ce52-4eb368a020769cc7&q=1&u=https%3A%2F%2Fwiki.lfnetworking.org%2Fdisplay%2FLN%2F2020%2BApril%2BTechnical%2BEvent%2BSchedule%3Fpreview%3D%2F34605773%2F34606179%2Fpassword_removal_update.pptx
<Versions</a>), please advise if your component is not compatible with
the recommended version.<br> <br> If you're using a **specific** (out of
common) version of a component listed in the page, you'll either have
to:<br> <br> * move to the common chart (**preferred**)<br> * deal
with the upgrade + explain why you can't use the common chart<br> <br> #
All upstream components should use an upstream (dockerhub, googlehub)
version<br> <br> All upstream components directly used (databases,
kafka, zookeeper, nginx, haproxy, ...) should use directly the upstream
version and not embed it into its own docker<br> <br> # AAF component is
optional<br> <br> AAF is used for certificates and RBAC. It's a non
mandatory requirement, meaning your component **must** be able to not
use it for both features (but this is *enabled* per default).<br> The
mode can be obviously degraded (no HTTPs, no rbac but basicAuth) when
AAF is disabled.<br> <br> Please bear in mind that we're working on
removing needs for AAF for certificates AND need for AAF for RBAC so
this will be tested. (see [AAF Removal]()<br> <br> # MSB component is
optional<br> <br> MSB is interesting when used on non Kubernetes
deployment. But the features proposed are an overlap of basic Kubernetes
features.<br> Furthermore, it prevents to correctly traces dependencies
between ONAP components.<br> <br> As such, your component **must** be
able to not use MSB<br> <br> # Password removal<br> <br> Full status: <a
href=>
Mariadb password removal is done. The work will continue on postgresql
and other common charts.
Work will be started on component (Policy is underway), be prepared to
receive so call for help.
# Pods requests/limits
Every container (including init containers) should have requests/limits
with "good" values (not too high, not too low).
A check will be done and some values will be changed.
A "simple" rule for setting the limits requests:
* request should be the mean use of the container
* limits you should be somewhat higher to expected MAX use.
* For memory, if you can set MAX values in the underlying program
(like in Java), set the max to 1.2 times this value. If not, set 1.5
times of max ever used.
* For CPU, set 2 or 3 times of max ever used as some CPU may be
significantly slower.
# Ingress use as default deployment (instead of node ports)
Instead of using NodePorts, we want to push the use by default of
Ingress. Significative changes will be made when this is ready
(especially removing "NodePort" as default service type from most of
ONAP services).
If your component is using "hardcoded" access to other components via
NodePort (like Portal), please advise as it won't be possible anymore
(but access will still be possible via ClusterIP for internal, Ingress
for external).
Using node ports will still possible, by configuring the different services.
# Dynamic PVC as default deployment (instead of /dockerdata-nfs)
Instead of pushing everything to /dockerdata-nfs, we'll default using
storage class(es). It will still be possible to use /dockerdata-nfs but
won't be the default way.
# Service Mesh PoC
Wiki page:
https://protect2.fireeye.com/url?k=585c7bb2-058f273d-585df0fd-0cc47a31ce52-48d524fa8904d2a1&q=1&u=https%3A%2F%2Fwiki.onap.org%2Fdisplay%2FDW%2FONAP%2BSecurity%2BModel
<https://protect2.fireeye.com/url?k=1fdfdb19-420c8796-1fde5056-0cc47a31ce52-31d79a5417632b74&q=1&u=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__wiki.onap.org_display_DW_ONAP-2BSecurity-2BModel%26d%3DDwQFAw%26c%3DLFYZ-o9_HUMeMTSQicvjIg%26r%3DqLcfee4a2vOwYSub0bljcQ%26m%3DHceOARp59TvXni8a6PJgJVBW81QZ4IHOTjrxRGuNiU0%26s%3D9hJXMi91c1ptqG-ttHA-7LTcSY7y9xAVHQSitAQ-itM%26e%3D>
A PoC using service mesh (Istio), Ingress (component tbd) and RBAC
management (keycloak) will be started. For that, we must be able for
your components to:
* use HTTP (and not HTTPs) as Istio handles the TLS part for both
**server** and **client** part
* disable RBAC / basicAuth as keycloak will do the RBAC for your
component (meaning if a request is coming on your component, it has
already been granted by keycloak) for both **server** and **client** part
Some call for help may be launched if we don't understand your component
behavior.
This work will be done starting by "Core" component (DMAAP, AAI, SDC,
SO, SDNC) and be extended.
I understand it's a very "bold" plan but I believe we can make it and
ONAP deployment will be a lot more consistent!
Best regards,
-- Sylvain
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
