History lesson: a long, long time ago we made a very big mistake. We
treated stack outputs as things that would be resolved dynamically when
you requested them, instead of having values fixed at the time the
template was created or updated. This makes performance of reading
outputs slow, especially for e.g. large stacks, because it requires
making ReST calls, and it can result in inconsistencies between Heat's
internal model of the world and what it actually outputs.
As unfortunate as this is, it's difficult to change the behaviour and be
certain that no existing users will get broken. For that reason, this
issue has never been addressed. Now is the time to address it.
Here's the tracker bug: https://bugs.launchpad.net/heat/+bug/1660831
It turns out that the correct fix is to store the attributes of a
resource in the DB - this accounts for the fact that outputs may contain
attributes of multiple resources, and that these resources might get
updated at different times. It also solves a related consistency issue,
which is that during a stack update a resource that is not updated may
nevertheless report new attribute values, and thus cause things
downstream to be updated, or to fail, unexpectedly (e.g.
https://bugzilla.redhat.com/show_bug.cgi?id=1430753#c13).
The proposal[1] is to make this change in Pike for convergence stacks
only. This is to allow some warning for existing users who might be
relying on the current behaviour - at least if they control their own
cloud then they can opt to keep convergence disabled, and even once they
opt to enable it for new stacks they can keep using existing stacks in
legacy mode until they are ready to convert them to convergence or
replace them. In addition, it avoids the difficulty of trying to get
consistency out of the legacy path's crazy backup-stack shenanigans -
there's basically no way to get the outputs to behave in exactly the
same way in the legacy path as they will in convergence.
This topic was raised at the Forum, and there was some feedback that:
1) There are users who require the old behaviour even after they move to
convergence.
2) Specifically, there are users who don't have public API endpoints for
services other than Heat, and who rely on Heat proxying requests to
other services to get any information at all about their resources o.O
3) There are users still using the legacy path (*cough*TripleO) that
want the performance benefits of quick output resolution.
The suggestion is that instead of tying the change to the convergence
flag, we should make it configurable by the user on a per-stack basis.
I am vehemently opposed to this suggestion.
It's a total cop-out to make the user decide. The existing behaviour is
clearly buggy and inconsistent. Users are not, and should not have to
be, sufficiently steeped in the inner workings of Heat to be able to
decide whether and when to subject themselves to random inconsistencies
and hope for the best. If we make the change the default then we'll
still break people, and if we don't we'll still be saying "OMG, you
forgot to enable the --not-suck option??!" 10 years from now.
Instead, this is what I'm proposing as the solution to the above feedback:
1) The 'show' attribute of each resource will be marked CACHE_NONE[2].
This ensures that the live data is always available via this attribute.
2) When showing a resource's attributes via the API (as opposed to
referencing them from within a template), always return live values.[3]
Since we only store the attribute values that are actually referenced in
the template anyway, we more or less have to do this if we want the
attributes output through this API to be consistent with each other.
3) Move to convergence. Seriously, the memory and database usage are
much improved, and there are even more memory improvements in the
pipeline,[4] and they might even get merged in Pike as long as we don't
have to stop and reimplement the attribute storage patches that they
depend on. If TripleO were to move to convergence in Queens, which I
believe is 100% feasible, then it would get the performance improvements
at least as soon as it would if we tried to implement attribute storage
in the legacy path.
Is anyone still dissatisfied? Speak now or... you know the drill.
cheers,
Zane.
[1]
https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bug/1660831
[2] https://review.openstack.org/#/c/422983/33/heat/engine/resource.py
[3] https://review.openstack.org/472501
[4]
https://review.openstack.org/#/q/status:open+project:openstack/heat+topic:bp/stack-definition
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
