Thanks for taking this on Ian! I'm fully on board with the effort. I like the consolidation and performance improvements. Storing t-h-t templates in Swift worked okay 3-4 years ago. Now that we have more templates, many of which need .j2 rendering the storage there has become quite a bottleneck.
Additionally, since we'd be sending commands to Heat via local filesystem template storage we could consider using softlinks again within t-h-t which should help with refactoring and deprecation efforts. Dan On Wed, Aug 1, 2018 at 7:35 PM Ian Main <im...@redhat.com> wrote: > > > Hey folks! > > So I've been working on some patches to speed up plan operations in TripleO. > This was originally driven by the UI needing to be able to perform a 'plan > upload' in something less than several minutes. :) > > https://review.openstack.org/#/c/581153/ > https://review.openstack.org/#/c/581141/ > > I have a functioning set of patches, and it actually cuts over 2 minutes off > the overcloud deployment time. > > Without patch: > + openstack overcloud plan create --templates > /home/stack/tripleo-heat-templates/ overcloud > Creating Swift container to store the plan > Creating plan from template files in: /home/stack/tripleo-heat-templates/ > Plan created. > real 3m3.415s > > With patch: > + openstack overcloud plan create --templates > /home/stack/tripleo-heat-templates/ overcloud > Creating Swift container to store the plan > Creating plan from template files in: /home/stack/tripleo-heat-templates/ > Plan created. > real 0m44.694s > > This is on VMs. On real hardware it now takes something like 15-20 seconds > to do the plan upload which is much more manageable from the UI standpoint. > > Some things about what this patch does: > > - It makes use of process-templates.py (written for the undercloud) to > process the jinjafied templates. This reduces replication with the existing > version in the code base and is very fast as it's all done on local disk. > - It stores the bulk of the templates as a tarball in swift. Any individual > files in swift take precedence over the contents of the tarball so it should > be backwards compatible. This is a great speed up as we're not accessing a > lot of individual files in swift. > > There's still some work to do; cleaning up and fixing the unit tests, testing > upgrades etc. I just wanted to get some feedback on the general idea and > hopefully some reviews and/or help - especially with the unit test stuff. > > Thanks everyone! > > Ian > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
