On 10/31/18 4:59 PM, Harald Jensås wrote:
On Wed, 2018-10-31 at 11:39 -0600, Wesley Hayutin wrote:
On Wed, Oct 31, 2018 at 11:21 AM Alex Schultz <aschu...@redhat.com>
wrote:
Hey everyone,
Based on previous emails around this[0][1], I have proposed a
possible
reducing in our usage by switching the scenario001--011 jobs to
non-voting and removing them from the gate[2]. This will reduce the
likelihood of causing gate resets and hopefully allow us to land
corrective patches sooner. In terms of risks, there is a risk that
we
might introduce breaking changes in the scenarios because they are
officially non-voting, and we will still be gating promotions on
these
scenarios. This means that if they are broken, they will need the
same attention and care to fix them so we should be vigilant when
the
jobs are failing.
The hope is that we can switch these scenarios out with voting
standalone versions in the next few weeks, but until that I think
we
should proceed by removing them from the gate. I know this is less
than ideal but as most failures with these jobs in the gate are
either
timeouts or unrelated to the changes (or gate queue), they are more
of
hindrance than a help at this point.
Thanks,
-Alex
I think I also have to agree.
Having to deploy with containers, update containers and run with two
nodes is no longer a very viable option upstream. It's not
impossible but it should be the exception and not the rule for all
our jobs.
afaict in my local environment, the container prep stuff takes ages
when adding the playbooks to update them with yum. We will still have
to do this for every standalone job right?
Also, I enabled profiling for ansible tasks on the undercloud and
noticed that the UndercloudPostDeploy was high on the list, actually
the longest running task when re-running the undercloud install ...
Moving from shell script using openstack cli to python reduced the time
for this task dramatically in my environment, see:
https://review.openstack.org/614540. 6 and half minutes reduced to 40
seconds ...
Everything old is new again:
https://github.com/openstack/instack-undercloud/commit/0eb1b59926c7dc46e321c56db29af95b3d755f34#diff-5602f1b710e86ca1eb7334cb0632f9ee
:-)
How much time would we save in the gates if we converted some of the
shell scripting to python, or if we want to stay in shell script we can
use the interactive shell or use the client-as-a-service[2]?
Interactive shell:
time openstack <<-EOC
server list
workflow list
workflow execution list
EOC
real 0m2.852s
time (openstack server list; \
openstack workflow list; \
openstack workflow execution list)
real 0m7.119s
The difference is significant.
We could cache a token[1], and specify the end-point on each command,
but doing so is still far from as effective as using the interactive.
There is an old thread[2] on the mailing list, which contain a
server/client solution. If we run this service in CI nodes and drop in
the replacement openstack command in /usr/local/bin/openstack we would
use ~1/5 of the time for each command.
(undercloud) [stack@leafs ~]$ time (/usr/bin/openstack network list -f
value -c ID; /usr/bin/openstack network segment list -f value -c ID;
/usr/bin/openstack subnet list -f value -c ID)
real 0m6.443s
user 0m2.171s
sys 0m0.366s
(undercloud) [stack@leafs ~]$ time (/usr/local/bin/openstack network
list -f value -c ID; /usr/local/bin/openstack network segment list -f
value -c ID; /usr/local/bin/openstack subnet list -f value -c ID)
real 0m1.698s
user 0m0.042s
sys 0m0.018s
I relize this is a kind of hacky approch, but it does seem to work and
it should be fairly quick to get in there. (With the Undercloud post
script I see 6 minutes returned, what can we get in CI, 10-15 minutes?
Then we could look at moving these scripts to python or use ansible
openstack modules which hopefully does'nt share the same issues with
loading as the python clients?
I'm personally a fan of using Python as then it is unit-testable, but
I'm not sure how that works with the tht-based code so maybe it's not a
factor.
[1] https://wiki.openstack.org/wiki/OpenStackClient/Authentication
[2]
http://lists.openstack.org/pipermail/openstack-dev/2016-April/092546.html
Thanks Alex
[0]
http://lists.openstack.org/pipermail/openstack-dev/2018-October/136141.html
[1]
http://lists.openstack.org/pipermail/openstack-dev/2018-October/135396.html
[2]
https://review.openstack.org/#/q/topic:reduce-tripleo-usage+(status:open+OR+status:merged
)
___________________________________________________________________
_______
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsu
bscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
WES HAYUTIN
ASSOCIATE MANAGER
Red Hat
whayu...@redhat.com T: +19194232509 IRC: weshay
View my calendar and check my availability for meetings HERE
_____________________________________________________________________
_____
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs
cribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
