[Openstack-operators] [openstack-dev][heat] We need more help for actions and review. And some PTG update for Heat

[Rico Lin]

Dear all

As we reach Stein and start to discuss what we should do in the next cycle,
I would like to raise voice for what kind of help we need and what target
we planning.
BTW, If you really can't start your contact with us with any English (we
don't mind any English skill you are)

*First of all, we need more developers, and reviewer. I would very much
like to give Heat core reviewer title to people if anyone provides fare
quality of reviews. So please help us review patches. Let me know if you
would like to be a part but got no clue on how can you started.*

Second, we need more help to achieve actions. Here I make a list of actions
base on what we discuss from PTG [1]. I mark some of them with (*) if it
looks like an easy contribution:

   - (*) Move interop tempest tests to a separate repo
   - Move python3 functional job to python3.6
   - (*) Implement upgrade check
   - (*) Copy templates from Cue project into the heat-templates repo
   - (*) Add Stein template versions
   - (*) Do document improvement or add documents for:
  - (*) Heat Event Notification list
 - Nice to have our own document and  provide a link to [2]
 - default heat service didn't enable notification, so might be
 mention and link to Notify page
  - (*) Autoscaling doc
  - (*) AutoHealing doc
  - (*) heat agent & heat container agent
  - (*) external resource
  - (*) Upgrade guideline
  - (*) Move document from wiki to in repo document
   - (*) Fix live properties (observe reality) feature and make sure all
   resource works
   - remove any legacy pattern from .zuul.yaml
   - Improve autoscaling and self-healing
   - Create Tempest test for self-healing scenario (around Heat integration)
   - (*) Examine all resource type and help to update if they do not sync
   up with physical resource

If you like to learn more of any above tasks, just reach out to me and
other core members, and we're more than happy to give you the background
and guideline to any of above tasks. Also, welcome to join our meeting and
raise topics for any tasks.
We actually got more tasks that need to be achieved (didn't list here maybe
because it's already start implemented or still under planning), so if you
didn't see any interesting task above, you can reach out to me and let me
know which specific area you're interested in. Also, you might want to go
through [1] or talk to other team members to see if any more comments added
in before you start working on any task.

Now here are some targets that we start to discuss or work in progress

   - Multi-cloud support
   - Within [5], we propose the ability to do multi-cloud orchestration,
  and the follow-on discussion is how can we provide the ability to use
  customized SSL options for multi-cloud or multi-region
  orchestration without violating any security concerns. What we plan to do
  now (after discussing with security sig (Barbican team)) is to
only support
  cacert for SSL which is less sensitive. Use a template file to store that
  cacert and give it to keystone session for providing SSL ability to
  connections. If that sounds like a good idea to all without much
concerns,
  I will implement it asap.
   - Autoscaling and self-healing improvement
  - This is a big complex task for sure and kind of relative to
  multiple projects. We got a fair number of users using
Autoscaling feature,
  but not much for self-healing for now. So we will focus on each
feature and
  the integration of two feature separately.
  - First, Heat got the ability to orchestrate autoscaling, but we need
  to improve the stability. Still go around our code base to see how can we
  modulize current implementation, and how can we improve from
here, but will
  update more information for all. We also starting to discuss autoscaling
  integration [3], which hopefully we can get a better solution and combine
  forces from Heat and Senlin as a long-term target. Please give your
  feedback if you also care about this target.
  - For self-healing, we propose some co-work on cross-project gatting
  in Self-healing-sig, which we still not generate tempest test out, but
  assume we can start to set up job and raise discussion for how
can we help
  projects to adopt that job. Also, we got discussions with Octavia team
  ([7], and [8]) and Monasca team about adopting the ability to
support event
  alarm/notification. Which we plan to put into actions. If anyone
also think
  those are important features, please provide your development
resources so
  we can get those feature done in this cycle.
  - For integrating two scenarios, I try to add more tasks into [6] and
  eliminate as many as we can. Also, plan to work on document
these scenarios
  down, so everyone can play with autoscaling+self-healing easily.
   - Glance resource update
  - We deprecate image resource in 

Re: [Openstack-operators] [openstack-dev] [Openstack-sigs] [tc]Global Reachout Proposal

[Anita Kuno]

On 2018-09-18 08:40 AM, Jeremy Stanley wrote:

On 2018-09-18 11:26:57 +0900 (+0900), Ghanshyam Mann wrote:
[...]

I can understand that IRC cannot be used in China which is very
painful and mostly it is used weChat.

[...]

I have yet to hear anyone provide first-hand confirmation that
access to Freenode's IRC servers is explicitly blocked by the
mainland Chinese government. There has been a lot of speculation
that the usual draconian corporate firewall policies (surprise, the
rest of the World gets to struggle with those too, it's not just a
problem in China) are blocking a variety of messaging protocols from
workplace networks and the people who encounter this can't tell the
difference because they're already accustomed to much of their other
communications being blocked at the border. I too have heard from
someone who's heard from someone that "IRC can't be used in China"
but the concrete reasons why continue to be missing from these
discussions.



I'll reply to this email arbitrarily in order to comply with Zhipeng 
Huang's wishes that the conversation concerned with understanding the 
actual obstacles to communication takes place on the mailing list. I do 
hope I am posting to the correct thread.


In response to part of your comment on the patch at 
https://review.openstack.org/#/c/602697/ which you posted about 5 hours 
ago you said "@Anita you are absolutely right it is only me stuck my 
head out speaks itself the problem I stated in the patch. Many of the 
community tools that we are comfortable with are not that accessible to 
a broader ecosystem. And please assured that I meant I refer the patch 
to the Chinese community, as Leong also did on the ML, to try to bring 
them over to join the convo." and I would like to reply.


I would like to say that I am honoured by your generosity. Thank you. 
Now, when the Chinese community consumes the patch, as well as the 
conversation in the comments, please encourage folks to ask for 
clarification if any descriptions or phrases don't make sense to them. 
One of the best ways of ensuring clear communication is to start off 
slowly and take the time to ask what the other side means. It can seem 
tedious and a waste of time, but I have found it to be very educational 
and helpful in understanding how the other person perceives the 
situation. It also helps me to understand how I am creating obstacles in 
ways that I talk.


Taking time to clarify helps me to adjust how I am speaking so that my 
meaning is more likely to be understood by the group to which I am 
trying to offer my perspective. I do appreciate that many people are 
trying to avoid embarrassment, but I have never found any way to 
understand people in a culture that is not the one I group up in, other 
than embarrassing myself and working through it. Usually I find the 
group I am wanting to understand is more than willing to rescue me from 
my embarrassment and support me in my learning. In a strange way, the 
embarrassment is kind of helpful in order to create understanding 
between myself and those people I am trying to understand.


Thank you, Anita

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] Fwd: Denver Ops Meetup post-mortem

[Jimmy McArthur]

Thanks for the thorough write-up as well as the detailed feedback.  I'm 
including some of my notes from the Ops Meetup Feedback session just a 
bit below, as well as some comments inline.


One of the critical things that would help both the Ops and Dev 
community is to have a holistic sense of what the Ops Meetup goals are.


 * Were the goals well defined ahead of the event?
 * Were they achieved and/or how can the larger OpenStack community
   help them achieve them?

From our discussion at the Feedback session, this isn't something that 
has been tracked in the past.  Having actionable, measurable goals 
coming out of the Ops Meetup could go a long way towards helping the 
projects realize them.  Per our discussion, being able to present this 
list to the User Committee would be a good step forward for each event.


I wasn't able to attend the entire time, but a couple of interesting notes:

 * The knowledge of deployment tools seemed pretty fragmented and it
   seemed like there was a desire for more clear and comprehensive
   documentation comparing the different deployment options, as well as
   documentation about how to get started with a POC.
 * Bare Metal in the Datacenter: It was clear that we need more Ironic
   101 content and education, including how to get started, system
   requirements, etc. We can dig up presentations from previous Summits
   and also talked to TheJulia about potentially hosting a community
   meeting or producing another video leading up to the Berlin Summit.
 * Here are the notes from the sessions in case anyone on the ops list
   is interested:
   https://etherpad.openstack.org/p/ops-meetup-ptg-denver-2018

It looks like there were some action items documented at the bottom of 
this etherpad: https://etherpad.openstack.org/p/ops-denver-2018-further-work


Ops Meetup Feedback Takeways from Feedback Session not covered below 
(mostly from https://etherpad.openstack.org/p/uc-stein-ptg)

Chris Morgan wrote:

--SNIP --

What went well

- some of the sessions were great and a lot of progress was made
- overall attendance in the ops room was good
We had to add 5 tables to accommodate the additional attendees. It was a 
great crowd!

- more developers were able to join the discussions
Given that this is something that wouldn't happen at a normal Ops 
Meetup, is there a way that would meet the Ops Community needs that we 
could help facilitate this int he future?

- facilities were generally fine
- some operators leveraged being at PTG to have useful involvement in 
other sessions/discussions such as Keystone, User Committee, 
Self-Healing SIG, not to mention the usual "hallway conversations", 
and similarly some project devs were able to bring pressing questions 
directly to operators.


What didn't go so well:

- Merging into upgrade SIG didn't go particularly well
This is a tough one b/c of the fluidity of the PTG. Agreed that one can 
end up missing a good chunk of the discussion.  OTOH, the flexibility of 
hte event is what allows great discussions to take place.  In the 
future, I think better coordination w/ specific project teams + updating 
the PTGBot could help make sure the schedules are in synch.

- fewer ops attended (in particular there were fewer from outside the US)
Do you have demographics on the Ops Meetup in Japan or NY?  Curious to 
know how those compare to what we saw in Denver.  If there is more 
promotion needed, or indeed these just end up being more 
continent/regionally focused?

- Some of the proposed sessions were not well vetted
Are there any suggestions on how to improve this moving forward?  
Perhaps a CFP style submission process, with a small vetting group, 
could help this situation?  My understanding was the Tokyo event, 
co-located with OpenStack Days, didn't suffer this problem.
- some ops who did attend stated the event identity was diluted, it 
was less attractive
I'd love some more info on this. Please have these people reach out to 
let me know how we can fix this in the future.  Even if we decide not to 
hold another Ops Meetup at a PTG, this is relevant to how we run events.
- we tried to adjust the day 2 schedule to include late submissions, 
however it was probably too late in some cases


I don't think it's so important to drill down into all the whys and 
wherefores of how we fell down here except to say that the ops meetups 
team is a small bunch of volunteers all with day jobs (presumably just 
like everyone else on this mailing list). The usual, basically.


Much more important : what will be done to improve things going forward:

- The User Committee has offered to get involved with the technical 
content. In particular to bring forward topics from other relevant 
events into the ops meetup planning process, and then take output from 
ops meetups forward to subsequent events. We (ops meetup team) have 
welcomed this.
This is super critical IMO.  One of the things we discussed at the Ops 
Meetup Feedback session (co-located w

[Openstack-operators] [openstack-dev] [horizon] Dashboard memory leaks

[Xingchao]

Hi All,

Recently, we found the server which hosts horizon dashboard had serveral
times OOM caused by horizon services. After restarting the dashboard, the
memory usage goes up very quickly if we access /project/network_topology/
path.

*How to reproduce*

Login into the dashboard and go to 'Network Topology' tab, then leave it
there (autorefresh 10s by default), now monitor the memory changes on the
host.

*Versions and Components*

Dashboard: Stable/Pike
Server: uWSGI 1.9.17-1
OS: Ubuntu 14.04 trusty
Python: 2.7.6

As the codes of memoized has little changes since Pike, if you use
Queen/Rocky release, you may also succeed to reproduce it.

*The investigation*

The root cause of the memory leak is the decorator
memorized(horizon/utils/memoized.py) which is used to cache function calls
in Horizon.

After disable it, the memory increases has been controlled.

The following is the comparison of memory change(with guppy) for each
request of /project/network_topology:

 - original (no code change) 684kb

 - do garbage collection manually 185kb

 - disable memorize cache 10kb

As we known, memoized uses weakref to cache objects. A weak reference to an
object is not enough to keep the object alive: when the only remaining
references to a referent are weak references, garbage collection is free to
destroy the referent and reuse its memory for something else.

In the memory, we could see lots of weakref stuffs, the following is a
example:




*Partition of a set of 394 objects. Total size = 37824 bytes. Index Count %
Size % Cumulative % Kind (class / dict of class) 0 197 50 18912 50
18912 50 _cffi_backend.CDataGCP 1 197 50 18912 50 37824 100
weakref.KeyedRefq*

But the rest of them are not. the following result is the memory objects
changes of per /project/network_topology access with garbage collection
manually.












*Partition of a set of 1017 objects. Total size = 183680 bytes. Index Count
% Size % Cumulative % Referrers by Kind (class / dict of class) 0 419
41 58320 32 58320 32 dict (no owner) 1 100 10 23416 13 81736 44
list 2 135 13 15184 8 96920 53  3 2 0 6704 4 103624 56
urllib3.connection.VerifiedHTTPSConnection 4 2 0 6704 4 110328 60
urllib3.connectionpool.HTTPSConnectionPool 5 1 0 3352 2 113680 62
novaclient.v2.client.Client 6 2 0 2096 1 115776 63
OpenSSL.SSL.Connection 7 2 0 2096 1 117872 64 OpenSSL.SSL.Context 8
2 0 2096 1 119968 65 Queue.LifoQueue 9 12 1 2096 1 122064 66 dict of
urllib3.connectionpool.HTTPSConnectionPool*

The most of them are dicts. Followings are the dicts sorted by class, as
you can see most of them are not weakref objects:








*Partition of a set of 419 objects. Total size = 58320 bytes. Index Count %
Size % Cumulative % Class 0 362 86 50712 87 50712 87 unicode 1 27 6
3736 6 54448 93 list 2 5 1 2168 4 56616 97 dict 3 22 5 1448 2 58064
100 str 4 2 0 192 0 58256 100 weakref.KeyedRef 5 1 0 64 0 58320 100
keystoneauth1.discover.Discover*

*The issue*

So the problem is that memoized does not work like what we expect. It
allocates memory to cache objects but some of them could not be released.
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[Openstack-operators] zun-ui install

[David Ivey]

Hi,

I am having issues getting zun-ui to work in my environment. it is a
multinode deployment with queens on Ubuntu16.04 . I installed zun-ui based
on the instructions from the stable/queens branch at
https://github.com/openstack/zun-ui. I can confirm that everything works
with openstack-dashboard, heat-dashboard, designate-dashboard before adding
zun-ui. Turning debug on gives me the following error.

Request Method: POST Request URL: http://10.10.5.161/horizon/auth/login/
Django Version: 1.11.15 Python Version: 2.7.12 Installed Applications:
['openstack_dashboard.dashboards.project', 'zun_ui', 'heat_dashboard',
'designatedashboard', 'openstack_dashboard.dashboards.admin',
'openstack_dashboard.dashboards.identity',
'openstack_dashboard.dashboards.settings', 'openstack_dashboard',
'django.contrib.contenttypes', 'django.contrib.auth',
'django.contrib.sessions', 'django.contrib.messages',
'django.contrib.staticfiles', 'django.contrib.humanize', 'django_pyscss',
'openstack_dashboard.django_pyscss_fix', 'compressor', 'horizon',
'openstack_auth'] Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'horizon.middleware.OperationLogMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'horizon.middleware.HorizonMiddleware', 'horizon.themes.ThemeMiddleware',
'django.middleware.locale.LocaleMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerClientMiddleware',
'openstack_dashboard.contrib.developer.profiler.middleware.ProfilerMiddleware')
Traceback: File
"/usr/local/lib/python2.7/dist-packages/django/core/handlers/exception.py"
in inner 41. response = get_response(request) File
"/usr/local/lib/python2.7/dist-packages/django/core/handlers/base.py" in
_legacy_get_response 244. response = middleware_method(request) File
"/usr/share/openstack-dashboard/horizon/middleware/base.py" in
process_request 52. if not hasattr(request, "user") or not
request.user.is_authenticated(): Exception Type: TypeError at /auth/login/
Exception Value: 'bool' object is not callable

Other than that I do not see any other errors except the "Something went
wrong! An unexpected error has occurred. Try refreshing the page. If that
doesn't help, contact your local administrator." when I go to the
dashboard..

Thanks in advance
David
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [Openstack-operators] [openstack-dev] [all] Consistent policy names

[Lance Bragstad]

johnsom (from octavia) had a good idea, which was to use the service types
that are defined already [0].

I like this for three reasons, specifically. First, it's already a known
convention for services that we can just reuse. Second, it includes a
spacing convention (e.g. load-balancer vs load_balancer). Third,
it's relatively short since it doesn't include "os" or "api".

So long as there isn't any objection to that, we can start figuring out how
we want to do the method and resource parts. I pulled some policies into a
place where I could try and query them for specific patterns and existing
usage [1]. With the representation that I have (nova, neutron, glance,
cinder, keystone, mistral, and octavia):

- *create* is favored over post (105 occurrences to 7)
- *list* is favored over get_all (74 occurrences to 28)
- *update* is favored over put/patch (91 occurrences to 10)

>From this perspective, using the HTTP method might be slightly redundant
for projects using the DocumentedRuleDefault object from oslo.policy since
it contains the URL and method for invoking the policy. It also might
differ depending on the service implementing the API (some might use put
instead of patch to update a resource). Conversely, using the HTTP method
in the policy name itself doesn't require use of DocumentedRuleDefault,
although its usage is still recommended.

Thoughts on using create, list, update, and delete as opposed to post, get,
put, patch, and delete in the naming convention?

[0] https://service-types.openstack.org/service-types.json
[1]
https://gist.github.com/lbragstad/5000b46f27342589701371c88262c35b#file-policy-names-yaml

On Sun, Sep 16, 2018 at 9:47 PM Lance Bragstad  wrote:

> If we consider dropping "os", should we entertain dropping "api", too? Do
> we have a good reason to keep "api"?
>
> I wouldn't be opposed to simple service types (e.g "compute" or
> "loadbalancer").
>
> On Sat, Sep 15, 2018 at 9:01 AM Morgan Fainberg 
> wrote:
>
>> I am generally opposed to needlessly prefixing things with "os".
>>
>> I would advocate to drop it.
>>
>>
>> On Fri, Sep 14, 2018, 20:17 Lance Bragstad  wrote:
>>
>>> Ok - yeah, I'm not sure what the history behind that is either...
>>>
>>> I'm mainly curious if that's something we can/should keep or if we are
>>> opposed to dropping 'os' and 'api' from the convention (e.g.
>>> load-balancer:loadbalancer:post as opposed to
>>> os_load-balancer_api:loadbalancer:post) and just sticking with the
>>> service-type?
>>>
>>> On Fri, Sep 14, 2018 at 2:16 PM Michael Johnson 
>>> wrote:
>>>
 I don't know for sure, but I assume it is short for "OpenStack" and
 prefixing OpenStack policies vs. third party plugin policies for
 documentation purposes.

 I am guilty of borrowing this from existing code examples[0].

 [0]
 http://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/policy-in-code.html

 Michael
 On Fri, Sep 14, 2018 at 8:46 AM Lance Bragstad 
 wrote:
 >
 >
 >
 > On Thu, Sep 13, 2018 at 5:46 PM Michael Johnson 
 wrote:
 >>
 >> In Octavia I selected[0] "os_load-balancer_api:loadbalancer:post"
 >> which maps to the "os--api::" format.
 >
 >
 > Thanks for explaining the justification, Michael.
 >
 > I'm curious if anyone has context on the "os-" part of the format?
 I've seen that pattern in a couple different projects. Does anyone know
 about its origin? Was it something we converted to our policy names because
 of API names/paths?
 >
 >>
 >>
 >> I selected it as it uses the service-type[1], references the API
 >> resource, and then the method. So it maps well to the API
 reference[2]
 >> for the service.
 >>
 >> [0]
 https://docs.openstack.org/octavia/latest/configuration/policy.html
 >> [1] https://service-types.openstack.org/
 >> [2]
 https://developer.openstack.org/api-ref/load-balancer/v2/index.html#create-a-load-balancer
 >>
 >> Michael
 >> On Wed, Sep 12, 2018 at 12:52 PM Tim Bell  wrote:
 >> >
 >> > So +1
 >> >
 >> >
 >> >
 >> > Tim
 >> >
 >> >
 >> >
 >> > From: Lance Bragstad 
 >> > Reply-To: "OpenStack Development Mailing List (not for usage
 questions)" 
 >> > Date: Wednesday, 12 September 2018 at 20:43
 >> > To: "OpenStack Development Mailing List (not for usage questions)"
 , OpenStack Operators <
 openstack-operators@lists.openstack.org>
 >> > Subject: [openstack-dev] [all] Consistent policy names
 >> >
 >> >
 >> >
 >> > The topic of having consistent policy names has popped up a few
 times this week. Ultimately, if we are to move forward with this, we'll
 need a convention. To help with that a little bit I started an etherpad [0]
 that includes links to policy references, basic conventions *within* that
 service, and some examples of each. I got through quite a few projects th

Re: [Openstack-operators] [openstack-dev] Forum Topic Submission Period

[Jimmy McArthur]

Sylvain Bauza wrote:



Le mer. 19 sept. 2018 à 00:41, Jimmy McArthur > a écrit :

SNIP



Same as I do :-) Unrelated point, for the first time in all the 
Summits I know, I wasn't able to know the track chairs for a specific 
track. Ideally, I'd love to reach them in order to know what they 
disliked in my proposal.
They were listed on an Etherpad that was listed under Presentation 
Selection Process in the CFP navigation. That has since been overwritten 
w/ Forum Selection Process, so let me try to dig that up.  We publish 
the Track Chairs every year.



SNIP


I have another question, do you know why we can't propose a Forum 
session with multiple speakers ? Is this a bug or an expected 
behaviour ? In general, there is only one moderator for a Forum 
session, but in the past, I clearly remember we had some sessions that 
were having multiple moderators (for various reasons).
Correct. Forum sessions aren't meant to have speakers like a normal 
presentation. They are all set up parliamentary style w/ one or more 
moderators. However, the moderator can manage the room any way they'd 
like.  If you want to promote the people that will be in the room, this 
can be added to the abstract.


-Sylvain


Cheers,
Jimmy




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators