Re: [openstack-dev] [keystone]PKI token VS Fernet token

[Clint Byrum]

Excerpts from joehuang's message of 2017-02-25 04:09:45 +:
> Hello, Matt,
> 
> Thank you for your reply, just as what you mentioned, for the slow changed 
> data, aync. replication should work. My concerns is that the impact of 
> replication delay, for example (though it's quite low chance to happen):
> 
> 1) Add new user/group/role in RegionOne, before the new user/group/role are 
> replicated to RegionTwo, the new user begin to access RegionTwo service, then 
> because the data has not arrived yet, the user's request to RegionTwo may be 
> rejected for the token vaildation failed in local KeyStone.
> 

I think this is entirely acceptable. You can even check with your
monitoring system to find out what the current replication lag is to
each region, and notify the user of how long it may take.

> 2)In token revoke case. If we remove the user'role in RegionOne, the token in 
> RegionOne will be invalid immediately, but before the remove operation 
> replicated to the RegionTwo, the user can still use the token to access the 
> services in RegionTwo. Although it may last in very short interval.
> 
> Is there someone can evaluate the security risk is affordable or not.
> 

The simple answer is that the window between a revocation event being
created, and being ubiquitous, is whatever the maximum replication lag
is between regions. So if you usually have 5 seconds of replication lag,
it will be 5 seconds. If you have a really write-heavy day, and you
suddenly have 5 minutes of replication lag, it will be 5 minutes.

The complicated component is that in async replication, reducing
replication lag is expensive. You don't have many options here. Reducing
writes on the master is one of them, but that isn't easy! Another is
filtering out tables on slaves so that you only replicate the tables
that you will be reading. But if there are lots of replication events,
that doesn't help.

One decent option is to switch to semi-sync replication:

https://dev.mysql.com/doc/refman/5.7/en/replication-semisync.html

That will at least make sure your writes aren't acknowledged until the
binlogs have been transferred everywhere. But if your master can take
writes a lot faster than your slaves, you may never catch up applying , no 
matter
how fast the binlogs are transferred.

The key is to evaluate your requirements and think through these
solutions. Good luck! :)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone]PKI token VS Fernet token

[Matt Fischer]

On Fri, Feb 24, 2017 at 9:09 PM, joehuang  wrote:

> Hello, Matt,
>
> Thank you for your reply, just as what you mentioned, for the slow changed
> data, aync. replication should work. My concerns is that the impact of
> replication delay, for example (though it's quite low chance to happen):
>
> 1) Add new user/group/role in RegionOne, before the new user/group/role
> are replicated to RegionTwo, the new user begin to access RegionTwo
> service, then because the data has not arrived yet, the user's request to
> RegionTwo may be rejected for the token vaildation failed in local
> KeyStone.
>
> 2)In token revoke case. If we remove the user'role in RegionOne, the token
> in RegionOne will be invalid immediately, but before the remove operation
> replicated to the RegionTwo, the user can still use the token to access the
> services in RegionTwo. Although it may last in very short interval.
>
> Is there someone can evaluate the security risk is affordable or not.
>
> Best Regards
> Chaoyi Huang (joehuang)
>
>

We actually had this happen for services like neutron even within a region,
where a network was created on one node and then immediately used on a
second node. We solved it by forcing haproxy to do transactions on one node
(with the others as backups). I only mention this because the scenario you
propose is possible to occur. If you are not dealing with a bunch of data
you could look into enabling causal reads (assuming you are using mysql
galera), but this will probably cause a perf hit (I did not test the
impact).

For scenario 2: I suppose you need to ask yourself, if I remove a user or
role, can I live with 2-5 seconds for that token to be revoked in all
regions? In our case it was not a major concern, but I worked on private
cloud.

For scenario 1: If I were you I think you should figure out whether or not
it's ever likely to really happen before you invest a bunch of time into
solving it. That will depend a lot on your sync time. We only had 2 regions
and we owned the pipes so it was not a major concern.

Sorry I don't have more definite answers for you.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone]PKI token VS Fernet token

[joehuang]

Hello, Matt,

Thank you for your reply, just as what you mentioned, for the slow changed 
data, aync. replication should work. My concerns is that the impact of 
replication delay, for example (though it's quite low chance to happen):

1) Add new user/group/role in RegionOne, before the new user/group/role are 
replicated to RegionTwo, the new user begin to access RegionTwo service, then 
because the data has not arrived yet, the user's request to RegionTwo may be 
rejected for the token vaildation failed in local KeyStone.

2)In token revoke case. If we remove the user'role in RegionOne, the token in 
RegionOne will be invalid immediately, but before the remove operation 
replicated to the RegionTwo, the user can still use the token to access the 
services in RegionTwo. Although it may last in very short interval.

Is there someone can evaluate the security risk is affordable or not.

Best Regards
Chaoyi Huang (joehuang)

From: Matt Fischer [m...@mattfischer.com]
Sent: 25 February 2017 11:38
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [keystone]PKI token VS Fernet token


At last, we still have one question:
For public cloud, it is very common that multi regions are deployed. And the 
distance is usually very far between the regions. So the transport delay is 
really a problem. Fernet token requires the data must be the same. Because of 
the slow connection and high time delay, in our opinion, it is unrealistic that 
let the keystones from different regions to use the same keystone datacenter. 
Any idea about this problem? Thanks.



There's nothing in Fernet tokens that would cause an issue with the 
transportation delay. You could mail the Fernet keys to each region and you're 
still fine, why? Because key rotation means that the "next key" is already in 
place on every box when you rotate keys. There is a widely held misconception 
that all keystone nodes must instantaneously sync keys in every region or it 
won't work, that is simply not true. In fact the main reason we switched to 
Fernet was to REDUCE the load on our cross-region replication. Without a 
database full of tokens to deal with, there's basically nothing to replicate as 
joe says below. User/group/role changes for us was more of a few times a day 
operation rather than getting a token which is thousands of times per second.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone]PKI token VS Fernet token

[Matt Fischer]

>
>
> At last, we still have one question:
> For public cloud, it is very common that multi regions are deployed. And
> the distance is usually very far between the regions. So the transport
> delay is really a problem. Fernet token requires the data must be the same.
> Because of the slow connection and high time delay, in our opinion, it is 
> unrealistic
> that let the keystones from different regions to use the same keystone
> datacenter. Any idea about this problem? Thanks.
>
>
>

There's nothing in Fernet tokens that would cause an issue with the
transportation delay. You could mail the Fernet keys to each region and
you're still fine, why? Because key rotation means that the "next key" is
already in place on every box when you rotate keys. There is a widely held
misconception that all keystone nodes must instantaneously sync keys in
every region or it won't work, that is simply not true. In fact the main
reason we switched to Fernet was to REDUCE the load on our cross-region
replication. Without a database full of tokens to deal with, there's
basically nothing to replicate as joe says below. User/group/role changes
for us was more of a few times a day operation rather than getting a token
which is thousands of times per second.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack] OpenStack Ocata for Ubuntu 16.04 LTS

[Emilien Macchi]

FYI Puppet OpenStack CI is deploying Ocata with the stable repo and it
works super well now.

Quick feedback, it worked pretty well during Ocata cycle and we didn't
have as much breakages like in the past (Newton or Mitaka...), so keep
rocking!

Thanks,

On Wed, Feb 22, 2017 at 10:09 PM, Corey Bryant
 wrote:
> Hi All,
>
>
> The Ubuntu OpenStack team at Canonical is pleased to announce the general
> availability of OpenStack Ocata on Ubuntu 16.04 LTS via the Ubuntu Cloud
> Archive. Ocata is the latest release of OpenStack and the first in the new
> OpenStack release cadence, with a focus on stabilization of existing
> features along with usability improvements. Details of the Ocata release can
> be found at:  https://www.openstack.org/software/ocata
>
>
> To get access to the Ubuntu Ocata packages:
>
>
> Ubuntu 16.04 LTS
>
> 
>
>
> You can enable the Ubuntu Cloud Archive pocket for OpenStack Ocata on Ubuntu
> 16.04 installations by running the following commands:
>
>
> sudo add-apt-repository cloud-archive:ocata
>
> sudo apt update
>
>
> The Ubuntu Cloud Archive for Ocata includes updates for:
>
>
> aodh, barbican, ceilometer, cinder, congress, designate,
> designate-dashboard, dpdk (16.11), glance, gnocchi, heat, horizon, ironic,
> libvirt (2.5.0), keystone, magnum, manila, manila-ui, mistral, murano,
> murano-dashboard, networking-ovn, networking-sfc, neutron,
> neutron-dynamic-routing, neutron-fwaas, neutron-lbaas,
> neutron-lbaas-dashboard, nova, nova-lxd, openstack-trove, panko, qemu (2.8),
> sahara, sahara-dashboard, senlin, swift, trove-dashboard, watcher and zaqar
>
>
> For a full list of packages and versions, please refer to [0].
>
>
> Migration considerations
>
> 
>
> * API’s now running under apache2 with mod_wsgi
>
>
> In Ocata we’ve updated the following APIs to run under apache2 with
> mod_wsgi: aodh-api, barbican-api, ceilometer-api, cinder-api, and the
> nova-placement-api. Please keep this in mind as the packages will no longer
> install systemd unit files for these services, and will instead install
> apache2 with corresponding apache2 site configuration.
>
>
> * libvirt 2.5.0 changes
>
>
> In this release of libvirt, the libvirt-bin systemd service has been renamed
> to libvirtd, and the unix_sock_group has changed from libvirtd to libvirt.
>
>
> * openstack-dashboard changes
>
>
> In Ocata we’ve moved static assets from
> /usr/share/openstack-dashboard/static to
> /var/lib/openstack-dashboard/static.
>
>
> Known Issues
>
> ---
>
> nova console log is empty with libvirt 2.5.0:
> https://bugs.launchpad.net/bugs/1667033
>
>
> Branch Package Builds
>
> ---
>
> If you would like to try out the latest updates to branches, we deliver
> continuously integrated packages on each upstream commit via the following
> PPA’s:
>
>
>   sudo add-apt-repository ppa:openstack-ubuntu-testing/liberty
>
>   sudo add-apt-repository ppa:openstack-ubuntu-testing/mitaka
>
>   sudo add-apt-repository ppa:openstack-ubuntu-testing/newton
>
>   sudo add-apt-repository ppa:openstack-ubuntu-testing/ocata
>
>
> Reporting bugs
>
> -
>
> If you have any issues please report bugs using the 'ubuntu-bug' tool to
> ensure that bugs get logged in the right place in Launchpad:
>
>
> sudo ubuntu-bug nova-conductor
>
>
> Thanks to everyone who has contributed to OpenStack Ocata, both upstream and
> downstream.  And special thanks to the puppet modules team for their
> continued early testing of OpenStack development releases.
>
>
> Have fun and see you in Pike!
>
>
> Regards,
>
> Corey
>
> (on behalf of the Ubuntu OpenStack team)
>
>
> [0]
> http://reqorts.qa.ubuntu.com/reports/ubuntu-server/cloud-archive/ocata_versions.html
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet] unit tests on puppet-keystone are broken

[Emilien Macchi]

I haven't spent time to debug yet but in case someone has an idea, the
unit tests on puppet-keystone are failing:
http://logs.openstack.org/16/438116/1/check/gate-puppet-keystone-puppet-unit-3.6-centos-7/7a9358f/console.html#_2017-02-24_23_29_26_977959

https://bugs.launchpad.net/puppet-keystone/+bug/1667866

Any help on this one is welcome,

Thanks!
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] PTG + Release status

[Emilien Macchi]

Before leaving in week-end, I thought useful to share some infos with the team.

First of all, it was awesome to meet people face to face, really. I'm
already looking forward to the next time.

About Ocata release, it seems like we're getting very close to be
ready for RC2, probably next week. I've been discussed with upgrade
team and we have defined most of blockers, that would hopefully be
fixed next week.
Also breaking news: the upgrade job is now testing Nova upgrades from
Newton to Ocata and we've got a successful pingtest:
http://logs.openstack.org/55/438055/1/check/gate-tripleo-ci-centos-7-multinode-upgrades-nv/353b16a/console.html#_2017-02-25_00_58_53_067521
First time we run an actual overcloud upgrade in CI with all our
critical services. Big kudos to the team!

Next week, I'll take some time to prepare the Launchpad for Pike
cycle. I'll probably reach some of you regarding the blueprints we
discussed at PTG.
I'll also write a summary on the mailing-list of what happened this week.

For those who travel this week-end have a safe trip back to home!

See you online next week,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all][qa] Use stable call_until_true()

[Ken'ichi Ohmichi]

Hi,

QA team defines stable interfaces of Tempest for using them from
outside(tempest plugins).
We are removing a deprecated call_until_true() method from unstable
module(test.py of Tempest), but some projects are still using it.
We have prepared patches[1] for soft-landing on these projects,
hopefully these patches are merged before a Tempest patch[2] which
removes the method.

Thanks
Ken Ohmichi

---
[1]: https://review.openstack.org/#/q/status:open+topic:switch-call_until_true
[2]: https://review.openstack.org/#/c/431701/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry][heat] Removal of Aodh combination alarms

[gordon chung]

On 24/02/17 06:15 AM, Julien Danjou wrote:
> Hey,
>
> We're about to remove the code for the deprecated combination alarm.
>
>   https://review.openstack.org/#/c/429405/
>
> I checked and as far as I can tell that should not break Heat gate, but
> I'd prefer to be sure. So, Heat developers if you have some code relying
> on it, let us know before we press the big +A button.
>

i mentioned this on patch, but it seems my dependent patch passes in 
heat: https://review.openstack.org/#/c/437192/

-- 
gord

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [I18n] Translation platform may be unstable or down during 2/27 - 3/3 due to upgrade

[Ian Y. Choi]

Hello all,


As I shared I18n summary in my previous e-mail [1],

there will be a major OpenStack translation platform upgrade during Feb 27
to Mar 3.


It implies that:


- Translation platform (https://translate.openstack.org) might be unstable
or might

  not work for some time slots during the upgrade period.


- “Imported Translation from Zanata” patch proposals [2] might not work.


- Translation sync jobs [3, 4] may fail during some period.


Infrastructure team kindly agreed to help a lot for the translation
platform upgrade (I truly appreciate it!), and I18n team & Zanata [5]
development team will have high attention on upgrade and changes.


Note that the current Zanata version is 3.7.3, and the target upgrade
version is 3.9.6.



With many thanks,


/Ian


[1]
http://lists.openstack.org/pipermail/openstack-dev/2017-February/112889.html

[2]
https://review.openstack.org/#/q/owner:%22OpenStack+Proposal+Bot%22+%22Imported+Translations+from+Zanata%22

[3]
http://status.openstack.org/openstack-health/#/g/build_queue/post?groupKey=build_queue=translation

[4]
http://status.openstack.org/openstack-health/#/g/build_queue/periodic?groupKey=build_queue=translation

[5] http://zanata.org
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [I18n] Pike PTG Summary

[Ian Y. Choi]

Hello all,


I18n team participated in PTG for horizontal discussions with the tight
collaboration of Documentation team on Monday and Tuesday.

Also, there were a few collaboration sessions with Infrastructure, Horizon,
Release Management, and OpenStackAnsible team on Wednesday & Thursday in a
reservable room.


The following is the summary of what I18n team discussed with other project
teams.

More details for I18n issues are available at:
https://etherpad.openstack.org/p/i18n-ptg-pike-topics



1. Documentation / I18n (Monday & Tuesday)


The details of all discussions with Docs & I18n teams are written in:
https://etherpad.openstack.org/p/docs-i18n-ptg-pike ,

and I would like to summarize mainly I18n issues (For Documentation topic
summary, please see [1]).


- For Horizon strings, Documentation team & I18n team identified

 : Documentation team wants to implement automated screenshot functionality
for user-guide. Then, stable DevStack

   instance would be fine with adding the functionality.

 : On the other hand, I18n team wants to implement translation checksite
which requires re-import of translated strings

   to review translated strings after around Soft & Hard StringFreezes.


- Agreed that archiving & versioning [2] translated documents are also the
target with consideration.


- Defined url policy for translated project specific install guides

 : https://docs.openstack.org (e.g.,
https://docs.openstack.org/project-install-guide/baremetal/ja/newton/ )


- I18n team will implement to generate translated PDF documents after pdf
implementation spec [3] is successfully landed.


- I18n team discussed the approach for implementing translation checksite,
I18n contributor guide translation,

  considering different release models, and solving Stackalytics
integration issues.



2. Infrastructure + I18n (Wednesday) + OpenStackAnsible + Horizon (Thursday)


- Translation platform ( https://translate.openstack.org ) is now ready for
upgrade.


- There will be upgrade during 2/27 - 3/3. Translation platform might not
work due to upgrade progress.


- openstackid-dev developer will test authentication with Zanata using
https://translate-dev.o.o from next week


- I18n team will automate to propose ATC status rather than current manual
proposal work as extra-ATCs using upgraded translation platform.


- Implementing translation checksite with OpenStackAnsible will be a
feasible solution in OpenStack Infrastructure. I18n team will start to
implement translation checksite using OpenStackAnsible from Pike cycle
(spec [4] needs to be changed).


- Discussed how to Improve algorithms to match Zanata ID with Launchpad &
Gerrit ID in Stackalytics



3. Horizon I18n collaboration session w/ Release Management team


- Shared current status of I18n team’s translation priority

  : I18n team sets the highest priority during Soft & Hard StringFreezes to
include more qualified translated strings

  as much as possible for Horizon & Horizon plugin projects.


- I18n team shared Ocata cycle experience: cycle-with-milestone,
cycle-with-intermediary, and cycle-trailing release models

  for Horizon plugin projects are not matched with Soft & Hard StringFreeze
periods.


- Discussed & drawn a new proposal to consider translations for Horizon
plugin projects

 : I18n team will consider Soft & Hard StringFreeze for the projects which
follow cycle-with-milestone release model.

 : I18n team will also consider translation with releases for the projects
which follow cycle-with-intermediary release model if they are fine
following deadlines.

 : I18n team will propose a specification to tag (e.g., with “translate”)
for projects which are the target of translations with managing

   stable branches in translation platform.

 : For cycle-trailing release models, I18n will consider translations with
managing stable versions in translation platform

if there is I18n cross-project liaison [5] for the project.

 : I18n team will higher priority for projects which have I18n
cross-project liaison.


- I18n team will ask to Zanata development team to automate to create new
stable versions in Zanata, copying master translations over.



I great appreciate all the help and support from especially Alex - Pike
Docs PTL to help successfully land collaboration room with I18n team, and
also many project team contributors to participate in the discussion and
draw nice, and cool approach for better internationalization.



With many thanks,


/Ian


[1]
http://lists.openstack.org/pipermail/openstack-dev/2017-February/112885.html

[2]
http://specs.openstack.org/openstack/docs-specs/specs/pike/archiving.html

[3]
http://specs.openstack.org/openstack/docs-specs/specs/ocata/build-pdf-from-rst-guides.html

[4]
http://specs.openstack.org/openstack-infra/infra-specs/specs/translation_check_site.html

[5] https://wiki.openstack.org/wiki/CrossProjectLiaisons#I18n
__

[openstack-dev] [tripleo] introducing python-tripleo-helper

[Gonéri Le Bouder]

Hi all,

TL/DR:

- use Python to do drive TripleO deployment
- RPM based
- used to be a bit specific, closer to upstream now (RDO)
- unit-test
- maybe a good candidate to join the TripleO umbrella

Following a discussion with Emilien, I would like to introduce
python-tripleo-helper.

python-tripleo-helper is a Python library that wrapper all the
operations required to get a working TripleO. The initial goal was to
have a solution to automate and validate the deployment of TripleO in
our lab environment.

Since the full deployment flow is based on a modern programming
language, it's also possible to write more complex operations.

For instance, this is a test that I did some month ago:
Once the Overcloud was running, we started a "chaos monkey" thread that
was randomly disconnecting controllers. We were running tempest in the
main thread to collect results. Python makes all these interactions
easy.

At this point, we support libvirt and the OpenStack as the target
environment. We use a couple of hacks to be able to use a regular
OpenStack cloud (OSP7). bare-metal is not supported yet even if it's
definitely a low-hanging fruit.

python-tripleo-helper relies on RPM packages but we are open to
contribution to support the other packaging solutions.

Till some weeks ago python-tripleo-helper was not really generical. This
is the reason why it's still maintained in the redhat-openstack
namespace. Nicolas Hicher pushed a couple of patches to be able to use
it with RDO and CentOS, I guess we can now consider a move to the
TripleO umbrella.

-- 
Gonéri Le Bouder


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-docs] [openstack-docs] [dev] What's up, Doc?

[Alexandra Settle]

Ah, thank you for calling it out :) it has indeed.

This is why you don’t write newsletters while you’re running around like a 
chicken with their head cut off.

One day I’ll get this email thing right. 

On 2/24/17, 1:36 PM, "Christian Berendt"  wrote:


> On 24 Feb 2017, at 13:32, Alexandra Settle  wrote:
> 
> Please vote for your fellow documenter below:

The community voting for Boston already closed.

Christian.

-- 
Christian Berendt
Chief Executive Officer (CEO)

Mail: bere...@betacloud-solutions.de
Web: https://www.betacloud-solutions.de

Betacloud Solutions GmbH
Teckstrasse 62 / 70190 Stuttgart / Deutschland

Geschäftsführer: Christian Berendt
Unternehmenssitz: Stuttgart
Amtsgericht: Stuttgart, HRB 756139



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-docs] [openstack-docs] [dev] What's up, Doc?

[Christian Berendt]

> On 24 Feb 2017, at 13:32, Alexandra Settle  wrote:
> 
> Please vote for your fellow documenter below:

The community voting for Boston already closed.

Christian.

-- 
Christian Berendt
Chief Executive Officer (CEO)

Mail: bere...@betacloud-solutions.de
Web: https://www.betacloud-solutions.de

Betacloud Solutions GmbH
Teckstrasse 62 / 70190 Stuttgart / Deutschland

Geschäftsführer: Christian Berendt
Unternehmenssitz: Stuttgart
Amtsgericht: Stuttgart, HRB 756139


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [openstack-docs] [dev] What's up, Doc?

[Alexandra Settle]

Team team team team team,

Welcome to my second edition of What's up, Doc!

We have had an extremely busy week in Atlanta for the PTG. We had a really 
enthusiastic attendance and I am truly impressed by how much we have managed to 
get done. This newsletter will be a short one and focus on a small PTG update!

Our bug triager for the first 2 weeks of Pike is Darren Chan. Looking for more 
people to sign up and help out: 
https://wiki.openstack.org/wiki/Documentation/SpecialityTeams#Bug_Triage_Team

== Progress towards Pike ==

Thanks to our amazing release managers, Maria, Brian and Lana for all their 
amazing hard work on the Ocata release. Big thanks to Anne for stepping in at 
the last minute!

* 44 days to go! 
https://www.timeanddate.com/countdown/newyear?p0=24=Pike+release=1=slab

== The Road to the Summit in Boston ==

Voting is now open for topics. You can find the list of topics here: 
https://www.openstack.org/summit/boston-2017/vote-for-speakers#/
I was able to find some documentation related topics. If anyone has any more 
topics I am unaware of, please reply to this email thread and share your talks!

Please vote for your fellow documenter below:

1. https://www.openstack.org/summit/boston-2017/vote-for-speakers#/17888
2. https://www.openstack.org/summit/boston-2017/vote-for-speakers#/18078
3. https://www.openstack.org/summit/boston-2017/vote-for-speakers#/17711

Event info is available here: http://www.openstack.org/ptg
Purchase tickets here: https://pikeptg.eventbrite.com/
Tickets for the Boston summit: https://www.openstack.org/summit/boston-2017/

== PTG Update ==

We had a very successful PTG! Thanks to all those that were able to attend. 
Thanks to Ian Choi and the I18n team for being wonderful collaborators. We were 
able to get a lot done in our horizontal sessions. For anyone that is 
interested, here's our etherpad with all the notes from the sessions: 
https://etherpad.openstack.org/p/docs-i18n-ptg-pike
Here are a few of the exciting things to look out for in Pike:


1.   High Availability Guide improvements

a.   Adam Spiers and Ianeta Hutchinson will be leading the charge on the HA 
guide with a massive restructure and up-to-date new content.

2.   Reno release notes for OpenStack manuals

a.   Adam Spiers has been working alongside John Davidge and the Infra team 
to work on implementing the reno release notes process for documentation. 
Including looking at implementing new reno templates to work specifically for 
documentation!

3.   Networking Guide improvements and more collaboration with the neutron 
team.

a.   John Davidge has been working with the neutron team to develop the 
best way to ensure information flows into the guide. Discussions around the use 
of the DocImpact tag were successful.

4.   Administrator Guide changes! Keep your eye out for exciting 
developments.

a.   Ildiko Vancsa and Joseph Robinson will be working together to come up 
with a new specification to implement in Queens for our Administration Guide. 
Development teams have been enthusiastic about implementing the 
project-install-guide and are looking for similar structures with the Admin 
Guide. Stay tuned for more news here!

I had the opportunity to report back to meeting attendees yesterday afternoon, 
if anyone would like to know more, here are the meeting minutes: 
http://eavesdrop.openstack.org/meetings/docteam/2017/docteam.2017-02-23-20.59.log.txt

== Doc team meeting ==

Our next meeting will be on Thursday 9 March at 2100 UTC in 
#openstack-meeting-alt.

Meeting chair will be me (Alexandra Settle - asettle)! \o/

For more meeting details, including minutes and the agenda: 
https://wiki.openstack.org/wiki/Meetings/DocTeamMeeting

--

Have a great weekend!

Alex
Lamp enthusiast and award winning marathon sleeper

IRC: asettle
Twitter: dewsday
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [dib] diskimage-builder v2 RC1 release; request for test

[Andre Florath]

Hello Ian,

I think I found a regression: since [1] only the first
package specified using '-p' is picked up.
You can find a proposal for a fix under [2].
Maybe it's worth to include this in V2?

Kind regards

Andre

[1] https://review.openstack.org/#/c/396702
[2] https://review.openstack.org/#/c/438010


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] - Neutron team social Pictures

[Vasudevan, Swaminathan (PNB Roseville)]

Hi Folks,
Here is the link to the pictures that I took during the Neutron Social in 
Atlanta.

https://souminathan.smugmug.com/OpenStack-Neutron-PTG-Atlanta-2017/

Thanks
Swami

From: Kevin Benton [mailto:ke...@benton.pub]
Sent: Thursday, February 23, 2017 4:09 PM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [neutron] - Neutron team social in Atlanta on 
Thursday

Hi everyone,

We will meet tonight at Mellow Mushroom at 6PM. It's about a 10-15 minute walk 
from the venue.

Here is the location:

https://goo.gl/maps/FAnmVEVEUSE2

On Feb 23, 2017 13:27, "Duarte Cardoso, Igor" 
> wrote:
+1

Best regards,
Igor.

From: Vasudevan, Swaminathan (PNB Roseville) 
[mailto:swaminathan.vasude...@hpe.com]
Sent: Friday, February 17, 2017 8:30 PM
To: OpenStack Development Mailing List (not for usage questions) 
>
Subject: Re: [openstack-dev] [neutron] - Neutron team social in Atlanta on 
Thursday

Count me in.

From: Kevin Benton [mailto:ke...@benton.pub]
Sent: Friday, February 17, 2017 11:19 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [neutron] - Neutron team social in Atlanta on Thursday

Hi all,

I'm organizing a Neutron social event for Thursday evening in Atlanta somewhere 
near the venue for dinner/drinks. If you're interested, please reply to this 
email with a "+1" so I can get a general count for a reservation.

Cheers,
Kevin Benton

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements] Managing project requirements

[Tony Breeds]

On Fri, Feb 24, 2017 at 02:09:33PM +0100, Andreas Scheuring wrote:
> Hi, I have a couple of questions in regards of how updates of
> requirements are handled and what happens if I release a new version of
> a library. I read along the README [1], but a few points are still
> unclear to me.
> 
> I have the following projects:
> 
> * os-dpm [4]
>   * is a openstack library that gets release when required
>   * release is done manually (creating a tag and pushing it to gerrit)
>   * listed in global requirements & upper constraints
> * zhmcclient [6]
>   * An "external" library that gets released to pypi
>   * listed in global requirements & upper constraints
> * nova-dpm [2] & networking-dpm [3]
>   * are the main projects and therefore not listed in the openstack
> requirements
>   * both specify the os-dpm and zhmcclient in their requirements.txt
>   * both are accepted in the projects.txt to allow the automated sync of
> global requirements
> 
> 
> Now the scenarios are
> 
> A new version of the external library 'zhmcclient' gets released
> 
> 
> What to do in order to get this reflected in upper constraints? At first
> I was pushing manual updates to requirements txt, but the latest version
> was pushed in by the OpenStack proposal bot around 10 days after the
> release [4]. What's the right approach here?
> 
> How can I ensure that I have to approve a new version for my projects,
> before it gets applied there? (Once it is in upper constraints, it is
> used in my project as well automatically). If I specify an upper
> constraint in my requirements.txt file, then the requirements job
> complains as the requirement does not exactly match the requirementused
> in global requirements...

So if you find that a new version of zhmcclient doesn't work for you the 2
options available to you are:
1) Fix you project to tolerate the new (and old) version
2) Ban (via !=) in openstack/requirements:global-requirements.txt, and revert
   the relevant parts of the upper-constraints.txt change, which will then
   propagate to your project once it merges (via the proposal-bot)

but fundamentally it boils down to we'll accept new versions into
upper-constraints.txt and if it breaks we'll back out/correct it.

If this proves to be overly problematic for you we can discuss ways for you to
more actively engage with requirements management to ensure you can approve
things before that land but that's atypical

> A new version of the openstack library 'os-dpm' gets released
> -
> 
> As this is an OpenStack project - I think the flow is slightly
> different: If a new release gets tagged with the corresponding metadata,
> a patch to upper constraints is submitted automatically. (I still need
> to figure out how that metdata should look like...)

Doug's already answer this better than I could.

> But the same question like above, can I control the upper constraints in
> my project?

You've opted into having your requirements managed by the requirements team.
So the way to set bans or (if really needed) caps is as I outlined above via
changes to global-requirements.txt.

The management of *constraints* is global and not per-project.

Yours Tony.


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo] PTG summary

[ChangBo Guo]

Oslo folks,

This is summary of  oslo PTG schedule during one and half days. You can
check the details in etherpad link:
https://etherpad.openstack.org/p/oslo-ptg-pike

1. oslo.messaging: consistent names for general driver configuration options
   Some of supported drviers have various option configuration names with
for common usage, This has led to a plethora of different configuration
options that complicates the   development of Day 1 deployment and
configuration tools. We propose that 'scrub' the configuration namespace
and come up with common generic names that can be shared among the various
drivers. Note: this is *not* calling for moving these configuration items
'up' to the DEFAULT group.  Each driver should be able to have its own
value for any given configuration setting.  This is necessary for the dual
backend deployment scenario.

2. Messaging Experiences towards hybrid backends
   Share experience and progress towards introducing dual messaging backend
operation :
https://docs.google.com/presentation/d/1Y-3gH8rxrU5KagvDQiqpByfvmgM8no-PLn1O0IT5PCs/edit?usp=sharing

3.How to make oslo work done in an efficient way
   1)Track bugs & reviews accoding to their priorities
   2)Record general core reviewers' focus to help get help from them.
https://etherpad.openstack.org/p/oslo-pike-tracking
   3) Don't migrate to StoryBoard until that's a community goal .

4.Add a new policy rule to check specific metadata key
  Decide to change nova codes instead of oslo.policy
https://etherpad.openstack.org/p/policy_support_for_specific_metadata_keys

5. Discuss oslo_messaging.rpc message encryption and security
Trove implements this function in its code base, wan to code be
included in oslo.messaging.
kgiusti - followup with amrith on possible API (message digest)

6. Feature updates & collect ideas
   1) Josh propose a new library to handle remote exceptions in an united
way: http://pypi.python.org/pypi/failure
   Need improve it in code and documentation , test in gate , then adop
it .
   gcb will talk with Nova PTL if nova want to adopt it.
  2) new functions from neutron(ihrachys): ability to cancel in
progress/blocked RPC requests in oslo.messaging
  ihrachys will file a bug, describe use cases to track this.

we also did bug smash to reduce bug numbers in rest of time.

-- 
ChangBo Guo(gcb)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements] Managing project requirements

[Matthew Thode]

On 02/24/2017 07:09 AM, Andreas Scheuring wrote:
> Hi, I have a couple of questions in regards of how updates of
> requirements are handled and what happens if I release a new version of
> a library. I read along the README [1], but a few points are still
> unclear to me.
> 
> I have the following projects:
> 
> * os-dpm [4]
>   * is a openstack library that gets release when required
>   * release is done manually (creating a tag and pushing it to gerrit)
>   * listed in global requirements & upper constraints
> * zhmcclient [6]
>   * An "external" library that gets released to pypi
>   * listed in global requirements & upper constraints
> * nova-dpm [2] & networking-dpm [3]
>   * are the main projects and therefore not listed in the openstack
> requirements
>   * both specify the os-dpm and zhmcclient in their requirements.txt
>   * both are accepted in the projects.txt to allow the automated sync of
> global requirements
> 
> 
> Now the scenarios are
> 
> A new version of the external library 'zhmcclient' gets released
> 
> 
> What to do in order to get this reflected in upper constraints? At first
> I was pushing manual updates to requirements txt, but the latest version
> was pushed in by the OpenStack proposal bot around 10 days after the
> release [4]. What's the right approach here?
> 
> How can I ensure that I have to approve a new version for my projects,
> before it gets applied there? (Once it is in upper constraints, it is
> used in my project as well automatically). If I specify an upper
> constraint in my requirements.txt file, then the requirements job
> complains as the requirement does not exactly match the requirementused
> in global requirements...
> 

It sounds like you want to approve of updates being done to specific
packages in upper-constraints.txt.  We generally don't do this because
with the amount of packages held in upper-constraints it'd become
unmanageable to have 'owners' for each package (or even just some of
them).  We do cross test updates though, if updates to zhmcclient
continually cause downstream projects to fail then there may be a case
to be made in adding the dowstream project to our cross-project gating.

As to your desire to cap/change/lock-down your requirements.txt
downstream projects that consume the requirements project are required
to be in sync with our constraints.  Also, all projects using
upper-constraints.txt to test means that we ensure all of those projects
are co-installable.

Our current policy for updating upper-constraints.txt is to require the
bot to do it unless also updating global-requirements.txt.  This is the
reduce conflicts that could be introduced between the bot updates and
on-off uc.txt changes.  Perhaps a policy change is needed?  If so, we'd
also have to change the daily bot update to submit one change at a time
to help reduce conflicts.

> 
> A new version of the openstack library 'os-dpm' gets released
> -
> 
> As this is an OpenStack project - I think the flow is slightly
> different: If a new release gets tagged with the corresponding metadata,
> a patch to upper constraints is submitted automatically. (I still need
> to figure out how that metdata should look like...)
> 
> But the same question like above, can I control the upper constraints in
> my project?
> 

Your project doesn't have an upper-constraints file, but a
(test-)requirements.txt file.  Projects test via upper-constraints.txt
in order to ensure co-installability.  So altering your requirements
would bring you out of sync with the rest of openstack (and we'd have to
remove you from gr-updates eventually as well).

> 
> Thanks!
> 
> 
> [1] https://github.com/openstack/requirements
> [2] https://github.com/openstack/nova-dpm
> [3] https://github.com/openstack/networking-dpm
> [4] https://github.com/openstack/os-dpm
> [5] https://review.openstack.org/#/c/426487/
> [6] https://github.com/zhmcclient/python-zhmcclient
> 

If you have more questions about requirements and your project don't
hesitate to ask, we are in #openstack-requirements as well.

-- 
Matthew Thode (prometheanfire)



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [keystone][api] Changing devstack to not set up keystone on :5000 and :35357

[Brant Knudson]

At the PTG there was some discussion about changing services to not listen
on ports[0]. I'd been working on this for devstack keystone off and on for
some time[1], so I thought I'd dust off that work again to see what the
problems were. Turns out things must have been fixed since, after rebasing
and cleaning up the change, it's now passing our gate tests. I split up the
change since there was some refactoring to do before making the final
change of having Apache HTTPd not listen on :5000 and :35357.

This will likely affect devstack plugins using KEYSTONE_SERVICE_PROTOCOL,
KEYSTONE_SERVICE_HOST, and KEYSTONE_AUTH_PORT instead of
KEYSTONE_SERVICE_URI[2].

[0] https://etherpad.openstack.org/p/service-catalog-pike
[1] https://review.openstack.org/#/c/312230/
[2] https://review.openstack.org/#/c/437760/

-- 
- Brant
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [PKG-Openstack-devel] The end of OpenStack packages in Debian?

[Allison Randal]

On 02/24/2017 06:25 AM, Ritesh Raj Sarraf wrote:
> rtslib-fb is a core component for the LIO(-fb) project. We already maintain
> configshell-fb and targetcli-fb under pkg-linux-target group.

Nod, that makes sense. I'd say you should take on maintenance of the
rtslib-fb packages, as long as you're willing to integrate with the
global Debian Python transitions and policy, and to handle the fact that
other projects (like the OpenStack Debian packagers) also use the
module. For OpenStack, we may occasionally need to update the rtslib-fb
packages to specific versions for compatibility with specific OpenStack
releases.

Allison



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] pbr and warnerrors status

[Doug Hellmann]

Excerpts from Ben Nemec's message of 2017-02-08 10:15:57 -0600:
> 
> On 02/08/2017 09:41 AM, Doug Hellmann wrote:
> > Excerpts from Ben Nemec's message of 2017-02-08 09:20:31 -0600:
> >>
> >> On 02/08/2017 01:53 AM, Andreas Jaeger wrote:
> >>> On 2017-02-08 00:56, Ian Cordasco  wrote:
> 
> 
>  On Feb 7, 2017 5:47 PM, "Joshua Harlow"   > wrote:
> 
>  Likely just never pulled the trigger.
> 
>  Seems like we should pull it though.
> 
> 
> 
>  Will have to wait until Pike given the library release freeze
> >>>
> >>> I've pushed a patch to release pbr so that we won't forget about it:
> >>>
> >>> https://review.openstack.org/430618
> >>>
> >>> Andreas
> >>>
> >>
> >> Great, thanks everybody!
> >>
> >
> > It would be useful for someone to look at the pbr change and verify that
> > releasing it as-is won't break the builds for a bunch of projects by
> > turning the flag back on. If it will, we're going to want to provide a
> > way to stage the roll out. If we change the name of the flag, for
> > example, we could release pbr for support for checking warnings, and
> > then projects could enable that when they have time and with the fixes
> > to the docs needed to make it work properly.
> 
> I think the only way to do this would be to run a docs build in every 
> project with warnerrors turned on, using the unreleased pbr.  Given the 
> amount of time it's been broken, there's a good chance some issues have 
> snuck in (they had in diskimage-builder).  So maybe deprecating the old 
> option name and requiring another explicit opt-in from everyone is the 
> safest way to go.
> 
> Thoughts on a new name?  fatalwarnings maybe?
> 

I've proposed [1] to rename the option.

Doug

[1] https://review.openstack.org/437929

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [glance] PTG summary, Thursday Feb 23

[Brian Rosmaita]

The Glance PTG schedule etherpad contains links to the etherpads for
each of the sessions discussed below:
https://etherpad.openstack.org/p/glance-pike-ptg-schedule


1. glance/glare/artifacts/images
The tldr; is Glance needs to complete image import before taking on
anything else, and the Glare project should continue to mature
independently.  Even though there is some conceptual overlap between the
two projects, Glance needs to focus on providing the OpenStack Image
Service.

2. multi-store use cases and multiple image locations
The tldr; is that efficient use of multiple stores via multiple image
locations is a complicated problem that requires a complicated solution,
possibly a microservice to provide location sorting instead of trying to
shoehorn this into a custom location strategy in Glance.  We're trying
to balance an effective strategy with a convenient way for operators to
modify strategies given environmental changes (e.g., a store goes offline).

3. organizing the Glance community
The consensus is that we need to introduce a few more events during the
cycle: a multi-day bug smash, virtual midcycle, and virtual operators'
midcycle (I mean "operators' virtual midcycle").  We're also going to
revive bi-weekly bug/patch reviews at a scheduled time TBD instead of
relying on everyone independently reviewing bugs/patches.  We'll also
continue the operators' surveys, next one will be about using the Glance
cache.

4. eliminate redundant downloads
hemanthm outlined an idea to make downloads more efficient along with
some ideas of how to eliminate a single point of failure that
ultra-efficient downloads from the backend could introduce.

5. image import, continued
We discussed a few unclarities in the current spec; rosmaita will put up
a patch to clear these up.

6. atomicity in race conditions
dharinic and nikhil outlined some race conditions that can occur around
the change in image status in Glance.  We discussed a few ways to fix
this; dharinic will explore these in order of complexity (least to most,
of course).  The good news is that these are mostly benign, and the one
condition that could be problematic only occurs in a non-default,
non-recommended configuration.


We'll be discussing priorities, roadmapping for Pike, and holding an
Ocata postmortem on Friday morning.


cheers,
brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements] Managing project requirements

[Doug Hellmann]

Excerpts from Andreas Scheuring's message of 2017-02-24 14:09:33 +0100:
> Hi, I have a couple of questions in regards of how updates of
> requirements are handled and what happens if I release a new version of
> a library. I read along the README [1], but a few points are still
> unclear to me.
> 
> I have the following projects:
> 
> * os-dpm [4]
>   * is a openstack library that gets release when required
>   * release is done manually (creating a tag and pushing it to gerrit)
>   * listed in global requirements & upper constraints
> * zhmcclient [6]
>   * An "external" library that gets released to pypi
>   * listed in global requirements & upper constraints
> * nova-dpm [2] & networking-dpm [3]
>   * are the main projects and therefore not listed in the openstack
> requirements
>   * both specify the os-dpm and zhmcclient in their requirements.txt
>   * both are accepted in the projects.txt to allow the automated sync of
> global requirements
> 
> 
> Now the scenarios are
> 
> A new version of the external library 'zhmcclient' gets released
> 
> 
> What to do in order to get this reflected in upper constraints? At first
> I was pushing manual updates to requirements txt, but the latest version
> was pushed in by the OpenStack proposal bot around 10 days after the
> release [4]. What's the right approach here?

Either works. It may be slightly faster if you update the constraint
by hand, though that depends on how the reviews end up being
prioritized. Another benefit of updating one constraint at a time
is if there is a test failure, it would be easy to debug because
there is only one package.

> How can I ensure that I have to approve a new version for my projects,
> before it gets applied there? (Once it is in upper constraints, it is
> used in my project as well automatically). If I specify an upper
> constraint in my requirements.txt file, then the requirements job
> complains as the requirement does not exactly match the requirementused
> in global requirements...

We run the unit tests (and maybe more?) for a few major projects with a
representative set of the requirements list. We cannot afford to run all
tests for all projects, though.

If you continue to manually patch upper-constraints.txt, then you could
use Depends-On to link any patch in your project to that constraints
change, which should cause the test jobs to run with the new constraint.

> A new version of the openstack library 'os-dpm' gets released
> -
> 
> As this is an OpenStack project - I think the flow is slightly
> different: If a new release gets tagged with the corresponding metadata,
> a patch to upper constraints is submitted automatically. (I still need
> to figure out how that metdata should look like...)

The scripts that build the metadata are in
openstack-infra/project-config/jenkins/scripts/release-tools. Take
a look at release_from_yaml.sh and release.sh in that directory.
You can run those scripts by hand instead of tagging directly, if
you want.

> But the same question like above, can I control the upper constraints in
> my project?

Not really. The point of upper-constraints is for all projects to
be testing with the same set of packages so we can ensure
co-installability for single-node deployments and so we can clearly
document what we are testing with for distros and other downstream
consumers.

Doug

> 
> 
> Thanks!
> 
> 
> [1] https://github.com/openstack/requirements
> [2] https://github.com/openstack/nova-dpm
> [3] https://github.com/openstack/networking-dpm
> [4] https://github.com/openstack/os-dpm
> [5] https://review.openstack.org/#/c/426487/
> [6] https://github.com/zhmcclient/python-zhmcclient
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [requirements] Managing project requirements

[Andreas Scheuring]

Hi, I have a couple of questions in regards of how updates of
requirements are handled and what happens if I release a new version of
a library. I read along the README [1], but a few points are still
unclear to me.

I have the following projects:

* os-dpm [4]
  * is a openstack library that gets release when required
  * release is done manually (creating a tag and pushing it to gerrit)
  * listed in global requirements & upper constraints
* zhmcclient [6]
  * An "external" library that gets released to pypi
  * listed in global requirements & upper constraints
* nova-dpm [2] & networking-dpm [3]
  * are the main projects and therefore not listed in the openstack
requirements
  * both specify the os-dpm and zhmcclient in their requirements.txt
  * both are accepted in the projects.txt to allow the automated sync of
global requirements


Now the scenarios are

A new version of the external library 'zhmcclient' gets released


What to do in order to get this reflected in upper constraints? At first
I was pushing manual updates to requirements txt, but the latest version
was pushed in by the OpenStack proposal bot around 10 days after the
release [4]. What's the right approach here?

How can I ensure that I have to approve a new version for my projects,
before it gets applied there? (Once it is in upper constraints, it is
used in my project as well automatically). If I specify an upper
constraint in my requirements.txt file, then the requirements job
complains as the requirement does not exactly match the requirementused
in global requirements...


A new version of the openstack library 'os-dpm' gets released
-

As this is an OpenStack project - I think the flow is slightly
different: If a new release gets tagged with the corresponding metadata,
a patch to upper constraints is submitted automatically. (I still need
to figure out how that metdata should look like...)

But the same question like above, can I control the upper constraints in
my project?


Thanks!


[1] https://github.com/openstack/requirements
[2] https://github.com/openstack/nova-dpm
[3] https://github.com/openstack/networking-dpm
[4] https://github.com/openstack/os-dpm
[5] https://review.openstack.org/#/c/426487/
[6] https://github.com/zhmcclient/python-zhmcclient

-- 
-
Andreas 
IRC: andreas_s





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] The end-user test suite for OpenStack clusters

[Ghanshyam Mann]

Yea, agree with dims.

Sampath ,

Thanks for taking over this, it is really great help. Please update the
current spec with approaches you have. Timur help will be great if he show
up sometime.

Also we will add destructive testing as one of weekly meeting agenda and
make sure you will get all help & required attention from QA team.

-gmann

On Fri, Feb 24, 2017 at 7:26 AM, Davanum Srinivas  wrote:

> Sampath,
>
> I am not sure if you will hear back from Timur soon as he may not be
> working on this stuff anymore (in Mirantis). So i'd recommend picking
> up the work if you don't hear from him soon.
>
> Thanks,
> Dims
>
> On Thu, Feb 23, 2017 at 3:41 PM, Sam P  wrote:
> > Hi Timur,
> >
> >  The current status of this work is,
> > 1) The QA user story for destructive testing of OpenStack cloud [1] is
> merged .
> > 2) The spec for a new framework which will focus on HA/failover and
> > destructive testing  [2] has no update since Nov 30 2016.
> > 3) The commit for the new repository [3]  has abandoned due to no
> > update after Nov 29 2016.
> >
> > Currently, I am working on 3rd party destructive/HA testing CI for
> > Masakari[4] and very much interested in this work.
> > I will keep working on [1] with PWG.
> > Please let me know your plans for [2], and [3].
> > If it is difficult for you to continue, I would love to continue your
> > work on [2], and [3].
> >
> > [1] https://review.openstack.org/#/c/396142
> > [2] https://review.openstack.org/#/c/399618
> > [3] https://review.openstack.org/#/c/374667
> > [4] https://wiki.openstack.org/wiki/Masakari
> > --- Regards,
> > Sampath
> >
> >
> >
> > On Mon, Nov 28, 2016 at 6:37 AM, Timur Nurlygayanov
> >  wrote:
> >> Hi team,
> >>
> >> here is a short update:
> >>
> >> 1) The QA user story for destructive testing of OpenStack cloud is on
> review
> >> [1].
> >> 2) The spec for a new framework which will focus on HA/failover and
> >> destructive testing is no review [2].
> >> 3) The commit for the new repository is on review [3] as well.
> >>
> >> [1] https://review.openstack.org/#/c/396142
> >> [2] https://review.openstack.org/#/c/399618
> >> [3] https://review.openstack.org/#/c/374667
> >>
> >>
> >> On Fri, Oct 14, 2016 at 3:47 AM, Ghanshyam Mann
> >>  wrote:
> >>>
> >>> I like os-faults library which can provide the abstraction over
> different
> >>> destructive actions like reboot/poweroff node etc.
> >>>
> >>>
> >>>
> >>> But not much clear about Stepler that what all tests it will contain.
> >>> Tempest do have scenario tests which can hit the production env with
> >>> significant way of testing scenario.
> >>>
> >>> It can cover the end user scenario also which involves the interaction
> of
> >>> public OpenStack APIs and always in enhancement state by adding more
> and
> >>> more scenario tests.
> >>>
> >>>
> >>>
> >>> Few query over Stepler as separate project:
> >>>
> >>> 1.   Is Stepler will cover only tests which hits the node level
> >>> action(reboot, HA etc)?
> >>>
> >>> 2.   This should not mix the scenario tests which are in Tempest
> scope
> >>> because that can make confusion for developers (where to add scenario
> tests)
> >>> as well as for tester(from where to run what scenario tests).
> >>>
> >>> 3.   How we make sure those tests run fine or at least run while
> >>> adding.
> >>>
> >>> a.   I think devstack enhancement for multi-node etc can do this as
> >>> mentioned by you also.
> >>>
> >>> b.  If so then why not adding those tests in Tempest only using
> >>> os-faults lib ?
> >>>
> >>>
> >>>
> >>> Overall I feel os-faults  as lib is really nice idea but tests scope
> can
> >>> go in Tempest under HW_scenario  (or something else) till it does not
> break
> >>> basic principle of Tempest.
> >>>
> >>>
> >>>
> >>> Thanks
> >>>
> >>> gmann
> >>>
> >>>
> >>>
> >>> From: Timur Nurlygayanov [mailto:tnurlygaya...@mirantis.com]
> >>> Sent: 06 October 2016 20:09
> >>> To: OpenStack Development Mailing List (not for usage questions)
> >>> 
> >>> Subject: Re: [openstack-dev] [QA] The end-user test suite for OpenStack
> >>> clusters
> >>>
> >>>
> >>>
> >>> Ken, it is a good idea!
> >>>
> >>>
> >>>
> >>> The plan is to develop os-faults as a library which will be able
> >>>
> >>> to manage cluster nodes and OpenStack services on these nodes.
> >>>
> >>> It is a good idea to add some Tempest tests which will use
> >>>
> >>> os-faults library as well for some API tests.
> >>>
> >>>
> >>>
> >>> The Stepler framework [1] will use os-faults to perform all destructive
> >>>
> >>> actions in the clouds (the reboot of nodes, restart of OpenStack
> services,
> >>>
> >>> enable/disable network interfaces or some firewall rules and etc).
> >>>
> >>>
> >>>
> >>> We need to get +1 from you in [1] to create the repository with
> >>>
> >>> advanced end-user scenario tests.
> >>>
> >>>
> >>>
> >>> Thank you!
> 

[openstack-dev] [all] DevStack gate local.conf support - action might be required to fix some jobs

[Sean Dague]

This week we've managed to get through one of the long overdue changes
to the test infrastructure, which is making devstack-gate /
project-config support and understand devstack's local.conf. Like all
big transitions, it did not go quite as smoothly as hoped.

We broke everything using devstack plugins for about 12 hours on
Thursday due to a line ending handling issue that wasn't caught during
review (but that's all fixed now). There were issues specific to
Ironic that took longer to resolve, but I think we're working there now.

Given that there aren't a ton of well defined interfaces around this,
some jobs might still be broken. Here is a bit of a guide on how to
correct those going forward.


There can only be one of local.conf / localrc
=

One of the things that people seem to have figured out is that under
some circumstances if you write both a localrc and a local.conf, they
both get used. That is mostly due to a bug, and stops really working
the moment local.conf has a [[local|localrc]] section in it.

If you are building a localrc file directly in something like a
pre-test hook, you'll need to write to a local.conf file instead. An
example of such an issue, and the fix, can be seen here on the Heat
project -
https://review.openstack.org/#/c/437686/1/heat_integrationtests/pre_test_hook.sh

Overall it's actually *not* recommended to configure things in
pre_test_hook.sh like this, because it's relying on some pass through
behavior which isn't a hard contract. It seems to be working for now,
but there are some other suggestions.


local_conf in project-config


A much requested feature was the ability to set local.conf (including
post-config sections) in project-config definitions. There is now a
way to do that.

::
   local_conf:
 conf: |
  [[local|localrc]]
  foo=a
  [[post-config|$NEUTRON_CONF]]
  [DEFAULT]
  global_physnet_mtu = 1400

is an example of what we are trying to enable here. We'll try to get
some jobs merged today using it so there will be some more examples
that people can work with.


devstack plugins


DevStack plugins allow you to load and change any settings you want,
and run arbitrary python code during the stack.sh run. A lot of what
is done in the pre_test_hook.sh by many projects, could rather be done
by an additional plugin, which also would let people reproduce those
configs for local testing very easily.


If you have other issues


If you find your jobs failing, and it looks like it is because the new
local.conf did not get generated correctly, please feel free to jump
into the #openstack-infra IRC channel and provide links to the jobs
and logs in question. We've been trying to debug as we go and figure
out situations that weren't anticipated.

Thanks to the Ironic and Heat teams to helping with some of those
unanticipated ones, and working to help us test solutions.

The fixes, like in the Heat case, may require projects to update their
testing (and backport that to stable branches). In an ideal world that
wouldn't be the case, but the lack of an interface to support
local.conf in devstack-gate caused folks to do lots of work arounds
using whatever seemed to work to get their job done. Some were more
inventive than others. The hope is that with the stable ``local_conf``
stanza in project-config we're going to have something we can support
going forward even as infrastructure evolves.

--
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] The end of OpenStack packages in Debian?

[Ritesh Raj Sarraf]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hello Allison,

On Thu, 2017-02-23 at 19:07 -0500, Allison Randal wrote:
> > For the LIO-fb target in Debian, we've been depending on the rtslib-fb
> package,
> > which you've maintained so far. Should we pick it up under the pkg-linux-
> target
> > team ?
> 
> Hi Ritesh,
> 
> For the sake of consistency, it probably makes more sense to maintain
> rtslib-fb within the Debian Python Modules Team. If you'd like to chat
> in more detail about which Debian team is the best fit, we can narrow
> this thread down to just the PKG OpenStack mailing list.

rtslib-fb is a core component for the LIO(-fb) project. We already maintain
configshell-fb and targetcli-fb under pkg-linux-target group.

It is perfectly fine to have it under DPMT, as long as it is maintained. Only
if in case there are no takers, then we can consider taking this package.

Thanks,
Ritesh

- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAliwGCYACgkQpjpYo/Lh
dWlhBA//VQ0wNnq5izj6G3CztB1PotY7RyUJOU85Vw8FF/xsOI6I3PEkw0aJ1H9x
FhHAxzbWSMduOw2NV/m4m7bOsoqktXwqZ6MKH+vrFJ+7EIQdN0woUFISPKasXRpU
+0Qg1m0btEZQNT+hnMtR6Oq0NDELREteQp7uHGws/XgsDzXg64eu2q19Pa51uWvG
l+zg5KsmC5ZPYew50B8o2yblIMj8csdRZ8jTknCS9O7B5GobIJoAXK7/aMTk1q87
J6JobsbWPk3BfX7ei5Hc73+9fNf11iABwvZnwFKzXxx1dDA3K+DuH9N95EyFY75Q
PhIkMcx+9qRGWnt2FxTgn0RyGaFKc4L5e4loNCS8Pip4EWBGU9kQjT+LX1kZs4xQ
8ymAq7cSO9TiHWlNB5e2eGs3tliAsv6YMD6JibZ1JaLq7ZeonLpXf9VQ90QFlqpl
tRK8KA4WVrXVx5svrYAIcDaiCzyxujZ/tj9U8fRyaC/Y1gcsvSqpNZzGyzy4GyQJ
AcGJZToMCsxgc/dpF842otluiQAPc+CWfZ2HQVoH9p7DnHbSDAayV9pfrx/P5T+w
pfVKnCxlqz1ysfETybNyQerNexOrLZjWYWsfUSGsI7JJKQnWNufH5KO6AmGnnNXI
wHp/yU3K89+ZrCLCoBlXxYdWzPL9CPq/LW/S937jRXlnPI7jJCw=
=+0Pk
-END PGP SIGNATURE-


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [telemetry][heat] Removal of Aodh combination alarms

[Julien Danjou]

Hey,

We're about to remove the code for the deprecated combination alarm.

  https://review.openstack.org/#/c/429405/

I checked and as far as I can tell that should not break Heat gate, but
I'd prefer to be sure. So, Heat developers if you have some code relying
on it, let us know before we press the big +A button.

Cheers,
-- 
Julien Danjou
# Free Software hacker
# https://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev