Re: [openstack-dev] [cinder] [ceilometer]

[Li, Xiaoyan]

The new notification reports kinds of capacity information, includes total, 
free, allocated, provisioned, visual_free. 
Detailed info please see the following specs: 

https://review.openstack.org/#/c/192542/2/specs/liberty/cinder-capacity-notifications.rst
https://review.openstack.org/#/c/249932/1/specs/mitaka/capacity-headroom.rst
https://review.openstack.org/#/c/206923/

Lisa

-Original Message-
From: Julien Danjou [mailto:jul...@danjou.info] 
Sent: Thursday, December 8, 2016 5:42 PM
To: Jiong Liu 
Cc: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [cinder] [ceilometer]

On Thu, Dec 08 2016, Jiong Liu wrote:

Hi Jeremy,

If I'm not mistaken, there's a new pollster in Ocata that uses the Cinder API 
to get the metrics now, so I think the volume usage audit is getting useless.

> Hello Cinder/Ceilometer community,
>
>  
>
> Is there any guideline on using `cinder-volume-usage-audit` command?
>
>  
>
> Search through cinder/ceilometer logs, I find some messages are sent 
> to ceilometer-collector.
>
> How do I check the output of this command in ceilometer? Do you have 
> any suggestion?
>
>  
>
> Looking forward to your comments. Thanks.
>
>  
>
> BR,
>
> Jeremy Liu
>
>
>

--
Julien Danjou
-- Free Software hacker
-- https://julien.danjou.info

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [gate] [cinder] A current major cause for gate failure - cinder backups

[Li, Xiaoyan]

Hi,

I noticed that as VolumesBackupsV1Test and VolumesBackupsV2Test use same volume 
to do backup creation test etc. 
When creating backup from volume, it needs to attach volume. As both two tests 
use same volume, they attach the volume at same time, and leads failure. 
I opened a bug https://bugs.launchpad.net/tempest/+bug/1616338, and will fix it.

Best wishes
Lisa

-Original Message-
From: Sean Dague [mailto:s...@dague.net] 
Sent: Wednesday, August 24, 2016 10:21 AM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: [openstack-dev] [gate] [cinder] A current major cause for gate failure 
- cinder backups

The gate is in a bad state, as people may have noticed. We're only at a 50% 
characterization for integrated-gate right now - 
http://status.openstack.org/elastic-recheck/data/integrated_gate.html
which means there are a lot of unknown bugs in there.

Spot checking one job - gate-tempest-dsvm-postgres-full-ubuntu-xenial -
6 of the 7 fails were failure of cinder backup -
http://logs.openstack.org/92/355392/4/gate/gate-tempest-dsvm-postgres-full-ubuntu-xenial/582fbd7/console.html#_2016-08-17_04_55_24_109972
- though they were often different tests.

With the current state of privsep logging (hundreds of lines at warn
level) it is making it difficult for me to narrow this down further. I do 
suspect this might be another concurrency shake out from os-brick, so it 
probably needs folks familiar to go through logs with a fine toothed comb to 
get to root cause. If anyone can jump on that, it would be great.

This is probably not the only big new issue, but it seems like a pretty 
concrete one that solving would help drop out merge window (which is 16 hours).

-Sean

--
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder]restore method while ceph backup backend is used

[Li, Xiaoyan]

Hi,

Cinder backup service discards excess bytes when restoring based on 
configuration.
https://github.com/openstack/cinder/blob/master/cinder/backup/drivers/ceph.py#L317
https://github.com/openstack/cinder/blob/master/cinder/backup/drivers/ceph.py#L284


Lisa

From: 刘庆 [mailto:win...@gmail.com]
Sent: Friday, June 24, 2016 11:05 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [cinder]restore method while ceph backup backend is 
used

Hi all,
This is Liu Qing, new to this community. I have a question about the ceph 
backup backend.
From the cinder/backup/manager.py, the restore process is done by attach the 
original device(restore dest) to the cinder service host. And in the ceph 
driver the full_restore will discard the additional space if restore dest(by 
volume extend) is larger than the backup. There are two ways to discard the 
unused space, by ceph discard or write zeroes to the unused space. As the dest 
is attached to the host, the dest is not recognized as an ceph volume. Ceph 
discard will never be used in volume restore, right? But ceph discard is much 
more efficient than writing zeroes to the unused space. Is there any plan to 
use the ceph discard way? Thanks.

Liu Qing
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] [nova] os-brick privsep failures and an upgrade strategy?

[Li, Xiaoyan]

Hi,

What is the discussion result of privsep issue?
When can we release next os-brick?

Best wishes
Lisa

From: Ivan Kolodyazhny [mailto:e...@e0ne.info]
Sent: Wednesday, July 13, 2016 9:55 PM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [cinder] [nova] os-brick privsep failures and an 
upgrade strategy?

Thanks for the update, Matt.

I will join our meeting next week.

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Tue, Jul 12, 2016 at 4:25 PM, Matt Riedemann 
> wrote:
On 7/12/2016 6:29 AM, Ivan Kolodyazhny wrote:
Hi team,

Do we have any decision on this issue? I've found few patches but both
of them are -1'ed.

From Cinder perspective, it blocks us to release new os-brick with
features, which are needed for other projects like Cinder and
python-brick-cinderclient-ext.

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/

On Wed, Jun 22, 2016 at 5:47 PM, Matt Riedemann
 
>> wrote:

On 6/21/2016 10:12 PM, Angus Lees wrote:

On Wed, 22 Jun 2016 at 05:59 Matt Riedemann
 
>



1840
WARNING oslo.privsep.daemon [-] privsep log:
/usr/local/bin/nova-rootwrap: Unauthorized command: privsep-helper
--config-file /etc/nova/nova.conf --privsep_context
os_brick.privileged.default --privsep_sock_path
/tmp/tmpV5w2VC/privsep.sock (no filter matched)

 .. so nova-rootwrap is rejecting the privsep-helper command line
because no filter matched.  This indicates the nova
compute.filters file
has not been updated, or is incorrect.


As was later pointed out by mtreinish, grenade is attempting to
run the
newton code against mitaka configs, and this includes using mitaka
rootwrap filters.   Unfortunately, the change to add privsep to
nova's
rootwrap filters wasn't approved until the newton cycle (so that
all the
os-brick privsep-related changes could be approved together), and so
this doesn't Just Work.

Digging in further, it appears that there *is* a mechanism in
grenade to
upgrade rootwrap filters between major releases, but this needs
to be
explicitly updated for each project+release and hasn't been for
nova+mitaka->newton.  I'm not sure how this is *meant* to work,
since
the grenade "theory of upgrade" doesn't mention when configs
should be
updated - the only mechanism provided is an "exception ... used
sparingly."


As noted in the review, my understanding of the config changes is
deprecation of options across release boundaries so that you can't
drop a config option that would break someone from release to
release without it being deprecated first. So deprecate option foo
in mitaka, people upgrading from liberty to mitaka aren't broken,
but they get warnings in mitaka so that when you drop the option in
newton it's not a surprise and consumers should have adjusted during
mitaka.

For rootwrap filters I agree this is more complicated.


Anyway, I added an upgrade step for nova mitaka->newton that updates
rootwrap filters appropriately(*).  Again, I'm not sure what this
communicates to deployers compared to cinder (which *did* have the
updated rootwrap filter merged in mitaka, but of course that update
still needs to be installed at some point).
(*) https://review.openstack.org/#/c/332610

 - Gus



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:

openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Alternatively Walter had a potential workaround to fallback to
rootwrap for 

Re: [openstack-dev] [cinder] Testing Cinder upgrades - c-bak upgrade

[Li, Xiaoyan]

@ DuncanT and @dule: 

I noticed from IRC log that you are discussing about c-bak upgrade, and I am 
working on this, please see following message. Hope I don't miss anything.

As you know, currently c-bak and c-vol are in same nodes. c-bak depends on 
c-vol service. 

But it is not necessary that all c-vols needs to be upgraded before c-backs.

The sequences can be random. As described in the patch 
https://review.openstack.org/#/c/269412/,
when c-api decides which c-bak service to create/restore, it checks the version 
of c-vol. If c-vol is new version, find a c-bak in new version.
If c-vol is in old version, find a c-bak in old version.

Let's us think a real case. Customers upgrade c-api, c-sch, and start to 
upgrade c-vol and c-bak.
There are two c-vol services c-vol1 and c-vol2, and two c-bak services c-bak1 
and c-bak2.
There are four typical upgrade sequences as following. 
Meanwhile, please notice that c-vol and c-bak are in same nodes in Liberty. So 
during upgrades, if c-vol went down to upgrade, c-bak is also down. 

1. c-vol1->c-bak1->c-vol2->c-bak2
The only insufficiency is when c-vol1 upgrades, and other c-bak services 
haven't upgraded, the request to create/restore backups from/to volumes in 
c-vol1 will fail with reason "no valid c-bak service
found". It is acceptable, as it is similar to scenario in Liberty: c-vol active 
and c-bak fails.

2. c-vol1->c-vol2->c-bak1->c-bak2
Before c-bak1 upgrades, no back request can be completed as no active c-bak 
services. This is reasonable.

3. c-bak1->c-vol1->c-bak2->c-vol2:
The issue is when c-bak2 upgrades, the request to create/restore backups 
from/to volumes in c-vol2 will fail with reason c-vol not active. This is 
consistent with behaviors in Liberty.

4. c-bak1->c-bak2->c-vol1->c-vol2: 
Before c-vol1 upgrades, no back request can be completed as c-vol services not 
active This is reasonable.


Best wishes
Lisa

On Jan 20, 2016 21:59, Michał Dulko wrote:
> Hi,
> 
> In Mitaka Cinder team is implementing rolling upgrades capabilities. I
> want to get some feedback on possibilities of implementing some partial or
> multinode grenade check for our purposes.
> 
> Basically Cinder consists of 4 services calling each other over RPC the in
> following fashion:
> 
>   +-+ c-api +-+
>   | + |
>   | | |
>   v v v
> c-sch <-> c-vol <-+ c-bak
> 
> The order of upgrades we're forced to use (at least in this release) is
> c-api->c-sch->c-vol->c-bak. I wonder how we can test interoperability of
> services in different versions in that model. I have three ideas:
> 
> 1. One idea would be to have two nodes - one with full Cinder deployment
> in latest master and second one running c-sch, c-vol and c-bak in latest
> stable version. That way we would test interoperability of the services
> versions. A problem I see is that tests would be strongly nondeterministic,
> as a particular test result would depend on which RPC service got the
> request. That would make debugging CI failures harder and may result in
> breaking patches slipping in.
> 
> 2. We may simply run a controller<->compute multinode, similar to how
> Nova is running multinode grenade job. Controller would run c-api, c-sch
> and compute c-vol, c-bak. Disadvantage of this model is the fact that we
> wouldn't test c-api->c-sch compatibility.
> 
> 3. Do a single upgrade for each of the services. That would mean testing
> master c-api with stable rest of services, then upgrading c-sch, retesting,
> upgrading c-vol, retesting, upgrading c-bak, retesting. That way we would
> test all the possibilities but such run would take a lot of time. Moreover if 
> I
> recall correctly such idea isn't possible in current state of Grenade.
> 
> Comments and feedback is very welcome.
> 
> Thanks,
> Michal (IRC: dulek)
> 
 _
 _ OpenStack Development Mailing List (not for usage
 questions) Unsubscribe: OpenStack-dev-
 requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Best wishes
Lisa



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder]Detach a encrypted volume won't cleanup the device patch

[Li, Xiaoyan]

Hi,

For ISCSIConnector, the path can be recovered by rescan before 
disconnect_volume:
https://github.com/openstack/os-brick/blob/master/os_brick/initiator/connector.py#L835

And for NFS volumes, it leads to following bug: 
https://bugs.launchpad.net/nova/+bug/1511255

Best wishes
Lisa

From: liuxinguo [mailto:liuxin...@huawei.com]
Sent: Tuesday, December 29, 2015 4:09 PM
To: walter.bor...@hp.com
Cc: OpenStack Development Mailing List; Zhangli (ISSP); Luozhen; Shenhong (C)
Subject: [openstack-dev] [nova][cinder]Detach a encrypted volume won't cleanup 
the device patch

Hi hemna:

When we attach a encrypted volume and then detach it, the device path created 
when the volume is attached won't be cleanup.
This is probably due to nova and cinder not work coordinately when detach a 
encrypted volume.
This will cause next volume attachment fails and also cause some test cases of 
CI fail because the not cleaned up device path.

I opened a bug in os-brick(not sure it's a nova or os-brick bug:)):
https://bugs.launchpad.net/os-brick/+bug/1528092

And want to see if you have any idea about this?

Thanks!
Wilson Liu
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Dependencies of snapshots on volumes

[Li, Xiaoyan]

On Dec 10, 2015 06:34, Mike Perez wrote:
> On 09:27 Dec 09, John Griffith wrote:
>> On Tue, Dec 8, 2015 at 9:10 PM, Li, Xiaoyan <xiaoyan...@intel.com>
> wrote:
> 
> 
> 
>>> As a result, this raises two concerns here:
>>> 1. Let such operations behavior same in Cinder.
>>> 2. I prefer to let storage driver decide the dependencies, not in
>>> the general core codes.
>>> 
>> 
>> ​I have and always will strongly disagree with this approach and your
>> proposal.  Sadly we've already started to allow more and more vendor
>> drivers just "do their own thing" and implement their own special API
>> methods.  This is in my opinion a horrible path and defeats the entire
>> purpose of have a Cinder abstraction layer.
>> 
>> This will make it impossible to have compatibility between clouds for
>> those that care about it, it will make it impossible for
>> operators/deployers to understand exactly what they can and should
>> expect in terms of the usage of their cloud.  Finally, it will also
>> mean that not OpenStack API functionality is COMPLETELY dependent on
>> backend device.  I know people are sick of hearing me say this, so I'll
>> keep it short and say it one more time: "Compatibility in the API
>> matters and should always be our priority"
> 
> +1
>

This seems that cinder needs to take more and more works, and vendor storages 
do what cinder asks them to. 
For example, cinder supports full and incremental snapshots in core codes, and 
it keeps the dependencies like backups. 

More general example is that storage vendors supports kinds of volumes, like 
normal, provisioned, and compressed etc. 
Cinder needs to implement such functions in core codes. Every storage vendor 
report their capability to cinder scheduler. 
When users create a volume, scheduler finds a storage vendor based on their 
capacities. 
(Of course these functions in cinder should be general and implemented by most 
of vendor storages. )

But currently cinder core codes do little, lots of this are in extra_specs, 
conf file which are handled in vendor drivers. 

This leads to a problem like extending volume. Extending a volume in an 
incremental snapshot fails
in vendor storage.  And then the cinder volume goes into error_extending 
status. From my opinion this is not good. 

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder] Dependencies of snapshots on volumes

[Li, Xiaoyan]

Hi all, 

Currently when deleting a volume, it checks whether there are snapshots created 
from it. If yes deletion is prohibited.  But it allows to extend  the volume, 
no check whether there are snapshots from it.

The two behaviors in Cinder are not consistent from my viewpoint. 

In backend storage, their behaviors are same.
For full snapshot, if still in copying progress, both extend and deletion are 
not allowed. If snapshot copying finishes, both extend and deletion are allowed.
For incremental snapshot, both extend and deletion are not allowed. 

As a result, this raises two concerns here: 
1. Let such operations behavior same in Cinder.
2. I prefer to let storage driver decide the dependencies, not in the general 
core codes. 

Meanwhile, if we let driver to decide the dependencies, the following changes 
need to do in Cinder: 
1. When creating a snapshot from volume, it needs copy all metadata of volume 
to snapshot. Currently it doesn't. 
Any other potential issues please let me know.

Any input will be appreciated. 

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick

[Li, Xiaoyan]

> -Original Message-
> From: Ben Swartzlander [mailto:b...@swartzlander.org]
> Sent: Friday, December 4, 2015 2:45 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick
> 
> On 12/03/2015 07:40 AM, Duncan Thomas wrote:
> > On 3 December 2015 at 11:14, Li, Xiaoyan <xiaoyan...@intel.com
> > <mailto:xiaoyan...@intel.com>> wrote:
> >
> > Just to clear the data operations cinder needs to touch plaintext
> > data are:
> >   1) Create volume from glance image
> >   2) Create glance image from volume
> >   3) Retype encrypted volumes. That is to change a volume from
> > unencrypted to encrypted, or vice visa.
> >
> > Backup/Restore doesn't need to decrypt data.
> >
> >
> > Backup / restore doesn't currently decrypt the data. There are some
> > people commenting that it is not useful for DR work to have a backup
> > that requires keys from a key service that is itself not backed up, so
> > there may be some proposal incoming about not encrypting backups, or
> > else giving them their own key rather than require access to the
> > original volume key during restore - needing that access also makes
> > things like re-keying the original volume difficult/impossible.
> >
> > Again, we have multiple use-cases for encryption, and they are not all
> > going to be solved by solved by draconian dictates that there shall
> > only be one way of doing things.
> 
> There are other very good reasons for multiple encryption keys for
> different purposes. Client side data encryption is known to prevent server-
> side compression and deduplication technologies from working at all, and
> it makes backups wildly less efficient. The upshot is that if choose to do
> implement security by encryption everything in the guest or hypervisor
> rather than in the storage controller, you're going to spend a ton more on
> disks.
> 
> Assuming your threat model involves evil people sniffing network wires,
> and pulling disks from machines in the datacenter, rather than assuming to
> storage admin himself is evil, you can devise schemes that involve separate
> encryption for in-flight data and at-rest data which allow the storage
> controller to do compression and deduplication and store your data in both
> a secure AND EFFICIENT manner.
> 
> The above isn't a future fantasy -- there are storage controllers that do this
> TODAY with unmodified cinder and nova. You just need a storage
> controller that features full-disk-encryption and also transport level
> security (such as blocks over Kerberized NFS) as well as the compression
> and deduplication technologies which are quickly becoming standardized.

Yeah, another point I can think is that when backup restores, it can use 
different encryptions, like key, key_size, and algorithm.
 
> 
> -Ben
> 
> 
> >
> _
> _
> >  OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> 
> _
> _
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick

[Li, Xiaoyan]

Thank you, Ben. I agree with you, and just to clear the cinder operations which 
needs to decrypt volumes in following.

On Dec 3, 2015 05:01, Ben Swartzlander wrote:
> On 11/30/2015 09:04 AM, Coffman, Joel M. wrote:
>> 
>> 
>> On 11/25/15, 11:33 AM, "Ben Swartzlander" > > wrote:
>> 
>> On 11/24/2015 03:27 PM, Nathan Reller wrote:
>> the cinder admin and the nova admin are ALWAYS the same
>> people
>> 
>> 
>> There is interest in hybrid clouds where the Nova and Cinder
>> services are managed by different providers. The customer would
>> place higher trust in Nova because you must trust the compute
>> service, and the customer would place less trust in Cinder. One
>> way to achieve this would be to have all encryption done by
>> Nova. Cinder would simply see encrypted data and provide a good
>> cheap storage solution for data.
>> 
>> Consider a company with sensitive data. They can run the
>> compute nodes themselves and offload Cinder service to some
>> third-party service. This way they are the only ones who can
>> manage the machines that see the plaintext.
>> 
>> If you have that level of paranoia, I suggest running LUKS inside
>> the guest VM and not relying on OpenStack to handle your
>> encryption. Then you don't have to worry about whether nova is
>> sharing your keys with cinder because even nova won't have them.
>> That approach isn't actually more secure - anyone with root access to
>> the compute host can dump the VM's memory to extract the encryption
> keys.
> 
> I agree, however in the above case there was implied trust in the compute
> infrastructure -- at least more so than in the storage infrastructure. If you
> don't trust your hypervisor admin to not read your VM memory and steal
> encryption keys, then relying on your hypervisor admin (or nova) to
> perform the encryption is kind of silly.
> In every case, the hypervisor admin can see the plaintext and the keys.
> 
> The suggestion was a way to achieve the goal of doing encryption
> WITHOUT trusting the storage admin and WITHOUT CHANGING ANY CODE.
> I assert that any attempt to implement encryption at the nova level and not
> sharing keys with cinder will break the existing model. There are 2
> solutions I can see:
> 1) don't break it (see above)
> 2) you break it, you fix it (nova takes over responsibility for all the
> operations cinder currently performs that involve plaintext).
> 
>> Trying to design a system where we expect nova to do data
>> encryption but not cinder will not work in the long run. The
>> eventual result will be that nova will have to take on most of the
>> functionality of cinder and we'll be back to the nova-volume days.
>> Could you explain further what you mean by "nova will have to take on
>> most of the functionality of cinder"? In the current design, Nova is
>> still passing data blocks to Cinder for storage - they're just
>> encrypted instead of plaintext. That doesn't seem to subvert the
>> functionality of Cinder or reimplement it.
> 
> I think Duncan covered it, but the data operations cinder currently
> performs, which require Cinder to touch plaintext data are:
> 1) Create volume from glance image
> 2) Create glance image from volume
> 3) Backup volume
> 4) Restore volume

Just to clear the data operations cinder needs to touch plaintext data are:
 1) Create volume from glance image
 2) Create glance image from volume
 3) Retype encrypted volumes. That is to change a volume from unencrypted to 
encrypted, or vice visa.

Backup/Restore doesn't need to decrypt data.

> 
> I'm not claiming that we can't redefine or alter the above operations to
> deal with encryption, but someone needs to propose how they should work
> differently or work at all when the volume isn't storing plaintext data.
> 
>> Also in case it's not obvious, if you use different providers for
>> compute and storage, your performance is going to be absolutely
>> terrible.
>> The general idea is probably separation of duties, which contradicts
>> the original statement that "the cinder admin and the nova admin are
>> ALWAYS the same people." Is there an operational reason that these
>> admins must be the same person, or is that just typical?
> 
> My assertion was to try to combat confusion about the roles of the "storage
> admin". Many people who don't deal with storage all the time tend to
> forgot that cinder is a management tool that's separate from actual
> hardware which stores bits. It's common to have a "storage admin"
> who is responsible for configuration and management of storage hardware
> and software but to have Cinder run by an "openstack admin" who is a
> customer/client of the storage admin.
> 
> My belief is that it's FAR more common for cinder to be installed and run
> by the same guy (or group) who installs/runs 

Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick

[Li, Xiaoyan]

From: Coffman, Joel M. [mailto:joel.coff...@jhuapl.edu]
Sent: Thursday, December 3, 2015 2:07 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick


From: "duncan.tho...@gmail.com" 
>
Reply-To: 
"openstack-dev@lists.openstack.org" 
>
Date: Monday, November 30, 2015 at 9:13 AM
To: 
"openstack-dev@lists.openstack.org" 
>
Subject: Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick

On 30 November 2015 at 16:04, Coffman, Joel M. 
> wrote:
On 11/25/15, 11:33 AM, "Ben Swartzlander" 
> wrote:

On 11/24/2015 03:27 PM, Nathan Reller wrote:
Trying to design a system where we expect nova to do data encryption but
not cinder will not work in the long run. The eventual result will be
that nova will have to take on most of the functionality of cinder and
we'll be back to the nova-volume days.
Could you explain further what you mean by "nova will have to take on most of 
the functionality of cinder"? In the current design, Nova is still passing data 
blocks to Cinder for storage – they're just encrypted instead of plaintext. 
That doesn't seem to subvert the functionality of Cinder or reimplement it.

The functionality of cinder is more than blindly storing blocks - in particular 
it has create-from/upload-to image, backup, and retype, all of which do some 
degree of manipulation of the data and/or volume encryption metadata.
From a security perspective, it is advantageous for users to be able to upload 
an encrypted image, copy that image to a volume, and boot from that volume 
without decrypting the image until it is booted.

This is not efficient, and user friendly.

We are suffering from somewhat incompatible requirements with encryption 
between those who want fully functional cinder and encryption on disk (the 
common case I think), and those who have enhanced security requirements.
The original design supports this distinction: there is a "control-location" 
parameter that indicates where encryption is to be performed (see 
http://docs.openstack.org/user-guide-admin/dashboard_manage_volumes.html).

It seems that it differentiates from Nova/Cinder and back-end block storage.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick

[Li, Xiaoyan]

Hi,

Except creating encrypted volume from images, uploading encrypted volumes to 
image, as Duncan said there is desire to migrate volumes between encrypted and 
unencrypted type.
https://review.openstack.org/#/c/248593/

And key magagment codes are duplicated in Cinder and Nova:
https://github.com/openstack/cinder/tree/master/cinder/keymgr
https://github.com/openstack/nova/tree/master/nova/keymgr


Best wishes
Lisa

From: Duncan Thomas [mailto:duncan.tho...@gmail.com]
Sent: Monday, November 23, 2015 8:29 PM
To: Daniel P. Berrange; OpenStack Development Mailing List (not for usage 
questions)
Subject: Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick

Hi Daniel
Much of this got discussed before.

Encrypted images uploaded to glance aren't shareable, and there is definitely a 
desire by many users to keep the usual glance functionality while having 
encryption at rest in cinder for e.g. regulatory purposes.
There is also some desire to be able to migrate unencrypted volume types to 
encrypted types inside cinder, which would require cinder to be able to create 
an encrypted volume in a similar way to creating from an image.

Managing access to the key data is, as far as I'm aware, the job of e.g. 
barbican/castellan, not nova per se. There are several usecases for encryption, 
and several of the less paranoid make good sense without requiring nova to be 
the only thing with access to the key material.


On 23 November 2015 at 13:21, Daniel P. Berrange 
> wrote:
On Fri, Nov 20, 2015 at 11:34:29AM -0800, Walter A. Boring IV wrote:
> On 11/20/2015 10:19 AM, Daniel P. Berrange wrote:
> >On Fri, Nov 20, 2015 at 02:45:15PM +0200, Duncan Thomas wrote:
> >>Brick does not have to take over the decisions in order to be a useful
> >>repository for the code. The motivation for this work is to avoid having
> >>the dm setup code copied wholesale into cinder, where it becomes difficult
> >>to keep in sync with the code in nova.
> >>
> >>Cinder needs a copy of this code since it is on the data path for certain
> >>operations (create from image, copy to image, backup/restore, migrate).
> >A core goal of using volume encryption in Nova to provide protection for
> >tenant data, from a malicious storage service. ie if the decryption key
> >is only ever used by Nova on the compute node, then cinder only ever sees
> >ciphertext, never plaintext.  Thus if cinder is compromised, then it can
> >not compromise any data stored in any encrypted volumes.
> >
> >If cinder is looking to get access to the dm-seutp code, this seems to
> >imply that cinder will be getting access to the plaintext data, which
> >feels to me like it de-values the volume encryption feature somewhat.
> >
> >I'm fuzzy on the details of just what code paths cinder needs to be
> >able to convert from plaintext to ciphertext or vica-verca, but in
> >general I think it is desirable if we can avoid any such operation
> >in cinder, and keep it so that only Nova compute nodes ever see the
> >decrypted data.
> Being able to limit the number of points where an encrypted volume can be
> used unencrypted
> is obviously a good goal.
> Unfortunately, it's entirely unrealistic to expect Cinder to never be able
> to have access that access.
>
> Cinder currently needs access to write data to volumes that are encrypted
> for several operations.
>
> 1) copy volume to image
If a volume is encrypted and it is being copied to an image, IMHO we
should not aim to decrypt it. We should copy the data as is and mark
the image as encrypted in glance, and then use it as is next time the
image is needed.

FYI, Nova is already aiming to consider both the glance data storage
and the glance service as a whole, as untrustworthy. The first step
in this is using cryptographic signatures to detect unauthorized
image data modification by a compromised glance. Encryption will be
a later step in the process.

> 2) copy image to volume

This is semi-plausible as a place where Cinder needs to go from
unencrypted image data to encrypted volume data, when a user is
creating a volume from an image ahead of time, distinct from any
VM boot attempt. In such a case it is desirable that Cinder not
be able to request any existing volume keys from the key server,
merely have the ability to upload new keys and throw away its
local copy thereafter.

> 3) backup

Cinder should really not try to decrypt volumes when backing them
up. If it conversely wants to encrypt volumes during backup, it
can do so with separate backup keys, distinct from those used for
primary volume encryption for use at runtime.


Regards,
Daniel
--
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick

[Li, Xiaoyan]

Hi all,

Let me summarize the most important items that need attach/detach encryptors in 
Cinder:

1.   Create an encrypted volume from images.

2.   Migrate an unencrypted volume to encrypted.


Except above, we may need to upload encrypted volume to Glance. If needs to 
change encryption keys or upload unencrypted images it needs encryptors.

If we can’t provide above functions, I think it will prevent the popular of 
Cinder encryption usage.

Best wishes
Lisa

From: Li, Xiaoyan [mailto:xiaoyan...@intel.com]
Sent: Monday, November 23, 2015 8:57 PM
To: OpenStack Development Mailing List (not for usage questions); Daniel P. 
Berrange
Subject: Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick

Hi,

Except creating encrypted volume from images, uploading encrypted volumes to 
image, as Duncan said there is desire to migrate volumes between encrypted and 
unencrypted type.
https://review.openstack.org/#/c/248593/

And key magagment codes are duplicated in Cinder and Nova:
https://github.com/openstack/cinder/tree/master/cinder/keymgr
https://github.com/openstack/nova/tree/master/nova/keymgr


Best wishes
Lisa

From: Duncan Thomas [mailto:duncan.tho...@gmail.com]
Sent: Monday, November 23, 2015 8:29 PM
To: Daniel P. Berrange; OpenStack Development Mailing List (not for usage 
questions)
Subject: Re: [openstack-dev] [cinder][nova]Move encryptors to os-brick

Hi Daniel
Much of this got discussed before.

Encrypted images uploaded to glance aren't shareable, and there is definitely a 
desire by many users to keep the usual glance functionality while having 
encryption at rest in cinder for e.g. regulatory purposes.
There is also some desire to be able to migrate unencrypted volume types to 
encrypted types inside cinder, which would require cinder to be able to create 
an encrypted volume in a similar way to creating from an image.
Managing access to the key data is, as far as I'm aware, the job of e.g. 
barbican/castellan, not nova per se. There are several usecases for encryption, 
and several of the less paranoid make good sense without requiring nova to be 
the only thing with access to the key material.


On 23 November 2015 at 13:21, Daniel P. Berrange 
<berra...@redhat.com<mailto:berra...@redhat.com>> wrote:
On Fri, Nov 20, 2015 at 11:34:29AM -0800, Walter A. Boring IV wrote:
> On 11/20/2015 10:19 AM, Daniel P. Berrange wrote:
> >On Fri, Nov 20, 2015 at 02:45:15PM +0200, Duncan Thomas wrote:
> >>Brick does not have to take over the decisions in order to be a useful
> >>repository for the code. The motivation for this work is to avoid having
> >>the dm setup code copied wholesale into cinder, where it becomes difficult
> >>to keep in sync with the code in nova.
> >>
> >>Cinder needs a copy of this code since it is on the data path for certain
> >>operations (create from image, copy to image, backup/restore, migrate).
> >A core goal of using volume encryption in Nova to provide protection for
> >tenant data, from a malicious storage service. ie if the decryption key
> >is only ever used by Nova on the compute node, then cinder only ever sees
> >ciphertext, never plaintext.  Thus if cinder is compromised, then it can
> >not compromise any data stored in any encrypted volumes.
> >
> >If cinder is looking to get access to the dm-seutp code, this seems to
> >imply that cinder will be getting access to the plaintext data, which
> >feels to me like it de-values the volume encryption feature somewhat.
> >
> >I'm fuzzy on the details of just what code paths cinder needs to be
> >able to convert from plaintext to ciphertext or vica-verca, but in
> >general I think it is desirable if we can avoid any such operation
> >in cinder, and keep it so that only Nova compute nodes ever see the
> >decrypted data.
> Being able to limit the number of points where an encrypted volume can be
> used unencrypted
> is obviously a good goal.
> Unfortunately, it's entirely unrealistic to expect Cinder to never be able
> to have access that access.
>
> Cinder currently needs access to write data to volumes that are encrypted
> for several operations.
>
> 1) copy volume to image
If a volume is encrypted and it is being copied to an image, IMHO we
should not aim to decrypt it. We should copy the data as is and mark
the image as encrypted in glance, and then use it as is next time the
image is needed.

FYI, Nova is already aiming to consider both the glance data storage
and the glance service as a whole, as untrustworthy. The first step
in this is using cryptographic signatures to detect unauthorized
image data modification by a compromised glance. Encryption will be
a later step in the process.

> 2) copy image to volume

This is semi-plausible as a place where Cinder needs to go from
unencrypted image data to enc

Re: [openstack-dev] [cinder][glance]Upload encrypted volumes to images

[Li, Xiaoyan]

On Nov 23, 2015 22:34, Daniel P. Berrange wrote:
> On Mon, Nov 23, 2015 at 07:05:05AM +0100, Philipp Marek wrote:
>>> About uploading encrypted volumes to image, there are three options:
>>> 1. Glance only keeps non-encrypted images. So when uploading
> encrypted
>>>volumes to image, cinder de-crypts the data and upload. 2. Glance
>>>maintain encrypted images. Cinder just upload the encrypted data to
>>>image.
>>> 3. Just prevent the function to upload encrypted volumes to images.
>>> 
>>> Option 1 No changes needed in Glance. But it may be not safe. As we
>>> decrypt the data, and upload it to images. Option 2 This imports
>>> encryption to Glance which needs to manage the encryption metadata.
>>> 
>>> Please add more if you have other suggestions. How do you think which
> one is preferred.
>> Well, IMO only option 1 is useful.
>> 
>> Option 2 means that the original volume, the image, and all derived
>> volumes will share the same key, right?
> 
> That depends on how you implement it really. If you directly upload the
> encrypted volume as-is, and then directly boot later VMs of the same
> image, as-is they'll obviously share the same key. It is possible though for
> cinder to re-encrypt the volume with a different key before uploading it, or
> more likely for Nova to re-encrypt the image with a different key after
> downloading it to boot an instance.

Could any people from Glance say anything? As I heard that there were problems
When nova boots from an encrypted image if Glance keeps encrypted images.

> 
>> That's not good. (Originally: "unacceptable")
> 
> If the images and all volumes are all owned by a single tenant user it is not
> a big deal if they have the same key. Depending on what threats you are
> protecting against, it may be more desirable than having the data stored
> unencrypted in glance.
> 
> Regards,
> Daniel


Best wishes
Lisa



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder][glance]Upload encrypted volumes to images

[Li, Xiaoyan]

Hi all,
More help about volume encryption is needed. 

About uploading encrypted volumes to image, there are three options:
1. Glance only keeps non-encrypted images. So when uploading encrypted volumes 
to image, cinder de-crypts the data and upload.
2. Glance maintain encrypted images. Cinder just upload the encrypted data to 
image. 
3. Just prevent the function to upload encrypted volumes to images.

Option 1 No changes needed in Glance. But it may be not safe. As we decrypt the 
data, and upload it to images. 
Option 2 This imports encryption to Glance which needs to manage the encryption 
metadata.

Please add more if you have other suggestions. How do you think which one is 
preferred.
Appreciate for your help.

Best wishes
Lisa



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder][nova]Move encryptors to os-brick

[Li, Xiaoyan]

Hi all,

To fix bug [1][2] in Cinder, Cinder needs to use nova/volume/encryptors[3] to 
attach/detach encrypted volumes. 

To decrease the code duplication, I raised a BP[4] to move encryptors to 
os-brick[5].

Once it is done, Nova needs to update to use the common library. This is BP 
raised. [6]

Meanwhile, except the above move, I found some bugs ,and plan to fix in this 
release. Please see [4].

Any help will be appreciated.

[1] https://bugs.launchpad.net/nova/+bug/1465656
[2] https://bugs.launchpad.net/cinder/+bug/1482464
[3] https://github.com/openstack/nova/tree/master/nova/volume/encryptors
[4] https://blueprints.launchpad.net/cinder/+spec/improve-encrypted-volume
[5] https://launchpad.net/os-brick
[6] https://blueprints.launchpad.net/nova/+spec/nova-os-brick-encryptors

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder]Review request for data transfer between encrypted volumes and images

[Li, Xiaoyan]

Hi, 

@Jay, 
As talked with you in IRC, I have updated the patches which fix the bug in 
Cinder about creating encrypted volumes from images, and uploading encrypted 
volumes to images.
Please help to review. 

https://review.openstack.org/#/c/216567/
https://review.openstack.org/#/c/217557/

Also welcome others' review. 

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Proposing Gorka Eguileor for core

[Li, Xiaoyan]

On Aug 14, 2015 03:13, Mike Perez wrote:
 It gives me great pleasure to nominate Gorka Eguileor for Cinder core.
 
 Gorka's contributions to Cinder core have been much apprecated:
 
 https://review.openstack.org/#/q/owner:%22Gorka+Eguileor%22+project:o
 penstack/cinder,p,0035b6410002dd11
 
 60/90 day review stats:
 
 http://russellbryant.net/openstack-stats/cinder-reviewers-60.txt
 http://russellbryant.net/openstack-stats/cinder-reviewers-90.txt
 
 Cinder core, please reply with a +1 for approval. This will be left open until
 August 19th. Assuming there are no objections, this will go forward after
 voting is closed.


My first package in cinder was reviewed by Gorka which helps me a lot. 
Thank you, Gorka.

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder]Cinder creates encrypted volume from image

[Li, Xiaoyan]

Hi all,

Currently when cinder creates a volume with an encrypted volume type from an 
image(which is unencrypted), it just reads data from image, and writes them
Into the volume.
As a result the encrypted volume contains unencrypted data, and Nova fails to 
boot from the volume.
https://bugs.launchpad.net/nova/+bug/1465656

I would like to implement the function that when creating an encrypted volume 
from an image, cinder reads data, encrypts and writes to the volume. 
So that the encrypted volume contains encrypted data as it should be.
https://blueprints.launchpad.net/cinder/+spec/encrypt-volume-with-image

Anyone else is working on it? 
Any suggestions?

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Cinder] encryption is not supported in ceph volume

[Li, Xiaoyan]

Hi all,

I created an encryption type, and create a volume in Ceph with the volume type.
 cinder encryption-type-create

But failed to attach it to a VM. The error message shows that no device_path in 
connection_info.

^[[01;31m2015-07-30 05:55:57.117 TRACE oslo_messaging.rpc.dispatcher 
^[[01;35m^[[00mself.symlink_path = connection_info['data']['device_path']^M
^[[01;31m2015-07-30 05:55:57.117 TRACE oslo_messaging.rpc.dispatcher 
^[[01;35m^[[00mKeyError: 'device_path'

Two questions:
1. Is it not supported to create volume in Ceph with encrypted volume type?
2. If yes, should we prohibit to create a Ceph volume with encrypted volume 
type.

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev