Re: Remove/Change SSH Keys from instance

[Nux!]

Depending on what you actually want to achieve, you could also register a bogus 
public key and reset to that one, knowing it will never really work.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Francois Gaudreault" 
> To: "Nux!" 
> Cc: dev@cloudstack.apache.org
> Sent: Monday, 16 February, 2015 23:53:52
> Subject: Re: Remove/Change SSH Keys from instance

> Thanks for looking at it and testing it.
> 
> That answers the key replacement. Looks like there is no way to actually
> removing the key from a created VM. I believe the only way left is the
> deletion. The updateVM call won't update it either :S
> 
> FG
> 
> On 2015-02-16 6:37 PM, Nux! wrote:
>> "resetSSHKeyForVirtualMachine
>> Resets the SSH Key for virtual machine. The virtual machine must be in a
>> "Stopped" state. [async]"
>>
>> Apologies for causing you extra stress. We all had a long Monday.
>>
>> I also never used this function before and here's how I tested.
>>
>> - empty or move /root/.ssh/authorized_keys on the VM prior to the operation
>> - in cloudmonkey
>>- stop virtualmachine id=XXX
>>- reset sshkeyforvirtualmachine keypair=newkey id=XXX
>>- start virtualmachine id=XXX
>> - after it came back I checked /root/.ssh/authorized_keys and .. lo & behold,
>> the new key was there.
>>
>> So, answer: yes, that's how to reset the SSH key and no, you can't remove it
>> that way - it will check if the provided key name actually exists and you 
>> must
>> provide one.
>>
>>   ☕ :-)
>>
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> - Original Message -
>>> From: "Francois Gaudreault" 
>>> To: "Nux!" 
>>> Cc: dev@cloudstack.apache.org
>>> Sent: Monday, 16 February, 2015 22:22:34
>>> Subject: Re: Remove/Change SSH Keys from instance
>>> Yes what? That API call is not clear at all.
>>>
>>> I guess a "try it yourself" is what you meant?
>>>
>>> Thanks for your great help.
>>>
>>> FG
>>> On Feb 16, 2015 5:04 PM, "Nux!"  wrote:
>>>
 yes?


 https://cloudstack.apache.org/docs/api/apidocs-4.4/user/resetSSHKeyForVirtualMachine.html

 --
 Sent from the Delta quadrant using Borg technology!

 Nux!
 www.nux.ro

 - Original Message -
> From: "Francois Gaudreault" 
> To: dev@cloudstack.apache.org
> Sent: Monday, 16 February, 2015 20:03:59
> Subject: Remove/Change SSH Keys from instance
> Hi,
>
> Quick question, using the API, how would a user remove an associated SSH
> keys to an instance? What call should we use? Seems that you can't based
> on the existing API calls.
>
> Furthermore, how can we change the SSH Key? Is it the
> resetSSHKeyForVirtualMachine call?
>
> Thanks.
>
> --
> Francois Gaudreault
> Gestionnaire de Produit | Product Manager - Cloud Platform & Services
> t:514-629-6775
>
> CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
> 420 rue Guy | Montreal | Quebec | H3J 1S6
> w: cloudops.com | tw: @CloudOps_
>>
> 
> 
> --
> Francois Gaudreault
> Gestionnaire de Produit | Product Manager - Cloud Platform & Services
> t:514-629-6775
> 
> CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
> 420 rue Guy | Montreal | Quebec | H3J 1S6
> w: cloudops.com | tw: @CloudOps_

Re: [MERGE] Redundant VPC routers and persistent router config

[Wilder Rodrigues]

Hi all,

I have been some tests on the branch in order to give you all some confidence.

During the tests I found 1 bug related to communication from VM A on Tier 1 to 
VM B on Tier 2 in a Single VPC. I can reproduce the bug and it disappears when 
I convert the Single VPC to a redundant one. I already talked to Ian and he is 
on it.

Results follow below.

Cheers,
Wilder

Environment:

Xen 6.2 running on VMware zone within our Betacloud (ACS 4.4.2)
MySQL running on MacBook Pro
Management Server on MacBook Pro

::: Manual Tests:::

Isolated Networks

* Create Network 
* Create 2 VMs using new Network
* Create FW rules
* Create PF rules
* SSH to the VMs
* SSH from one VM onto the other in the same isolated network
* Destroy Master router
* Restart the Network
* Restart the Network with Clean-up option
* Repeat steps above

Redundant Isolated Networks

* Create Redundant Network Offering
* Create 2 VMs using new offering
* Create FW rules
* Create PF rules
* SSH to the VMs
* SSH from one VM onto the other in the same redundant isolated network
* Destroy Master router
* Restart the Network
* Stop the Master Router

Single VPC

* Create VPC
* Create 2 Tiers
* Create ACLS
* Create 1 Vm for each Tier
* Associate 2 IP address
* Add PF rules
* SSH onto VMs
* SSH from 1 VM onto another
* Restart VPC - Make it redundant
* Repeat steps above

Redundant VPC

* Create VPC
* Create 2 Tiers
* Create ACLS
* Create 1 Vm for each Tier
* Associate 2 IP address
* Add PF rules
* SSH onto VMs
* SSH from 1 VM onto another
* Stop/Destroy the Master Router
* Observe the Backup router became Master
* SSH again onto the VMs
* Restart VPC (without clean-up)
* Observer only 1 new router is created
* New router is started as Backup
* SSH onto VMs
* Restart VPC (with clean-up)
* Observer only 2 new routers are created
* SSH onto VMs

::: Automated Tests :::

Test Create Account and user for that account ... === TestName: 
test_01_create_account | Status : SUCCESS ===
ok
Test Sub domain allowed to launch VM  when a Domain level zone is created ... 
=== TestName: test_01_add_vm_to_subdomain | Status : SUCCESS ===
ok
Test delete domain without force option ... === TestName: test_DeleteDomain | 
Status : SUCCESS ===
ok
Test delete domain with force option ... === TestName: test_forceDeleteDomain | 
Status : SUCCESS ===
ok
Test update admin details ... === TestName: test_updateAdminDetails | Status : 
SUCCESS ===
ok
Test update domain admin details ... === TestName: 
test_updateDomainAdminDetails | Status : SUCCESS ===
ok
Test user update API ... === TestName: test_updateUserDetails | Status : 
SUCCESS ===
ok
Test login API with domain ... === TestName: test_LoginApiDomain | Status : 
SUCCESS ===
ok
Test if Login API does not return UUID's ... === TestName: 
test_LoginApiUuidResponse | Status : SUCCESS ===
ok

--
Ran 9 tests in 1140.977s

OK

Test reset virtual machine on reboot ... === TestName: 
test_01_reset_vm_on_reboot | Status : SUCCESS ===
ok

--
Ran 1 test in 216.907s

OK

Test advanced zone virtual router ... === TestName: test_advZoneVirtualRouter | 
Status : SUCCESS ===
ok
Test Deploy Virtual Machine ... === TestName: test_deploy_vm | Status : SUCCESS 
===
ok
Test Multiple Deploy Virtual Machine ... === TestName: test_deploy_vm_multiple 
| Status : SUCCESS ===
ok
Test Stop Virtual Machine ... === TestName: test_01_stop_vm | Status : SUCCESS 
===
ok
Test Start Virtual Machine ... === TestName: test_02_start_vm | Status : 
SUCCESS ===
ok
Test Reboot Virtual Machine ... === TestName: test_03_reboot_vm | Status : 
SUCCESS ===
ok
Test destroy Virtual Machine ... === TestName: test_06_destroy_vm | Status : 
SUCCESS ===
ok
Test recover Virtual Machine ... === TestName: test_07_restore_vm | Status : 
SUCCESS ===
ok
Test migrate VM ... SKIP: At least two hosts should be present in the zone for 
migration
Test destroy(expunge) Virtual Machine ... === TestName: test_09_expunge_vm | 
Status : SUCCESS ===
ok

--
Ran 10 tests in 851.022s

OK (SKIP=1)

Test router internal advanced zone ... SKIP: Marvin configuration has no host 
credentials to check router services
Test restart network ... === TestName: test_03_restart_network_cleanup | Status 
: SUCCESS ===
ok
Test router basic setup ... === TestName: test_05_router_basic | Status : 
SUCCESS ===
ok
Test router advanced setup ... === TestName: test_06_router_advanced | Status : 
SUCCESS ===
ok
Test stop router ... === TestName: test_07_stop_router | Status : SUCCESS ===
ok
Test start router ... === TestName: test_08_start_router | Status : SUCCESS ===
ok
Test reboot router ... === TestName: test_09_reboot_router | Status : SUCCESS 
===
ok

--
Ran 7 tests in 454.519s

OK (SKIP=1)

Test to create service offering ... === TestName

Re: Remove/Change SSH Keys from instance

[Francois Gaudreault]

Thanks for looking at it and testing it.

That answers the key replacement. Looks like there is no way to actually 
removing the key from a created VM. I believe the only way left is the 
deletion. The updateVM call won't update it either :S


FG

On 2015-02-16 6:37 PM, Nux! wrote:

"resetSSHKeyForVirtualMachine
Resets the SSH Key for virtual machine. The virtual machine must be in a "Stopped" 
state. [async]"

Apologies for causing you extra stress. We all had a long Monday.

I also never used this function before and here's how I tested.

- empty or move /root/.ssh/authorized_keys on the VM prior to the operation
- in cloudmonkey
   - stop virtualmachine id=XXX
   - reset sshkeyforvirtualmachine keypair=newkey id=XXX
   - start virtualmachine id=XXX
- after it came back I checked /root/.ssh/authorized_keys and .. lo & behold, 
the new key was there.

So, answer: yes, that's how to reset the SSH key and no, you can't remove it 
that way - it will check if the provided key name actually exists and you must 
provide one.

  ☕ :-)


--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: "Francois Gaudreault" 
To: "Nux!" 
Cc: dev@cloudstack.apache.org
Sent: Monday, 16 February, 2015 22:22:34
Subject: Re: Remove/Change SSH Keys from instance
Yes what? That API call is not clear at all.

I guess a "try it yourself" is what you meant?

Thanks for your great help.

FG
On Feb 16, 2015 5:04 PM, "Nux!"  wrote:


yes?


https://cloudstack.apache.org/docs/api/apidocs-4.4/user/resetSSHKeyForVirtualMachine.html

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: "Francois Gaudreault" 
To: dev@cloudstack.apache.org
Sent: Monday, 16 February, 2015 20:03:59
Subject: Remove/Change SSH Keys from instance
Hi,

Quick question, using the API, how would a user remove an associated SSH
keys to an instance? What call should we use? Seems that you can't based
on the existing API calls.

Furthermore, how can we change the SSH Key? Is it the
resetSSHKeyForVirtualMachine call?

Thanks.

--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_





--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_

Re: Remove/Change SSH Keys from instance

[Nux!]

"resetSSHKeyForVirtualMachine
Resets the SSH Key for virtual machine. The virtual machine must be in a 
"Stopped" state. [async]"

Apologies for causing you extra stress. We all had a long Monday.

I also never used this function before and here's how I tested.

- empty or move /root/.ssh/authorized_keys on the VM prior to the operation
- in cloudmonkey
  - stop virtualmachine id=XXX
  - reset sshkeyforvirtualmachine keypair=newkey id=XXX
  - start virtualmachine id=XXX
- after it came back I checked /root/.ssh/authorized_keys and .. lo & behold, 
the new key was there.

So, answer: yes, that's how to reset the SSH key and no, you can't remove it 
that way - it will check if the provided key name actually exists and you must 
provide one.

 ☕ :-)


--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Francois Gaudreault" 
> To: "Nux!" 
> Cc: dev@cloudstack.apache.org
> Sent: Monday, 16 February, 2015 22:22:34
> Subject: Re: Remove/Change SSH Keys from instance

> Yes what? That API call is not clear at all.
> 
> I guess a "try it yourself" is what you meant?
> 
> Thanks for your great help.
> 
> FG
> On Feb 16, 2015 5:04 PM, "Nux!"  wrote:
> 
>> yes?
>>
>>
>> https://cloudstack.apache.org/docs/api/apidocs-4.4/user/resetSSHKeyForVirtualMachine.html
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> - Original Message -
>> > From: "Francois Gaudreault" 
>> > To: dev@cloudstack.apache.org
>> > Sent: Monday, 16 February, 2015 20:03:59
>> > Subject: Remove/Change SSH Keys from instance
>>
>> > Hi,
>> >
>> > Quick question, using the API, how would a user remove an associated SSH
>> > keys to an instance? What call should we use? Seems that you can't based
>> > on the existing API calls.
>> >
>> > Furthermore, how can we change the SSH Key? Is it the
>> > resetSSHKeyForVirtualMachine call?
>> >
>> > Thanks.
>> >
>> > --
>> > Francois Gaudreault
>> > Gestionnaire de Produit | Product Manager - Cloud Platform & Services
>> > t:514-629-6775
>> >
>> > CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
>> > 420 rue Guy | Montreal | Quebec | H3J 1S6
>> > w: cloudops.com | tw: @CloudOps_

Re: Remove/Change SSH Keys from instance

[Sebastien Goasguen]

I think nux put a ? In there, which means he is not sure ...

So his yes is a maybe , maybe ?

-Sebastien

> On 16 Feb 2015, at 23:22, Francois Gaudreault  
> wrote:
> 
> Yes what? That API call is not clear at all.
> 
> I guess a "try it yourself" is what you meant?
> 
> Thanks for your great help.
> 
> FG
>> On Feb 16, 2015 5:04 PM, "Nux!"  wrote:
>> 
>> yes?
>> 
>> 
>> https://cloudstack.apache.org/docs/api/apidocs-4.4/user/resetSSHKeyForVirtualMachine.html
>> 
>> --
>> Sent from the Delta quadrant using Borg technology!
>> 
>> Nux!
>> www.nux.ro
>> 
>> - Original Message -
>>> From: "Francois Gaudreault" 
>>> To: dev@cloudstack.apache.org
>>> Sent: Monday, 16 February, 2015 20:03:59
>>> Subject: Remove/Change SSH Keys from instance
>> 
>>> Hi,
>>> 
>>> Quick question, using the API, how would a user remove an associated SSH
>>> keys to an instance? What call should we use? Seems that you can't based
>>> on the existing API calls.
>>> 
>>> Furthermore, how can we change the SSH Key? Is it the
>>> resetSSHKeyForVirtualMachine call?
>>> 
>>> Thanks.
>>> 
>>> --
>>> Francois Gaudreault
>>> Gestionnaire de Produit | Product Manager - Cloud Platform & Services
>>> t:514-629-6775
>>> 
>>> CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
>>> 420 rue Guy | Montreal | Quebec | H3J 1S6
>>> w: cloudops.com | tw: @CloudOps_
>> 

Re: Remove/Change SSH Keys from instance

[Francois Gaudreault]

Look...I apologize for the sarcasm. I was a little bit irritated. I just 
wanted to get some help.


I read the API documentation, I read the User guide. I just wanted to 
same some time and ask on this list.


Thanks.

FG

On 2015-02-16 5:04 PM, Nux! wrote:

yes?

https://cloudstack.apache.org/docs/api/apidocs-4.4/user/resetSSHKeyForVirtualMachine.html

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: "Francois Gaudreault" 
To: dev@cloudstack.apache.org
Sent: Monday, 16 February, 2015 20:03:59
Subject: Remove/Change SSH Keys from instance
Hi,

Quick question, using the API, how would a user remove an associated SSH
keys to an instance? What call should we use? Seems that you can't based
on the existing API calls.

Furthermore, how can we change the SSH Key? Is it the
resetSSHKeyForVirtualMachine call?

Thanks.

--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_



--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_

Re: Remove/Change SSH Keys from instance

[Francois Gaudreault]

Yes what? That API call is not clear at all.

I guess a "try it yourself" is what you meant?

Thanks for your great help.

FG
On Feb 16, 2015 5:04 PM, "Nux!"  wrote:

> yes?
>
>
> https://cloudstack.apache.org/docs/api/apidocs-4.4/user/resetSSHKeyForVirtualMachine.html
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Francois Gaudreault" 
> > To: dev@cloudstack.apache.org
> > Sent: Monday, 16 February, 2015 20:03:59
> > Subject: Remove/Change SSH Keys from instance
>
> > Hi,
> >
> > Quick question, using the API, how would a user remove an associated SSH
> > keys to an instance? What call should we use? Seems that you can't based
> > on the existing API calls.
> >
> > Furthermore, how can we change the SSH Key? Is it the
> > resetSSHKeyForVirtualMachine call?
> >
> > Thanks.
> >
> > --
> > Francois Gaudreault
> > Gestionnaire de Produit | Product Manager - Cloud Platform & Services
> > t:514-629-6775
> >
> > CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
> > 420 rue Guy | Montreal | Quebec | H3J 1S6
> > w: cloudops.com | tw: @CloudOps_
>

Re: Remove/Change SSH Keys from instance

[Nux!]

yes?

https://cloudstack.apache.org/docs/api/apidocs-4.4/user/resetSSHKeyForVirtualMachine.html

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Francois Gaudreault" 
> To: dev@cloudstack.apache.org
> Sent: Monday, 16 February, 2015 20:03:59
> Subject: Remove/Change SSH Keys from instance

> Hi,
> 
> Quick question, using the API, how would a user remove an associated SSH
> keys to an instance? What call should we use? Seems that you can't based
> on the existing API calls.
> 
> Furthermore, how can we change the SSH Key? Is it the
> resetSSHKeyForVirtualMachine call?
> 
> Thanks.
> 
> --
> Francois Gaudreault
> Gestionnaire de Produit | Product Manager - Cloud Platform & Services
> t:514-629-6775
> 
> CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
> 420 rue Guy | Montreal | Quebec | H3J 1S6
> w: cloudops.com | tw: @CloudOps_

Re: Your thoughts on using Primary Storage for keeping snapshots

[Mike Tutkowski]

Whatever way you think makes the most sense.

Either way, I'm working on this for XenServer and ESXi (eventually on KVM,
I expect) for managed storage (SolidFire is an example of managed storage).

On Mon, Feb 16, 2015 at 2:38 PM, Andrei Mikhailovsky 
wrote:

> I am happy to see the discussion is taking its pace and a lot of people
> tend to agree that we should address this area. I have done the ticket for
> that, but I am not sure if this should be dealt in a more general way as
> suggested. Or perhaps having individual tickets for each hypervisor would
> achieve a faster response from the community?
>
> Andrei
>
> - Original Message -
>
> > From: "Mike Tutkowski" 
> > To: dev@cloudstack.apache.org
> > Sent: Monday, 16 February, 2015 9:17:26 PM
> > Subject: Re: Your thoughts on using Primary Storage for keeping
> > snapshots
>
> > Well...count me in on the general-purpose part (I'm already working
> > on that
> > and have much of it working).
>
> > If someone is interested in implementing the RBD part, he/she can
> > sync with
> > me and see if there is any overlapping work that I've already
> > implementing
> > from a general-purpose standpoint.
>
> > On Mon, Feb 16, 2015 at 1:39 PM, Ian Rae  wrote:
>
> > > Agree with Logan. As fans of Ceph as well as SolidFire, we are
> > > interested
> > > in seeing this particular use case (RBD/KVM) being well
> > > implemented,
> > > however the concept of volume snapshots residing only on primary
> > > storage vs
> > > being transferred to secondary storage is a more generally useful
> > > one that
> > > is worth solving with the same terminology and interfaces, even if
> > > the
> > > mechanisms may be specific to the storage type and hypervisor.
> > >
> > > It its not practical then its not practical, but seems like it
> > > would be
> > > worth trying.
> > >
> > > On Mon, Feb 16, 2015 at 1:02 PM, Logan Barfield
> > > 
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > I agree it is a general CloudStack issue that can be addressed
> > > > across
> > > > multiple primary storage options. It's a two stage issue since
> > > > some
> > > > changes will need to be implemented to support these features
> > > > across
> > > > the board, and others will need to be made to each storage
> > > > option.
> > > >
> > > > It would be nice to see a single issue opened to cover this
> > > > across all
> > > > available storage options. Maybe have a community vote on what
> > > > support they want to see, and not consider the feature complete
> > > > until
> > > > all of the desired options are implemented? That would slow down
> > > > development for sure, but it would ensure that it was supported
> > > > where
> > > > it needs to be.
> > > >
> > > > Thank You,
> > > >
> > > > Logan Barfield
> > > > Tranquil Hosting
> > > >
> > > >
> > > > On Mon, Feb 16, 2015 at 12:42 PM, Mike Tutkowski
> > > >  wrote:
> > > > > For example, Punith from CloudByte sent out an e-mail yesterday
> > > > > that
> > > was
> > > > > very similar to this thread, but he was wondering how to
> > > > > implement
> > > such a
> > > > > concept on his company's SAN technology.
> > > > >
> > > > > On Mon, Feb 16, 2015 at 10:40 AM, Mike Tutkowski <
> > > > > mike.tutkow...@solidfire.com> wrote:
> > > > >
> > > > >> Yeah, I think it's a similar concept, though.
> > > > >>
> > > > >> You would want to take snapshots on Ceph (or some other
> > > > >> backend system
> > > > >> that acts as primary storage) instead of copying data to
> > > > >> secondary
> > > > storage
> > > > >> and calling it a snapshot.
> > > > >>
> > > > >> For Ceph or any other backend system like that, the idea is to
> > > > >> speed
> > > up
> > > > >> snapshots by not requiring CPU cycles on the front end or
> > > > >> network
> > > > bandwidth
> > > > >> to transfer the data.
> > > > >>
> > > > >> In that sense, this is a general-purpose CloudStack problem
> > > > >> and it
> > > > appears
> > > > >> you are intending on discussing only the Ceph implementation
> > > > >> here,
> > > > which is
> > > > >> fine.
> > > > >>
> > > > >> On Mon, Feb 16, 2015 at 10:34 AM, Logan Barfield <
> > > > lbarfi...@tqhosting.com>
> > > > >> wrote:
> > > > >>
> > > > >>> Hi Mike,
> > > > >>>
> > > > >>> I think the interest in this issue is primarily for Ceph RBD,
> > > > >>> which
> > > > >>> doesn't use iSCSI or SAN concepts in general. As well I
> > > > >>> believe RBD
> > > > >>> is only currently supported in KVM (and VMware?). QEMU has
> > > > >>> native
> > > RBD
> > > > >>> support, so it attaches the devices directly to the VMs in
> > > > >>> question.
> > > > >>> It also natively supports snapshotting, which is what this
> > > > >>> discussion
> > > > >>> is about.
> > > > >>>
> > > > >>> Thank You,
> > > > >>>
> > > > >>> Logan Barfield
> > > > >>> Tranquil Hosting
> > > > >>>
> > > > >>>
> > > > >>> On Mon, Feb 16, 2015 at 11:46 AM, Mike Tutkowski
> > > > >>>  wrote:
> > > > >>> > I should have also commented on KVM (since that was t

Re: Your thoughts on using Primary Storage for keeping snapshots

[Andrei Mikhailovsky]

I am happy to see the discussion is taking its pace and a lot of people tend to 
agree that we should address this area. I have done the ticket for that, but I 
am not sure if this should be dealt in a more general way as suggested. Or 
perhaps having individual tickets for each hypervisor would achieve a faster 
response from the community? 

Andrei 

- Original Message -

> From: "Mike Tutkowski" 
> To: dev@cloudstack.apache.org
> Sent: Monday, 16 February, 2015 9:17:26 PM
> Subject: Re: Your thoughts on using Primary Storage for keeping
> snapshots

> Well...count me in on the general-purpose part (I'm already working
> on that
> and have much of it working).

> If someone is interested in implementing the RBD part, he/she can
> sync with
> me and see if there is any overlapping work that I've already
> implementing
> from a general-purpose standpoint.

> On Mon, Feb 16, 2015 at 1:39 PM, Ian Rae  wrote:

> > Agree with Logan. As fans of Ceph as well as SolidFire, we are
> > interested
> > in seeing this particular use case (RBD/KVM) being well
> > implemented,
> > however the concept of volume snapshots residing only on primary
> > storage vs
> > being transferred to secondary storage is a more generally useful
> > one that
> > is worth solving with the same terminology and interfaces, even if
> > the
> > mechanisms may be specific to the storage type and hypervisor.
> >
> > It its not practical then its not practical, but seems like it
> > would be
> > worth trying.
> >
> > On Mon, Feb 16, 2015 at 1:02 PM, Logan Barfield
> > 
> > wrote:
> >
> > > Hi Mike,
> > >
> > > I agree it is a general CloudStack issue that can be addressed
> > > across
> > > multiple primary storage options. It's a two stage issue since
> > > some
> > > changes will need to be implemented to support these features
> > > across
> > > the board, and others will need to be made to each storage
> > > option.
> > >
> > > It would be nice to see a single issue opened to cover this
> > > across all
> > > available storage options. Maybe have a community vote on what
> > > support they want to see, and not consider the feature complete
> > > until
> > > all of the desired options are implemented? That would slow down
> > > development for sure, but it would ensure that it was supported
> > > where
> > > it needs to be.
> > >
> > > Thank You,
> > >
> > > Logan Barfield
> > > Tranquil Hosting
> > >
> > >
> > > On Mon, Feb 16, 2015 at 12:42 PM, Mike Tutkowski
> > >  wrote:
> > > > For example, Punith from CloudByte sent out an e-mail yesterday
> > > > that
> > was
> > > > very similar to this thread, but he was wondering how to
> > > > implement
> > such a
> > > > concept on his company's SAN technology.
> > > >
> > > > On Mon, Feb 16, 2015 at 10:40 AM, Mike Tutkowski <
> > > > mike.tutkow...@solidfire.com> wrote:
> > > >
> > > >> Yeah, I think it's a similar concept, though.
> > > >>
> > > >> You would want to take snapshots on Ceph (or some other
> > > >> backend system
> > > >> that acts as primary storage) instead of copying data to
> > > >> secondary
> > > storage
> > > >> and calling it a snapshot.
> > > >>
> > > >> For Ceph or any other backend system like that, the idea is to
> > > >> speed
> > up
> > > >> snapshots by not requiring CPU cycles on the front end or
> > > >> network
> > > bandwidth
> > > >> to transfer the data.
> > > >>
> > > >> In that sense, this is a general-purpose CloudStack problem
> > > >> and it
> > > appears
> > > >> you are intending on discussing only the Ceph implementation
> > > >> here,
> > > which is
> > > >> fine.
> > > >>
> > > >> On Mon, Feb 16, 2015 at 10:34 AM, Logan Barfield <
> > > lbarfi...@tqhosting.com>
> > > >> wrote:
> > > >>
> > > >>> Hi Mike,
> > > >>>
> > > >>> I think the interest in this issue is primarily for Ceph RBD,
> > > >>> which
> > > >>> doesn't use iSCSI or SAN concepts in general. As well I
> > > >>> believe RBD
> > > >>> is only currently supported in KVM (and VMware?). QEMU has
> > > >>> native
> > RBD
> > > >>> support, so it attaches the devices directly to the VMs in
> > > >>> question.
> > > >>> It also natively supports snapshotting, which is what this
> > > >>> discussion
> > > >>> is about.
> > > >>>
> > > >>> Thank You,
> > > >>>
> > > >>> Logan Barfield
> > > >>> Tranquil Hosting
> > > >>>
> > > >>>
> > > >>> On Mon, Feb 16, 2015 at 11:46 AM, Mike Tutkowski
> > > >>>  wrote:
> > > >>> > I should have also commented on KVM (since that was the
> > > >>> > hypervisor
> > > >>> called
> > > >>> > out in the initial e-mail).
> > > >>> >
> > > >>> > In my situation, most of my customers use XenServer and/or
> > > >>> > ESXi, so
> > > KVM
> > > >>> has
> > > >>> > received the fewest of my cycles with regards to those
> > > >>> > three
> > > >>> hypervisors.
> > > >>> >
> > > >>> > KVM, though, is actually the simplest hypervisor for which
> > > >>> > to
> > > implement
> > > >>> > these changes (since I am using the iSCSI adapter of the
> > > >>> > KVM agent
> > >

Re: Your thoughts on using Primary Storage for keeping snapshots

[Mike Tutkowski]

Well...count me in on the general-purpose part (I'm already working on that
and have much of it working).

If someone is interested in implementing the RBD part, he/she can sync with
me and see if there is any overlapping work that I've already implementing
from a general-purpose standpoint.

On Mon, Feb 16, 2015 at 1:39 PM, Ian Rae  wrote:

> Agree with Logan. As fans of Ceph as well as SolidFire, we are interested
> in seeing this particular use case (RBD/KVM) being well implemented,
> however the concept of volume snapshots residing only on primary storage vs
> being transferred to secondary storage is a more generally useful one that
> is worth solving with the same terminology and interfaces, even if the
> mechanisms may be specific to the storage type and hypervisor.
>
> It its not practical then its not practical, but seems like it would be
> worth trying.
>
> On Mon, Feb 16, 2015 at 1:02 PM, Logan Barfield 
> wrote:
>
> > Hi Mike,
> >
> > I agree it is a general CloudStack issue that can be addressed across
> > multiple primary storage options.  It's a two stage issue since some
> > changes will need to be implemented to support these features across
> > the board, and others will need to be made to each storage option.
> >
> > It would be nice to see a single issue opened to cover this across all
> > available storage options.  Maybe have a community vote on what
> > support they want to see, and not consider the feature complete until
> > all of the desired options are implemented?  That would slow down
> > development for sure, but it would ensure that it was supported where
> > it needs to be.
> >
> > Thank You,
> >
> > Logan Barfield
> > Tranquil Hosting
> >
> >
> > On Mon, Feb 16, 2015 at 12:42 PM, Mike Tutkowski
> >  wrote:
> > > For example, Punith from CloudByte sent out an e-mail yesterday that
> was
> > > very similar to this thread, but he was wondering how to implement
> such a
> > > concept on his company's SAN technology.
> > >
> > > On Mon, Feb 16, 2015 at 10:40 AM, Mike Tutkowski <
> > > mike.tutkow...@solidfire.com> wrote:
> > >
> > >> Yeah, I think it's a similar concept, though.
> > >>
> > >> You would want to take snapshots on Ceph (or some other backend system
> > >> that acts as primary storage) instead of copying data to secondary
> > storage
> > >> and calling it a snapshot.
> > >>
> > >> For Ceph or any other backend system like that, the idea is to speed
> up
> > >> snapshots by not requiring CPU cycles on the front end or network
> > bandwidth
> > >> to transfer the data.
> > >>
> > >> In that sense, this is a general-purpose CloudStack problem and it
> > appears
> > >> you are intending on discussing only the Ceph implementation here,
> > which is
> > >> fine.
> > >>
> > >> On Mon, Feb 16, 2015 at 10:34 AM, Logan Barfield <
> > lbarfi...@tqhosting.com>
> > >> wrote:
> > >>
> > >>> Hi Mike,
> > >>>
> > >>> I think the interest in this issue is primarily for Ceph RBD, which
> > >>> doesn't use iSCSI or SAN concepts in general.  As well I believe RBD
> > >>> is only currently supported in KVM (and VMware?).  QEMU has native
> RBD
> > >>> support, so it attaches the devices directly to the VMs in question.
> > >>> It also natively supports snapshotting, which is what this discussion
> > >>> is about.
> > >>>
> > >>> Thank You,
> > >>>
> > >>> Logan Barfield
> > >>> Tranquil Hosting
> > >>>
> > >>>
> > >>> On Mon, Feb 16, 2015 at 11:46 AM, Mike Tutkowski
> > >>>  wrote:
> > >>> > I should have also commented on KVM (since that was the hypervisor
> > >>> called
> > >>> > out in the initial e-mail).
> > >>> >
> > >>> > In my situation, most of my customers use XenServer and/or ESXi, so
> > KVM
> > >>> has
> > >>> > received the fewest of my cycles with regards to those three
> > >>> hypervisors.
> > >>> >
> > >>> > KVM, though, is actually the simplest hypervisor for which to
> > implement
> > >>> > these changes (since I am using the iSCSI adapter of the KVM agent
> > and
> > >>> it
> > >>> > just essentially passes my LUN to the VM in question).
> > >>> >
> > >>> > For KVM, there is no clustered file system applied to my backend
> LUN,
> > >>> so I
> > >>> > don't have to "worry" about that layer.
> > >>> >
> > >>> > I don't see any hurdles like *immutable* UUIDs of SRs and VDIs
> (such
> > is
> > >>> the
> > >>> > case with XenServer) or having to re-signature anything (such is
> the
> > >>> case
> > >>> > with ESXi).
> > >>> >
> > >>> > On Mon, Feb 16, 2015 at 9:33 AM, Mike Tutkowski <
> > >>> > mike.tutkow...@solidfire.com> wrote:
> > >>> >
> > >>> >> I have been working on this on and off for a while now (as time
> > >>> permits).
> > >>> >>
> > >>> >> Here is an e-mail I sent to a customer of ours that helps describe
> > >>> some of
> > >>> >> the issues:
> > >>> >>
> > >>> >> *** Beginning of e-mail ***
> > >>> >>
> > >>> >> The main requests were around the following features:
> > >>> >>
> > >>> >> * The ability to leverage SolidFire snapshots.
> > >>> >>
> > >>> >> *

Re: Your thoughts on using Primary Storage for keeping snapshots

[Ian Rae]

Agree with Logan. As fans of Ceph as well as SolidFire, we are interested
in seeing this particular use case (RBD/KVM) being well implemented,
however the concept of volume snapshots residing only on primary storage vs
being transferred to secondary storage is a more generally useful one that
is worth solving with the same terminology and interfaces, even if the
mechanisms may be specific to the storage type and hypervisor.

It its not practical then its not practical, but seems like it would be
worth trying.

On Mon, Feb 16, 2015 at 1:02 PM, Logan Barfield 
wrote:

> Hi Mike,
>
> I agree it is a general CloudStack issue that can be addressed across
> multiple primary storage options.  It's a two stage issue since some
> changes will need to be implemented to support these features across
> the board, and others will need to be made to each storage option.
>
> It would be nice to see a single issue opened to cover this across all
> available storage options.  Maybe have a community vote on what
> support they want to see, and not consider the feature complete until
> all of the desired options are implemented?  That would slow down
> development for sure, but it would ensure that it was supported where
> it needs to be.
>
> Thank You,
>
> Logan Barfield
> Tranquil Hosting
>
>
> On Mon, Feb 16, 2015 at 12:42 PM, Mike Tutkowski
>  wrote:
> > For example, Punith from CloudByte sent out an e-mail yesterday that was
> > very similar to this thread, but he was wondering how to implement such a
> > concept on his company's SAN technology.
> >
> > On Mon, Feb 16, 2015 at 10:40 AM, Mike Tutkowski <
> > mike.tutkow...@solidfire.com> wrote:
> >
> >> Yeah, I think it's a similar concept, though.
> >>
> >> You would want to take snapshots on Ceph (or some other backend system
> >> that acts as primary storage) instead of copying data to secondary
> storage
> >> and calling it a snapshot.
> >>
> >> For Ceph or any other backend system like that, the idea is to speed up
> >> snapshots by not requiring CPU cycles on the front end or network
> bandwidth
> >> to transfer the data.
> >>
> >> In that sense, this is a general-purpose CloudStack problem and it
> appears
> >> you are intending on discussing only the Ceph implementation here,
> which is
> >> fine.
> >>
> >> On Mon, Feb 16, 2015 at 10:34 AM, Logan Barfield <
> lbarfi...@tqhosting.com>
> >> wrote:
> >>
> >>> Hi Mike,
> >>>
> >>> I think the interest in this issue is primarily for Ceph RBD, which
> >>> doesn't use iSCSI or SAN concepts in general.  As well I believe RBD
> >>> is only currently supported in KVM (and VMware?).  QEMU has native RBD
> >>> support, so it attaches the devices directly to the VMs in question.
> >>> It also natively supports snapshotting, which is what this discussion
> >>> is about.
> >>>
> >>> Thank You,
> >>>
> >>> Logan Barfield
> >>> Tranquil Hosting
> >>>
> >>>
> >>> On Mon, Feb 16, 2015 at 11:46 AM, Mike Tutkowski
> >>>  wrote:
> >>> > I should have also commented on KVM (since that was the hypervisor
> >>> called
> >>> > out in the initial e-mail).
> >>> >
> >>> > In my situation, most of my customers use XenServer and/or ESXi, so
> KVM
> >>> has
> >>> > received the fewest of my cycles with regards to those three
> >>> hypervisors.
> >>> >
> >>> > KVM, though, is actually the simplest hypervisor for which to
> implement
> >>> > these changes (since I am using the iSCSI adapter of the KVM agent
> and
> >>> it
> >>> > just essentially passes my LUN to the VM in question).
> >>> >
> >>> > For KVM, there is no clustered file system applied to my backend LUN,
> >>> so I
> >>> > don't have to "worry" about that layer.
> >>> >
> >>> > I don't see any hurdles like *immutable* UUIDs of SRs and VDIs (such
> is
> >>> the
> >>> > case with XenServer) or having to re-signature anything (such is the
> >>> case
> >>> > with ESXi).
> >>> >
> >>> > On Mon, Feb 16, 2015 at 9:33 AM, Mike Tutkowski <
> >>> > mike.tutkow...@solidfire.com> wrote:
> >>> >
> >>> >> I have been working on this on and off for a while now (as time
> >>> permits).
> >>> >>
> >>> >> Here is an e-mail I sent to a customer of ours that helps describe
> >>> some of
> >>> >> the issues:
> >>> >>
> >>> >> *** Beginning of e-mail ***
> >>> >>
> >>> >> The main requests were around the following features:
> >>> >>
> >>> >> * The ability to leverage SolidFire snapshots.
> >>> >>
> >>> >> * The ability to create CloudStack templates from SolidFire
> snapshots.
> >>> >>
> >>> >> I had these on my roadmap, but bumped the priority up and began
> work on
> >>> >> them for the CS 4.6 release.
> >>> >>
> >>> >> During design, I realized there were issues with the way XenServer
> is
> >>> >> architected that prevented me from directly using SolidFire
> snapshots.
> >>> >>
> >>> >> I could definitely take a SolidFire snapshot of a SolidFire volume,
> but
> >>> >> this snapshot would not be usable from XenServer if the original
> >>> volume was
> >>> >> still in use.
> >>> >>
> >>> >> Here is t

Remove/Change SSH Keys from instance

[Francois Gaudreault]

Hi,

Quick question, using the API, how would a user remove an associated SSH 
keys to an instance? What call should we use? Seems that you can't based 
on the existing API calls.


Furthermore, how can we change the SSH Key? Is it the 
resetSSHKeyForVirtualMachine call?


Thanks.

--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_

Re: Your thoughts on using Primary Storage for keeping snapshots

[Logan Barfield]

Hi Mike,

I agree it is a general CloudStack issue that can be addressed across
multiple primary storage options.  It's a two stage issue since some
changes will need to be implemented to support these features across
the board, and others will need to be made to each storage option.

It would be nice to see a single issue opened to cover this across all
available storage options.  Maybe have a community vote on what
support they want to see, and not consider the feature complete until
all of the desired options are implemented?  That would slow down
development for sure, but it would ensure that it was supported where
it needs to be.

Thank You,

Logan Barfield
Tranquil Hosting


On Mon, Feb 16, 2015 at 12:42 PM, Mike Tutkowski
 wrote:
> For example, Punith from CloudByte sent out an e-mail yesterday that was
> very similar to this thread, but he was wondering how to implement such a
> concept on his company's SAN technology.
>
> On Mon, Feb 16, 2015 at 10:40 AM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> Yeah, I think it's a similar concept, though.
>>
>> You would want to take snapshots on Ceph (or some other backend system
>> that acts as primary storage) instead of copying data to secondary storage
>> and calling it a snapshot.
>>
>> For Ceph or any other backend system like that, the idea is to speed up
>> snapshots by not requiring CPU cycles on the front end or network bandwidth
>> to transfer the data.
>>
>> In that sense, this is a general-purpose CloudStack problem and it appears
>> you are intending on discussing only the Ceph implementation here, which is
>> fine.
>>
>> On Mon, Feb 16, 2015 at 10:34 AM, Logan Barfield 
>> wrote:
>>
>>> Hi Mike,
>>>
>>> I think the interest in this issue is primarily for Ceph RBD, which
>>> doesn't use iSCSI or SAN concepts in general.  As well I believe RBD
>>> is only currently supported in KVM (and VMware?).  QEMU has native RBD
>>> support, so it attaches the devices directly to the VMs in question.
>>> It also natively supports snapshotting, which is what this discussion
>>> is about.
>>>
>>> Thank You,
>>>
>>> Logan Barfield
>>> Tranquil Hosting
>>>
>>>
>>> On Mon, Feb 16, 2015 at 11:46 AM, Mike Tutkowski
>>>  wrote:
>>> > I should have also commented on KVM (since that was the hypervisor
>>> called
>>> > out in the initial e-mail).
>>> >
>>> > In my situation, most of my customers use XenServer and/or ESXi, so KVM
>>> has
>>> > received the fewest of my cycles with regards to those three
>>> hypervisors.
>>> >
>>> > KVM, though, is actually the simplest hypervisor for which to implement
>>> > these changes (since I am using the iSCSI adapter of the KVM agent and
>>> it
>>> > just essentially passes my LUN to the VM in question).
>>> >
>>> > For KVM, there is no clustered file system applied to my backend LUN,
>>> so I
>>> > don't have to "worry" about that layer.
>>> >
>>> > I don't see any hurdles like *immutable* UUIDs of SRs and VDIs (such is
>>> the
>>> > case with XenServer) or having to re-signature anything (such is the
>>> case
>>> > with ESXi).
>>> >
>>> > On Mon, Feb 16, 2015 at 9:33 AM, Mike Tutkowski <
>>> > mike.tutkow...@solidfire.com> wrote:
>>> >
>>> >> I have been working on this on and off for a while now (as time
>>> permits).
>>> >>
>>> >> Here is an e-mail I sent to a customer of ours that helps describe
>>> some of
>>> >> the issues:
>>> >>
>>> >> *** Beginning of e-mail ***
>>> >>
>>> >> The main requests were around the following features:
>>> >>
>>> >> * The ability to leverage SolidFire snapshots.
>>> >>
>>> >> * The ability to create CloudStack templates from SolidFire snapshots.
>>> >>
>>> >> I had these on my roadmap, but bumped the priority up and began work on
>>> >> them for the CS 4.6 release.
>>> >>
>>> >> During design, I realized there were issues with the way XenServer is
>>> >> architected that prevented me from directly using SolidFire snapshots.
>>> >>
>>> >> I could definitely take a SolidFire snapshot of a SolidFire volume, but
>>> >> this snapshot would not be usable from XenServer if the original
>>> volume was
>>> >> still in use.
>>> >>
>>> >> Here is the gist of the problem:
>>> >>
>>> >> When XenServer leverages an iSCSI target such as a SolidFire volume, it
>>> >> applies a clustered files system to it, which they call a storage
>>> >> repository (SR). An SR has an *immutable* UUID associated with it.
>>> >>
>>> >> The virtual volume (which a VM sees as a disk) is represented by a
>>> virtual
>>> >> disk image (VDI) in the SR. A VDI also has an *immutable* UUID
>>> associated
>>> >> with it.
>>> >>
>>> >> If I take a snapshot (or a clone) of the SolidFire volume and then
>>> later
>>> >> try to use that snapshot from XenServer, XenServer complains that the
>>> SR on
>>> >> the snapshot has a UUID that conflicts with an existing UUID.
>>> >>
>>> >> In other words, it is not possible to use the original SR and the
>>> snapshot
>>> >> of this SR from XenServer at the same time, which is crit

Re: Your thoughts on using Primary Storage for keeping snapshots

[Mike Tutkowski]

Hey Ian,

Since it looks like the intent of this particular thread is just to discuss
RBD and snapshots (which I don't think your business uses), you would be
more interested in the "Query on snapshot and cloning for managed storage"
thread as that one talks about this issue at a more general level.

Talk to you later!
Mike

On Mon, Feb 16, 2015 at 10:42 AM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> For example, Punith from CloudByte sent out an e-mail yesterday that was
> very similar to this thread, but he was wondering how to implement such a
> concept on his company's SAN technology.
>
> On Mon, Feb 16, 2015 at 10:40 AM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> Yeah, I think it's a similar concept, though.
>>
>> You would want to take snapshots on Ceph (or some other backend system
>> that acts as primary storage) instead of copying data to secondary storage
>> and calling it a snapshot.
>>
>> For Ceph or any other backend system like that, the idea is to speed up
>> snapshots by not requiring CPU cycles on the front end or network bandwidth
>> to transfer the data.
>>
>> In that sense, this is a general-purpose CloudStack problem and it
>> appears you are intending on discussing only the Ceph implementation here,
>> which is fine.
>>
>> On Mon, Feb 16, 2015 at 10:34 AM, Logan Barfield > > wrote:
>>
>>> Hi Mike,
>>>
>>> I think the interest in this issue is primarily for Ceph RBD, which
>>> doesn't use iSCSI or SAN concepts in general.  As well I believe RBD
>>> is only currently supported in KVM (and VMware?).  QEMU has native RBD
>>> support, so it attaches the devices directly to the VMs in question.
>>> It also natively supports snapshotting, which is what this discussion
>>> is about.
>>>
>>> Thank You,
>>>
>>> Logan Barfield
>>> Tranquil Hosting
>>>
>>>
>>> On Mon, Feb 16, 2015 at 11:46 AM, Mike Tutkowski
>>>  wrote:
>>> > I should have also commented on KVM (since that was the hypervisor
>>> called
>>> > out in the initial e-mail).
>>> >
>>> > In my situation, most of my customers use XenServer and/or ESXi, so
>>> KVM has
>>> > received the fewest of my cycles with regards to those three
>>> hypervisors.
>>> >
>>> > KVM, though, is actually the simplest hypervisor for which to implement
>>> > these changes (since I am using the iSCSI adapter of the KVM agent and
>>> it
>>> > just essentially passes my LUN to the VM in question).
>>> >
>>> > For KVM, there is no clustered file system applied to my backend LUN,
>>> so I
>>> > don't have to "worry" about that layer.
>>> >
>>> > I don't see any hurdles like *immutable* UUIDs of SRs and VDIs (such
>>> is the
>>> > case with XenServer) or having to re-signature anything (such is the
>>> case
>>> > with ESXi).
>>> >
>>> > On Mon, Feb 16, 2015 at 9:33 AM, Mike Tutkowski <
>>> > mike.tutkow...@solidfire.com> wrote:
>>> >
>>> >> I have been working on this on and off for a while now (as time
>>> permits).
>>> >>
>>> >> Here is an e-mail I sent to a customer of ours that helps describe
>>> some of
>>> >> the issues:
>>> >>
>>> >> *** Beginning of e-mail ***
>>> >>
>>> >> The main requests were around the following features:
>>> >>
>>> >> * The ability to leverage SolidFire snapshots.
>>> >>
>>> >> * The ability to create CloudStack templates from SolidFire snapshots.
>>> >>
>>> >> I had these on my roadmap, but bumped the priority up and began work
>>> on
>>> >> them for the CS 4.6 release.
>>> >>
>>> >> During design, I realized there were issues with the way XenServer is
>>> >> architected that prevented me from directly using SolidFire snapshots.
>>> >>
>>> >> I could definitely take a SolidFire snapshot of a SolidFire volume,
>>> but
>>> >> this snapshot would not be usable from XenServer if the original
>>> volume was
>>> >> still in use.
>>> >>
>>> >> Here is the gist of the problem:
>>> >>
>>> >> When XenServer leverages an iSCSI target such as a SolidFire volume,
>>> it
>>> >> applies a clustered files system to it, which they call a storage
>>> >> repository (SR). An SR has an *immutable* UUID associated with it.
>>> >>
>>> >> The virtual volume (which a VM sees as a disk) is represented by a
>>> virtual
>>> >> disk image (VDI) in the SR. A VDI also has an *immutable* UUID
>>> associated
>>> >> with it.
>>> >>
>>> >> If I take a snapshot (or a clone) of the SolidFire volume and then
>>> later
>>> >> try to use that snapshot from XenServer, XenServer complains that the
>>> SR on
>>> >> the snapshot has a UUID that conflicts with an existing UUID.
>>> >>
>>> >> In other words, it is not possible to use the original SR and the
>>> snapshot
>>> >> of this SR from XenServer at the same time, which is critical in a
>>> cloud
>>> >> environment (to enable creating templates from snapshots).
>>> >>
>>> >> The way I have proposed circumventing this issue is not ideal, but
>>> >> technically works (this code is checked into the CS 4.6 branch):
>>> >>
>>> >> When the time comes to take a CloudStack snapshot of a CloudStack

Re: Query on snapshot and cloning for managed storage

[Mike Tutkowski]

Hi Punith,

I sent this e-mail (below) in a different CloudStack dev@ thread, but it
seems they are only interested in RBD in that thread, so I will include
that e-mail text in this thread so that we keep track on the mailing list
of current issues in XenServer and ESXi with regards to leveraging backend
SAN snapshots:

*** Beginning of e-mail ***

The main requests were around the following features:

* The ability to leverage SolidFire snapshots.

* The ability to create CloudStack templates from SolidFire snapshots.

I had these on my roadmap, but bumped the priority up and began work on
them for the CS 4.6 release.

During design, I realized there were issues with the way XenServer is
architected that prevented me from directly using SolidFire snapshots.

I could definitely take a SolidFire snapshot of a SolidFire volume, but
this snapshot would not be usable from XenServer if the original volume was
still in use.

Here is the gist of the problem:

When XenServer leverages an iSCSI target such as a SolidFire volume, it
applies a clustered files system to it, which they call a storage
repository (SR). An SR has an *immutable* UUID associated with it.

The virtual volume (which a VM sees as a disk) is represented by a virtual
disk image (VDI) in the SR. A VDI also has an *immutable* UUID associated
with it.

If I take a snapshot (or a clone) of the SolidFire volume and then later
try to use that snapshot from XenServer, XenServer complains that the SR on
the snapshot has a UUID that conflicts with an existing UUID.

In other words, it is not possible to use the original SR and the snapshot
of this SR from XenServer at the same time, which is critical in a cloud
environment (to enable creating templates from snapshots).

The way I have proposed circumventing this issue is not ideal, but
technically works (this code is checked into the CS 4.6 branch):

When the time comes to take a CloudStack snapshot of a CloudStack volume
that is backed by SolidFire storage via the storage plug-in, the plug-in
will create a new SolidFire volume with characteristics (size and IOPS)
equal to those of the original volume.

We then have XenServer attach to this new SolidFire volume, create a *new*
SR on it, and then copy the VDI from the source SR to the destination SR
(the new SR).

This leads to us having a copy of the VDI (a "snapshot" of sorts), but it
requires CPU cycles on the compute cluster as well as network bandwidth to
write to the SAN (thus it is slower and more resource intensive than a
SolidFire snapshot).

I spoke with Tim Mackey (who works on XenServer at Citrix) concerning this
issue before and during the CloudStack Collaboration Conference in Budapest
in November. He agreed that this is a legitimate issue with the way
XenServer is designed and could not think of a way (other than what I was
doing) to get around it in current versions of XenServer.

One thought is to have a feature added to XenServer that enables you to
change the UUID of an SR and of a VDI.

If I could do that, then I could take a SolidFire snapshot of the SolidFire
volume and issue commands to XenServer to have it change the UUIDs of the
original SR and the original VDI. I could then recored the necessary UUID
info in the CS DB.

*** End of e-mail ***

Talk to you later,
Mike

On Sun, Feb 15, 2015 at 11:31 PM, Punith S  wrote:

> thanks mike, i'll go through the classes and i'll try to test it on a
> master(4.6) setup for the better understanding.
>
> On Mon, Feb 16, 2015 at 11:57 AM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> I think it might be easier if you examine my plug-in along with two new
>> classes:
>>
>> StorageSystemSnapshotStrategy
>> StorageSystemDataMotionStrategy
>>
>> Those last two classes are intended for managed storage in general (i.e.
>> they are not SolidFire specific).
>>
>> That code currently only handles XenServer.
>>
>> I'm currently working on equivalent support in ESXi.
>>
>>
>> On Sunday, February 15, 2015, Punith S  wrote:
>>
>>> thanks for the info mike.i'll go through your commits of 4.6.
>>>
>>> the updated plugin i pushed for 4.5 used to only take backend storage
>>> snapshot in cloudbyte. but when i'm trying to create a volume out of that
>>> snapshot it is going to motion service where it is trying to copy the vmfs
>>> or vmdk file but since there is no destination( storage volume cloned out
>>> of snapshot) it is failed to create a volume.
>>>
>>> mike can you please brief me about when we have to make use of canCopy()
>>> and copyAsync() interfaces of datastore interfaces.
>>>
>>> thanks
>>>
>>> On Mon, Feb 16, 2015 at 11:40 AM, Mike Tutkowski <
>>> mike.tutkow...@solidfire.com> wrote:
>>>
 I would recommend you take a look at a bunch of my most recent 4.6
 commits.


 On Sunday, February 15, 2015, Mike Tutkowski <
 mike.tutkow...@solidfire.com> wrote:

> I am working on creating the necessary infrastructure to support
> managed snapshots on XenSe

Re: Your thoughts on using Primary Storage for keeping snapshots

[Mike Tutkowski]

For example, Punith from CloudByte sent out an e-mail yesterday that was
very similar to this thread, but he was wondering how to implement such a
concept on his company's SAN technology.

On Mon, Feb 16, 2015 at 10:40 AM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> Yeah, I think it's a similar concept, though.
>
> You would want to take snapshots on Ceph (or some other backend system
> that acts as primary storage) instead of copying data to secondary storage
> and calling it a snapshot.
>
> For Ceph or any other backend system like that, the idea is to speed up
> snapshots by not requiring CPU cycles on the front end or network bandwidth
> to transfer the data.
>
> In that sense, this is a general-purpose CloudStack problem and it appears
> you are intending on discussing only the Ceph implementation here, which is
> fine.
>
> On Mon, Feb 16, 2015 at 10:34 AM, Logan Barfield 
> wrote:
>
>> Hi Mike,
>>
>> I think the interest in this issue is primarily for Ceph RBD, which
>> doesn't use iSCSI or SAN concepts in general.  As well I believe RBD
>> is only currently supported in KVM (and VMware?).  QEMU has native RBD
>> support, so it attaches the devices directly to the VMs in question.
>> It also natively supports snapshotting, which is what this discussion
>> is about.
>>
>> Thank You,
>>
>> Logan Barfield
>> Tranquil Hosting
>>
>>
>> On Mon, Feb 16, 2015 at 11:46 AM, Mike Tutkowski
>>  wrote:
>> > I should have also commented on KVM (since that was the hypervisor
>> called
>> > out in the initial e-mail).
>> >
>> > In my situation, most of my customers use XenServer and/or ESXi, so KVM
>> has
>> > received the fewest of my cycles with regards to those three
>> hypervisors.
>> >
>> > KVM, though, is actually the simplest hypervisor for which to implement
>> > these changes (since I am using the iSCSI adapter of the KVM agent and
>> it
>> > just essentially passes my LUN to the VM in question).
>> >
>> > For KVM, there is no clustered file system applied to my backend LUN,
>> so I
>> > don't have to "worry" about that layer.
>> >
>> > I don't see any hurdles like *immutable* UUIDs of SRs and VDIs (such is
>> the
>> > case with XenServer) or having to re-signature anything (such is the
>> case
>> > with ESXi).
>> >
>> > On Mon, Feb 16, 2015 at 9:33 AM, Mike Tutkowski <
>> > mike.tutkow...@solidfire.com> wrote:
>> >
>> >> I have been working on this on and off for a while now (as time
>> permits).
>> >>
>> >> Here is an e-mail I sent to a customer of ours that helps describe
>> some of
>> >> the issues:
>> >>
>> >> *** Beginning of e-mail ***
>> >>
>> >> The main requests were around the following features:
>> >>
>> >> * The ability to leverage SolidFire snapshots.
>> >>
>> >> * The ability to create CloudStack templates from SolidFire snapshots.
>> >>
>> >> I had these on my roadmap, but bumped the priority up and began work on
>> >> them for the CS 4.6 release.
>> >>
>> >> During design, I realized there were issues with the way XenServer is
>> >> architected that prevented me from directly using SolidFire snapshots.
>> >>
>> >> I could definitely take a SolidFire snapshot of a SolidFire volume, but
>> >> this snapshot would not be usable from XenServer if the original
>> volume was
>> >> still in use.
>> >>
>> >> Here is the gist of the problem:
>> >>
>> >> When XenServer leverages an iSCSI target such as a SolidFire volume, it
>> >> applies a clustered files system to it, which they call a storage
>> >> repository (SR). An SR has an *immutable* UUID associated with it.
>> >>
>> >> The virtual volume (which a VM sees as a disk) is represented by a
>> virtual
>> >> disk image (VDI) in the SR. A VDI also has an *immutable* UUID
>> associated
>> >> with it.
>> >>
>> >> If I take a snapshot (or a clone) of the SolidFire volume and then
>> later
>> >> try to use that snapshot from XenServer, XenServer complains that the
>> SR on
>> >> the snapshot has a UUID that conflicts with an existing UUID.
>> >>
>> >> In other words, it is not possible to use the original SR and the
>> snapshot
>> >> of this SR from XenServer at the same time, which is critical in a
>> cloud
>> >> environment (to enable creating templates from snapshots).
>> >>
>> >> The way I have proposed circumventing this issue is not ideal, but
>> >> technically works (this code is checked into the CS 4.6 branch):
>> >>
>> >> When the time comes to take a CloudStack snapshot of a CloudStack
>> volume
>> >> that is backed by SolidFire storage via the storage plug-in, the
>> plug-in
>> >> will create a new SolidFire volume with characteristics (size and IOPS)
>> >> equal to those of the original volume.
>> >>
>> >> We then have XenServer attach to this new SolidFire volume, create a
>> *new*
>> >> SR on it, and then copy the VDI from the source SR to the destination
>> SR
>> >> (the new SR).
>> >>
>> >> This leads to us having a copy of the VDI (a "snapshot" of sorts), but
>> it
>> >> requires CPU cycles on the compute cluster as well as

Re: Your thoughts on using Primary Storage for keeping snapshots

[Mike Tutkowski]

Yeah, I think it's a similar concept, though.

You would want to take snapshots on Ceph (or some other backend system that
acts as primary storage) instead of copying data to secondary storage and
calling it a snapshot.

For Ceph or any other backend system like that, the idea is to speed up
snapshots by not requiring CPU cycles on the front end or network bandwidth
to transfer the data.

In that sense, this is a general-purpose CloudStack problem and it appears
you are intending on discussing only the Ceph implementation here, which is
fine.

On Mon, Feb 16, 2015 at 10:34 AM, Logan Barfield 
wrote:

> Hi Mike,
>
> I think the interest in this issue is primarily for Ceph RBD, which
> doesn't use iSCSI or SAN concepts in general.  As well I believe RBD
> is only currently supported in KVM (and VMware?).  QEMU has native RBD
> support, so it attaches the devices directly to the VMs in question.
> It also natively supports snapshotting, which is what this discussion
> is about.
>
> Thank You,
>
> Logan Barfield
> Tranquil Hosting
>
>
> On Mon, Feb 16, 2015 at 11:46 AM, Mike Tutkowski
>  wrote:
> > I should have also commented on KVM (since that was the hypervisor called
> > out in the initial e-mail).
> >
> > In my situation, most of my customers use XenServer and/or ESXi, so KVM
> has
> > received the fewest of my cycles with regards to those three hypervisors.
> >
> > KVM, though, is actually the simplest hypervisor for which to implement
> > these changes (since I am using the iSCSI adapter of the KVM agent and it
> > just essentially passes my LUN to the VM in question).
> >
> > For KVM, there is no clustered file system applied to my backend LUN, so
> I
> > don't have to "worry" about that layer.
> >
> > I don't see any hurdles like *immutable* UUIDs of SRs and VDIs (such is
> the
> > case with XenServer) or having to re-signature anything (such is the case
> > with ESXi).
> >
> > On Mon, Feb 16, 2015 at 9:33 AM, Mike Tutkowski <
> > mike.tutkow...@solidfire.com> wrote:
> >
> >> I have been working on this on and off for a while now (as time
> permits).
> >>
> >> Here is an e-mail I sent to a customer of ours that helps describe some
> of
> >> the issues:
> >>
> >> *** Beginning of e-mail ***
> >>
> >> The main requests were around the following features:
> >>
> >> * The ability to leverage SolidFire snapshots.
> >>
> >> * The ability to create CloudStack templates from SolidFire snapshots.
> >>
> >> I had these on my roadmap, but bumped the priority up and began work on
> >> them for the CS 4.6 release.
> >>
> >> During design, I realized there were issues with the way XenServer is
> >> architected that prevented me from directly using SolidFire snapshots.
> >>
> >> I could definitely take a SolidFire snapshot of a SolidFire volume, but
> >> this snapshot would not be usable from XenServer if the original volume
> was
> >> still in use.
> >>
> >> Here is the gist of the problem:
> >>
> >> When XenServer leverages an iSCSI target such as a SolidFire volume, it
> >> applies a clustered files system to it, which they call a storage
> >> repository (SR). An SR has an *immutable* UUID associated with it.
> >>
> >> The virtual volume (which a VM sees as a disk) is represented by a
> virtual
> >> disk image (VDI) in the SR. A VDI also has an *immutable* UUID
> associated
> >> with it.
> >>
> >> If I take a snapshot (or a clone) of the SolidFire volume and then later
> >> try to use that snapshot from XenServer, XenServer complains that the
> SR on
> >> the snapshot has a UUID that conflicts with an existing UUID.
> >>
> >> In other words, it is not possible to use the original SR and the
> snapshot
> >> of this SR from XenServer at the same time, which is critical in a cloud
> >> environment (to enable creating templates from snapshots).
> >>
> >> The way I have proposed circumventing this issue is not ideal, but
> >> technically works (this code is checked into the CS 4.6 branch):
> >>
> >> When the time comes to take a CloudStack snapshot of a CloudStack volume
> >> that is backed by SolidFire storage via the storage plug-in, the plug-in
> >> will create a new SolidFire volume with characteristics (size and IOPS)
> >> equal to those of the original volume.
> >>
> >> We then have XenServer attach to this new SolidFire volume, create a
> *new*
> >> SR on it, and then copy the VDI from the source SR to the destination SR
> >> (the new SR).
> >>
> >> This leads to us having a copy of the VDI (a "snapshot" of sorts), but
> it
> >> requires CPU cycles on the compute cluster as well as network bandwidth
> to
> >> write to the SAN (thus it is slower and more resource intensive than a
> >> SolidFire snapshot).
> >>
> >> I spoke with Tim Mackey (who works on XenServer at Citrix) concerning
> this
> >> issue before and during the CloudStack Collaboration Conference in
> Budapest
> >> in November. He agreed that this is a legitimate issue with the way
> >> XenServer is designed and could not think of a way (other than w

Re: Your thoughts on using Primary Storage for keeping snapshots

[Logan Barfield]

Hi Mike,

I think the interest in this issue is primarily for Ceph RBD, which
doesn't use iSCSI or SAN concepts in general.  As well I believe RBD
is only currently supported in KVM (and VMware?).  QEMU has native RBD
support, so it attaches the devices directly to the VMs in question.
It also natively supports snapshotting, which is what this discussion
is about.

Thank You,

Logan Barfield
Tranquil Hosting


On Mon, Feb 16, 2015 at 11:46 AM, Mike Tutkowski
 wrote:
> I should have also commented on KVM (since that was the hypervisor called
> out in the initial e-mail).
>
> In my situation, most of my customers use XenServer and/or ESXi, so KVM has
> received the fewest of my cycles with regards to those three hypervisors.
>
> KVM, though, is actually the simplest hypervisor for which to implement
> these changes (since I am using the iSCSI adapter of the KVM agent and it
> just essentially passes my LUN to the VM in question).
>
> For KVM, there is no clustered file system applied to my backend LUN, so I
> don't have to "worry" about that layer.
>
> I don't see any hurdles like *immutable* UUIDs of SRs and VDIs (such is the
> case with XenServer) or having to re-signature anything (such is the case
> with ESXi).
>
> On Mon, Feb 16, 2015 at 9:33 AM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> I have been working on this on and off for a while now (as time permits).
>>
>> Here is an e-mail I sent to a customer of ours that helps describe some of
>> the issues:
>>
>> *** Beginning of e-mail ***
>>
>> The main requests were around the following features:
>>
>> * The ability to leverage SolidFire snapshots.
>>
>> * The ability to create CloudStack templates from SolidFire snapshots.
>>
>> I had these on my roadmap, but bumped the priority up and began work on
>> them for the CS 4.6 release.
>>
>> During design, I realized there were issues with the way XenServer is
>> architected that prevented me from directly using SolidFire snapshots.
>>
>> I could definitely take a SolidFire snapshot of a SolidFire volume, but
>> this snapshot would not be usable from XenServer if the original volume was
>> still in use.
>>
>> Here is the gist of the problem:
>>
>> When XenServer leverages an iSCSI target such as a SolidFire volume, it
>> applies a clustered files system to it, which they call a storage
>> repository (SR). An SR has an *immutable* UUID associated with it.
>>
>> The virtual volume (which a VM sees as a disk) is represented by a virtual
>> disk image (VDI) in the SR. A VDI also has an *immutable* UUID associated
>> with it.
>>
>> If I take a snapshot (or a clone) of the SolidFire volume and then later
>> try to use that snapshot from XenServer, XenServer complains that the SR on
>> the snapshot has a UUID that conflicts with an existing UUID.
>>
>> In other words, it is not possible to use the original SR and the snapshot
>> of this SR from XenServer at the same time, which is critical in a cloud
>> environment (to enable creating templates from snapshots).
>>
>> The way I have proposed circumventing this issue is not ideal, but
>> technically works (this code is checked into the CS 4.6 branch):
>>
>> When the time comes to take a CloudStack snapshot of a CloudStack volume
>> that is backed by SolidFire storage via the storage plug-in, the plug-in
>> will create a new SolidFire volume with characteristics (size and IOPS)
>> equal to those of the original volume.
>>
>> We then have XenServer attach to this new SolidFire volume, create a *new*
>> SR on it, and then copy the VDI from the source SR to the destination SR
>> (the new SR).
>>
>> This leads to us having a copy of the VDI (a "snapshot" of sorts), but it
>> requires CPU cycles on the compute cluster as well as network bandwidth to
>> write to the SAN (thus it is slower and more resource intensive than a
>> SolidFire snapshot).
>>
>> I spoke with Tim Mackey (who works on XenServer at Citrix) concerning this
>> issue before and during the CloudStack Collaboration Conference in Budapest
>> in November. He agreed that this is a legitimate issue with the way
>> XenServer is designed and could not think of a way (other than what I was
>> doing) to get around it in current versions of XenServer.
>>
>> One thought is to have a feature added to XenServer that enables you to
>> change the UUID of an SR and of a VDI.
>>
>> If I could do that, then I could take a SolidFire snapshot of the
>> SolidFire volume and issue commands to XenServer to have it change the
>> UUIDs of the original SR and the original VDI. I could then recored the
>> necessary UUID info in the CS DB.
>>
>> *** End of e-mail ***
>>
>> I have since investigated this on ESXi.
>>
>> ESXi does have a way for us to "re-signature" a datastore, so backend
>> snapshots can be taken and effectively used on this hypervisor.
>>
>> On Mon, Feb 16, 2015 at 8:19 AM, Logan Barfield 
>> wrote:
>>
>>> I'm just going to stick with the qemu-img option change for RBD for
>>> now (which should cut

[MERGE] Redundant VPC routers and persistent router config

[Daan Hoogland]

H,

I will merge our feature/systemvm-persistent-config into master. If
you have objections please let me know before tomorrow.

@john: your comment was addressed in the present day version.

-- 
Daan

Re: Your thoughts on using Primary Storage for keeping snapshots

[Mike Tutkowski]

I should have also commented on KVM (since that was the hypervisor called
out in the initial e-mail).

In my situation, most of my customers use XenServer and/or ESXi, so KVM has
received the fewest of my cycles with regards to those three hypervisors.

KVM, though, is actually the simplest hypervisor for which to implement
these changes (since I am using the iSCSI adapter of the KVM agent and it
just essentially passes my LUN to the VM in question).

For KVM, there is no clustered file system applied to my backend LUN, so I
don't have to "worry" about that layer.

I don't see any hurdles like *immutable* UUIDs of SRs and VDIs (such is the
case with XenServer) or having to re-signature anything (such is the case
with ESXi).

On Mon, Feb 16, 2015 at 9:33 AM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> I have been working on this on and off for a while now (as time permits).
>
> Here is an e-mail I sent to a customer of ours that helps describe some of
> the issues:
>
> *** Beginning of e-mail ***
>
> The main requests were around the following features:
>
> * The ability to leverage SolidFire snapshots.
>
> * The ability to create CloudStack templates from SolidFire snapshots.
>
> I had these on my roadmap, but bumped the priority up and began work on
> them for the CS 4.6 release.
>
> During design, I realized there were issues with the way XenServer is
> architected that prevented me from directly using SolidFire snapshots.
>
> I could definitely take a SolidFire snapshot of a SolidFire volume, but
> this snapshot would not be usable from XenServer if the original volume was
> still in use.
>
> Here is the gist of the problem:
>
> When XenServer leverages an iSCSI target such as a SolidFire volume, it
> applies a clustered files system to it, which they call a storage
> repository (SR). An SR has an *immutable* UUID associated with it.
>
> The virtual volume (which a VM sees as a disk) is represented by a virtual
> disk image (VDI) in the SR. A VDI also has an *immutable* UUID associated
> with it.
>
> If I take a snapshot (or a clone) of the SolidFire volume and then later
> try to use that snapshot from XenServer, XenServer complains that the SR on
> the snapshot has a UUID that conflicts with an existing UUID.
>
> In other words, it is not possible to use the original SR and the snapshot
> of this SR from XenServer at the same time, which is critical in a cloud
> environment (to enable creating templates from snapshots).
>
> The way I have proposed circumventing this issue is not ideal, but
> technically works (this code is checked into the CS 4.6 branch):
>
> When the time comes to take a CloudStack snapshot of a CloudStack volume
> that is backed by SolidFire storage via the storage plug-in, the plug-in
> will create a new SolidFire volume with characteristics (size and IOPS)
> equal to those of the original volume.
>
> We then have XenServer attach to this new SolidFire volume, create a *new*
> SR on it, and then copy the VDI from the source SR to the destination SR
> (the new SR).
>
> This leads to us having a copy of the VDI (a "snapshot" of sorts), but it
> requires CPU cycles on the compute cluster as well as network bandwidth to
> write to the SAN (thus it is slower and more resource intensive than a
> SolidFire snapshot).
>
> I spoke with Tim Mackey (who works on XenServer at Citrix) concerning this
> issue before and during the CloudStack Collaboration Conference in Budapest
> in November. He agreed that this is a legitimate issue with the way
> XenServer is designed and could not think of a way (other than what I was
> doing) to get around it in current versions of XenServer.
>
> One thought is to have a feature added to XenServer that enables you to
> change the UUID of an SR and of a VDI.
>
> If I could do that, then I could take a SolidFire snapshot of the
> SolidFire volume and issue commands to XenServer to have it change the
> UUIDs of the original SR and the original VDI. I could then recored the
> necessary UUID info in the CS DB.
>
> *** End of e-mail ***
>
> I have since investigated this on ESXi.
>
> ESXi does have a way for us to "re-signature" a datastore, so backend
> snapshots can be taken and effectively used on this hypervisor.
>
> On Mon, Feb 16, 2015 at 8:19 AM, Logan Barfield 
> wrote:
>
>> I'm just going to stick with the qemu-img option change for RBD for
>> now (which should cut snapshot time down drastically), and look
>> forward to this in the future.  I'd be happy to help get this moving,
>> but I'm not enough of a developer to lead the charge.
>>
>> As far as renaming goes, I agree that maybe backups isn't the right
>> word.  That being said calling a full-sized copy of a volume a
>> "snapshot" also isn't the right word.  Maybe "image" would be better?
>>
>> I've also got my reservations about "accounts" vs "users" (I think
>> "departments" and "accounts or users" respectively is less confusing),
>> but that's a different thread.
>>
>> Thank You,
>>
>>

[GitHub] cloudstack pull request: Latest changes on feature/systemvm persis...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/79


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Your thoughts on using Primary Storage for keeping snapshots

[Mike Tutkowski]

I have been working on this on and off for a while now (as time permits).

Here is an e-mail I sent to a customer of ours that helps describe some of
the issues:

*** Beginning of e-mail ***

The main requests were around the following features:

* The ability to leverage SolidFire snapshots.

* The ability to create CloudStack templates from SolidFire snapshots.

I had these on my roadmap, but bumped the priority up and began work on
them for the CS 4.6 release.

During design, I realized there were issues with the way XenServer is
architected that prevented me from directly using SolidFire snapshots.

I could definitely take a SolidFire snapshot of a SolidFire volume, but
this snapshot would not be usable from XenServer if the original volume was
still in use.

Here is the gist of the problem:

When XenServer leverages an iSCSI target such as a SolidFire volume, it
applies a clustered files system to it, which they call a storage
repository (SR). An SR has an *immutable* UUID associated with it.

The virtual volume (which a VM sees as a disk) is represented by a virtual
disk image (VDI) in the SR. A VDI also has an *immutable* UUID associated
with it.

If I take a snapshot (or a clone) of the SolidFire volume and then later
try to use that snapshot from XenServer, XenServer complains that the SR on
the snapshot has a UUID that conflicts with an existing UUID.

In other words, it is not possible to use the original SR and the snapshot
of this SR from XenServer at the same time, which is critical in a cloud
environment (to enable creating templates from snapshots).

The way I have proposed circumventing this issue is not ideal, but
technically works (this code is checked into the CS 4.6 branch):

When the time comes to take a CloudStack snapshot of a CloudStack volume
that is backed by SolidFire storage via the storage plug-in, the plug-in
will create a new SolidFire volume with characteristics (size and IOPS)
equal to those of the original volume.

We then have XenServer attach to this new SolidFire volume, create a *new*
SR on it, and then copy the VDI from the source SR to the destination SR
(the new SR).

This leads to us having a copy of the VDI (a "snapshot" of sorts), but it
requires CPU cycles on the compute cluster as well as network bandwidth to
write to the SAN (thus it is slower and more resource intensive than a
SolidFire snapshot).

I spoke with Tim Mackey (who works on XenServer at Citrix) concerning this
issue before and during the CloudStack Collaboration Conference in Budapest
in November. He agreed that this is a legitimate issue with the way
XenServer is designed and could not think of a way (other than what I was
doing) to get around it in current versions of XenServer.

One thought is to have a feature added to XenServer that enables you to
change the UUID of an SR and of a VDI.

If I could do that, then I could take a SolidFire snapshot of the SolidFire
volume and issue commands to XenServer to have it change the UUIDs of the
original SR and the original VDI. I could then recored the necessary UUID
info in the CS DB.

*** End of e-mail ***

I have since investigated this on ESXi.

ESXi does have a way for us to "re-signature" a datastore, so backend
snapshots can be taken and effectively used on this hypervisor.

On Mon, Feb 16, 2015 at 8:19 AM, Logan Barfield 
wrote:

> I'm just going to stick with the qemu-img option change for RBD for
> now (which should cut snapshot time down drastically), and look
> forward to this in the future.  I'd be happy to help get this moving,
> but I'm not enough of a developer to lead the charge.
>
> As far as renaming goes, I agree that maybe backups isn't the right
> word.  That being said calling a full-sized copy of a volume a
> "snapshot" also isn't the right word.  Maybe "image" would be better?
>
> I've also got my reservations about "accounts" vs "users" (I think
> "departments" and "accounts or users" respectively is less confusing),
> but that's a different thread.
>
> Thank You,
>
> Logan Barfield
> Tranquil Hosting
>
>
> On Mon, Feb 16, 2015 at 10:04 AM, Wido den Hollander 
> wrote:
> >
> >
> > On 16-02-15 15:38, Logan Barfield wrote:
> >> I like this idea a lot for Ceph RBD.  I do think there should still be
> >> support for copying snapshots to secondary storage as needed (for
> >> transfers between zones, etc.).  I really think that this could be
> >> part of a larger move to clarify the naming conventions used for disk
> >> operations.  Currently "Volume Snapshots" should probably really be
> >> called "Backups".  So having "snapshot" functionality, and a "convert
> >> snapshot to backup/template" would be a good move.
> >>
> >
> > I fully agree that this would be a very great addition.
> >
> > I won't be able to work on this any time soon though.
> >
> > Wido
> >
> >> Thank You,
> >>
> >> Logan Barfield
> >> Tranquil Hosting
> >>
> >>
> >> On Mon, Feb 16, 2015 at 9:16 AM, Andrija Panic 
> wrote:
> >>> BIG +1
> >>>
> >>> My team should 

Re: Cloudmonkey question

[Mike Tutkowski]

We have such a weird way of storing storage tags in the DB.

They get stored as individual rows in the storage_pool_details table (which
is fine). If the value in the "value" column is "true", then we assume the
"key" column contains the name of a storage tag.

This makes it more difficult than it should be to store arbitrary details
that are booleans.

On Mon, Feb 16, 2015 at 7:25 AM, Andrei Mikhailovsky 
wrote:

> After digging a bit further and getting hints from Rohit, the correct
> syntax to achieve this without using any additional scripting would be:
>
> list volumes tags[0].key=remote_backup tags[0].value=yes
>
> This will only list the volumes with the tag remote_backup=yes
>
> Thanks for your help and ideas
>
> Andrei
>
> - Original Message -
>
> > From: "Mike Tutkowski" 
> > To: dev@cloudstack.apache.org
> > Sent: Monday, 16 February, 2015 5:34:26 AM
> > Subject: Re: Cloudmonkey question
>
> > Yeah, that's true - it does look like that should work.
>
> > On Sunday, February 15, 2015, Ian Duffy  wrote:
>
> > > Assuming I'm reading and understanding the API docs correctly at
> > >
> > >
> http://cloudstack.apache.org/docs/api/apidocs-4.1/root_admin/listVolumes.html
> > >
> > > list volumes tags=remote_backup
> > >
> > > should list everything with the tag remote_backup
> > >
> > > On 15 February 2015 at 23:50, Mike Tutkowski
> > > > wrote:
> > > > I believe you'd need to write a script to invoke CloudMonkey to
> > > > execute
> > > the
> > > > list command and then parse the results of the command (keeping
> > > > only what
> > > > you're interested in).
> > > >
> > > > On Sunday, February 15, 2015, Andrei Mikhailovsky
> > > >  > > > wrote:
> > > >
> > > >> Hello guys,
> > > >>
> > > >> I have a silly question; can't really find an answer by
> > > >> googling. How
> > > do I
> > > >> use tags when I want to query something. For instance, if I want
> > > >> to
> > > query
> > > >> volumes using "list volumes" command. If i would like to get
> > > >> only the
> > > >> results containing a certain tag, like a tag with key
> > > >> remote_backup and
> > > >> value of yes; how would the list volumes command should look
> > > >> like?
> > > >>
> > > >> Thanks
> > > >>
> > > >> Andrei
> > > >>
> > > >
> > > >
> > > > --
> > > > *Mike Tutkowski*
> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > e: mike.tutkow...@solidfire.com 
> > > > o: 303.746.7302
> > > > Advancing the way the world uses the cloud
> > > > *™*
> > >
>
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkow...@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the cloud
> > *™*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Re: Disable HA temporary ?

[Andrija Panic]

I agree...and understand :)

But would this means, that VMs will not be provisioned anywhere during HA
kicking in ? I guess so...
So I avoid having started another copy of the same VM, that is alrady
running on disconnected hosts - I need this as the temporary solution,
during CEPH backfilling, so not sure if this heavy hack is good , or will
case me even more trouble...

cheers


On 16 February 2015 at 16:58, Logan Barfield 
wrote:

> Hi Andrija,
>
> The way I understand it (and have seen in practice) is that by default
> the MGMT server will use any available server for HA.  Setting the HA
> tag on a hosts just dedicates that host to HA, meaning that during
> normal provisioning no VMs will use that host, it will only be used
> for HA purposes.  In other words, the "HA" tag is not required for HA
> to work.
>
> Thank You,
>
> Logan Barfield
> Tranquil Hosting
>
>
> On Mon, Feb 16, 2015 at 10:43 AM, Andrija Panic 
> wrote:
> > Seems to me, that I'm about to issue something similar to:   update
> > cloud.vm_instance set ha = 0 where ha =1...
> >
> > Now seriously, wondering, per the manual - if you define HA host tag on
> the
> > global config level, and then have NO hosts with that tag - MGMT will not
> > be able to start VMs on other hosts, since there are no hosts that are
> > dedicated got HA destination ?
> >
> > Does this makes sense ? I guess the VMs will be just marked as Stopped in
> > the GUI/databse, but unable to start them...
> > Stupid proposal, but... ?
> >
> > On 16 February 2015 at 16:22, Logan Barfield 
> > wrote:
> >
> >> Some sort of fencing independent of the management server is
> >> definitely needed.  HA in general (particularly on KVM) is all kinds
> >> of unpredictable/buggy right now.
> >>
> >> I like the idea of having a switch that an admin can flip to stop HA.
> >> In fact I think a better job control system in general (e.g., being
> >> able to stop/restart/manually start tasks) would be awesome, if it's
> >> feasible.
> >>
> >> Thank You,
> >>
> >> Logan Barfield
> >> Tranquil Hosting
> >>
> >>
> >> On Mon, Feb 16, 2015 at 10:05 AM, Wido den Hollander 
> >> wrote:
> >> >
> >> >
> >> > On 16-02-15 13:16, Andrei Mikhailovsky wrote:
> >> >> I had similar issues at least two or thee times. The host agent would
> >> disconnect from the management server. The agent would not connect back
> to
> >> the management server without manual intervention, however, it would
> >> happily continue running the vms. The management server would initiate
> the
> >> HA and fire up vms, which are already running on the disconnected host.
> I
> >> ended up with a handful of vms and virtual routers being ran on two
> >> hypervisors, thus corrupting the disk and having all sorts of issues
> ((( .
> >> >>
> >> >> I think there has to be a better way of dealing with this case. At
> >> least on an image level. Perhaps a host should keep some sort of lock
> file
> >> or a file for every image where it would record a time stamp. Something
> >> like:
> >> >>
> >> >> f5ffa8b0-d852-41c8-a386-6efb8241e2e7 and
> >> >> f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp
> >> >>
> >> >> Thus, the f5ffa8b0-d852-41c8-a386-6efb8241e2e7 is the name of the
> disk
> >> image and f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp is the image's
> >> time stamp.
> >> >>
> >> >> The hypervisor should record the time stamp in this file while the vm
> >> is running. Let's say every 5-10 seconds. If the timestamp is old, we
> can
> >> assume that the volume is no longer used by the hypervisor.
> >> >>
> >> >> When a vm is started, the timestamp file should be checked and if the
> >> timestamp is recent, the vm should not start, otherwise, the vm should
> >> start and the timestamp file should be regularly updated.
> >> >>
> >> >> I am sure there are better ways of doing this, but at least this
> method
> >> should not allow two vms running on different hosts to use the same
> volume
> >> and corrupt the data.
> >> >>
> >> >> In ceph, as far as I remember, a new feature is being developed to
> >> provide a locking mechanism of an rbd image. Not sure if this will do
> the
> >> job?
> >> >>
> >> >
> >> > Something like this is still on my wishlist for Ceph/RBD, something
> like
> >> > you propose.
> >> >
> >> > For NFS we currently have this in place, but for Ceph/RBD we don't.
> It's
> >> > a matter of code in the Agent and the investigators inside the
> >> > Management Server which decide if HA should kick in.
> >> >
> >> > Wido
> >> >
> >> >> Andrei
> >> >>
> >> >> - Original Message -
> >> >>
> >> >>> From: "Wido den Hollander" 
> >> >>> To: dev@cloudstack.apache.org
> >> >>> Sent: Monday, 16 February, 2015 11:32:13 AM
> >> >>> Subject: Re: Disable HA temporary ?
> >> >>
> >> >>> On 16-02-15 11:00, Andrija Panic wrote:
> >>  Hi team,
> >> 
> >>  I just had funny behaviour few days ago - one of my hosts was under
> >>  heavy
> >>  load (some disk/network load) and it went disconnected from MGMT
> >>  s

Re: Disable HA temporary ?

[Logan Barfield]

Hi Andrija,

The way I understand it (and have seen in practice) is that by default
the MGMT server will use any available server for HA.  Setting the HA
tag on a hosts just dedicates that host to HA, meaning that during
normal provisioning no VMs will use that host, it will only be used
for HA purposes.  In other words, the "HA" tag is not required for HA
to work.

Thank You,

Logan Barfield
Tranquil Hosting


On Mon, Feb 16, 2015 at 10:43 AM, Andrija Panic  wrote:
> Seems to me, that I'm about to issue something similar to:   update
> cloud.vm_instance set ha = 0 where ha =1...
>
> Now seriously, wondering, per the manual - if you define HA host tag on the
> global config level, and then have NO hosts with that tag - MGMT will not
> be able to start VMs on other hosts, since there are no hosts that are
> dedicated got HA destination ?
>
> Does this makes sense ? I guess the VMs will be just marked as Stopped in
> the GUI/databse, but unable to start them...
> Stupid proposal, but... ?
>
> On 16 February 2015 at 16:22, Logan Barfield 
> wrote:
>
>> Some sort of fencing independent of the management server is
>> definitely needed.  HA in general (particularly on KVM) is all kinds
>> of unpredictable/buggy right now.
>>
>> I like the idea of having a switch that an admin can flip to stop HA.
>> In fact I think a better job control system in general (e.g., being
>> able to stop/restart/manually start tasks) would be awesome, if it's
>> feasible.
>>
>> Thank You,
>>
>> Logan Barfield
>> Tranquil Hosting
>>
>>
>> On Mon, Feb 16, 2015 at 10:05 AM, Wido den Hollander 
>> wrote:
>> >
>> >
>> > On 16-02-15 13:16, Andrei Mikhailovsky wrote:
>> >> I had similar issues at least two or thee times. The host agent would
>> disconnect from the management server. The agent would not connect back to
>> the management server without manual intervention, however, it would
>> happily continue running the vms. The management server would initiate the
>> HA and fire up vms, which are already running on the disconnected host. I
>> ended up with a handful of vms and virtual routers being ran on two
>> hypervisors, thus corrupting the disk and having all sorts of issues ((( .
>> >>
>> >> I think there has to be a better way of dealing with this case. At
>> least on an image level. Perhaps a host should keep some sort of lock file
>> or a file for every image where it would record a time stamp. Something
>> like:
>> >>
>> >> f5ffa8b0-d852-41c8-a386-6efb8241e2e7 and
>> >> f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp
>> >>
>> >> Thus, the f5ffa8b0-d852-41c8-a386-6efb8241e2e7 is the name of the disk
>> image and f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp is the image's
>> time stamp.
>> >>
>> >> The hypervisor should record the time stamp in this file while the vm
>> is running. Let's say every 5-10 seconds. If the timestamp is old, we can
>> assume that the volume is no longer used by the hypervisor.
>> >>
>> >> When a vm is started, the timestamp file should be checked and if the
>> timestamp is recent, the vm should not start, otherwise, the vm should
>> start and the timestamp file should be regularly updated.
>> >>
>> >> I am sure there are better ways of doing this, but at least this method
>> should not allow two vms running on different hosts to use the same volume
>> and corrupt the data.
>> >>
>> >> In ceph, as far as I remember, a new feature is being developed to
>> provide a locking mechanism of an rbd image. Not sure if this will do the
>> job?
>> >>
>> >
>> > Something like this is still on my wishlist for Ceph/RBD, something like
>> > you propose.
>> >
>> > For NFS we currently have this in place, but for Ceph/RBD we don't. It's
>> > a matter of code in the Agent and the investigators inside the
>> > Management Server which decide if HA should kick in.
>> >
>> > Wido
>> >
>> >> Andrei
>> >>
>> >> - Original Message -
>> >>
>> >>> From: "Wido den Hollander" 
>> >>> To: dev@cloudstack.apache.org
>> >>> Sent: Monday, 16 February, 2015 11:32:13 AM
>> >>> Subject: Re: Disable HA temporary ?
>> >>
>> >>> On 16-02-15 11:00, Andrija Panic wrote:
>>  Hi team,
>> 
>>  I just had funny behaviour few days ago - one of my hosts was under
>>  heavy
>>  load (some disk/network load) and it went disconnected from MGMT
>>  server.
>> 
>>  Then MGMT server stared doing HA thing, but without being able to
>>  make sure
>>  that the VMs on the disconnected hosts are really shutdown (and
>>  they were
>>  NOT).
>> 
>>  So MGMT started again some VMs on other hosts, thus resulting in
>>  having 2
>>  copies of the same VM, using shared strage - so corruption happened
>>  on the
>>  disk.
>> 
>>  Is there a way to temporary disable HA feature on global level, or
>>  anything
>>  similar ?
>> >>
>> >>> Not that I'm aware of, but this is something I also ran in to a
>> >>> couple
>> >>> of times.
>> >>
>> >>> It would indeed be nice if there c

Re: Disable HA temporary ?

[Andrija Panic]

Seems to me, that I'm about to issue something similar to:   update
cloud.vm_instance set ha = 0 where ha =1...

Now seriously, wondering, per the manual - if you define HA host tag on the
global config level, and then have NO hosts with that tag - MGMT will not
be able to start VMs on other hosts, since there are no hosts that are
dedicated got HA destination ?

Does this makes sense ? I guess the VMs will be just marked as Stopped in
the GUI/databse, but unable to start them...
Stupid proposal, but... ?

On 16 February 2015 at 16:22, Logan Barfield 
wrote:

> Some sort of fencing independent of the management server is
> definitely needed.  HA in general (particularly on KVM) is all kinds
> of unpredictable/buggy right now.
>
> I like the idea of having a switch that an admin can flip to stop HA.
> In fact I think a better job control system in general (e.g., being
> able to stop/restart/manually start tasks) would be awesome, if it's
> feasible.
>
> Thank You,
>
> Logan Barfield
> Tranquil Hosting
>
>
> On Mon, Feb 16, 2015 at 10:05 AM, Wido den Hollander 
> wrote:
> >
> >
> > On 16-02-15 13:16, Andrei Mikhailovsky wrote:
> >> I had similar issues at least two or thee times. The host agent would
> disconnect from the management server. The agent would not connect back to
> the management server without manual intervention, however, it would
> happily continue running the vms. The management server would initiate the
> HA and fire up vms, which are already running on the disconnected host. I
> ended up with a handful of vms and virtual routers being ran on two
> hypervisors, thus corrupting the disk and having all sorts of issues ((( .
> >>
> >> I think there has to be a better way of dealing with this case. At
> least on an image level. Perhaps a host should keep some sort of lock file
> or a file for every image where it would record a time stamp. Something
> like:
> >>
> >> f5ffa8b0-d852-41c8-a386-6efb8241e2e7 and
> >> f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp
> >>
> >> Thus, the f5ffa8b0-d852-41c8-a386-6efb8241e2e7 is the name of the disk
> image and f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp is the image's
> time stamp.
> >>
> >> The hypervisor should record the time stamp in this file while the vm
> is running. Let's say every 5-10 seconds. If the timestamp is old, we can
> assume that the volume is no longer used by the hypervisor.
> >>
> >> When a vm is started, the timestamp file should be checked and if the
> timestamp is recent, the vm should not start, otherwise, the vm should
> start and the timestamp file should be regularly updated.
> >>
> >> I am sure there are better ways of doing this, but at least this method
> should not allow two vms running on different hosts to use the same volume
> and corrupt the data.
> >>
> >> In ceph, as far as I remember, a new feature is being developed to
> provide a locking mechanism of an rbd image. Not sure if this will do the
> job?
> >>
> >
> > Something like this is still on my wishlist for Ceph/RBD, something like
> > you propose.
> >
> > For NFS we currently have this in place, but for Ceph/RBD we don't. It's
> > a matter of code in the Agent and the investigators inside the
> > Management Server which decide if HA should kick in.
> >
> > Wido
> >
> >> Andrei
> >>
> >> - Original Message -
> >>
> >>> From: "Wido den Hollander" 
> >>> To: dev@cloudstack.apache.org
> >>> Sent: Monday, 16 February, 2015 11:32:13 AM
> >>> Subject: Re: Disable HA temporary ?
> >>
> >>> On 16-02-15 11:00, Andrija Panic wrote:
>  Hi team,
> 
>  I just had funny behaviour few days ago - one of my hosts was under
>  heavy
>  load (some disk/network load) and it went disconnected from MGMT
>  server.
> 
>  Then MGMT server stared doing HA thing, but without being able to
>  make sure
>  that the VMs on the disconnected hosts are really shutdown (and
>  they were
>  NOT).
> 
>  So MGMT started again some VMs on other hosts, thus resulting in
>  having 2
>  copies of the same VM, using shared strage - so corruption happened
>  on the
>  disk.
> 
>  Is there a way to temporary disable HA feature on global level, or
>  anything
>  similar ?
> >>
> >>> Not that I'm aware of, but this is something I also ran in to a
> >>> couple
> >>> of times.
> >>
> >>> It would indeed be nice if there could be a way to stop the HA
> >>> process
> >>> completely as an Admin.
> >>
> >>> Wido
> >>
>  Thanks
> 
> >>
>



-- 

Andrija Panić

Re: Disable HA temporary ?

[Logan Barfield]

Some sort of fencing independent of the management server is
definitely needed.  HA in general (particularly on KVM) is all kinds
of unpredictable/buggy right now.

I like the idea of having a switch that an admin can flip to stop HA.
In fact I think a better job control system in general (e.g., being
able to stop/restart/manually start tasks) would be awesome, if it's
feasible.

Thank You,

Logan Barfield
Tranquil Hosting


On Mon, Feb 16, 2015 at 10:05 AM, Wido den Hollander  wrote:
>
>
> On 16-02-15 13:16, Andrei Mikhailovsky wrote:
>> I had similar issues at least two or thee times. The host agent would 
>> disconnect from the management server. The agent would not connect back to 
>> the management server without manual intervention, however, it would happily 
>> continue running the vms. The management server would initiate the HA and 
>> fire up vms, which are already running on the disconnected host. I ended up 
>> with a handful of vms and virtual routers being ran on two hypervisors, thus 
>> corrupting the disk and having all sorts of issues ((( .
>>
>> I think there has to be a better way of dealing with this case. At least on 
>> an image level. Perhaps a host should keep some sort of lock file or a file 
>> for every image where it would record a time stamp. Something like:
>>
>> f5ffa8b0-d852-41c8-a386-6efb8241e2e7 and
>> f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp
>>
>> Thus, the f5ffa8b0-d852-41c8-a386-6efb8241e2e7 is the name of the disk image 
>> and f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp is the image's time stamp.
>>
>> The hypervisor should record the time stamp in this file while the vm is 
>> running. Let's say every 5-10 seconds. If the timestamp is old, we can 
>> assume that the volume is no longer used by the hypervisor.
>>
>> When a vm is started, the timestamp file should be checked and if the 
>> timestamp is recent, the vm should not start, otherwise, the vm should start 
>> and the timestamp file should be regularly updated.
>>
>> I am sure there are better ways of doing this, but at least this method 
>> should not allow two vms running on different hosts to use the same volume 
>> and corrupt the data.
>>
>> In ceph, as far as I remember, a new feature is being developed to provide a 
>> locking mechanism of an rbd image. Not sure if this will do the job?
>>
>
> Something like this is still on my wishlist for Ceph/RBD, something like
> you propose.
>
> For NFS we currently have this in place, but for Ceph/RBD we don't. It's
> a matter of code in the Agent and the investigators inside the
> Management Server which decide if HA should kick in.
>
> Wido
>
>> Andrei
>>
>> - Original Message -
>>
>>> From: "Wido den Hollander" 
>>> To: dev@cloudstack.apache.org
>>> Sent: Monday, 16 February, 2015 11:32:13 AM
>>> Subject: Re: Disable HA temporary ?
>>
>>> On 16-02-15 11:00, Andrija Panic wrote:
 Hi team,

 I just had funny behaviour few days ago - one of my hosts was under
 heavy
 load (some disk/network load) and it went disconnected from MGMT
 server.

 Then MGMT server stared doing HA thing, but without being able to
 make sure
 that the VMs on the disconnected hosts are really shutdown (and
 they were
 NOT).

 So MGMT started again some VMs on other hosts, thus resulting in
 having 2
 copies of the same VM, using shared strage - so corruption happened
 on the
 disk.

 Is there a way to temporary disable HA feature on global level, or
 anything
 similar ?
>>
>>> Not that I'm aware of, but this is something I also ran in to a
>>> couple
>>> of times.
>>
>>> It would indeed be nice if there could be a way to stop the HA
>>> process
>>> completely as an Admin.
>>
>>> Wido
>>
 Thanks

>>

[GitHub] cloudstack pull request: Latest changes on feature/systemvm persis...

[wilderrodrigues]

GitHub user wilderrodrigues opened a pull request:

https://github.com/apache/cloudstack/pull/79

Latest changes on feature/systemvm persistent config

@DaanHoogland

Latest commits on this branch. Could you please have a look at this pull 
request?
I will execute all the tests and add the results here. Once it happen, you 
can create the PR towards master.

Cheers,
Wilder

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/schubergphilis/cloudstack 
feature/systemvm-persistent-config-4

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/79.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #79


commit 4e03444b8aec86152472ebb2d9101e8fb1f01cc4
Author: Ian Southam 
Date:   2014-07-29T15:04:31Z

Add the Python bits

commit f1ab827fec22e77e436986237ec306110af90481
Author: Ian Southam 
Date:   2014-07-29T15:17:07Z

Added cs_ip module
Corrected syntax error in merge.py

commit 78ee9e0cf16d0a8e0aaebf68da11bb14cd059274
Author: Ian Southam 
Date:   2014-07-29T15:24:07Z

Use json naming standards instead of camelCase

commit 06749f1150033707373564bc2c02a0acf6ba94ff
Author: Ian Southam 
Date:   2014-07-29T15:28:01Z

Changed from camelCase to json_case

commit aabe6c413aa8aca8ac55908584fc4163fbbaac88
Author: Hugo Trippaers 
Date:   2014-07-29T15:31:23Z

Create a json file for SetNetworkACL

commit 3d1315c9f4e7c8a0a354efb6998b6c8649454ea5
Author: Leo Simons 
Date:   2014-07-29T15:34:17Z

Remove unsupported pre-veewee systemvm build.

commit 8d380a6938490a34a6701f241f7294082514
Author: Ian Southam 
Date:   2014-07-29T15:37:07Z

Only ip_association files for now

commit e655457c5e3e824caf4f5c30fdb21d25969742e2
Author: Ian Southam 
Date:   2014-07-29T17:05:09Z

Can now read the ips out of the cmdline databag (if present)

commit 97e05f73a0d72cb482a3b3de8b6cdd191ca0d39a
Author: Ian Southam 
Date:   2014-07-29T17:06:03Z

Correct small typo in error message

commit 7ccd73a80db6e8bc4db90761380a6a88e7820c67
Author: Hugo Trippaers 
Date:   2014-07-29T15:56:51Z

Include a type field in all json configuration objects

commit cf4ccada5915dff814485040c7a698d692f95f5b
Author: Ian Southam 
Date:   2014-07-29T17:07:00Z

Remove debug code

commit 2cbccd1273d7cb24cc81dce8b3e718d98f857c54
Author: Hugo Trippaers 
Date:   2014-07-30T08:37:22Z

Switch ip associations to new model and update the recipes

commit f403e29b1696b6ff060042a00784ff1e32d9e45b
Author: Hugo Trippaers 
Date:   2014-07-30T08:41:56Z

Disable cmdline check until it's fixed

commit a9264f67d3b60b121affb729308fbe5ec6a8da35
Author: Ian Southam 
Date:   2014-07-30T11:16:27Z

1.  Completed provider for ip rules (fwmark)
2.  Added merge routine for guestnetwork config messages
3.  Updated test script

commit efaaea095ca40cdcc4d8b332672154510fbf5ced
Author: Ian Southam 
Date:   2014-07-30T12:10:03Z

Corrected a hole in my logic

commit 0d757809ef0665804aa78e359785a83e40217ed9
Author: Hugo Trippaers 
Date:   2014-07-30T12:13:24Z

Rewrite networkacl model to have separate entries for each rule

commit 426203c9d2785fae5d9f712016f1581b0f3409f6
Author: Hugo Trippaers 
Date:   2014-07-30T14:04:35Z

Add some debug logging to keep track of timing

commit 441751d8a8b497e0481f13f3e18083ccf9117bb9
Author: Hugo Trippaers 
Date:   2014-07-30T14:05:41Z

Change vmdata to the new config system

commit fe49361f66d09e4dd1eebbdc7468aacf43c484e2
Author: Leo Simons 
Date:   2014-07-30T15:38:39Z

A working test-kitchen setup for testing systemvm boxes.

commit 44c4ea311e4021d18ad1ff48f7f1b8d5dd28ed46
Author: Ian Southam 
Date:   2014-07-30T15:46:06Z

Include the guestnetwork code
This takes the guestnetwork object and also creates an ip object

commit 0f5333971f268caf27c4154deefa3f5ca4c2912e
Author: Ian Southam 
Date:   2014-07-30T16:03:35Z

Split Databag in to separate class as I would now need this

commit 1e0f21fc8df25d7b36f8f0096d782579ad711c1d
Author: Leo Simons 
Date:   2014-07-31T11:40:41Z

junit report output for vagrant systemvm tests

commit 41abcb9a0f6a3b7b643b8b7512ecac68556bbf4a
Author: Leo Simons 
Date:   2014-07-31T14:00:15Z

Use bundler to exec test-kitchen

commit 077c649ad4a29fa486cd517e85c8b58fc20082f1
Author: Leo Simons 
Date:   2014-07-31T14:04:29Z

Commit missing .kitchen.yml

commit 45d9acb351517d3b6bf591c7204498c96a2aa96b
Author: Leo Simons 
Date:   2014-08-01T12:16:26Z

Massively simpler serverspec invocation

Give up on using test-kitchen, busser, and more of its complexity and
simply run serverspec directly, via SSH.

commit ef6366c5450f1b7d565ab7955ab1270be5b324a2
Author: Leo Simons 
Date:   2014-08-01T12:43:35Z

Missing gem for vagrant magic

commit 0b998969fd1ddb04cc34e5cfef287e97121b3

Re: Your thoughts on using Primary Storage for keeping snapshots

[Logan Barfield]

I'm just going to stick with the qemu-img option change for RBD for
now (which should cut snapshot time down drastically), and look
forward to this in the future.  I'd be happy to help get this moving,
but I'm not enough of a developer to lead the charge.

As far as renaming goes, I agree that maybe backups isn't the right
word.  That being said calling a full-sized copy of a volume a
"snapshot" also isn't the right word.  Maybe "image" would be better?

I've also got my reservations about "accounts" vs "users" (I think
"departments" and "accounts or users" respectively is less confusing),
but that's a different thread.

Thank You,

Logan Barfield
Tranquil Hosting


On Mon, Feb 16, 2015 at 10:04 AM, Wido den Hollander  wrote:
>
>
> On 16-02-15 15:38, Logan Barfield wrote:
>> I like this idea a lot for Ceph RBD.  I do think there should still be
>> support for copying snapshots to secondary storage as needed (for
>> transfers between zones, etc.).  I really think that this could be
>> part of a larger move to clarify the naming conventions used for disk
>> operations.  Currently "Volume Snapshots" should probably really be
>> called "Backups".  So having "snapshot" functionality, and a "convert
>> snapshot to backup/template" would be a good move.
>>
>
> I fully agree that this would be a very great addition.
>
> I won't be able to work on this any time soon though.
>
> Wido
>
>> Thank You,
>>
>> Logan Barfield
>> Tranquil Hosting
>>
>>
>> On Mon, Feb 16, 2015 at 9:16 AM, Andrija Panic  
>> wrote:
>>> BIG +1
>>>
>>> My team should submit some patch to ACS for better KVM snapshots, including
>>> whole VM snapshot etc...but it's too early to give details...
>>> best
>>>
>>> On 16 February 2015 at 13:01, Andrei Mikhailovsky  wrote:
>>>
 Hello guys,

 I was hoping to have some feedback from the community on the subject of
 having an ability to keep snapshots on the primary storage where it is
 supported by the storage backend.

 The idea behind this functionality is to improve how snapshots are
 currently handled on KVM hypervisors with Ceph primary storage. At the
 moment, the snapshots are taken on the primary storage and being copied to
 the secondary storage. This method is very slow and inefficient even on
 small infrastructure. Even on medium deployments using snapshots in KVM
 becomes nearly impossible. If you have tens or hundreds concurrent
 snapshots taking place you will have a bunch of timeouts and errors, your
 network becomes clogged, etc. In addition, using these snapshots for
 creating new volumes or reverting back vms also slow and inefficient. As
 above, when you have tens or hundreds concurrent operations it will not
 succeed and you will have a majority of tasks with errors or timeouts.

 At the moment, taking a single snapshot of relatively small volumes (200GB
 or 500GB for instance) takes tens if not hundreds of minutes. Taking a
 snapshot of the same volume on ceph primary storage takes a few seconds at
 most! Similarly, converting a snapshot to a volume takes tens if not
 hundreds of minutes when secondary storage is involved; compared with
 seconds if done directly on the primary storage.

 I suggest that the CloudStack should have the ability to keep volume
 snapshots on the primary storage where this is supported by the storage.
 Perhaps having a per primary storage setting that enables this
 functionality. This will be beneficial for Ceph primary storage on KVM
 hypervisors and perhaps on XenServer when Ceph will be supported in a near
 future.

 This will greatly speed up the process of using snapshots on KVM and users
 will actually start using snapshotting rather than giving up with
 frustration.

 I have opened the ticket CLOUDSTACK-8256, so please cast your vote if you
 are in agreement.

 Thanks for your input

 Andrei





>>>
>>>
>>> --
>>>
>>> Andrija Panić

[GitHub] cloudstack pull request: Feature/systemvm persistent config 4

[wilderrodrigues]

Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/78#issuecomment-74522767
  
It seems I made a mistake with the branch. Will close this one and create a 
new PR.

Cheers,
Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Feature/systemvm persistent config 4

[wilderrodrigues]

Github user wilderrodrigues closed the pull request at:

https://github.com/apache/cloudstack/pull/78


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Feature/systemvm persistent config 4

[wilderrodrigues]

GitHub user wilderrodrigues opened a pull request:

https://github.com/apache/cloudstack/pull/78

Feature/systemvm persistent config 4

Hi Daan,

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/schubergphilis/cloudstack 
feature/systemvm-persistent-config-4

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack/pull/78.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #78


commit 4e03444b8aec86152472ebb2d9101e8fb1f01cc4
Author: Ian Southam 
Date:   2014-07-29T15:04:31Z

Add the Python bits

commit f1ab827fec22e77e436986237ec306110af90481
Author: Ian Southam 
Date:   2014-07-29T15:17:07Z

Added cs_ip module
Corrected syntax error in merge.py

commit 78ee9e0cf16d0a8e0aaebf68da11bb14cd059274
Author: Ian Southam 
Date:   2014-07-29T15:24:07Z

Use json naming standards instead of camelCase

commit 06749f1150033707373564bc2c02a0acf6ba94ff
Author: Ian Southam 
Date:   2014-07-29T15:28:01Z

Changed from camelCase to json_case

commit aabe6c413aa8aca8ac55908584fc4163fbbaac88
Author: Hugo Trippaers 
Date:   2014-07-29T15:31:23Z

Create a json file for SetNetworkACL

commit 3d1315c9f4e7c8a0a354efb6998b6c8649454ea5
Author: Leo Simons 
Date:   2014-07-29T15:34:17Z

Remove unsupported pre-veewee systemvm build.

commit 8d380a6938490a34a6701f241f7294082514
Author: Ian Southam 
Date:   2014-07-29T15:37:07Z

Only ip_association files for now

commit e655457c5e3e824caf4f5c30fdb21d25969742e2
Author: Ian Southam 
Date:   2014-07-29T17:05:09Z

Can now read the ips out of the cmdline databag (if present)

commit 97e05f73a0d72cb482a3b3de8b6cdd191ca0d39a
Author: Ian Southam 
Date:   2014-07-29T17:06:03Z

Correct small typo in error message

commit 7ccd73a80db6e8bc4db90761380a6a88e7820c67
Author: Hugo Trippaers 
Date:   2014-07-29T15:56:51Z

Include a type field in all json configuration objects

commit cf4ccada5915dff814485040c7a698d692f95f5b
Author: Ian Southam 
Date:   2014-07-29T17:07:00Z

Remove debug code

commit 2cbccd1273d7cb24cc81dce8b3e718d98f857c54
Author: Hugo Trippaers 
Date:   2014-07-30T08:37:22Z

Switch ip associations to new model and update the recipes

commit f403e29b1696b6ff060042a00784ff1e32d9e45b
Author: Hugo Trippaers 
Date:   2014-07-30T08:41:56Z

Disable cmdline check until it's fixed

commit a9264f67d3b60b121affb729308fbe5ec6a8da35
Author: Ian Southam 
Date:   2014-07-30T11:16:27Z

1.  Completed provider for ip rules (fwmark)
2.  Added merge routine for guestnetwork config messages
3.  Updated test script

commit efaaea095ca40cdcc4d8b332672154510fbf5ced
Author: Ian Southam 
Date:   2014-07-30T12:10:03Z

Corrected a hole in my logic

commit 0d757809ef0665804aa78e359785a83e40217ed9
Author: Hugo Trippaers 
Date:   2014-07-30T12:13:24Z

Rewrite networkacl model to have separate entries for each rule

commit 426203c9d2785fae5d9f712016f1581b0f3409f6
Author: Hugo Trippaers 
Date:   2014-07-30T14:04:35Z

Add some debug logging to keep track of timing

commit 441751d8a8b497e0481f13f3e18083ccf9117bb9
Author: Hugo Trippaers 
Date:   2014-07-30T14:05:41Z

Change vmdata to the new config system

commit fe49361f66d09e4dd1eebbdc7468aacf43c484e2
Author: Leo Simons 
Date:   2014-07-30T15:38:39Z

A working test-kitchen setup for testing systemvm boxes.

commit 44c4ea311e4021d18ad1ff48f7f1b8d5dd28ed46
Author: Ian Southam 
Date:   2014-07-30T15:46:06Z

Include the guestnetwork code
This takes the guestnetwork object and also creates an ip object

commit 0f5333971f268caf27c4154deefa3f5ca4c2912e
Author: Ian Southam 
Date:   2014-07-30T16:03:35Z

Split Databag in to separate class as I would now need this

commit 1e0f21fc8df25d7b36f8f0096d782579ad711c1d
Author: Leo Simons 
Date:   2014-07-31T11:40:41Z

junit report output for vagrant systemvm tests

commit 41abcb9a0f6a3b7b643b8b7512ecac68556bbf4a
Author: Leo Simons 
Date:   2014-07-31T14:00:15Z

Use bundler to exec test-kitchen

commit 077c649ad4a29fa486cd517e85c8b58fc20082f1
Author: Leo Simons 
Date:   2014-07-31T14:04:29Z

Commit missing .kitchen.yml

commit 45d9acb351517d3b6bf591c7204498c96a2aa96b
Author: Leo Simons 
Date:   2014-08-01T12:16:26Z

Massively simpler serverspec invocation

Give up on using test-kitchen, busser, and more of its complexity and
simply run serverspec directly, via SSH.

commit ef6366c5450f1b7d565ab7955ab1270be5b324a2
Author: Leo Simons 
Date:   2014-08-01T12:43:35Z

Missing gem for vagrant magic

commit 0b998969fd1ddb04cc34e5cfef287e97121b3280
Author: Leo Simons 
Date:   2014-08-01T13:27:06Z

Documentation and license headers for new systemvm testing tools.

commit e3f883d508963f27735af1b7284d0bd7bbe8046b
Author: Ian Southam 
Date:   2014-08-01T14:44:49Z

Plan B
Replace chef wit

Re: Disable HA temporary ?

[Wido den Hollander]

On 16-02-15 13:16, Andrei Mikhailovsky wrote:
> I had similar issues at least two or thee times. The host agent would 
> disconnect from the management server. The agent would not connect back to 
> the management server without manual intervention, however, it would happily 
> continue running the vms. The management server would initiate the HA and 
> fire up vms, which are already running on the disconnected host. I ended up 
> with a handful of vms and virtual routers being ran on two hypervisors, thus 
> corrupting the disk and having all sorts of issues ((( . 
> 
> I think there has to be a better way of dealing with this case. At least on 
> an image level. Perhaps a host should keep some sort of lock file or a file 
> for every image where it would record a time stamp. Something like: 
> 
> f5ffa8b0-d852-41c8-a386-6efb8241e2e7 and 
> f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp 
> 
> Thus, the f5ffa8b0-d852-41c8-a386-6efb8241e2e7 is the name of the disk image 
> and f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp is the image's time stamp. 
> 
> The hypervisor should record the time stamp in this file while the vm is 
> running. Let's say every 5-10 seconds. If the timestamp is old, we can assume 
> that the volume is no longer used by the hypervisor. 
> 
> When a vm is started, the timestamp file should be checked and if the 
> timestamp is recent, the vm should not start, otherwise, the vm should start 
> and the timestamp file should be regularly updated. 
> 
> I am sure there are better ways of doing this, but at least this method 
> should not allow two vms running on different hosts to use the same volume 
> and corrupt the data. 
> 
> In ceph, as far as I remember, a new feature is being developed to provide a 
> locking mechanism of an rbd image. Not sure if this will do the job? 
>

Something like this is still on my wishlist for Ceph/RBD, something like
you propose.

For NFS we currently have this in place, but for Ceph/RBD we don't. It's
a matter of code in the Agent and the investigators inside the
Management Server which decide if HA should kick in.

Wido

> Andrei 
> 
> - Original Message -
> 
>> From: "Wido den Hollander" 
>> To: dev@cloudstack.apache.org
>> Sent: Monday, 16 February, 2015 11:32:13 AM
>> Subject: Re: Disable HA temporary ?
> 
>> On 16-02-15 11:00, Andrija Panic wrote:
>>> Hi team,
>>>
>>> I just had funny behaviour few days ago - one of my hosts was under
>>> heavy
>>> load (some disk/network load) and it went disconnected from MGMT
>>> server.
>>>
>>> Then MGMT server stared doing HA thing, but without being able to
>>> make sure
>>> that the VMs on the disconnected hosts are really shutdown (and
>>> they were
>>> NOT).
>>>
>>> So MGMT started again some VMs on other hosts, thus resulting in
>>> having 2
>>> copies of the same VM, using shared strage - so corruption happened
>>> on the
>>> disk.
>>>
>>> Is there a way to temporary disable HA feature on global level, or
>>> anything
>>> similar ?
> 
>> Not that I'm aware of, but this is something I also ran in to a
>> couple
>> of times.
> 
>> It would indeed be nice if there could be a way to stop the HA
>> process
>> completely as an Admin.
> 
>> Wido
> 
>>> Thanks
>>>
> 

Re: Your thoughts on using Primary Storage for keeping snapshots

[Wido den Hollander]

On 16-02-15 15:38, Logan Barfield wrote:
> I like this idea a lot for Ceph RBD.  I do think there should still be
> support for copying snapshots to secondary storage as needed (for
> transfers between zones, etc.).  I really think that this could be
> part of a larger move to clarify the naming conventions used for disk
> operations.  Currently "Volume Snapshots" should probably really be
> called "Backups".  So having "snapshot" functionality, and a "convert
> snapshot to backup/template" would be a good move.
> 

I fully agree that this would be a very great addition.

I won't be able to work on this any time soon though.

Wido

> Thank You,
> 
> Logan Barfield
> Tranquil Hosting
> 
> 
> On Mon, Feb 16, 2015 at 9:16 AM, Andrija Panic  
> wrote:
>> BIG +1
>>
>> My team should submit some patch to ACS for better KVM snapshots, including
>> whole VM snapshot etc...but it's too early to give details...
>> best
>>
>> On 16 February 2015 at 13:01, Andrei Mikhailovsky  wrote:
>>
>>> Hello guys,
>>>
>>> I was hoping to have some feedback from the community on the subject of
>>> having an ability to keep snapshots on the primary storage where it is
>>> supported by the storage backend.
>>>
>>> The idea behind this functionality is to improve how snapshots are
>>> currently handled on KVM hypervisors with Ceph primary storage. At the
>>> moment, the snapshots are taken on the primary storage and being copied to
>>> the secondary storage. This method is very slow and inefficient even on
>>> small infrastructure. Even on medium deployments using snapshots in KVM
>>> becomes nearly impossible. If you have tens or hundreds concurrent
>>> snapshots taking place you will have a bunch of timeouts and errors, your
>>> network becomes clogged, etc. In addition, using these snapshots for
>>> creating new volumes or reverting back vms also slow and inefficient. As
>>> above, when you have tens or hundreds concurrent operations it will not
>>> succeed and you will have a majority of tasks with errors or timeouts.
>>>
>>> At the moment, taking a single snapshot of relatively small volumes (200GB
>>> or 500GB for instance) takes tens if not hundreds of minutes. Taking a
>>> snapshot of the same volume on ceph primary storage takes a few seconds at
>>> most! Similarly, converting a snapshot to a volume takes tens if not
>>> hundreds of minutes when secondary storage is involved; compared with
>>> seconds if done directly on the primary storage.
>>>
>>> I suggest that the CloudStack should have the ability to keep volume
>>> snapshots on the primary storage where this is supported by the storage.
>>> Perhaps having a per primary storage setting that enables this
>>> functionality. This will be beneficial for Ceph primary storage on KVM
>>> hypervisors and perhaps on XenServer when Ceph will be supported in a near
>>> future.
>>>
>>> This will greatly speed up the process of using snapshots on KVM and users
>>> will actually start using snapshotting rather than giving up with
>>> frustration.
>>>
>>> I have opened the ticket CLOUDSTACK-8256, so please cast your vote if you
>>> are in agreement.
>>>
>>> Thanks for your input
>>>
>>> Andrei
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>>
>> Andrija Panić

Re: Your thoughts on using Primary Storage for keeping snapshots

[Ian Rae]

Totally agreed that there is high value in having both the ability to do
rapid, lightweight snapshots on primary storage as well as the ability to
transfer those snapshots to secondary storage for highly durable long-term
use, template creation etc... Glad to hear that others see a distinction
between these use cases, will ask the CloudOps team and Mike T to engage on
this.

On Monday, February 16, 2015, Andrei Mikhailovsky  wrote:

> Hello guys,
>
> I was hoping to have some feedback from the community on the subject of
> having an ability to keep snapshots on the primary storage where it is
> supported by the storage backend.
>
> The idea behind this functionality is to improve how snapshots are
> currently handled on KVM hypervisors with Ceph primary storage. At the
> moment, the snapshots are taken on the primary storage and being copied to
> the secondary storage. This method is very slow and inefficient even on
> small infrastructure. Even on medium deployments using snapshots in KVM
> becomes nearly impossible. If you have tens or hundreds concurrent
> snapshots taking place you will have a bunch of timeouts and errors, your
> network becomes clogged, etc. In addition, using these snapshots for
> creating new volumes or reverting back vms also slow and inefficient. As
> above, when you have tens or hundreds concurrent operations it will not
> succeed and you will have a majority of tasks with errors or timeouts.
>
> At the moment, taking a single snapshot of relatively small volumes (200GB
> or 500GB for instance) takes tens if not hundreds of minutes. Taking a
> snapshot of the same volume on ceph primary storage takes a few seconds at
> most! Similarly, converting a snapshot to a volume takes tens if not
> hundreds of minutes when secondary storage is involved; compared with
> seconds if done directly on the primary storage.
>
> I suggest that the CloudStack should have the ability to keep volume
> snapshots on the primary storage where this is supported by the storage.
> Perhaps having a per primary storage setting that enables this
> functionality. This will be beneficial for Ceph primary storage on KVM
> hypervisors and perhaps on XenServer when Ceph will be supported in a near
> future.
>
> This will greatly speed up the process of using snapshots on KVM and users
> will actually start using snapshotting rather than giving up with
> frustration.
>
> I have opened the ticket CLOUDSTACK-8256, so please cast your vote if you
> are in agreement.
>
> Thanks for your input
>
> Andrei
>
>
>
>
>

-- 
*Ian Rae*
PDG *| *CEO
t *514.944.4008*

*CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
w cloudops.com  *|* 420 rue Guy *|* Montreal *|*
 Quebec *|* H3J 1S6

Re: Your thoughts on using Primary Storage for keeping snapshots

[Andrei Mikhailovsky]

+1 for renaming the Snapshot into something more logical. 

However, for many ppl Backups kind of means the functionality on a more 
granular level (like ability to restore files, etc.) Not sure if Backup should 
be the right term for the current volume Snapshots. 

I agree, there should be ability to copy snapshots to the secondary storage. 
perhaps even both if one requires. If someone wants to have a backup copy of 
the snapshot on the secondary storage, they might choose to have this option. 

Andrei 
- Original Message -

> From: "Logan Barfield" 
> To: dev@cloudstack.apache.org
> Sent: Monday, 16 February, 2015 2:38:00 PM
> Subject: Re: Your thoughts on using Primary Storage for keeping
> snapshots

> I like this idea a lot for Ceph RBD. I do think there should still be
> support for copying snapshots to secondary storage as needed (for
> transfers between zones, etc.). I really think that this could be
> part of a larger move to clarify the naming conventions used for disk
> operations. Currently "Volume Snapshots" should probably really be
> called "Backups". So having "snapshot" functionality, and a "convert
> snapshot to backup/template" would be a good move.

> Thank You,

> Logan Barfield
> Tranquil Hosting

> On Mon, Feb 16, 2015 at 9:16 AM, Andrija Panic
>  wrote:
> > BIG +1
> >
> > My team should submit some patch to ACS for better KVM snapshots,
> > including
> > whole VM snapshot etc...but it's too early to give details...
> > best
> >
> > On 16 February 2015 at 13:01, Andrei Mikhailovsky
> >  wrote:
> >
> >> Hello guys,
> >>
> >> I was hoping to have some feedback from the community on the
> >> subject of
> >> having an ability to keep snapshots on the primary storage where
> >> it is
> >> supported by the storage backend.
> >>
> >> The idea behind this functionality is to improve how snapshots are
> >> currently handled on KVM hypervisors with Ceph primary storage. At
> >> the
> >> moment, the snapshots are taken on the primary storage and being
> >> copied to
> >> the secondary storage. This method is very slow and inefficient
> >> even on
> >> small infrastructure. Even on medium deployments using snapshots
> >> in KVM
> >> becomes nearly impossible. If you have tens or hundreds concurrent
> >> snapshots taking place you will have a bunch of timeouts and
> >> errors, your
> >> network becomes clogged, etc. In addition, using these snapshots
> >> for
> >> creating new volumes or reverting back vms also slow and
> >> inefficient. As
> >> above, when you have tens or hundreds concurrent operations it
> >> will not
> >> succeed and you will have a majority of tasks with errors or
> >> timeouts.
> >>
> >> At the moment, taking a single snapshot of relatively small
> >> volumes (200GB
> >> or 500GB for instance) takes tens if not hundreds of minutes.
> >> Taking a
> >> snapshot of the same volume on ceph primary storage takes a few
> >> seconds at
> >> most! Similarly, converting a snapshot to a volume takes tens if
> >> not
> >> hundreds of minutes when secondary storage is involved; compared
> >> with
> >> seconds if done directly on the primary storage.
> >>
> >> I suggest that the CloudStack should have the ability to keep
> >> volume
> >> snapshots on the primary storage where this is supported by the
> >> storage.
> >> Perhaps having a per primary storage setting that enables this
> >> functionality. This will be beneficial for Ceph primary storage on
> >> KVM
> >> hypervisors and perhaps on XenServer when Ceph will be supported
> >> in a near
> >> future.
> >>
> >> This will greatly speed up the process of using snapshots on KVM
> >> and users
> >> will actually start using snapshotting rather than giving up with
> >> frustration.
> >>
> >> I have opened the ticket CLOUDSTACK-8256, so please cast your vote
> >> if you
> >> are in agreement.
> >>
> >> Thanks for your input
> >>
> >> Andrei
> >>
> >>
> >>
> >>
> >>
> >
> >
> > --
> >
> > Andrija Panić

Re: Your thoughts on using Primary Storage for keeping snapshots

[Logan Barfield]

I like this idea a lot for Ceph RBD.  I do think there should still be
support for copying snapshots to secondary storage as needed (for
transfers between zones, etc.).  I really think that this could be
part of a larger move to clarify the naming conventions used for disk
operations.  Currently "Volume Snapshots" should probably really be
called "Backups".  So having "snapshot" functionality, and a "convert
snapshot to backup/template" would be a good move.

Thank You,

Logan Barfield
Tranquil Hosting


On Mon, Feb 16, 2015 at 9:16 AM, Andrija Panic  wrote:
> BIG +1
>
> My team should submit some patch to ACS for better KVM snapshots, including
> whole VM snapshot etc...but it's too early to give details...
> best
>
> On 16 February 2015 at 13:01, Andrei Mikhailovsky  wrote:
>
>> Hello guys,
>>
>> I was hoping to have some feedback from the community on the subject of
>> having an ability to keep snapshots on the primary storage where it is
>> supported by the storage backend.
>>
>> The idea behind this functionality is to improve how snapshots are
>> currently handled on KVM hypervisors with Ceph primary storage. At the
>> moment, the snapshots are taken on the primary storage and being copied to
>> the secondary storage. This method is very slow and inefficient even on
>> small infrastructure. Even on medium deployments using snapshots in KVM
>> becomes nearly impossible. If you have tens or hundreds concurrent
>> snapshots taking place you will have a bunch of timeouts and errors, your
>> network becomes clogged, etc. In addition, using these snapshots for
>> creating new volumes or reverting back vms also slow and inefficient. As
>> above, when you have tens or hundreds concurrent operations it will not
>> succeed and you will have a majority of tasks with errors or timeouts.
>>
>> At the moment, taking a single snapshot of relatively small volumes (200GB
>> or 500GB for instance) takes tens if not hundreds of minutes. Taking a
>> snapshot of the same volume on ceph primary storage takes a few seconds at
>> most! Similarly, converting a snapshot to a volume takes tens if not
>> hundreds of minutes when secondary storage is involved; compared with
>> seconds if done directly on the primary storage.
>>
>> I suggest that the CloudStack should have the ability to keep volume
>> snapshots on the primary storage where this is supported by the storage.
>> Perhaps having a per primary storage setting that enables this
>> functionality. This will be beneficial for Ceph primary storage on KVM
>> hypervisors and perhaps on XenServer when Ceph will be supported in a near
>> future.
>>
>> This will greatly speed up the process of using snapshots on KVM and users
>> will actually start using snapshotting rather than giving up with
>> frustration.
>>
>> I have opened the ticket CLOUDSTACK-8256, so please cast your vote if you
>> are in agreement.
>>
>> Thanks for your input
>>
>> Andrei
>>
>>
>>
>>
>>
>
>
> --
>
> Andrija Panić

Re: Cloudmonkey question

[Andrei Mikhailovsky]

After digging a bit further and getting hints from Rohit, the correct syntax to 
achieve this without using any additional scripting would be: 

list volumes tags[0].key=remote_backup tags[0].value=yes 

This will only list the volumes with the tag remote_backup=yes 

Thanks for your help and ideas 

Andrei 

- Original Message -

> From: "Mike Tutkowski" 
> To: dev@cloudstack.apache.org
> Sent: Monday, 16 February, 2015 5:34:26 AM
> Subject: Re: Cloudmonkey question

> Yeah, that's true - it does look like that should work.

> On Sunday, February 15, 2015, Ian Duffy  wrote:

> > Assuming I'm reading and understanding the API docs correctly at
> >
> > http://cloudstack.apache.org/docs/api/apidocs-4.1/root_admin/listVolumes.html
> >
> > list volumes tags=remote_backup
> >
> > should list everything with the tag remote_backup
> >
> > On 15 February 2015 at 23:50, Mike Tutkowski
> > > wrote:
> > > I believe you'd need to write a script to invoke CloudMonkey to
> > > execute
> > the
> > > list command and then parse the results of the command (keeping
> > > only what
> > > you're interested in).
> > >
> > > On Sunday, February 15, 2015, Andrei Mikhailovsky
> > >  > > wrote:
> > >
> > >> Hello guys,
> > >>
> > >> I have a silly question; can't really find an answer by
> > >> googling. How
> > do I
> > >> use tags when I want to query something. For instance, if I want
> > >> to
> > query
> > >> volumes using "list volumes" command. If i would like to get
> > >> only the
> > >> results containing a certain tag, like a tag with key
> > >> remote_backup and
> > >> value of yes; how would the list volumes command should look
> > >> like?
> > >>
> > >> Thanks
> > >>
> > >> Andrei
> > >>
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkow...@solidfire.com 
> > > o: 303.746.7302
> > > Advancing the way the world uses the cloud
> > > *™*
> >

> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud
> *™*

Re: Your thoughts on using Primary Storage for keeping snapshots

[Andrija Panic]

BIG +1

My team should submit some patch to ACS for better KVM snapshots, including
whole VM snapshot etc...but it's too early to give details...
best

On 16 February 2015 at 13:01, Andrei Mikhailovsky  wrote:

> Hello guys,
>
> I was hoping to have some feedback from the community on the subject of
> having an ability to keep snapshots on the primary storage where it is
> supported by the storage backend.
>
> The idea behind this functionality is to improve how snapshots are
> currently handled on KVM hypervisors with Ceph primary storage. At the
> moment, the snapshots are taken on the primary storage and being copied to
> the secondary storage. This method is very slow and inefficient even on
> small infrastructure. Even on medium deployments using snapshots in KVM
> becomes nearly impossible. If you have tens or hundreds concurrent
> snapshots taking place you will have a bunch of timeouts and errors, your
> network becomes clogged, etc. In addition, using these snapshots for
> creating new volumes or reverting back vms also slow and inefficient. As
> above, when you have tens or hundreds concurrent operations it will not
> succeed and you will have a majority of tasks with errors or timeouts.
>
> At the moment, taking a single snapshot of relatively small volumes (200GB
> or 500GB for instance) takes tens if not hundreds of minutes. Taking a
> snapshot of the same volume on ceph primary storage takes a few seconds at
> most! Similarly, converting a snapshot to a volume takes tens if not
> hundreds of minutes when secondary storage is involved; compared with
> seconds if done directly on the primary storage.
>
> I suggest that the CloudStack should have the ability to keep volume
> snapshots on the primary storage where this is supported by the storage.
> Perhaps having a per primary storage setting that enables this
> functionality. This will be beneficial for Ceph primary storage on KVM
> hypervisors and perhaps on XenServer when Ceph will be supported in a near
> future.
>
> This will greatly speed up the process of using snapshots on KVM and users
> will actually start using snapshotting rather than giving up with
> frustration.
>
> I have opened the ticket CLOUDSTACK-8256, so please cast your vote if you
> are in agreement.
>
> Thanks for your input
>
> Andrei
>
>
>
>
>


-- 

Andrija Panić

Re: Ans: About Instance Storage Live Migration on VMware

[Star Guo]

Ilya

Thanks a lot ! I try to read the source code.

Best Regards,
Star Guo

-邮件原件-
发件人: ilya musayev [mailto:ilya.mailing.li...@gmail.com] 
发送时间: 2015年2月15日 15:00
收件人: dev@cloudstack.apache.org
主题: Re: Ans: About Instance Storage Live Migration on VMware

Star,

There is an API call "MigrateVirtualMachineWithVolume",

http://cloudstack.apache.org/docs/api/apidocs-4.4/root_admin/migrateVirtualMachineWithVolume.html

It only works over NFS, Citrix promised to add support for VMFS in the near 
future.

Regards
ilya

On 2/13/15 4:15 AM, Erik Weber wrote:
> According to https://issues.apache.org/jira/browse/CLOUDSTACK-2701 it 
> should be supported.
> Haven't tried it myself though.
>

Re: Disable HA temporary ?

[Andrei Mikhailovsky]

I had similar issues at least two or thee times. The host agent would 
disconnect from the management server. The agent would not connect back to the 
management server without manual intervention, however, it would happily 
continue running the vms. The management server would initiate the HA and fire 
up vms, which are already running on the disconnected host. I ended up with a 
handful of vms and virtual routers being ran on two hypervisors, thus 
corrupting the disk and having all sorts of issues ((( . 

I think there has to be a better way of dealing with this case. At least on an 
image level. Perhaps a host should keep some sort of lock file or a file for 
every image where it would record a time stamp. Something like: 

f5ffa8b0-d852-41c8-a386-6efb8241e2e7 and 
f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp 

Thus, the f5ffa8b0-d852-41c8-a386-6efb8241e2e7 is the name of the disk image 
and f5ffa8b0-d852-41c8-a386-6efb8241e2e7-timestamp is the image's time stamp. 

The hypervisor should record the time stamp in this file while the vm is 
running. Let's say every 5-10 seconds. If the timestamp is old, we can assume 
that the volume is no longer used by the hypervisor. 

When a vm is started, the timestamp file should be checked and if the timestamp 
is recent, the vm should not start, otherwise, the vm should start and the 
timestamp file should be regularly updated. 

I am sure there are better ways of doing this, but at least this method should 
not allow two vms running on different hosts to use the same volume and corrupt 
the data. 

In ceph, as far as I remember, a new feature is being developed to provide a 
locking mechanism of an rbd image. Not sure if this will do the job? 

Andrei 

- Original Message -

> From: "Wido den Hollander" 
> To: dev@cloudstack.apache.org
> Sent: Monday, 16 February, 2015 11:32:13 AM
> Subject: Re: Disable HA temporary ?

> On 16-02-15 11:00, Andrija Panic wrote:
> > Hi team,
> >
> > I just had funny behaviour few days ago - one of my hosts was under
> > heavy
> > load (some disk/network load) and it went disconnected from MGMT
> > server.
> >
> > Then MGMT server stared doing HA thing, but without being able to
> > make sure
> > that the VMs on the disconnected hosts are really shutdown (and
> > they were
> > NOT).
> >
> > So MGMT started again some VMs on other hosts, thus resulting in
> > having 2
> > copies of the same VM, using shared strage - so corruption happened
> > on the
> > disk.
> >
> > Is there a way to temporary disable HA feature on global level, or
> > anything
> > similar ?

> Not that I'm aware of, but this is something I also ran in to a
> couple
> of times.

> It would indeed be nice if there could be a way to stop the HA
> process
> completely as an Admin.

> Wido

> > Thanks
> >

Re: Review Request 30660: CLOUDSTACK-8219: Marvin: Correct code related to getting free vlan in the setup

[Gaurav Aradhye]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30660/#review72610
---


This is pending for review.

- Gaurav Aradhye


On Feb. 5, 2015, 4:26 p.m., Gaurav Aradhye wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30660/
> ---
> 
> (Updated Feb. 5, 2015, 4:26 p.m.)
> 
> 
> Review request for cloudstack and SrikanteswaraRao Talluri.
> 
> 
> Bugs: CLOUDSTACK-8219
> https://issues.apache.org/jira/browse/CLOUDSTACK-8219
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> The function "get_free_vlan" lists all the networks in the setup and 
> generates a vlan in the physical vlan range which is not used by any of the 
> listed networks.
> 
> However the method lists only shared networks. It should list all the 
> networks so that the vlan does not overlap with any of the used vlans.
> 
> 
> Diffs
> -
> 
>   tools/marvin/marvin/lib/common.py 28ec024 
> 
> Diff: https://reviews.apache.org/r/30660/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> Test Shared Network ALL ... === TestName: test_createSharedNetwork_All | 
> Status : SUCCESS ===
> ok
> Test Shared Network with scope account ... === TestName: 
> test_createSharedNetwork_accountSpecific | Status : SUCCESS ===
> ok
> Test Shared Network with scope domain ... === TestName: 
> test_createSharedNetwork_domainSpecific | Status : SUCCESS ===
> ok
> 
> 
> Thanks,
> 
> Gaurav Aradhye
> 
>

Review Request 31081: CLOUDSTACK-8257: test_iso.py - Removed assertion on Iso name when random character s are appended to test data before creating Iso

[Gaurav Aradhye]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31081/
---

Review request for cloudstack and SrikanteswaraRao Talluri.


Bugs: CLOUDSTACK-8258
https://issues.apache.org/jira/browse/CLOUDSTACK-8258


Repository: cloudstack-git


Description
---

Create method in Class Iso in base library now appends random characters (6) to 
the Iso name passed from test data so that each Iso created through test case 
has different name.

Hence assertios based on Iso name should be removed from test case as they 
won't hold true anymore (Iso name will not match with that passed through the 
test data).

Other changes:
1. Pep8 fixes
2. Import fixes
3. Removing white-spaces
4. Removing unused variables


Diffs
-

  test/integration/smoke/test_iso.py 4bd66b5 

Diff: https://reviews.apache.org/r/31081/diff/


Testing
---

Yes. One test case failed in downloading Iso due to DNS issue in the setup. 
Otherwise should pass.
The test case which failed in BVT build passed in my testing.


Thanks,

Gaurav Aradhye

Your thoughts on using Primary Storage for keeping snapshots

[Andrei Mikhailovsky]

Hello guys, 

I was hoping to have some feedback from the community on the subject of having 
an ability to keep snapshots on the primary storage where it is supported by 
the storage backend. 

The idea behind this functionality is to improve how snapshots are currently 
handled on KVM hypervisors with Ceph primary storage. At the moment, the 
snapshots are taken on the primary storage and being copied to the secondary 
storage. This method is very slow and inefficient even on small infrastructure. 
Even on medium deployments using snapshots in KVM becomes nearly impossible. If 
you have tens or hundreds concurrent snapshots taking place you will have a 
bunch of timeouts and errors, your network becomes clogged, etc. In addition, 
using these snapshots for creating new volumes or reverting back vms also slow 
and inefficient. As above, when you have tens or hundreds concurrent operations 
it will not succeed and you will have a majority of tasks with errors or 
timeouts. 

At the moment, taking a single snapshot of relatively small volumes (200GB or 
500GB for instance) takes tens if not hundreds of minutes. Taking a snapshot of 
the same volume on ceph primary storage takes a few seconds at most! Similarly, 
converting a snapshot to a volume takes tens if not hundreds of minutes when 
secondary storage is involved; compared with seconds if done directly on the 
primary storage. 

I suggest that the CloudStack should have the ability to keep volume snapshots 
on the primary storage where this is supported by the storage. Perhaps having a 
per primary storage setting that enables this functionality. This will be 
beneficial for Ceph primary storage on KVM hypervisors and perhaps on XenServer 
when Ceph will be supported in a near future. 

This will greatly speed up the process of using snapshots on KVM and users will 
actually start using snapshotting rather than giving up with frustration. 

I have opened the ticket CLOUDSTACK-8256, so please cast your vote if you are 
in agreement. 

Thanks for your input 

Andrei 

Re: Disable HA temporary ?

[Wido den Hollander]

On 16-02-15 11:00, Andrija Panic wrote:
> Hi team,
> 
> I just had funny behaviour few days ago - one of my hosts was under heavy
> load (some disk/network load) and it went disconnected from MGMT server.
> 
> Then MGMT server stared doing HA thing, but without being able to make sure
> that the VMs on the disconnected hosts are really shutdown (and they were
> NOT).
> 
> So MGMT started again some VMs on other hosts, thus resulting in having 2
> copies of the same VM, using shared strage  - so corruption happened on the
> disk.
> 
> Is there a way to temporary disable HA feature on global level, or anything
> similar ?

Not that I'm aware of, but this is something I also ran in to a couple
of times.

It would indeed be nice if there could be a way to stop the HA process
completely as an Admin.

Wido

> Thanks
> 

[GitHub] cloudstack pull request: One of the routers is not running, so we ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/cloudstack/pull/77


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

redundant vpc merge

[Daan Hoogland]

H,

This is a pre-announcement;) I want to rebase our code this week and
then merge into master. This code is functionally tested against xen.
It is bound to be less perfect in other peoples environments so please
test if you have doubts. We (Wilder, Ian, me) will support after the
merge but let's prevent issues rather then solve them.

regards,
-- 
Daan

All-about jobids

[Radu Stefanache]

Hello everyone,

I am trying to understand a couple of things but since I am not a Java
developer I need your help .

1. How are the jobids generated ? Based on what ?

I presume that after a new job is spawned and a new id generated, that goes
in the database .
I also assume that if you have multiple management servers, in different
locations, and you have some latency you can end up with duplicate job ids
or no jobs at all (?)

2. If you have multiple management servers, let's say in the same
datacenter so we take out the latency factor, how are these servers sharing
the jobs ? Is there any mechanism of who is processing what ?

If during an upgrade you have in a pool management servers of different
versions (bring down old version server, patch, bring up..basic procedure)
will the jobs get affected ?


Thanks a lot!

Cloudstack development architect wanted

[Octavian Popescu]

Ladies & gents,

Interoute is looking for a top-notch Cloudstack developer to join our rapidly 
expanding team in London. More details here - 
http://www.interoutejobs.com/vacancies/vacancy-cloudstack-development-architect-411928-31.html
 , feel free to drop me a line directly if I can help with additional details 
about the role.

Thanks,
Octavian

Disable HA temporary ?

[Andrija Panic]

Hi team,

I just had funny behaviour few days ago - one of my hosts was under heavy
load (some disk/network load) and it went disconnected from MGMT server.

Then MGMT server stared doing HA thing, but without being able to make sure
that the VMs on the disconnected hosts are really shutdown (and they were
NOT).

So MGMT started again some VMs on other hosts, thus resulting in having 2
copies of the same VM, using shared strage  - so corruption happened on the
disk.

Is there a way to temporary disable HA feature on global level, or anything
similar ?
Thanks

-- 

Andrija Panić

[GitHub] cloudstack pull request: CLOUDSTACK-7908: Add user_id column to vm...

[bhaisaab]

Github user bhaisaab closed the pull request at:

https://github.com/apache/cloudstack/pull/48


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---