By the way, in case you're looking at the diff and wondering why I took out
a StopCommand check and call to execute in LibvirtComputingResource, there
were two of them.
- } else if (cmd instanceof StopCommand) {
- return execute((StopCommand) cmd);
On Thu, Oct 24, 2013 at 5:26 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:
> This is a newer link, Marcus:
>
>
> https://github.com/mike-tutkowski/incubator-cloudstack/commit/e84de7577ad38fea88d1492d4949672d1989889b
>
> I just rebased on top of master today.
>
>
> On Thu, Oct 24, 2013 at 3:46 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> Hey Marcus,
>>
>> If you're interested in running the simulator against your code + my
>> code, I have it on GitHub here:
>>
>>
>> https://github.com/mike-tutkowski/incubator-cloudstack/commit/776f3eb9dda45745f43e6765b026d34fbc6be072
>>
>> Thanks!
>>
>>
>> On Wed, Oct 23, 2013 at 11:06 PM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com> wrote:
>>
>>> I'm trying to attach my diff to https://reviews.apache.org/r/13865/,
>>> but I don't see the necessary buttons.
>>>
>>> I wonder if I need to get edit access back again? We had trouble with
>>> the Wiki. Was this also impacted?
>>>
>>>
>>> On Wed, Oct 23, 2013 at 10:47 PM, Mike Tutkowski <
>>> mike.tutkow...@solidfire.com> wrote:
>>>
>>>> Sure, I can create a diff file and attach it to Review Board.
>>>>
>>>>
>>>> On Wed, Oct 23, 2013 at 10:40 PM, Marcus Sorensen
>>>> <shadow...@gmail.com>wrote:
>>>>
>>>>> Sure. The majority of it only affects people who are on your storage
>>>>> anyway. Perhaps you can post a patch and I can run it through the
>>>>> simulator to verify that the minor change to the existing code hasn't
>>>>> broken the standard storages. I don't think it is since I've
>>>>> thoroughly tested the code I posted, but I know there were some
>>>>> additional changes.
>>>>>
>>>>> On Wed, Oct 23, 2013 at 10:35 PM, Mike Tutkowski
>>>>> <mike.tutkow...@solidfire.com> wrote:
>>>>> > OK, Marcus, I made the change to detect my volumes and it seems to
>>>>> work just
>>>>> > fine.
>>>>> >
>>>>> > Perhaps another day of testing and we can check this code in. What
>>>>> do you
>>>>> > think?
>>>>> >
>>>>> >
>>>>> > On Wed, Oct 23, 2013 at 9:14 PM, Mike Tutkowski
>>>>> > <mike.tutkow...@solidfire.com> wrote:
>>>>> >>
>>>>> >> Thanks, Marcus...I hadn't read that note, but that makes sense.
>>>>> >>
>>>>> >> Yes, that must be the root disk for the VM. I can put in code, as
>>>>> you
>>>>> >> recommend, to handle only my volumes.
>>>>> >>
>>>>> >>
>>>>> >> On Wed, Oct 23, 2013 at 5:37 PM, Marcus Sorensen <
>>>>> shadow...@gmail.com>
>>>>> >> wrote:
>>>>> >>>
>>>>> >>> It should be sending the path info for each disk per the XML of the
>>>>> >>> VM... so it will send all disks regardless of whether or not your
>>>>> >>> adaptor manages that disk, and it's up to your adaptor to ignore
>>>>> any
>>>>> >>> that aren't managed by it. There should be notes to that effect in
>>>>> the
>>>>> >>> code near the disconnectPhysicalDisk interface in StorageAdaptor:
>>>>> >>>
>>>>> >>> // given local path to file/device (per Libvirt XML), 1) check
>>>>> >>> that device is
>>>>> >>> // handled by your adaptor, return false if not. 2) clean up
>>>>> >>> device, return true
>>>>> >>> public boolean disconnectPhysicalDiskByPath(String localPath);
>>>>> >>>
>>>>> >>> Since we only have XML disk definitions when we stop or migrate a
>>>>> VM,
>>>>> >>> we have to try all adaptors against all defined disks. So in your
>>>>> >>> disconnectPhysicalDisk you might do something like check that the
>>>>> path
>>>>> >>> starts with '/dev/disk/by-path' and contains 'iscs-iqn' (maybe
>>>>> there's
>>>>> >>> some way that's more robust like checking the full path against a
>>>>> lun
>>>>> >>> listing or something). If it doesn't match, then your
>>>>> >>> disconnectPhysicalDisk just does nothing.
>>>>> >>>
>>>>> >>> I assume this is a root disk or some other local storage disk. If
>>>>> it's
>>>>> >>> not, then your VM XML is messed up somehow.
>>>>> >>>
>>>>> >>> On Wed, Oct 23, 2013 at 5:01 PM, Mike Tutkowski
>>>>> >>> <mike.tutkow...@solidfire.com> wrote:
>>>>> >>> > I found the problem.
>>>>> >>> >
>>>>> >>> > disconnectPhysicalDiskByPath is being passed in (in my
>>>>> situation) the
>>>>> >>> > following:
>>>>> >>> >
>>>>> >>> > /var/lib/libvirt/images/9887d511-8dc7-4cb4-96f9-01230fe4bbb6
>>>>> >>> >
>>>>> >>> > Due to the name of the method, my code was expecting data such
>>>>> as the
>>>>> >>> > following:
>>>>> >>> >
>>>>> >>> >
>>>>> >>> >
>>>>> /dev/disk/by-path/ip-192.168.233.10:3260-iscsi-iqn.2012-03.com.solidfire:volume1-lun-0
>>>>> >>> >
>>>>> >>> > Was it intentional to send the data into this method in the
>>>>> current
>>>>> >>> > way?
>>>>> >>> >
>>>>> >>> >
>>>>> >>> > On Wed, Oct 23, 2013 at 1:57 PM, Mike Tutkowski
>>>>> >>> > <mike.tutkow...@solidfire.com> wrote:
>>>>> >>> >>
>>>>> >>> >> You know, I forgot we supposed to be doing that! :)
>>>>> Multi-tasking too
>>>>> >>> >> much
>>>>> >>> >> today, I guess.
>>>>> >>> >>
>>>>> >>> >> Anyways, it must not be working because I still had a hypervisor
>>>>> >>> >> connection after I shut down the VM.
>>>>> >>> >>
>>>>> >>> >> Let me investigate.
>>>>> >>> >>
>>>>> >>> >>
>>>>> >>> >> On Wed, Oct 23, 2013 at 1:48 PM, Marcus Sorensen <
>>>>> shadow...@gmail.com>
>>>>> >>> >> wrote:
>>>>> >>> >>>
>>>>> >>> >>> Are we not disconnecting when we stop the vm? There's a method
>>>>> for
>>>>> >>> >>> it, we
>>>>> >>> >>> should be. disconnectPhysicalDiskViaVmSpec
>>>>> >>> >>>
>>>>> >>> >>> On Oct 23, 2013 1:28 PM, "Mike Tutkowski"
>>>>> >>> >>> <mike.tutkow...@solidfire.com>
>>>>> >>> >>> wrote:
>>>>> >>> >>>>
>>>>> >>> >>>> I see one problem for us now, Marcus.
>>>>> >>> >>>>
>>>>> >>> >>>> * You have a running VM that you attach a volume to.
>>>>> >>> >>>> * You stop the VM.
>>>>> >>> >>>> * You detach the volume.
>>>>> >>> >>>> * You start up the VM.
>>>>> >>> >>>>
>>>>> >>> >>>> The VM will not be connected to the volume (which is good),
>>>>> but the
>>>>> >>> >>>> hypervisor will still be connected to the volume.
>>>>> >>> >>>>
>>>>> >>> >>>> It would be great if we actually sent a command to the last
>>>>> host ID
>>>>> >>> >>>> of
>>>>> >>> >>>> the stopped VM when detaching a volume (to have the hypervisor
>>>>> >>> >>>> disconnect
>>>>> >>> >>>> from the volume).
>>>>> >>> >>>>
>>>>> >>> >>>> What do you think about that?
>>>>> >>> >>>>
>>>>> >>> >>>>
>>>>> >>> >>>> On Wed, Oct 23, 2013 at 1:15 PM, Mike Tutkowski
>>>>> >>> >>>> <mike.tutkow...@solidfire.com> wrote:
>>>>> >>> >>>>>
>>>>> >>> >>>>> OK, whatever way you prefer then, Marcus (createVdb first or
>>>>> >>> >>>>> second).
>>>>> >>> >>>>>
>>>>> >>> >>>>> If I leave createVdb first and return 0, it does seem to
>>>>> work.
>>>>> >>> >>>>>
>>>>> >>> >>>>>
>>>>> >>> >>>>> On Wed, Oct 23, 2013 at 11:13 AM, Marcus Sorensen
>>>>> >>> >>>>> <shadow...@gmail.com>
>>>>> >>> >>>>> wrote:
>>>>> >>> >>>>>>
>>>>> >>> >>>>>> I think we could flip-flop these two lines if necessary:
>>>>> >>> >>>>>>
>>>>> >>> >>>>>> createVbd(conn, vmSpec, vmName, vm);
>>>>> >>> >>>>>>
>>>>> >>> >>>>>>
>>>>> _storagePoolMgr.connectPhysicalDisksViaVmSpec(vmSpec);
>>>>> >>> >>>>>>
>>>>> >>> >>>>>> I haven't actually tried it though. But in general I don't
>>>>> see the
>>>>> >>> >>>>>> Libvirt DiskDef using size at all, which is what createVbd
>>>>> does
>>>>> >>> >>>>>> (creates XML definitions for disks to attach to the VM
>>>>> >>> >>>>>> definition). It
>>>>> >>> >>>>>> just takes the device at it's native advertised size when it
>>>>> >>> >>>>>> actually
>>>>> >>> >>>>>> goes to use it.
>>>>> >>> >>>>>>
>>>>> >>> >>>>>> On Wed, Oct 23, 2013 at 10:52 AM, Mike Tutkowski
>>>>> >>> >>>>>> <mike.tutkow...@solidfire.com> wrote:
>>>>> >>> >>>>>> > Little problem that I wanted to get your take on, Marcus.
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> > When a VM is being started, we call createVdb before
>>>>> calling
>>>>> >>> >>>>>> > connectPhysicalDisksViaVmSpec.
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> > The problem is that createVdb calls getPhysicalDisk and my
>>>>> >>> >>>>>> > volume
>>>>> >>> >>>>>> > has not
>>>>> >>> >>>>>> > yet been connected because connectPhysicalDisksViaVmSpec
>>>>> has not
>>>>> >>> >>>>>> > yet
>>>>> >>> >>>>>> > been
>>>>> >>> >>>>>> > called.
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> > When I try to read up the size of the disk to populate a
>>>>> >>> >>>>>> > PhysicalDisk, I get
>>>>> >>> >>>>>> > an error, of course, because the path does not yet exist.
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> > I could populate a 0 for the size of the physical disk
>>>>> and then
>>>>> >>> >>>>>> > the
>>>>> >>> >>>>>> > next
>>>>> >>> >>>>>> > time getPhysicalDisk is called, it should be filled in
>>>>> with a
>>>>> >>> >>>>>> > proper
>>>>> >>> >>>>>> > size.
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> > Do you see a problem with that approach?
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> > On Tue, Oct 22, 2013 at 6:40 PM, Marcus Sorensen
>>>>> >>> >>>>>> > <shadow...@gmail.com>
>>>>> >>> >>>>>> > wrote:
>>>>> >>> >>>>>> >>
>>>>> >>> >>>>>> >> That's right. All should be well.
>>>>> >>> >>>>>> >>
>>>>> >>> >>>>>> >> On Oct 22, 2013 6:03 PM, "Mike Tutkowski"
>>>>> >>> >>>>>> >> <mike.tutkow...@solidfire.com>
>>>>> >>> >>>>>> >> wrote:
>>>>> >>> >>>>>> >>>
>>>>> >>> >>>>>> >>> Looks like we disconnect physical disks when the VM is
>>>>> >>> >>>>>> >>> stopped.
>>>>> >>> >>>>>> >>>
>>>>> >>> >>>>>> >>> I didn't see that before.
>>>>> >>> >>>>>> >>>
>>>>> >>> >>>>>> >>> I suppose that means the disks are physically
>>>>> disconnected
>>>>> >>> >>>>>> >>> when
>>>>> >>> >>>>>> >>> the VM is
>>>>> >>> >>>>>> >>> stopped, but the CloudStack DB still has the VM
>>>>> associated
>>>>> >>> >>>>>> >>> with
>>>>> >>> >>>>>> >>> the disks
>>>>> >>> >>>>>> >>> for the next time the VM may be started up (unless
>>>>> someone
>>>>> >>> >>>>>> >>> does a
>>>>> >>> >>>>>> >>> disconnect
>>>>> >>> >>>>>> >>> while the VM is in the Stopped State).
>>>>> >>> >>>>>> >>>
>>>>> >>> >>>>>> >>>
>>>>> >>> >>>>>> >>> On Tue, Oct 22, 2013 at 4:19 PM, Mike Tutkowski
>>>>> >>> >>>>>> >>> <mike.tutkow...@solidfire.com> wrote:
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> Hey Marcus,
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> Quick question for you related to attaching/detaching
>>>>> volumes
>>>>> >>> >>>>>> >>>> when the
>>>>> >>> >>>>>> >>>> VM is in the Stopped State.
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> If I detach a volume from a VM that is in the Stopped
>>>>> State,
>>>>> >>> >>>>>> >>>> the
>>>>> >>> >>>>>> >>>> DB
>>>>> >>> >>>>>> >>>> seems to get updated, but I don't see a command going
>>>>> to the
>>>>> >>> >>>>>> >>>> KVM
>>>>> >>> >>>>>> >>>> hypervisor
>>>>> >>> >>>>>> >>>> that leads to the removal of the iSCSI target.
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> It seems the iSCSI target is only removed the next
>>>>> time the
>>>>> >>> >>>>>> >>>> VM is
>>>>> >>> >>>>>> >>>> started.
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> Do you know if this is true?
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> If it is, I'm concerned that the volume could be
>>>>> attached to
>>>>> >>> >>>>>> >>>> another VM
>>>>> >>> >>>>>> >>>> before the Stopped VM is re-started and when the
>>>>> Stopped VM
>>>>> >>> >>>>>> >>>> gets
>>>>> >>> >>>>>> >>>> restarted
>>>>> >>> >>>>>> >>>> that it would disconnect the iSCSI volume from
>>>>> underneath the
>>>>> >>> >>>>>> >>>> VM
>>>>> >>> >>>>>> >>>> that now
>>>>> >>> >>>>>> >>>> has the volume attached.
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> I still want to perform some tests on this, but am
>>>>> first
>>>>> >>> >>>>>> >>>> trying
>>>>> >>> >>>>>> >>>> to get a
>>>>> >>> >>>>>> >>>> VM to start after I've attached a volume to it when it
>>>>> was in
>>>>> >>> >>>>>> >>>> the
>>>>> >>> >>>>>> >>>> Stopped
>>>>> >>> >>>>>> >>>> State.
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> Thanks,
>>>>> >>> >>>>>> >>>> Mike
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> On Mon, Oct 21, 2013 at 10:57 PM, Mike Tutkowski
>>>>> >>> >>>>>> >>>> <mike.tutkow...@solidfire.com> wrote:
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> Thanks for that info, Marcus.
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> By the way, I wanted to see if I could attach my
>>>>> volume to a
>>>>> >>> >>>>>> >>>>> VM
>>>>> >>> >>>>>> >>>>> in the
>>>>> >>> >>>>>> >>>>> Stopped State.
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> The attach logic didn't trigger any exceptions;
>>>>> however,
>>>>> >>> >>>>>> >>>>> when I
>>>>> >>> >>>>>> >>>>> started
>>>>> >>> >>>>>> >>>>> the VM, I received an Insufficient Capacity exception.
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> If I detach the volume and then start the VM, the VM
>>>>> starts
>>>>> >>> >>>>>> >>>>> just
>>>>> >>> >>>>>> >>>>> fine.
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> I noticed a problem here (in StoragePoolHostDaoImpl):
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> @Override
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> public StoragePoolHostVO findByPoolHost(long
>>>>> poolId,
>>>>> >>> >>>>>> >>>>> long
>>>>> >>> >>>>>> >>>>> hostId) {
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> SearchCriteria<StoragePoolHostVO> sc =
>>>>> >>> >>>>>> >>>>> PoolHostSearch.create();
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> sc.setParameters("pool_id", poolId);
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> sc.setParameters("host_id", hostId);
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> return findOneIncludingRemovedBy(sc);
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> }
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> The findOneIncludingRemovedBy method returns null (the
>>>>> >>> >>>>>> >>>>> poolId is
>>>>> >>> >>>>>> >>>>> my
>>>>> >>> >>>>>> >>>>> storage pool's ID and the hostId is the expected host
>>>>> ID).
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> I'm not sure what this method is trying to do.
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> I looked in the storage_pool_host_ref table (is that
>>>>> the
>>>>> >>> >>>>>> >>>>> correct
>>>>> >>> >>>>>> >>>>> table?) and it only has one row, which maps the local
>>>>> >>> >>>>>> >>>>> storage
>>>>> >>> >>>>>> >>>>> pool of the
>>>>> >>> >>>>>> >>>>> KVM host to the KVM host (which explains why no match
>>>>> is
>>>>> >>> >>>>>> >>>>> found
>>>>> >>> >>>>>> >>>>> for my
>>>>> >>> >>>>>> >>>>> situation).
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> Do you understand what this logic is trying to do?
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> Thanks!
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> On Mon, Oct 21, 2013 at 8:08 PM, Marcus Sorensen
>>>>> >>> >>>>>> >>>>> <shadow...@gmail.com>
>>>>> >>> >>>>>> >>>>> wrote:
>>>>> >>> >>>>>> >>>>>>
>>>>> >>> >>>>>> >>>>>> Do you have the capability to clone the root disk?
>>>>> Normally
>>>>> >>> >>>>>> >>>>>> the
>>>>> >>> >>>>>> >>>>>> template is installed to primary, and then cloned
>>>>> for each
>>>>> >>> >>>>>> >>>>>> root
>>>>> >>> >>>>>> >>>>>> disk.
>>>>> >>> >>>>>> >>>>>> In some cases (such as CLVM), this isn't efficient
>>>>> and so
>>>>> >>> >>>>>> >>>>>> the
>>>>> >>> >>>>>> >>>>>> template
>>>>> >>> >>>>>> >>>>>> is copied fresh to populate each root disk.
>>>>> >>> >>>>>> >>>>>>
>>>>> >>> >>>>>> >>>>>> I'm actually not 100% sure how this works in the new
>>>>> code.
>>>>> >>> >>>>>> >>>>>> It
>>>>> >>> >>>>>> >>>>>> used to
>>>>> >>> >>>>>> >>>>>> be handled by copyPhysicalDisk in the storage
>>>>> adaptor,
>>>>> >>> >>>>>> >>>>>> called
>>>>> >>> >>>>>> >>>>>> by
>>>>> >>> >>>>>> >>>>>> copyTemplateToPrimaryStorage, which runs on the
>>>>> agent. It
>>>>> >>> >>>>>> >>>>>> would
>>>>> >>> >>>>>> >>>>>> pass
>>>>> >>> >>>>>> >>>>>> template/secondary storage info, and the destination
>>>>> >>> >>>>>> >>>>>> volume/primary
>>>>> >>> >>>>>> >>>>>> storage info, and copyPhysicalDisk would do the work
>>>>> of
>>>>> >>> >>>>>> >>>>>> installing the
>>>>> >>> >>>>>> >>>>>> image to the destination. Then subsequent root
>>>>> disks would
>>>>> >>> >>>>>> >>>>>> be
>>>>> >>> >>>>>> >>>>>> cloned
>>>>> >>> >>>>>> >>>>>> in CreateCommand by calling createDiskFromTemplate.
>>>>> >>> >>>>>> >>>>>>
>>>>> >>> >>>>>> >>>>>> In master it looks like this was moved to
>>>>> >>> >>>>>> >>>>>> KVMStorageProcessor
>>>>> >>> >>>>>> >>>>>> 'cloneVolumeFromBaseTemplate', although I think this
>>>>> just
>>>>> >>> >>>>>> >>>>>> takes
>>>>> >>> >>>>>> >>>>>> over
>>>>> >>> >>>>>> >>>>>> as default, and there's something in your storage
>>>>> driver
>>>>> >>> >>>>>> >>>>>> that
>>>>> >>> >>>>>> >>>>>> should
>>>>> >>> >>>>>> >>>>>> be capable of cloning templates on the mgmt server
>>>>> side.
>>>>> >>> >>>>>> >>>>>> I'm
>>>>> >>> >>>>>> >>>>>> less sure
>>>>> >>> >>>>>> >>>>>> about how the template gets to primary storage in
>>>>> the first
>>>>> >>> >>>>>> >>>>>> place, I
>>>>> >>> >>>>>> >>>>>> assume copyTemplateToPrimaryStorage in
>>>>> KVMStorageProcessor
>>>>> >>> >>>>>> >>>>>> calling
>>>>> >>> >>>>>> >>>>>> copyPhysicalDisk in your adaptor. It's a bit tough
>>>>> for me
>>>>> >>> >>>>>> >>>>>> to
>>>>> >>> >>>>>> >>>>>> tell
>>>>> >>> >>>>>> >>>>>> since our earlier storage adaptor did everything on
>>>>> the
>>>>> >>> >>>>>> >>>>>> host it
>>>>> >>> >>>>>> >>>>>> mostly
>>>>> >>> >>>>>> >>>>>> just worked with the default stuff.
>>>>> >>> >>>>>> >>>>>>
>>>>> >>> >>>>>> >>>>>> On Mon, Oct 21, 2013 at 4:49 PM, Mike Tutkowski
>>>>> >>> >>>>>> >>>>>> <mike.tutkow...@solidfire.com> wrote:
>>>>> >>> >>>>>> >>>>>> > Hey Marcus,
>>>>> >>> >>>>>> >>>>>> >
>>>>> >>> >>>>>> >>>>>> > So...now that this works well for data disks, I was
>>>>> >>> >>>>>> >>>>>> > wondering
>>>>> >>> >>>>>> >>>>>> > what
>>>>> >>> >>>>>> >>>>>> > might be
>>>>> >>> >>>>>> >>>>>> > involved in getting this process to work for root
>>>>> disks.
>>>>> >>> >>>>>> >>>>>> >
>>>>> >>> >>>>>> >>>>>> > Can you point me in the right direction as far as
>>>>> what
>>>>> >>> >>>>>> >>>>>> > gets
>>>>> >>> >>>>>> >>>>>> > invoked
>>>>> >>> >>>>>> >>>>>> > when a
>>>>> >>> >>>>>> >>>>>> > VM is being created on KVM (so that its root disk
>>>>> can be
>>>>> >>> >>>>>> >>>>>> > created and
>>>>> >>> >>>>>> >>>>>> > the
>>>>> >>> >>>>>> >>>>>> > necessary template laid down or ISO installed)?
>>>>> >>> >>>>>> >>>>>> >
>>>>> >>> >>>>>> >>>>>> > Thanks!
>>>>> >>> >>>>>> >>>>>> >
>>>>> >>> >>>>>> >>>>>> >
>>>>> >>> >>>>>> >>>>>> > On Mon, Oct 21, 2013 at 1:14 PM, Mike Tutkowski
>>>>> >>> >>>>>> >>>>>> > <mike.tutkow...@solidfire.com> wrote:
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> Hey Marcus,
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> Just wanted to let you know the branch of mine
>>>>> that has
>>>>> >>> >>>>>> >>>>>> >> your
>>>>> >>> >>>>>> >>>>>> >> code
>>>>> >>> >>>>>> >>>>>> >> and mine
>>>>> >>> >>>>>> >>>>>> >> appears to work well with regards to attaching a
>>>>> data
>>>>> >>> >>>>>> >>>>>> >> disk
>>>>> >>> >>>>>> >>>>>> >> to a
>>>>> >>> >>>>>> >>>>>> >> running VM:
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> fdisk -l from hypervisor:
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> http://i.imgur.com/NkP5fo0.png
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> fdisk -l from within VM:
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> http://i.imgur.com/8YwiiC7.png
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> I plan to do more testing on this over the coming
>>>>> days.
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> If all goes well, perhaps we can check this code
>>>>> in by
>>>>> >>> >>>>>> >>>>>> >> the
>>>>> >>> >>>>>> >>>>>> >> end of
>>>>> >>> >>>>>> >>>>>> >> the
>>>>> >>> >>>>>> >>>>>> >> week?
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> Talk to you later,
>>>>> >>> >>>>>> >>>>>> >> Mike
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> On Sun, Oct 20, 2013 at 10:23 PM, Mike Tutkowski
>>>>> >>> >>>>>> >>>>>> >> <mike.tutkow...@solidfire.com> wrote:
>>>>> >>> >>>>>> >>>>>> >>>
>>>>> >>> >>>>>> >>>>>> >>> Don't ask me, but it works now (I've been having
>>>>> this
>>>>> >>> >>>>>> >>>>>> >>> trouble
>>>>> >>> >>>>>> >>>>>> >>> quite a
>>>>> >>> >>>>>> >>>>>> >>> while today).
>>>>> >>> >>>>>> >>>>>> >>>
>>>>> >>> >>>>>> >>>>>> >>> I guess the trick is to send you an e-mail. :)
>>>>> >>> >>>>>> >>>>>> >>>
>>>>> >>> >>>>>> >>>>>> >>>
>>>>> >>> >>>>>> >>>>>> >>> On Sun, Oct 20, 2013 at 10:05 PM, Marcus Sorensen
>>>>> >>> >>>>>> >>>>>> >>> <shadow...@gmail.com>
>>>>> >>> >>>>>> >>>>>> >>> wrote:
>>>>> >>> >>>>>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>>>> >>>> Did you create a service offering that uses
>>>>> local
>>>>> >>> >>>>>> >>>>>> >>>> storage,
>>>>> >>> >>>>>> >>>>>> >>>> or add
>>>>> >>> >>>>>> >>>>>> >>>> a
>>>>> >>> >>>>>> >>>>>> >>>> shared primary storage? By default there is no
>>>>> storage
>>>>> >>> >>>>>> >>>>>> >>>> that
>>>>> >>> >>>>>> >>>>>> >>>> matches the
>>>>> >>> >>>>>> >>>>>> >>>> built in offerings.
>>>>> >>> >>>>>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>>>> >>>> On Oct 20, 2013 9:39 PM, "Mike Tutkowski"
>>>>> >>> >>>>>> >>>>>> >>>> <mike.tutkow...@solidfire.com>
>>>>> >>> >>>>>> >>>>>> >>>> wrote:
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>> Hey Marcus,
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>> So, I went back to the branch of mine that has
>>>>> your
>>>>> >>> >>>>>> >>>>>> >>>>> code
>>>>> >>> >>>>>> >>>>>> >>>>> and
>>>>> >>> >>>>>> >>>>>> >>>>> mine and
>>>>> >>> >>>>>> >>>>>> >>>>> was able to create a new CloudStack install
>>>>> from
>>>>> >>> >>>>>> >>>>>> >>>>> scratch
>>>>> >>> >>>>>> >>>>>> >>>>> with it
>>>>> >>> >>>>>> >>>>>> >>>>> (once
>>>>> >>> >>>>>> >>>>>> >>>>> again, after manually deleting what was in
>>>>> >>> >>>>>> >>>>>> >>>>> /var/lib/libvirt/images to the
>>>>> >>> >>>>>> >>>>>> >>>>> get system VMs to start).
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>> Anyways, my system VMs are running now and I
>>>>> tried to
>>>>> >>> >>>>>> >>>>>> >>>>> kick off a
>>>>> >>> >>>>>> >>>>>> >>>>> VM
>>>>> >>> >>>>>> >>>>>> >>>>> using the CentOS 6.3 image you provided me a
>>>>> while
>>>>> >>> >>>>>> >>>>>> >>>>> back.
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>> The virtual router has a Status of Running;
>>>>> however,
>>>>> >>> >>>>>> >>>>>> >>>>> my
>>>>> >>> >>>>>> >>>>>> >>>>> VM fails
>>>>> >>> >>>>>> >>>>>> >>>>> to
>>>>> >>> >>>>>> >>>>>> >>>>> start (with the generic message of Insufficient
>>>>> >>> >>>>>> >>>>>> >>>>> Capacity).
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>> I've not seen this exception before (related
>>>>> to the
>>>>> >>> >>>>>> >>>>>> >>>>> VR).
>>>>> >>> >>>>>> >>>>>> >>>>> Do you
>>>>> >>> >>>>>> >>>>>> >>>>> have
>>>>> >>> >>>>>> >>>>>> >>>>> any insight into this?:
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.exception.ResourceUnavailableException:
>>>>> >>> >>>>>> >>>>>> >>>>> Resource
>>>>> >>> >>>>>> >>>>>> >>>>> [Pod:1] is
>>>>> >>> >>>>>> >>>>>> >>>>> unreachable: Unable to apply userdata and
>>>>> password
>>>>> >>> >>>>>> >>>>>> >>>>> entry
>>>>> >>> >>>>>> >>>>>> >>>>> on
>>>>> >>> >>>>>> >>>>>> >>>>> router
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3793)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyUserData(VirtualNetworkApplianceManagerImpl.java:3017)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.network.element.VirtualRouterElement.addPasswordAndUserdata(VirtualRouterElement.java:933)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareElement(NetworkOrchestrator.java:1172)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepareNic(NetworkOrchestrator.java:1288)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.prepare(NetworkOrchestrator.java:1224)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:826)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:508)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3338)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2919)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2905)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:421)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$1.runInContext(AsyncJobManagerImpl.java:532)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>> >>> >>>>>> >>>>>> >>>>> at
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>> >>> >>>>>> >>>>>> >>>>> at java.lang.Thread.run(Thread.java:724)
>>>>> >>> >>>>>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>> >>>>> Thanks!
>>>>> >>> >>>>>> >>>>>> >>>
>>>>> >>> >>>>>> >>>>>> >>>
>>>>> >>> >>>>>> >>>>>> >>>
>>>>> >>> >>>>>> >>>>>> >>>
>>>>> >>> >>>>>> >>>>>> >>> --
>>>>> >>> >>>>>> >>>>>> >>> Mike Tutkowski
>>>>> >>> >>>>>> >>>>>> >>> Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >>>>>> >>>>>> >>> e: mike.tutkow...@solidfire.com
>>>>> >>> >>>>>> >>>>>> >>> o: 303.746.7302
>>>>> >>> >>>>>> >>>>>> >>> Advancing the way the world uses the cloud™
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >>
>>>>> >>> >>>>>> >>>>>> >> --
>>>>> >>> >>>>>> >>>>>> >> Mike Tutkowski
>>>>> >>> >>>>>> >>>>>> >> Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >>>>>> >>>>>> >> e: mike.tutkow...@solidfire.com
>>>>> >>> >>>>>> >>>>>> >> o: 303.746.7302
>>>>> >>> >>>>>> >>>>>> >> Advancing the way the world uses the cloud™
>>>>> >>> >>>>>> >>>>>> >
>>>>> >>> >>>>>> >>>>>> >
>>>>> >>> >>>>>> >>>>>> >
>>>>> >>> >>>>>> >>>>>> >
>>>>> >>> >>>>>> >>>>>> > --
>>>>> >>> >>>>>> >>>>>> > Mike Tutkowski
>>>>> >>> >>>>>> >>>>>> > Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >>>>>> >>>>>> > e: mike.tutkow...@solidfire.com
>>>>> >>> >>>>>> >>>>>> > o: 303.746.7302
>>>>> >>> >>>>>> >>>>>> > Advancing the way the world uses the cloud™
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>>
>>>>> >>> >>>>>> >>>>> --
>>>>> >>> >>>>>> >>>>> Mike Tutkowski
>>>>> >>> >>>>>> >>>>> Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >>>>>> >>>>> e: mike.tutkow...@solidfire.com
>>>>> >>> >>>>>> >>>>> o: 303.746.7302
>>>>> >>> >>>>>> >>>>> Advancing the way the world uses the cloud™
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>>
>>>>> >>> >>>>>> >>>> --
>>>>> >>> >>>>>> >>>> Mike Tutkowski
>>>>> >>> >>>>>> >>>> Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >>>>>> >>>> e: mike.tutkow...@solidfire.com
>>>>> >>> >>>>>> >>>> o: 303.746.7302
>>>>> >>> >>>>>> >>>> Advancing the way the world uses the cloud™
>>>>> >>> >>>>>> >>>
>>>>> >>> >>>>>> >>>
>>>>> >>> >>>>>> >>>
>>>>> >>> >>>>>> >>>
>>>>> >>> >>>>>> >>> --
>>>>> >>> >>>>>> >>> Mike Tutkowski
>>>>> >>> >>>>>> >>> Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >>>>>> >>> e: mike.tutkow...@solidfire.com
>>>>> >>> >>>>>> >>> o: 303.746.7302
>>>>> >>> >>>>>> >>> Advancing the way the world uses the cloud™
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> >
>>>>> >>> >>>>>> > --
>>>>> >>> >>>>>> > Mike Tutkowski
>>>>> >>> >>>>>> > Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >>>>>> > e: mike.tutkow...@solidfire.com
>>>>> >>> >>>>>> > o: 303.746.7302
>>>>> >>> >>>>>> > Advancing the way the world uses the cloud™
>>>>> >>> >>>>>
>>>>> >>> >>>>>
>>>>> >>> >>>>>
>>>>> >>> >>>>>
>>>>> >>> >>>>> --
>>>>> >>> >>>>> Mike Tutkowski
>>>>> >>> >>>>> Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >>>>> e: mike.tutkow...@solidfire.com
>>>>> >>> >>>>> o: 303.746.7302
>>>>> >>> >>>>> Advancing the way the world uses the cloud™
>>>>> >>> >>>>
>>>>> >>> >>>>
>>>>> >>> >>>>
>>>>> >>> >>>>
>>>>> >>> >>>> --
>>>>> >>> >>>> Mike Tutkowski
>>>>> >>> >>>> Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >>>> e: mike.tutkow...@solidfire.com
>>>>> >>> >>>> o: 303.746.7302
>>>>> >>> >>>> Advancing the way the world uses the cloud™
>>>>> >>> >>
>>>>> >>> >>
>>>>> >>> >>
>>>>> >>> >>
>>>>> >>> >> --
>>>>> >>> >> Mike Tutkowski
>>>>> >>> >> Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> >> e: mike.tutkow...@solidfire.com
>>>>> >>> >> o: 303.746.7302
>>>>> >>> >> Advancing the way the world uses the cloud™
>>>>> >>> >
>>>>> >>> >
>>>>> >>> >
>>>>> >>> >
>>>>> >>> > --
>>>>> >>> > Mike Tutkowski
>>>>> >>> > Senior CloudStack Developer, SolidFire Inc.
>>>>> >>> > e: mike.tutkow...@solidfire.com
>>>>> >>> > o: 303.746.7302
>>>>> >>> > Advancing the way the world uses the cloud™
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Mike Tutkowski
>>>>> >> Senior CloudStack Developer, SolidFire Inc.
>>>>> >> e: mike.tutkow...@solidfire.com
>>>>> >> o: 303.746.7302
>>>>> >> Advancing the way the world uses the cloud™
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Mike Tutkowski
>>>>> > Senior CloudStack Developer, SolidFire Inc.
>>>>> > e: mike.tutkow...@solidfire.com
>>>>> > o: 303.746.7302
>>>>> > Advancing the way the world uses the cloud™
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkow...@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the
>>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>>> *™*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkow...@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud<http://solidfire.com/solution/overview/?video=play>
>>> *™*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud<http://solidfire.com/solution/overview/?video=play>
>> *™*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud<http://solidfire.com/solution/overview/?video=play>
> *™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
