I've been narrowing it down by putting in a million print-to-log statements.
Do you know if it is a problem that value ends up null (in a constructor
for Agent)?
String value = _shell.getPersistentProperty(getResourceName(), "id");
In that same constructor, this line never finishes:
if (!_resource.configure(getResourceName(), params)) {
I need to dig into the configure method to see what's going on there.
On Mon, Sep 23, 2013 at 5:45 PM, Marcus Sorensen <shadow...@gmail.com>wrote:
> It might be a centos specific thing. These are created by the init scripts.
> Check your agent init script on Ubuntu and see I'd you can decipher where
> it sends stdout.
> On Sep 23, 2013 5:21 PM, "Mike Tutkowski" <mike.tutkow...@solidfire.com>
> wrote:
>
> > Weird...no such file exists.
> >
> >
> > On Mon, Sep 23, 2013 at 4:54 PM, Marcus Sorensen <shadow...@gmail.com
> > >wrote:
> >
> > > maybe cloudstack-agent.out
> > >
> > > On Mon, Sep 23, 2013 at 4:44 PM, Mike Tutkowski
> > > <mike.tutkow...@solidfire.com> wrote:
> > > > OK, so, nothing is screaming out in the logs. I did notice the
> > following:
> > > >
> > > > From setup.log:
> > > >
> > > > DEBUG:root:execute:apparmor_status |grep libvirt
> > > >
> > > > DEBUG:root:Failed to execute:
> > > >
> > > >
> > > > DEBUG:root:execute:sudo /usr/sbin/service cloudstack-agent status
> > > >
> > > > DEBUG:root:Failed to execute: * could not access PID file for
> > > > cloudstack-agent
> > > >
> > > >
> > > > This is the final line in this log file:
> > > >
> > > > DEBUG:root:execute:sudo /usr/sbin/service cloudstack-agent start
> > > >
> > > >
> > > > This is from agent.log:
> > > >
> > > > 2013-09-23 15:30:55,549 DEBUG [cloud.agent.AgentShell] (main:null)
> > > Checking
> > > > to see if agent.pid exists.
> > > >
> > > > 2013-09-23 15:30:55,655 DEBUG [cloud.utils.ProcessUtil] (main:null)
> > > > Executing: bash -c echo $PPID
> > > >
> > > > 2013-09-23 15:30:55,742 DEBUG [cloud.utils.ProcessUtil] (main:null)
> > > > Execution is successful.
> > > >
> > > > 2013-09-23 15:30:56,000 INFO [cloud.agent.Agent] (main:null) id is
> > > >
> > > > 2013-09-23 15:30:56,000 DEBUG [cloud.resource.ServerResourceBase]
> > > > (main:null) Retrieving network interface: cloudbr0
> > > >
> > > > 2013-09-23 15:30:56,016 DEBUG [cloud.resource.ServerResourceBase]
> > > > (main:null) Retrieving network interface: cloudbr0
> > > >
> > > > 2013-09-23 15:30:56,016 DEBUG [cloud.resource.ServerResourceBase]
> > > > (main:null) Retrieving network interface: null
> > > >
> > > > 2013-09-23 15:30:56,017 DEBUG [cloud.resource.ServerResourceBase]
> > > > (main:null) Retrieving network interface: null
> > > >
> > > >
> > > > The following kinds of lines are repeated for a bunch of different
> .sh
> > > > files. I think they often end up being found here:
> > > > /usr/share/cloudstack-common/scripts/network/domr, so this is
> probably
> > > not
> > > > an issue.
> > > >
> > > >
> > > > 2013-09-23 15:30:56,111 DEBUG [utils.script.Script] (main:null)
> Looking
> > > for
> > > > call_firewall.sh in the classpath
> > > >
> > > > 2013-09-23 15:30:56,112 DEBUG [utils.script.Script] (main:null)
> System
> > > > resource: null
> > > >
> > > > 2013-09-23 15:30:56,113 DEBUG [utils.script.Script] (main:null)
> > Classpath
> > > > resource: null
> > > >
> > > > 2013-09-23 15:30:56,123 DEBUG [utils.script.Script] (main:null)
> Looking
> > > for
> > > > call_firewall.sh
> > > >
> > > >
> > > > Is there a log file for the Java code that I could write stuff out to
> > and
> > > > see how far we get?
> > > >
> > > >
> > > > On Mon, Sep 23, 2013 at 3:17 PM, Mike Tutkowski <
> > > > mike.tutkow...@solidfire.com> wrote:
> > > >
> > > >> Thanks, Marcus
> > > >>
> > > >> I've been developing on Windows for most of my time, so a bunch of
> > these
> > > >> Linux-type commands are new to me and I don't always interpret the
> > > output
> > > >> correctly. Getting there. :)
> > > >>
> > > >>
> > > >> On Mon, Sep 23, 2013 at 2:37 PM, Marcus Sorensen <
> shadow...@gmail.com
> > > >wrote:
> > > >>
> > > >>> Nope, not running. That's just your grep process. It would look
> like:
> > > >>>
> > > >>> root 24429 24428 1 14:25 ? 00:00:08 jsvc.exec -cp
> > > >>>
> > > >>>
> > >
> >
> /usr/share/java/commons-daemon.jar:/usr/share/java/jna.jar:/usr/share/cloudstack-agent/lib/activation-1.1.jar:/usr/share/cloudstack-agent/lib/antisamy-1.4.3.jar:/usr/share/cloudstack-agent/lib/aopalliance-1.0.jar:/usr/share/cloudstack-agent/lib/apache-log4j-extras-1.1.jar:/usr/share/cloudstack-agent/lib/aspectjrt-1.7.
> > > >>>
> > > >>> Your agent log should tell you why it failed to start if you set it
> > in
> > > >>> debug and try to start... or maybe cloudstack-agent.out if it
> doesn't
> > > >>> get far enough (say it's missing a class or something and can't
> > > >>> start).
> > > >>>
> > > >>> On Mon, Sep 23, 2013 at 2:33 PM, Mike Tutkowski
> > > >>> <mike.tutkow...@solidfire.com> wrote:
> > > >>> > Looks like it's running, though:
> > > >>> >
> > > >>> > mtutkowski@ubuntu:~$ ps -ef | grep jsvc
> > > >>> > 1000 7097 7013 0 14:32 pts/1 00:00:00 grep --color=auto
> > > jsvc
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > On Mon, Sep 23, 2013 at 2:31 PM, Mike Tutkowski <
> > > >>> > mike.tutkow...@solidfire.com> wrote:
> > > >>> >
> > > >>> >> Hey Marcus,
> > > >>> >>
> > > >>> >> Maybe you could give me a better idea of what the "flow" is when
> > > >>> adding a
> > > >>> >> KVM host.
> > > >>> >>
> > > >>> >> It looks like we SSH into the potential KVM host and execute a
> > > startup
> > > >>> >> script (giving it necessary info about the cloud and the
> > management
> > > >>> server
> > > >>> >> it should talk to).
> > > >>> >>
> > > >>> >> After this, is the Java VM started?
> > > >>> >>
> > > >>> >> After a reboot, I assume the JVM is started automatically?
> > > >>> >>
> > > >>> >> How do you debug your KVM-side Java code?
> > > >>> >>
> > > >>> >> Been looking through the logs and nothing obvious sticks out. I
> > will
> > > >>> have
> > > >>> >> another look.
> > > >>> >>
> > > >>> >> Thanks
> > > >>> >>
> > > >>> >>
> > > >>> >> On Mon, Sep 23, 2013 at 2:15 PM, Mike Tutkowski <
> > > >>> >> mike.tutkow...@solidfire.com> wrote:
> > > >>> >>
> > > >>> >>> Hey Marcus,
> > > >>> >>>
> > > >>> >>> I've been investigating my issue with not being able to add a
> KVM
> > > >>> host to
> > > >>> >>> CS.
> > > >>> >>>
> > > >>> >>> For what it's worth, this comes back successful:
> > > >>> >>>
> > > >>> >>> SSHCmdHelper.sshExecuteCmd(sshConnection,
> "cloudstack-setup-agent
> > > " +
> > > >>> >>> parameters, 3);
> > > >>> >>>
> > > >>> >>> This is what the command looks like:
> > > >>> >>>
> > > >>> >>> cloudstack-setup-agent -m 192.168.233.1 -z 1 -p 1 -c 1 -g
> > > >>> >>> 6b4aa1c2-2ac9-3c60-aabe-704aed40c684 -a --pubNic=cloudbr0
> > > >>> --prvNic=cloudbr0
> > > >>> >>> --guestNic=cloudbr0
> > > >>> >>>
> > > >>> >>> The problem is this method in LibvirtServerDiscoverer never
> > finds a
> > > >>> >>> matching host in the DB:
> > > >>> >>>
> > > >>> >>> waitForHostConnect(long dcId, long podId, long clusterId,
> String
> > > guid)
> > > >>> >>>
> > > >>> >>> I assume once the KVM host is up and running that it's supposed
> > to
> > > >>> call
> > > >>> >>> into the CS MS so the DB can be updated as such?
> > > >>> >>>
> > > >>> >>> If so, the problem must be on the KVM side.
> > > >>> >>>
> > > >>> >>> I did run this again (from the KVM host) to see if the
> connection
> > > was
> > > >>> in
> > > >>> >>> place:
> > > >>> >>>
> > > >>> >>> mtutkowski@ubuntu:~$ telnet 192.168.233.1 8250
> > > >>> >>>
> > > >>> >>> Trying 192.168.233.1...
> > > >>> >>>
> > > >>> >>> Connected to 192.168.233.1.
> > > >>> >>>
> > > >>> >>> Escape character is '^]'.
> > > >>> >>> So that looks good.
> > > >>> >>>
> > > >>> >>> I turned on more info in the debug log, but nothing obvious
> jumps
> > > out
> > > >>> as
> > > >>> >>> of yet.
> > > >>> >>>
> > > >>> >>> If you have any thoughts on this, please shoot them my way. :)
> > > >>> >>>
> > > >>> >>> Thanks!
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> On Sun, Sep 22, 2013 at 12:11 AM, Mike Tutkowski <
> > > >>> >>> mike.tutkow...@solidfire.com> wrote:
> > > >>> >>>
> > > >>> >>>> First step is for me to get this working for KVM, though. :)
> > > >>> >>>>
> > > >>> >>>> Once I do that, I can perhaps make modifications to the
> storage
> > > >>> >>>> framework and hypervisor plug-ins to refactor the logic and
> > such.
> > > >>> >>>>
> > > >>> >>>>
> > > >>> >>>> On Sun, Sep 22, 2013 at 12:09 AM, Mike Tutkowski <
> > > >>> >>>> mike.tutkow...@solidfire.com> wrote:
> > > >>> >>>>
> > > >>> >>>>> Same would work for KVM.
> > > >>> >>>>>
> > > >>> >>>>> If CreateCommand and DestroyCommand were called at the
> > > appropriate
> > > >>> >>>>> times by the storage framework, I could move my connect and
> > > >>> disconnect
> > > >>> >>>>> logic out of the attach/detach logic.
> > > >>> >>>>>
> > > >>> >>>>>
> > > >>> >>>>> On Sun, Sep 22, 2013 at 12:08 AM, Mike Tutkowski <
> > > >>> >>>>> mike.tutkow...@solidfire.com> wrote:
> > > >>> >>>>>
> > > >>> >>>>>> Conversely, if the storage framework called the
> DestroyCommand
> > > for
> > > >>> >>>>>> managed storage after the DetachCommand, then I could have
> had
> > > my
> > > >>> remove
> > > >>> >>>>>> SR/datastore logic placed in the DestroyCommand handling
> > rather
> > > >>> than in the
> > > >>> >>>>>> DetachCommand handling.
> > > >>> >>>>>>
> > > >>> >>>>>>
> > > >>> >>>>>> On Sun, Sep 22, 2013 at 12:06 AM, Mike Tutkowski <
> > > >>> >>>>>> mike.tutkow...@solidfire.com> wrote:
> > > >>> >>>>>>
> > > >>> >>>>>>> Edison's plug-in calls the CreateCommand. Mine does not.
> > > >>> >>>>>>>
> > > >>> >>>>>>> The initial approach that was discussed during 4.2 was for
> me
> > > to
> > > >>> >>>>>>> modify the attach/detach logic only in the XenServer and
> > VMware
> > > >>> hypervisor
> > > >>> >>>>>>> plug-ins.
> > > >>> >>>>>>>
> > > >>> >>>>>>> Now that I think about it more, though, I kind of would
> have
> > > >>> liked to
> > > >>> >>>>>>> have the storage framework send a CreateCommand to the
> > > hypervisor
> > > >>> before
> > > >>> >>>>>>> sending the AttachCommand if the storage in question was
> > > managed.
> > > >>> >>>>>>>
> > > >>> >>>>>>> Then I could have created my SR/datastore in the
> > CreateCommand
> > > and
> > > >>> >>>>>>> the AttachCommand would have had the SR/datastore that it
> was
> > > >>> always
> > > >>> >>>>>>> expecting (and I wouldn't have had to create the
> SR/datastore
> > > in
> > > >>> the
> > > >>> >>>>>>> AttachCommand).
> > > >>> >>>>>>>
> > > >>> >>>>>>>
> > > >>> >>>>>>> On Sat, Sep 21, 2013 at 11:56 PM, Marcus Sorensen <
> > > >>> >>>>>>> shadow...@gmail.com> wrote:
> > > >>> >>>>>>>
> > > >>> >>>>>>>> Yeah, I think it probably is as well, but I figured you'd
> be
> > > in a
> > > >>> >>>>>>>> better position to tell.
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> I see that copyAsync is unsupported in your current 4.2
> > > driver,
> > > >>> does
> > > >>> >>>>>>>> that mean that there's no template support? Or is it some
> > > other
> > > >>> call
> > > >>> >>>>>>>> that does templating now? I'm still getting up to speed on
> > all
> > > >>> of the
> > > >>> >>>>>>>> 4.2 changes. I was just looking at CreateCommand in
> > > >>> >>>>>>>> LibvirtComputingResource, since that's the only place
> > > >>> >>>>>>>> createPhysicalDisk is called, and it occurred to me that
> > > >>> >>>>>>>> CreateCommand
> > > >>> >>>>>>>> might be skipped altogether when utilizing storage
> plugins.
> > > >>> >>>>>>>>
> > > >>> >>>>>>>> On Sat, Sep 21, 2013 at 11:38 PM, Mike Tutkowski
> > > >>> >>>>>>>> <mike.tutkow...@solidfire.com> wrote:
> > > >>> >>>>>>>> > That's an interesting comment, Marcus.
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> > It was my intent that it should work with any CloudStack
> > > >>> "managed"
> > > >>> >>>>>>>> storage
> > > >>> >>>>>>>> > that uses an iSCSI target. Even though I'm using CHAP, I
> > > wrote
> > > >>> the
> > > >>> >>>>>>>> code so
> > > >>> >>>>>>>> > CHAP didn't have to be used.
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> > As I'm doing my testing, I can try to think about
> whether
> > > it is
> > > >>> >>>>>>>> generic
> > > >>> >>>>>>>> > enough to keep those names or not.
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> > My expectation is that it is generic enough.
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> > On Sat, Sep 21, 2013 at 11:32 PM, Marcus Sorensen <
> > > >>> >>>>>>>> shadow...@gmail.com>wrote:
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> >> I added a comment to your diff. In general I think it
> > looks
> > > >>> good,
> > > >>> >>>>>>>> >> though I obviously can't vouch for whether or not it
> will
> > > >>> work.
> > > >>> >>>>>>>> One
> > > >>> >>>>>>>> >> thing I do have reservations about is the adaptor/pool
> > > >>> naming. If
> > > >>> >>>>>>>> you
> > > >>> >>>>>>>> >> think the code is generic enough that it will work for
> > > anyone
> > > >>> who
> > > >>> >>>>>>>> does
> > > >>> >>>>>>>> >> an iscsi LUN-per-volume plugin, then it's OK, but if
> > > there's
> > > >>> >>>>>>>> anything
> > > >>> >>>>>>>> >> about it that's specific to YOUR iscsi target or how it
> > > likes
> > > >>> to
> > > >>> >>>>>>>> be
> > > >>> >>>>>>>> >> treated then I'd say that they should be named
> something
> > > less
> > > >>> >>>>>>>> generic
> > > >>> >>>>>>>> >> than iScsiAdmStorage.
> > > >>> >>>>>>>> >>
> > > >>> >>>>>>>> >> On Sat, Sep 21, 2013 at 7:23 PM, Mike Tutkowski
> > > >>> >>>>>>>> >> <mike.tutkow...@solidfire.com> wrote:
> > > >>> >>>>>>>> >> > Great - thanks!
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> > Just to give you an overview of what my code does
> (for
> > > when
> > > >>> you
> > > >>> >>>>>>>> get a
> > > >>> >>>>>>>> >> > chance to review it):
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> > SolidFireHostListener is registered in
> > > >>> >>>>>>>> SolidfirePrimaryDataStoreProvider.
> > > >>> >>>>>>>> >> > Its hostConnect method is invoked when a host
> connects
> > > with
> > > >>> the
> > > >>> >>>>>>>> CS MS. If
> > > >>> >>>>>>>> >> > the host is running KVM, the listener sends a
> > > >>> >>>>>>>> ModifyStoragePoolCommand to
> > > >>> >>>>>>>> >> > the host. This logic was based off of
> > > DefaultHostListener.
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> > The handling of ModifyStoragePoolCommand is
> unchanged.
> > It
> > > >>> >>>>>>>> invokes
> > > >>> >>>>>>>> >> > createStoragePool on the KVMStoragePoolManager. The
> > > >>> >>>>>>>> KVMStoragePoolManager
> > > >>> >>>>>>>> >> > asks for an adaptor and finds my new one:
> > > >>> >>>>>>>> iScsiAdmStorageAdaptor (which
> > > >>> >>>>>>>> >> was
> > > >>> >>>>>>>> >> > registered in the constructor for
> KVMStoragePoolManager
> > > >>> under
> > > >>> >>>>>>>> the key of
> > > >>> >>>>>>>> >> > StoragePoolType.Iscsi.toString()).
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> > iScsiAdmStorageAdaptor.createStoragePool just makes
> an
> > > >>> instance
> > > >>> >>>>>>>> of
> > > >>> >>>>>>>> >> > iScsiAdmStoragePool, adds it to a map, and returns
> the
> > > >>> pointer
> > > >>> >>>>>>>> to the
> > > >>> >>>>>>>> >> > iScsiAdmStoragePool object. The key of the map is the
> > > UUID
> > > >>> of
> > > >>> >>>>>>>> the storage
> > > >>> >>>>>>>> >> > pool.
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> > When a volume is attached, createPhysicalDisk is
> > invoked
> > > for
> > > >>> >>>>>>>> managed
> > > >>> >>>>>>>> >> > storage rather than getPhysicalDisk.
> createPhysicalDisk
> > > uses
> > > >>> >>>>>>>> iscsiadm to
> > > >>> >>>>>>>> >> > establish the iSCSI connection to the volume on the
> SAN
> > > and
> > > >>> a
> > > >>> >>>>>>>> >> > KVMPhysicalDisk is returned to be used in the attach
> > > logic
> > > >>> that
> > > >>> >>>>>>>> follows.
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> > When a volume is detached, getPhysicalDisk is invoked
> > > with
> > > >>> the
> > > >>> >>>>>>>> IQN of the
> > > >>> >>>>>>>> >> > volume if the storage pool in question is managed
> > > storage.
> > > >>> >>>>>>>> Otherwise, the
> > > >>> >>>>>>>> >> > normal vol.getPath() is used.
> > > >>> >>>>>>>> iScsiAdmStorageAdaptor.getPhysicalDisk just
> > > >>> >>>>>>>> >> > returns a new instance of KVMPhysicalDisk to be used
> in
> > > the
> > > >>> >>>>>>>> detach logic.
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> > Once the volume has been detached,
> > > >>> >>>>>>>> iScsiAdmStoragePool.deletePhysicalDisk
> > > >>> >>>>>>>> >> > is invoked if the storage pool is managed.
> > > >>> deletePhysicalDisk
> > > >>> >>>>>>>> removes the
> > > >>> >>>>>>>> >> > iSCSI connection to the volume using iscsiadm.
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> > On Sat, Sep 21, 2013 at 5:46 PM, Marcus Sorensen <
> > > >>> >>>>>>>> shadow...@gmail.com
> > > >>> >>>>>>>> >> >wrote:
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> >> Its the log4j properties file in
> /etc/cloudstack/agent
> > > >>> change
> > > >>> >>>>>>>> all INFO
> > > >>> >>>>>>>> >> to
> > > >>> >>>>>>>> >> >> DEBUG. I imagine the agent just isn't starting, you
> > can
> > > >>> tail
> > > >>> >>>>>>>> the log
> > > >>> >>>>>>>> >> when
> > > >>> >>>>>>>> >> >> you try to start the service, or maybe it will spit
> > > >>> something
> > > >>> >>>>>>>> out into
> > > >>> >>>>>>>> >> one
> > > >>> >>>>>>>> >> >> of the other files in /var/log/cloudstack/agent
> > > >>> >>>>>>>> >> >> On Sep 21, 2013 5:19 PM, "Mike Tutkowski" <
> > > >>> >>>>>>>> mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> >> wrote:
> > > >>> >>>>>>>> >> >>
> > > >>> >>>>>>>> >> >> > This is how I've been trying to query for the
> status
> > > of
> > > >>> the
> > > >>> >>>>>>>> service (I
> > > >>> >>>>>>>> >> >> > assume it could be started this way, as well, by
> > > changing
> > > >>> >>>>>>>> "status" to
> > > >>> >>>>>>>> >> >> > "start" or "restart"?):
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> > mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo
> > > >>> >>>>>>>> /usr/sbin/service
> > > >>> >>>>>>>> >> >> > cloudstack-agent status
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> > I get this back:
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> > Failed to execute: * could not access PID file for
> > > >>> >>>>>>>> cloudstack-agent
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> > I've made a bunch of code changes recently,
> though,
> > > so I
> > > >>> >>>>>>>> think I'm
> > > >>> >>>>>>>> >> going
> > > >>> >>>>>>>> >> >> to
> > > >>> >>>>>>>> >> >> > rebuild and redeploy everything.
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> > The debug info sounds helpful. Where can I set
> > > >>> enable.debug?
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> > Thanks, Marcus!
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> > On Sat, Sep 21, 2013 at 4:11 PM, Marcus Sorensen <
> > > >>> >>>>>>>> shadow...@gmail.com
> > > >>> >>>>>>>> >> >> > >wrote:
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> > > OK, will check it out in the next few days. As
> > > >>> mentioned,
> > > >>> >>>>>>>> you can
> > > >>> >>>>>>>> >> set
> > > >>> >>>>>>>> >> >> up
> > > >>> >>>>>>>> >> >> > > your Ubuntu vm as the management server as well
> if
> > > all
> > > >>> >>>>>>>> else fails.
> > > >>> >>>>>>>> >> If
> > > >>> >>>>>>>> >> >> > you
> > > >>> >>>>>>>> >> >> > > can get to the mgmt server on 8250 from the KVM
> > > host,
> > > >>> then
> > > >>> >>>>>>>> you need
> > > >>> >>>>>>>> >> to
> > > >>> >>>>>>>> >> >> > > enable.debug on the agent. It won't run without
> > > >>> >>>>>>>> complaining loudly
> > > >>> >>>>>>>> >> if
> > > >>> >>>>>>>> >> >> it
> > > >>> >>>>>>>> >> >> > > can't get to the mgmt server, and I didn't see
> > that
> > > in
> > > >>> >>>>>>>> your agent
> > > >>> >>>>>>>> >> log,
> > > >>> >>>>>>>> >> >> so
> > > >>> >>>>>>>> >> >> > > perhaps its not running. I assume you know how
> to
> > > >>> >>>>>>>> stop/start the
> > > >>> >>>>>>>> >> agent
> > > >>> >>>>>>>> >> >> on
> > > >>> >>>>>>>> >> >> > > KVM via 'service cloud stacks agent'.
> > > >>> >>>>>>>> >> >> > > On Sep 21, 2013 3:02 PM, "Mike Tutkowski" <
> > > >>> >>>>>>>> >> >> mike.tutkow...@solidfire.com>
> > > >>> >>>>>>>> >> >> > > wrote:
> > > >>> >>>>>>>> >> >> > >
> > > >>> >>>>>>>> >> >> > > > Hey Marcus,
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > > I haven't yet been able to test my new code,
> > but I
> > > >>> >>>>>>>> thought you
> > > >>> >>>>>>>> >> would
> > > >>> >>>>>>>> >> >> > be a
> > > >>> >>>>>>>> >> >> > > > good person to ask to review it:
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > >
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >>
> > > >>> >>>>>>>> >>
> > > >>> >>>>>>>>
> > > >>>
> > >
> >
> https://github.com/mike-tutkowski/incubator-cloudstack/commit/ea74b312a8a36801994500407fd54f0cdda55e37
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > > All it is supposed to do is attach and detach
> a
> > > data
> > > >>> >>>>>>>> disk (that
> > > >>> >>>>>>>> >> has
> > > >>> >>>>>>>> >> >> > > > guaranteed IOPS) with KVM as the hypervisor.
> The
> > > data
> > > >>> >>>>>>>> disk
> > > >>> >>>>>>>> >> happens to
> > > >>> >>>>>>>> >> >> > be
> > > >>> >>>>>>>> >> >> > > > from SolidFire-backed storage - where we have
> a
> > > 1:1
> > > >>> >>>>>>>> mapping
> > > >>> >>>>>>>> >> between a
> > > >>> >>>>>>>> >> >> > > > CloudStack volume and a data disk.
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > > There is no support for hypervisor snapshots
> or
> > > stuff
> > > >>> >>>>>>>> like that
> > > >>> >>>>>>>> >> >> > (likely a
> > > >>> >>>>>>>> >> >> > > > future release)...just attaching and
> detaching a
> > > data
> > > >>> >>>>>>>> disk in 4.3.
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > > Thanks!
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > > On Fri, Sep 20, 2013 at 11:05 PM, Mike
> > Tutkowski <
> > > >>> >>>>>>>> >> >> > > > mike.tutkow...@solidfire.com> wrote:
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > > > When I re-deployed the DEBs, I didn't remove
> > > >>> >>>>>>>> cloudstack-agent
> > > >>> >>>>>>>> >> >> first.
> > > >>> >>>>>>>> >> >> > > > Would
> > > >>> >>>>>>>> >> >> > > > > that be a problem? I just did a sudo apt-get
> > > >>> install
> > > >>> >>>>>>>> >> >> > cloudstack-agent.
> > > >>> >>>>>>>> >> >> > > > >
> > > >>> >>>>>>>> >> >> > > > >
> > > >>> >>>>>>>> >> >> > > > > On Fri, Sep 20, 2013 at 11:03 PM, Mike
> > > Tutkowski <
> > > >>> >>>>>>>> >> >> > > > > mike.tutkow...@solidfire.com> wrote:
> > > >>> >>>>>>>> >> >> > > > >
> > > >>> >>>>>>>> >> >> > > > >> I get the same error running the command
> > > manually:
> > > >>> >>>>>>>> >> >> > > > >>
> > > >>> >>>>>>>> >> >> > > > >> mtutkowski@ubuntu:/etc/cloudstack/agent$
> > sudo
> > > >>> >>>>>>>> >> /usr/sbin/service
> > > >>> >>>>>>>> >> >> > > > >> cloudstack-agent status
> > > >>> >>>>>>>> >> >> > > > >> * could not access PID file for
> > > cloudstack-agent
> > > >>> >>>>>>>> >> >> > > > >>
> > > >>> >>>>>>>> >> >> > > > >>
> > > >>> >>>>>>>> >> >> > > > >> On Fri, Sep 20, 2013 at 11:00 PM, Mike
> > > Tutkowski <
> > > >>> >>>>>>>> >> >> > > > >> mike.tutkow...@solidfire.com> wrote:
> > > >>> >>>>>>>> >> >> > > > >>
> > > >>> >>>>>>>> >> >> > > > >>> agent.log looks OK to me:
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,010 INFO
> > > >>> >>>>>>>> [cloud.agent.AgentShell]
> > > >>> >>>>>>>> >> >> (main:null)
> > > >>> >>>>>>>> >> >> > > > Agent
> > > >>> >>>>>>>> >> >> > > > >>> started
> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,011 INFO
> > > >>> >>>>>>>> [cloud.agent.AgentShell]
> > > >>> >>>>>>>> >> >> (main:null)
> > > >>> >>>>>>>> >> >> > > > >>> Implementation Version is 4.3.0-SNAPSHOT
> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,015 INFO
> > > >>> >>>>>>>> [cloud.agent.AgentShell]
> > > >>> >>>>>>>> >> >> (main:null)
> > > >>> >>>>>>>> >> >> > > > >>> agent.properties found at
> > > >>> >>>>>>>> >> /etc/cloudstack/agent/agent.properties
> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,023 INFO
> > > >>> >>>>>>>> [cloud.agent.AgentShell]
> > > >>> >>>>>>>> >> >> (main:null)
> > > >>> >>>>>>>> >> >> > > > >>> Defaulting to using properties file for
> > > storage
> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,024 INFO
> > > >>> >>>>>>>> [cloud.agent.AgentShell]
> > > >>> >>>>>>>> >> >> (main:null)
> > > >>> >>>>>>>> >> >> > > > >>> Defaulting to the constant time backoff
> > > algorithm
> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,029 INFO
> > > >>> [cloud.utils.LogUtils]
> > > >>> >>>>>>>> >> (main:null)
> > > >>> >>>>>>>> >> >> > > log4j
> > > >>> >>>>>>>> >> >> > > > >>> configuration found at
> > > >>> >>>>>>>> /etc/cloudstack/agent/log4j-cloud.xml
> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,163 INFO
> > > [cloud.agent.Agent]
> > > >>> >>>>>>>> (main:null)
> > > >>> >>>>>>>> >> id
> > > >>> >>>>>>>> >> >> > is 3
> > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,197 INFO
> > > >>> >>>>>>>> >> >> > > > >>>
> > > [resource.virtualnetwork.VirtualRoutingResource]
> > > >>> >>>>>>>> (main:null)
> > > >>> >>>>>>>> >> >> > > > >>> VirtualRoutingResource _scriptDir to use:
> > > >>> >>>>>>>> >> >> scripts/network/domr/kvm
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>> However, I wasn't aware that setup.log was
> > > >>> >>>>>>>> important. This
> > > >>> >>>>>>>> >> seems
> > > >>> >>>>>>>> >> >> to
> > > >>> >>>>>>>> >> >> > > be
> > > >>> >>>>>>>> >> >> > > > a
> > > >>> >>>>>>>> >> >> > > > >>> problem, but I'm not sure what it might
> > > indicate:
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service
> > > >>> >>>>>>>> cloudstack-agent
> > > >>> >>>>>>>> >> status
> > > >>> >>>>>>>> >> >> > > > >>> DEBUG:root:Failed to execute: * could not
> > > access
> > > >>> PID
> > > >>> >>>>>>>> file for
> > > >>> >>>>>>>> >> >> > > > >>> cloudstack-agent
> > > >>> >>>>>>>> >> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service
> > > >>> >>>>>>>> cloudstack-agent
> > > >>> >>>>>>>> >> start
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>> On Fri, Sep 20, 2013 at 10:53 PM, Marcus
> > > >>> Sorensen <
> > > >>> >>>>>>>> >> >> > > shadow...@gmail.com
> > > >>> >>>>>>>> >> >> > > > >wrote:
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>>> Sorry, I saw that in the log, I thought
> it
> > > was
> > > >>> the
> > > >>> >>>>>>>> agent log
> > > >>> >>>>>>>> >> for
> > > >>> >>>>>>>> >> >> > > some
> > > >>> >>>>>>>> >> >> > > > >>>> reason. Is the agent started? That might
> be
> > > the
> > > >>> >>>>>>>> place to
> > > >>> >>>>>>>> >> look.
> > > >>> >>>>>>>> >> >> > There
> > > >>> >>>>>>>> >> >> > > > is
> > > >>> >>>>>>>> >> >> > > > >>>> an
> > > >>> >>>>>>>> >> >> > > > >>>> agent log for the agent and one for the
> > setup
> > > >>> when
> > > >>> >>>>>>>> it adds
> > > >>> >>>>>>>> >> the
> > > >>> >>>>>>>> >> >> > host,
> > > >>> >>>>>>>> >> >> > > > >>>> both
> > > >>> >>>>>>>> >> >> > > > >>>> in /var/log
> > > >>> >>>>>>>> >> >> > > > >>>> On Sep 20, 2013 10:42 PM, "Mike
> Tutkowski"
> > <
> > > >>> >>>>>>>> >> >> > > > >>>> mike.tutkow...@solidfire.com>
> > > >>> >>>>>>>> >> >> > > > >>>> wrote:
> > > >>> >>>>>>>> >> >> > > > >>>>
> > > >>> >>>>>>>> >> >> > > > >>>> > Is it saying that the MS is at the IP
> > > address
> > > >>> or
> > > >>> >>>>>>>> the KVM
> > > >>> >>>>>>>> >> host?
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> > The KVM host is at 192.168.233.10.
> > > >>> >>>>>>>> >> >> > > > >>>> > The MS host is at 192.168.233.1.
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> > I see this for my host Global Settings
> > > >>> parameter:
> > > >>> >>>>>>>> >> >> > > > >>>> > hostThe ip address of management
> > > >>> >>>>>>>> server192.168.233.1
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> > /etc/cloudstack/agent/agent.properties
> > has
> > > a
> > > >>> >>>>>>>> >> >> host=192.168.233.1
> > > >>> >>>>>>>> >> >> > > > value.
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> > On Fri, Sep 20, 2013 at 10:21 PM,
> Marcus
> > > >>> Sorensen
> > > >>> >>>>>>>> <
> > > >>> >>>>>>>> >> >> > > > >>>> shadow...@gmail.com
> > > >>> >>>>>>>> >> >> > > > >>>> > >wrote:
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> > > The log says your mgmt server is
> > > >>> >>>>>>>> 192.168.233.10? But you
> > > >>> >>>>>>>> >> >> tried
> > > >>> >>>>>>>> >> >> > > to
> > > >>> >>>>>>>> >> >> > > > >>>> telnet
> > > >>> >>>>>>>> >> >> > > > >>>> > to
> > > >>> >>>>>>>> >> >> > > > >>>> > > 192.168.233.1? It might be enough to
> > > change
> > > >>> >>>>>>>> that in
> > > >>> >>>>>>>> >> >> > > > >>>> > >
> /etc/cloudstack/agent/agent.properties,
> > > but
> > > >>> you
> > > >>> >>>>>>>> may want
> > > >>> >>>>>>>> >> to
> > > >>> >>>>>>>> >> >> > edit
> > > >>> >>>>>>>> >> >> > > > the
> > > >>> >>>>>>>> >> >> > > > >>>> > config
> > > >>> >>>>>>>> >> >> > > > >>>> > > as well to tell it the real ms IP.
> > > >>> >>>>>>>> >> >> > > > >>>> > > On Sep 20, 2013 10:12 PM, "Mike
> > > Tutkowski" <
> > > >>> >>>>>>>> >> >> > > > >>>> mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> >> > > > >>>> > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > wrote:
> > > >>> >>>>>>>> >> >> > > > >>>> > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > Here's what my
> > /etc/network/interfaces
> > > >>> file
> > > >>> >>>>>>>> looks
> > > >>> >>>>>>>> >> like, if
> > > >>> >>>>>>>> >> >> > > that
> > > >>> >>>>>>>> >> >> > > > >>>> is of
> > > >>> >>>>>>>> >> >> > > > >>>> > > > interest (the 192.168.233.0 network
> > is
> > > the
> > > >>> >>>>>>>> NAT network
> > > >>> >>>>>>>> >> >> > VMware
> > > >>> >>>>>>>> >> >> > > > >>>> Fusion
> > > >>> >>>>>>>> >> >> > > > >>>> > set
> > > >>> >>>>>>>> >> >> > > > >>>> > > > up):
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > auto lo
> > > >>> >>>>>>>> >> >> > > > >>>> > > > iface lo inet loopback
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > auto eth0
> > > >>> >>>>>>>> >> >> > > > >>>> > > > iface eth0 inet manual
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > auto cloudbr0
> > > >>> >>>>>>>> >> >> > > > >>>> > > > iface cloudbr0 inet static
> > > >>> >>>>>>>> >> >> > > > >>>> > > > address 192.168.233.10
> > > >>> >>>>>>>> >> >> > > > >>>> > > > netmask 255.255.255.0
> > > >>> >>>>>>>> >> >> > > > >>>> > > > network 192.168.233.0
> > > >>> >>>>>>>> >> >> > > > >>>> > > > broadcast 192.168.233.255
> > > >>> >>>>>>>> >> >> > > > >>>> > > > dns-nameservers 8.8.8.8
> > > >>> >>>>>>>> >> >> > > > >>>> > > > bridge_ports eth0
> > > >>> >>>>>>>> >> >> > > > >>>> > > > bridge_fd 5
> > > >>> >>>>>>>> >> >> > > > >>>> > > > bridge_stp off
> > > >>> >>>>>>>> >> >> > > > >>>> > > > bridge_maxwait 1
> > > >>> >>>>>>>> >> >> > > > >>>> > > > post-up route add default gw
> > > >>> >>>>>>>> 192.168.233.2 metric 1
> > > >>> >>>>>>>> >> >> > > > >>>> > > > pre-down route del default gw
> > > >>> >>>>>>>> 192.168.233.2
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > On Fri, Sep 20, 2013 at 10:08 PM,
> > Mike
> > > >>> >>>>>>>> Tutkowski <
> > > >>> >>>>>>>> >> >> > > > >>>> > > > mike.tutkow...@solidfire.com>
> wrote:
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > You appear to be correct. This is
> > > from
> > > >>> the
> > > >>> >>>>>>>> MS log
> > > >>> >>>>>>>> >> >> (below).
> > > >>> >>>>>>>> >> >> > > > >>>> Discovery
> > > >>> >>>>>>>> >> >> > > > >>>> > > > timed
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > out.
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I'm not sure why this would be.
> My
> > > >>> network
> > > >>> >>>>>>>> settings
> > > >>> >>>>>>>> >> >> > > shouldn't
> > > >>> >>>>>>>> >> >> > > > >>>> have
> > > >>> >>>>>>>> >> >> > > > >>>> > > > changed
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > since the last time I tried this.
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I am able to ping the KVM host
> from
> > > the
> > > >>> MS
> > > >>> >>>>>>>> host and
> > > >>> >>>>>>>> >> vice
> > > >>> >>>>>>>> >> >> > > > versa.
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I'm even able to manually kick
> off
> > a
> > > VM
> > > >>> on
> > > >>> >>>>>>>> the KVM
> > > >>> >>>>>>>> >> host
> > > >>> >>>>>>>> >> >> > and
> > > >>> >>>>>>>> >> >> > > > >>>> ping from
> > > >>> >>>>>>>> >> >> > > > >>>> > > it
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > to the MS host.
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I am using NAT from my Mac OS X
> > host
> > > >>> (also
> > > >>> >>>>>>>> running
> > > >>> >>>>>>>> >> the
> > > >>> >>>>>>>> >> >> CS
> > > >>> >>>>>>>> >> >> > > MS)
> > > >>> >>>>>>>> >> >> > > > >>>> to the
> > > >>> >>>>>>>> >> >> > > > >>>> > VM
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > running KVM (VMware Fusion).
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,141 DEBUG
> > > >>> >>>>>>>> >> >> > > > >>>> [c.c.h.k.d.LibvirtServerDiscoverer]
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3
> > > >>> :ctx-6b28dc48)
> > > >>> >>>>>>>> Timeout,
> > > >>> >>>>>>>> >> to
> > > >>> >>>>>>>> >> >> > wait
> > > >>> >>>>>>>> >> >> > > > for
> > > >>> >>>>>>>> >> >> > > > >>>> the
> > > >>> >>>>>>>> >> >> > > > >>>> > > host
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > connecting to mgt svr, assuming
> it
> > is
> > > >>> failed
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,144 WARN
> > > >>> >>>>>>>> >> >> [c.c.r.ResourceManagerImpl]
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3
> > > >>> :ctx-6b28dc48)
> > > >>> >>>>>>>> Unable to
> > > >>> >>>>>>>> >> >> find
> > > >>> >>>>>>>> >> >> > > the
> > > >>> >>>>>>>> >> >> > > > >>>> server
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > resources at
> http://192.168.233.10
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,145 INFO
> > > >>> >>>>>>>> >> >> > [c.c.u.e.CSExceptionErrorCode]
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3
> > > >>> :ctx-6b28dc48)
> > > >>> >>>>>>>> Could not
> > > >>> >>>>>>>> >> >> find
> > > >>> >>>>>>>> >> >> > > > >>>> exception:
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > com.cloud.exception.DiscoveryException
> > > >>> in
> > > >>> >>>>>>>> error code
> > > >>> >>>>>>>> >> >> list
> > > >>> >>>>>>>> >> >> > > for
> > > >>> >>>>>>>> >> >> > > > >>>> > > exceptions
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,147 WARN
> > > >>> >>>>>>>> >> [o.a.c.a.c.a.h.AddHostCmd]
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3
> > > >>> :ctx-6b28dc48)
> > > >>> >>>>>>>> Exception:
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > com.cloud.exception.DiscoveryException:
> > > >>> >>>>>>>> Unable to add
> > > >>> >>>>>>>> >> >> the
> > > >>> >>>>>>>> >> >> > > host
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > at
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > >
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>>
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > >
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >>
> > > >>> >>>>>>>> >>
> > > >>> >>>>>>>>
> > > >>>
> > >
> >
> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:778)
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > I do seem to be able to telnet in
> > > from
> > > >>> my
> > > >>> >>>>>>>> KVM host to
> > > >>> >>>>>>>> >> >> the
> > > >>> >>>>>>>> >> >> > MS
> > > >>> >>>>>>>> >> >> > > > >>>> host's
> > > >>> >>>>>>>> >> >> > > > >>>> > > 8250
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > port:
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > mtutkowski@ubuntu:~$ telnet
> > > >>> 192.168.233.1
> > > >>> >>>>>>>> 8250
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > Trying 192.168.233.1...
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > Connected to 192.168.233.1.
> > > >>> >>>>>>>> >> >> > > > >>>> > > > > Escape character is '^]'.
> > > >>> >>>>>>>> >> >> > > > >>>> > > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > > > --
> > > >>> >>>>>>>> >> >> > > > >>>> > > > *Mike Tutkowski*
> > > >>> >>>>>>>> >> >> > > > >>>> > > > *Senior CloudStack Developer,
> > SolidFire
> > > >>> Inc.*
> > > >>> >>>>>>>> >> >> > > > >>>> > > > e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> >> > > > >>>> > > > o: 303.746.7302
> > > >>> >>>>>>>> >> >> > > > >>>> > > > Advancing the way the world uses
> the
> > > >>> >>>>>>>> >> >> > > > >>>> > > > cloud<
> > > >>> >>>>>>>> >> http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>>>>>> >> >> > > > >>>> > > > *™*
> > > >>> >>>>>>>> >> >> > > > >>>> > > >
> > > >>> >>>>>>>> >> >> > > > >>>> > >
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>> > --
> > > >>> >>>>>>>> >> >> > > > >>>> > *Mike Tutkowski*
> > > >>> >>>>>>>> >> >> > > > >>>> > *Senior CloudStack Developer, SolidFire
> > > Inc.*
> > > >>> >>>>>>>> >> >> > > > >>>> > e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> >> > > > >>>> > o: 303.746.7302
> > > >>> >>>>>>>> >> >> > > > >>>> > Advancing the way the world uses the
> > > >>> >>>>>>>> >> >> > > > >>>> > cloud<
> > > >>> >>>>>>>> http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>>>>>> >> >> > > > >>>> > *™*
> > > >>> >>>>>>>> >> >> > > > >>>> >
> > > >>> >>>>>>>> >> >> > > > >>>>
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>> --
> > > >>> >>>>>>>> >> >> > > > >>> *Mike Tutkowski*
> > > >>> >>>>>>>> >> >> > > > >>> *Senior CloudStack Developer, SolidFire
> > Inc.*
> > > >>> >>>>>>>> >> >> > > > >>> e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> >> > > > >>> o: 303.746.7302
> > > >>> >>>>>>>> >> >> > > > >>> Advancing the way the world uses the
> cloud<
> > > >>> >>>>>>>> >> >> > > >
> > > http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>>>>>> >> >> > > > >>> *™*
> > > >>> >>>>>>>> >> >> > > > >>>
> > > >>> >>>>>>>> >> >> > > > >>
> > > >>> >>>>>>>> >> >> > > > >>
> > > >>> >>>>>>>> >> >> > > > >>
> > > >>> >>>>>>>> >> >> > > > >> --
> > > >>> >>>>>>>> >> >> > > > >> *Mike Tutkowski*
> > > >>> >>>>>>>> >> >> > > > >> *Senior CloudStack Developer, SolidFire
> Inc.*
> > > >>> >>>>>>>> >> >> > > > >> e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> >> > > > >> o: 303.746.7302
> > > >>> >>>>>>>> >> >> > > > >> Advancing the way the world uses the cloud<
> > > >>> >>>>>>>> >> >> > > >
> > > http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>>>>>> >> >> > > > >> *™*
> > > >>> >>>>>>>> >> >> > > > >>
> > > >>> >>>>>>>> >> >> > > > >
> > > >>> >>>>>>>> >> >> > > > >
> > > >>> >>>>>>>> >> >> > > > >
> > > >>> >>>>>>>> >> >> > > > > --
> > > >>> >>>>>>>> >> >> > > > > *Mike Tutkowski*
> > > >>> >>>>>>>> >> >> > > > > *Senior CloudStack Developer, SolidFire
> Inc.*
> > > >>> >>>>>>>> >> >> > > > > e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> >> > > > > o: 303.746.7302
> > > >>> >>>>>>>> >> >> > > > > Advancing the way the world uses the cloud<
> > > >>> >>>>>>>> >> >> > > >
> > > http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>>>>>> >> >> > > > > *™*
> > > >>> >>>>>>>> >> >> > > > >
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > > > --
> > > >>> >>>>>>>> >> >> > > > *Mike Tutkowski*
> > > >>> >>>>>>>> >> >> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >>>>>>>> >> >> > > > e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> >> > > > o: 303.746.7302
> > > >>> >>>>>>>> >> >> > > > Advancing the way the world uses the
> > > >>> >>>>>>>> >> >> > > > cloud<
> > > >>> http://solidfire.com/solution/overview/?video=play
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> >> >> > > > *™*
> > > >>> >>>>>>>> >> >> > > >
> > > >>> >>>>>>>> >> >> > >
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >> > --
> > > >>> >>>>>>>> >> >> > *Mike Tutkowski*
> > > >>> >>>>>>>> >> >> > *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >>>>>>>> >> >> > e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> >> > o: 303.746.7302
> > > >>> >>>>>>>> >> >> > Advancing the way the world uses the
> > > >>> >>>>>>>> >> >> > cloud<
> > > http://solidfire.com/solution/overview/?video=play
> > > >>> >
> > > >>> >>>>>>>> >> >> > *™*
> > > >>> >>>>>>>> >> >> >
> > > >>> >>>>>>>> >> >>
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> >
> > > >>> >>>>>>>> >> > --
> > > >>> >>>>>>>> >> > *Mike Tutkowski*
> > > >>> >>>>>>>> >> > *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >>>>>>>> >> > e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> >> > o: 303.746.7302
> > > >>> >>>>>>>> >> > Advancing the way the world uses the
> > > >>> >>>>>>>> >> > cloud<
> > http://solidfire.com/solution/overview/?video=play
> > > >
> > > >>> >>>>>>>> >> > *™*
> > > >>> >>>>>>>> >>
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> >
> > > >>> >>>>>>>> > --
> > > >>> >>>>>>>> > *Mike Tutkowski*
> > > >>> >>>>>>>> > *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >>>>>>>> > e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>>> > o: 303.746.7302
> > > >>> >>>>>>>> > Advancing the way the world uses the
> > > >>> >>>>>>>> > cloud<
> http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>>>>>> > *™*
> > > >>> >>>>>>>>
> > > >>> >>>>>>>
> > > >>> >>>>>>>
> > > >>> >>>>>>>
> > > >>> >>>>>>> --
> > > >>> >>>>>>> *Mike Tutkowski*
> > > >>> >>>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >>>>>>> e: mike.tutkow...@solidfire.com
> > > >>> >>>>>>> o: 303.746.7302
> > > >>> >>>>>>> Advancing the way the world uses the cloud<
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>>>>> *™*
> > > >>> >>>>>>>
> > > >>> >>>>>>
> > > >>> >>>>>>
> > > >>> >>>>>>
> > > >>> >>>>>> --
> > > >>> >>>>>> *Mike Tutkowski*
> > > >>> >>>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >>>>>> e: mike.tutkow...@solidfire.com
> > > >>> >>>>>> o: 303.746.7302
> > > >>> >>>>>> Advancing the way the world uses the cloud<
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>>>> *™*
> > > >>> >>>>>>
> > > >>> >>>>>
> > > >>> >>>>>
> > > >>> >>>>>
> > > >>> >>>>> --
> > > >>> >>>>> *Mike Tutkowski*
> > > >>> >>>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >>>>> e: mike.tutkow...@solidfire.com
> > > >>> >>>>> o: 303.746.7302
> > > >>> >>>>> Advancing the way the world uses the cloud<
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>>> *™*
> > > >>> >>>>>
> > > >>> >>>>
> > > >>> >>>>
> > > >>> >>>>
> > > >>> >>>> --
> > > >>> >>>> *Mike Tutkowski*
> > > >>> >>>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >>>> e: mike.tutkow...@solidfire.com
> > > >>> >>>> o: 303.746.7302
> > > >>> >>>> Advancing the way the world uses the cloud<
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>> >>>> *™*
> > > >>> >>>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>>
> > > >>> >>> --
> > > >>> >>> *Mike Tutkowski*
> > > >>> >>> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >>> e: mike.tutkow...@solidfire.com
> > > >>> >>> o: 303.746.7302
> > > >>> >>> Advancing the way the world uses the cloud<
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>> >>> *™*
> > > >>> >>>
> > > >>> >>
> > > >>> >>
> > > >>> >>
> > > >>> >> --
> > > >>> >> *Mike Tutkowski*
> > > >>> >> *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> >> e: mike.tutkow...@solidfire.com
> > > >>> >> o: 303.746.7302
> > > >>> >> Advancing the way the world uses the cloud<
> > > >>> http://solidfire.com/solution/overview/?video=play>
> > > >>> >> *™*
> > > >>> >>
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > *Mike Tutkowski*
> > > >>> > *Senior CloudStack Developer, SolidFire Inc.*
> > > >>> > e: mike.tutkow...@solidfire.com
> > > >>> > o: 303.746.7302
> > > >>> > Advancing the way the world uses the
> > > >>> > cloud<http://solidfire.com/solution/overview/?video=play>
> > > >>> > *™*
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> *Mike Tutkowski*
> > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > >> e: mike.tutkow...@solidfire.com
> > > >> o: 303.746.7302
> > > >> Advancing the way the world uses the cloud<
> > > http://solidfire.com/solution/overview/?video=play>
> > > >> *™*
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > *Mike Tutkowski*
> > > > *Senior CloudStack Developer, SolidFire Inc.*
> > > > e: mike.tutkow...@solidfire.com
> > > > o: 303.746.7302
> > > > Advancing the way the world uses the
> > > > cloud<http://solidfire.com/solution/overview/?video=play>
> > > > *™*
> > >
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkow...@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud<http://solidfire.com/solution/overview/?video=play>
> > *™*
> >
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud<http://solidfire.com/solution/overview/?video=play>
*™*
