Thanks, Marcus I've been developing on Windows for most of my time, so a bunch of these Linux-type commands are new to me and I don't always interpret the output correctly. Getting there. :)
On Mon, Sep 23, 2013 at 2:37 PM, Marcus Sorensen <shadow...@gmail.com>wrote: > Nope, not running. That's just your grep process. It would look like: > > root 24429 24428 1 14:25 ? 00:00:08 jsvc.exec -cp > > /usr/share/java/commons-daemon.jar:/usr/share/java/jna.jar:/usr/share/cloudstack-agent/lib/activation-1.1.jar:/usr/share/cloudstack-agent/lib/antisamy-1.4.3.jar:/usr/share/cloudstack-agent/lib/aopalliance-1.0.jar:/usr/share/cloudstack-agent/lib/apache-log4j-extras-1.1.jar:/usr/share/cloudstack-agent/lib/aspectjrt-1.7. > > Your agent log should tell you why it failed to start if you set it in > debug and try to start... or maybe cloudstack-agent.out if it doesn't > get far enough (say it's missing a class or something and can't > start). > > On Mon, Sep 23, 2013 at 2:33 PM, Mike Tutkowski > <mike.tutkow...@solidfire.com> wrote: > > Looks like it's running, though: > > > > mtutkowski@ubuntu:~$ ps -ef | grep jsvc > > 1000 7097 7013 0 14:32 pts/1 00:00:00 grep --color=auto jsvc > > > > > > > > On Mon, Sep 23, 2013 at 2:31 PM, Mike Tutkowski < > > mike.tutkow...@solidfire.com> wrote: > > > >> Hey Marcus, > >> > >> Maybe you could give me a better idea of what the "flow" is when adding > a > >> KVM host. > >> > >> It looks like we SSH into the potential KVM host and execute a startup > >> script (giving it necessary info about the cloud and the management > server > >> it should talk to). > >> > >> After this, is the Java VM started? > >> > >> After a reboot, I assume the JVM is started automatically? > >> > >> How do you debug your KVM-side Java code? > >> > >> Been looking through the logs and nothing obvious sticks out. I will > have > >> another look. > >> > >> Thanks > >> > >> > >> On Mon, Sep 23, 2013 at 2:15 PM, Mike Tutkowski < > >> mike.tutkow...@solidfire.com> wrote: > >> > >>> Hey Marcus, > >>> > >>> I've been investigating my issue with not being able to add a KVM host > to > >>> CS. > >>> > >>> For what it's worth, this comes back successful: > >>> > >>> SSHCmdHelper.sshExecuteCmd(sshConnection, "cloudstack-setup-agent " + > >>> parameters, 3); > >>> > >>> This is what the command looks like: > >>> > >>> cloudstack-setup-agent -m 192.168.233.1 -z 1 -p 1 -c 1 -g > >>> 6b4aa1c2-2ac9-3c60-aabe-704aed40c684 -a --pubNic=cloudbr0 > --prvNic=cloudbr0 > >>> --guestNic=cloudbr0 > >>> > >>> The problem is this method in LibvirtServerDiscoverer never finds a > >>> matching host in the DB: > >>> > >>> waitForHostConnect(long dcId, long podId, long clusterId, String guid) > >>> > >>> I assume once the KVM host is up and running that it's supposed to call > >>> into the CS MS so the DB can be updated as such? > >>> > >>> If so, the problem must be on the KVM side. > >>> > >>> I did run this again (from the KVM host) to see if the connection was > in > >>> place: > >>> > >>> mtutkowski@ubuntu:~$ telnet 192.168.233.1 8250 > >>> > >>> Trying 192.168.233.1... > >>> > >>> Connected to 192.168.233.1. > >>> > >>> Escape character is '^]'. > >>> So that looks good. > >>> > >>> I turned on more info in the debug log, but nothing obvious jumps out > as > >>> of yet. > >>> > >>> If you have any thoughts on this, please shoot them my way. :) > >>> > >>> Thanks! > >>> > >>> > >>> On Sun, Sep 22, 2013 at 12:11 AM, Mike Tutkowski < > >>> mike.tutkow...@solidfire.com> wrote: > >>> > >>>> First step is for me to get this working for KVM, though. :) > >>>> > >>>> Once I do that, I can perhaps make modifications to the storage > >>>> framework and hypervisor plug-ins to refactor the logic and such. > >>>> > >>>> > >>>> On Sun, Sep 22, 2013 at 12:09 AM, Mike Tutkowski < > >>>> mike.tutkow...@solidfire.com> wrote: > >>>> > >>>>> Same would work for KVM. > >>>>> > >>>>> If CreateCommand and DestroyCommand were called at the appropriate > >>>>> times by the storage framework, I could move my connect and > disconnect > >>>>> logic out of the attach/detach logic. > >>>>> > >>>>> > >>>>> On Sun, Sep 22, 2013 at 12:08 AM, Mike Tutkowski < > >>>>> mike.tutkow...@solidfire.com> wrote: > >>>>> > >>>>>> Conversely, if the storage framework called the DestroyCommand for > >>>>>> managed storage after the DetachCommand, then I could have had my > remove > >>>>>> SR/datastore logic placed in the DestroyCommand handling rather > than in the > >>>>>> DetachCommand handling. > >>>>>> > >>>>>> > >>>>>> On Sun, Sep 22, 2013 at 12:06 AM, Mike Tutkowski < > >>>>>> mike.tutkow...@solidfire.com> wrote: > >>>>>> > >>>>>>> Edison's plug-in calls the CreateCommand. Mine does not. > >>>>>>> > >>>>>>> The initial approach that was discussed during 4.2 was for me to > >>>>>>> modify the attach/detach logic only in the XenServer and VMware > hypervisor > >>>>>>> plug-ins. > >>>>>>> > >>>>>>> Now that I think about it more, though, I kind of would have liked > to > >>>>>>> have the storage framework send a CreateCommand to the hypervisor > before > >>>>>>> sending the AttachCommand if the storage in question was managed. > >>>>>>> > >>>>>>> Then I could have created my SR/datastore in the CreateCommand and > >>>>>>> the AttachCommand would have had the SR/datastore that it was > always > >>>>>>> expecting (and I wouldn't have had to create the SR/datastore in > the > >>>>>>> AttachCommand). > >>>>>>> > >>>>>>> > >>>>>>> On Sat, Sep 21, 2013 at 11:56 PM, Marcus Sorensen < > >>>>>>> shadow...@gmail.com> wrote: > >>>>>>> > >>>>>>>> Yeah, I think it probably is as well, but I figured you'd be in a > >>>>>>>> better position to tell. > >>>>>>>> > >>>>>>>> I see that copyAsync is unsupported in your current 4.2 driver, > does > >>>>>>>> that mean that there's no template support? Or is it some other > call > >>>>>>>> that does templating now? I'm still getting up to speed on all of > the > >>>>>>>> 4.2 changes. I was just looking at CreateCommand in > >>>>>>>> LibvirtComputingResource, since that's the only place > >>>>>>>> createPhysicalDisk is called, and it occurred to me that > >>>>>>>> CreateCommand > >>>>>>>> might be skipped altogether when utilizing storage plugins. > >>>>>>>> > >>>>>>>> On Sat, Sep 21, 2013 at 11:38 PM, Mike Tutkowski > >>>>>>>> <mike.tutkow...@solidfire.com> wrote: > >>>>>>>> > That's an interesting comment, Marcus. > >>>>>>>> > > >>>>>>>> > It was my intent that it should work with any CloudStack > "managed" > >>>>>>>> storage > >>>>>>>> > that uses an iSCSI target. Even though I'm using CHAP, I wrote > the > >>>>>>>> code so > >>>>>>>> > CHAP didn't have to be used. > >>>>>>>> > > >>>>>>>> > As I'm doing my testing, I can try to think about whether it is > >>>>>>>> generic > >>>>>>>> > enough to keep those names or not. > >>>>>>>> > > >>>>>>>> > My expectation is that it is generic enough. > >>>>>>>> > > >>>>>>>> > > >>>>>>>> > On Sat, Sep 21, 2013 at 11:32 PM, Marcus Sorensen < > >>>>>>>> shadow...@gmail.com>wrote: > >>>>>>>> > > >>>>>>>> >> I added a comment to your diff. In general I think it looks > good, > >>>>>>>> >> though I obviously can't vouch for whether or not it will work. > >>>>>>>> One > >>>>>>>> >> thing I do have reservations about is the adaptor/pool naming. > If > >>>>>>>> you > >>>>>>>> >> think the code is generic enough that it will work for anyone > who > >>>>>>>> does > >>>>>>>> >> an iscsi LUN-per-volume plugin, then it's OK, but if there's > >>>>>>>> anything > >>>>>>>> >> about it that's specific to YOUR iscsi target or how it likes > to > >>>>>>>> be > >>>>>>>> >> treated then I'd say that they should be named something less > >>>>>>>> generic > >>>>>>>> >> than iScsiAdmStorage. > >>>>>>>> >> > >>>>>>>> >> On Sat, Sep 21, 2013 at 7:23 PM, Mike Tutkowski > >>>>>>>> >> <mike.tutkow...@solidfire.com> wrote: > >>>>>>>> >> > Great - thanks! > >>>>>>>> >> > > >>>>>>>> >> > Just to give you an overview of what my code does (for when > you > >>>>>>>> get a > >>>>>>>> >> > chance to review it): > >>>>>>>> >> > > >>>>>>>> >> > SolidFireHostListener is registered in > >>>>>>>> SolidfirePrimaryDataStoreProvider. > >>>>>>>> >> > Its hostConnect method is invoked when a host connects with > the > >>>>>>>> CS MS. If > >>>>>>>> >> > the host is running KVM, the listener sends a > >>>>>>>> ModifyStoragePoolCommand to > >>>>>>>> >> > the host. This logic was based off of DefaultHostListener. > >>>>>>>> >> > > >>>>>>>> >> > The handling of ModifyStoragePoolCommand is unchanged. It > >>>>>>>> invokes > >>>>>>>> >> > createStoragePool on the KVMStoragePoolManager. The > >>>>>>>> KVMStoragePoolManager > >>>>>>>> >> > asks for an adaptor and finds my new one: > >>>>>>>> iScsiAdmStorageAdaptor (which > >>>>>>>> >> was > >>>>>>>> >> > registered in the constructor for KVMStoragePoolManager under > >>>>>>>> the key of > >>>>>>>> >> > StoragePoolType.Iscsi.toString()). > >>>>>>>> >> > > >>>>>>>> >> > iScsiAdmStorageAdaptor.createStoragePool just makes an > instance > >>>>>>>> of > >>>>>>>> >> > iScsiAdmStoragePool, adds it to a map, and returns the > pointer > >>>>>>>> to the > >>>>>>>> >> > iScsiAdmStoragePool object. The key of the map is the UUID of > >>>>>>>> the storage > >>>>>>>> >> > pool. > >>>>>>>> >> > > >>>>>>>> >> > When a volume is attached, createPhysicalDisk is invoked for > >>>>>>>> managed > >>>>>>>> >> > storage rather than getPhysicalDisk. createPhysicalDisk uses > >>>>>>>> iscsiadm to > >>>>>>>> >> > establish the iSCSI connection to the volume on the SAN and a > >>>>>>>> >> > KVMPhysicalDisk is returned to be used in the attach logic > that > >>>>>>>> follows. > >>>>>>>> >> > > >>>>>>>> >> > When a volume is detached, getPhysicalDisk is invoked with > the > >>>>>>>> IQN of the > >>>>>>>> >> > volume if the storage pool in question is managed storage. > >>>>>>>> Otherwise, the > >>>>>>>> >> > normal vol.getPath() is used. > >>>>>>>> iScsiAdmStorageAdaptor.getPhysicalDisk just > >>>>>>>> >> > returns a new instance of KVMPhysicalDisk to be used in the > >>>>>>>> detach logic. > >>>>>>>> >> > > >>>>>>>> >> > Once the volume has been detached, > >>>>>>>> iScsiAdmStoragePool.deletePhysicalDisk > >>>>>>>> >> > is invoked if the storage pool is managed. deletePhysicalDisk > >>>>>>>> removes the > >>>>>>>> >> > iSCSI connection to the volume using iscsiadm. > >>>>>>>> >> > > >>>>>>>> >> > > >>>>>>>> >> > On Sat, Sep 21, 2013 at 5:46 PM, Marcus Sorensen < > >>>>>>>> shadow...@gmail.com > >>>>>>>> >> >wrote: > >>>>>>>> >> > > >>>>>>>> >> >> Its the log4j properties file in /etc/cloudstack/agent > change > >>>>>>>> all INFO > >>>>>>>> >> to > >>>>>>>> >> >> DEBUG. I imagine the agent just isn't starting, you can > tail > >>>>>>>> the log > >>>>>>>> >> when > >>>>>>>> >> >> you try to start the service, or maybe it will spit > something > >>>>>>>> out into > >>>>>>>> >> one > >>>>>>>> >> >> of the other files in /var/log/cloudstack/agent > >>>>>>>> >> >> On Sep 21, 2013 5:19 PM, "Mike Tutkowski" < > >>>>>>>> mike.tutkow...@solidfire.com > >>>>>>>> >> > > >>>>>>>> >> >> wrote: > >>>>>>>> >> >> > >>>>>>>> >> >> > This is how I've been trying to query for the status of > the > >>>>>>>> service (I > >>>>>>>> >> >> > assume it could be started this way, as well, by changing > >>>>>>>> "status" to > >>>>>>>> >> >> > "start" or "restart"?): > >>>>>>>> >> >> > > >>>>>>>> >> >> > mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo > >>>>>>>> /usr/sbin/service > >>>>>>>> >> >> > cloudstack-agent status > >>>>>>>> >> >> > > >>>>>>>> >> >> > I get this back: > >>>>>>>> >> >> > > >>>>>>>> >> >> > Failed to execute: * could not access PID file for > >>>>>>>> cloudstack-agent > >>>>>>>> >> >> > > >>>>>>>> >> >> > I've made a bunch of code changes recently, though, so I > >>>>>>>> think I'm > >>>>>>>> >> going > >>>>>>>> >> >> to > >>>>>>>> >> >> > rebuild and redeploy everything. > >>>>>>>> >> >> > > >>>>>>>> >> >> > The debug info sounds helpful. Where can I set > enable.debug? > >>>>>>>> >> >> > > >>>>>>>> >> >> > Thanks, Marcus! > >>>>>>>> >> >> > > >>>>>>>> >> >> > > >>>>>>>> >> >> > On Sat, Sep 21, 2013 at 4:11 PM, Marcus Sorensen < > >>>>>>>> shadow...@gmail.com > >>>>>>>> >> >> > >wrote: > >>>>>>>> >> >> > > >>>>>>>> >> >> > > OK, will check it out in the next few days. As > mentioned, > >>>>>>>> you can > >>>>>>>> >> set > >>>>>>>> >> >> up > >>>>>>>> >> >> > > your Ubuntu vm as the management server as well if all > >>>>>>>> else fails. > >>>>>>>> >> If > >>>>>>>> >> >> > you > >>>>>>>> >> >> > > can get to the mgmt server on 8250 from the KVM host, > then > >>>>>>>> you need > >>>>>>>> >> to > >>>>>>>> >> >> > > enable.debug on the agent. It won't run without > >>>>>>>> complaining loudly > >>>>>>>> >> if > >>>>>>>> >> >> it > >>>>>>>> >> >> > > can't get to the mgmt server, and I didn't see that in > >>>>>>>> your agent > >>>>>>>> >> log, > >>>>>>>> >> >> so > >>>>>>>> >> >> > > perhaps its not running. I assume you know how to > >>>>>>>> stop/start the > >>>>>>>> >> agent > >>>>>>>> >> >> on > >>>>>>>> >> >> > > KVM via 'service cloud stacks agent'. > >>>>>>>> >> >> > > On Sep 21, 2013 3:02 PM, "Mike Tutkowski" < > >>>>>>>> >> >> mike.tutkow...@solidfire.com> > >>>>>>>> >> >> > > wrote: > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > Hey Marcus, > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > I haven't yet been able to test my new code, but I > >>>>>>>> thought you > >>>>>>>> >> would > >>>>>>>> >> >> > be a > >>>>>>>> >> >> > > > good person to ask to review it: > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > >>>>>>>> >> >> > >>>>>>>> >> > >>>>>>>> > https://github.com/mike-tutkowski/incubator-cloudstack/commit/ea74b312a8a36801994500407fd54f0cdda55e37 > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > All it is supposed to do is attach and detach a data > >>>>>>>> disk (that > >>>>>>>> >> has > >>>>>>>> >> >> > > > guaranteed IOPS) with KVM as the hypervisor. The data > >>>>>>>> disk > >>>>>>>> >> happens to > >>>>>>>> >> >> > be > >>>>>>>> >> >> > > > from SolidFire-backed storage - where we have a 1:1 > >>>>>>>> mapping > >>>>>>>> >> between a > >>>>>>>> >> >> > > > CloudStack volume and a data disk. > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > There is no support for hypervisor snapshots or stuff > >>>>>>>> like that > >>>>>>>> >> >> > (likely a > >>>>>>>> >> >> > > > future release)...just attaching and detaching a data > >>>>>>>> disk in 4.3. > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > Thanks! > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > On Fri, Sep 20, 2013 at 11:05 PM, Mike Tutkowski < > >>>>>>>> >> >> > > > mike.tutkow...@solidfire.com> wrote: > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > When I re-deployed the DEBs, I didn't remove > >>>>>>>> cloudstack-agent > >>>>>>>> >> >> first. > >>>>>>>> >> >> > > > Would > >>>>>>>> >> >> > > > > that be a problem? I just did a sudo apt-get install > >>>>>>>> >> >> > cloudstack-agent. > >>>>>>>> >> >> > > > > > >>>>>>>> >> >> > > > > > >>>>>>>> >> >> > > > > On Fri, Sep 20, 2013 at 11:03 PM, Mike Tutkowski < > >>>>>>>> >> >> > > > > mike.tutkow...@solidfire.com> wrote: > >>>>>>>> >> >> > > > > > >>>>>>>> >> >> > > > >> I get the same error running the command manually: > >>>>>>>> >> >> > > > >> > >>>>>>>> >> >> > > > >> mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo > >>>>>>>> >> /usr/sbin/service > >>>>>>>> >> >> > > > >> cloudstack-agent status > >>>>>>>> >> >> > > > >> * could not access PID file for cloudstack-agent > >>>>>>>> >> >> > > > >> > >>>>>>>> >> >> > > > >> > >>>>>>>> >> >> > > > >> On Fri, Sep 20, 2013 at 11:00 PM, Mike Tutkowski < > >>>>>>>> >> >> > > > >> mike.tutkow...@solidfire.com> wrote: > >>>>>>>> >> >> > > > >> > >>>>>>>> >> >> > > > >>> agent.log looks OK to me: > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,010 INFO > >>>>>>>> [cloud.agent.AgentShell] > >>>>>>>> >> >> (main:null) > >>>>>>>> >> >> > > > Agent > >>>>>>>> >> >> > > > >>> started > >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,011 INFO > >>>>>>>> [cloud.agent.AgentShell] > >>>>>>>> >> >> (main:null) > >>>>>>>> >> >> > > > >>> Implementation Version is 4.3.0-SNAPSHOT > >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,015 INFO > >>>>>>>> [cloud.agent.AgentShell] > >>>>>>>> >> >> (main:null) > >>>>>>>> >> >> > > > >>> agent.properties found at > >>>>>>>> >> /etc/cloudstack/agent/agent.properties > >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,023 INFO > >>>>>>>> [cloud.agent.AgentShell] > >>>>>>>> >> >> (main:null) > >>>>>>>> >> >> > > > >>> Defaulting to using properties file for storage > >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,024 INFO > >>>>>>>> [cloud.agent.AgentShell] > >>>>>>>> >> >> (main:null) > >>>>>>>> >> >> > > > >>> Defaulting to the constant time backoff algorithm > >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,029 INFO > [cloud.utils.LogUtils] > >>>>>>>> >> (main:null) > >>>>>>>> >> >> > > log4j > >>>>>>>> >> >> > > > >>> configuration found at > >>>>>>>> /etc/cloudstack/agent/log4j-cloud.xml > >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,163 INFO [cloud.agent.Agent] > >>>>>>>> (main:null) > >>>>>>>> >> id > >>>>>>>> >> >> > is 3 > >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,197 INFO > >>>>>>>> >> >> > > > >>> [resource.virtualnetwork.VirtualRoutingResource] > >>>>>>>> (main:null) > >>>>>>>> >> >> > > > >>> VirtualRoutingResource _scriptDir to use: > >>>>>>>> >> >> scripts/network/domr/kvm > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >>> However, I wasn't aware that setup.log was > >>>>>>>> important. This > >>>>>>>> >> seems > >>>>>>>> >> >> to > >>>>>>>> >> >> > > be > >>>>>>>> >> >> > > > a > >>>>>>>> >> >> > > > >>> problem, but I'm not sure what it might indicate: > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service > >>>>>>>> cloudstack-agent > >>>>>>>> >> status > >>>>>>>> >> >> > > > >>> DEBUG:root:Failed to execute: * could not access > PID > >>>>>>>> file for > >>>>>>>> >> >> > > > >>> cloudstack-agent > >>>>>>>> >> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service > >>>>>>>> cloudstack-agent > >>>>>>>> >> start > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >>> On Fri, Sep 20, 2013 at 10:53 PM, Marcus Sorensen > < > >>>>>>>> >> >> > > shadow...@gmail.com > >>>>>>>> >> >> > > > >wrote: > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >>>> Sorry, I saw that in the log, I thought it was > the > >>>>>>>> agent log > >>>>>>>> >> for > >>>>>>>> >> >> > > some > >>>>>>>> >> >> > > > >>>> reason. Is the agent started? That might be the > >>>>>>>> place to > >>>>>>>> >> look. > >>>>>>>> >> >> > There > >>>>>>>> >> >> > > > is > >>>>>>>> >> >> > > > >>>> an > >>>>>>>> >> >> > > > >>>> agent log for the agent and one for the setup > when > >>>>>>>> it adds > >>>>>>>> >> the > >>>>>>>> >> >> > host, > >>>>>>>> >> >> > > > >>>> both > >>>>>>>> >> >> > > > >>>> in /var/log > >>>>>>>> >> >> > > > >>>> On Sep 20, 2013 10:42 PM, "Mike Tutkowski" < > >>>>>>>> >> >> > > > >>>> mike.tutkow...@solidfire.com> > >>>>>>>> >> >> > > > >>>> wrote: > >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > Is it saying that the MS is at the IP address > or > >>>>>>>> the KVM > >>>>>>>> >> host? > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > The KVM host is at 192.168.233.10. > >>>>>>>> >> >> > > > >>>> > The MS host is at 192.168.233.1. > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > I see this for my host Global Settings > parameter: > >>>>>>>> >> >> > > > >>>> > hostThe ip address of management > >>>>>>>> server192.168.233.1 > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > /etc/cloudstack/agent/agent.properties has a > >>>>>>>> >> >> host=192.168.233.1 > >>>>>>>> >> >> > > > value. > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > On Fri, Sep 20, 2013 at 10:21 PM, Marcus > Sorensen > >>>>>>>> < > >>>>>>>> >> >> > > > >>>> shadow...@gmail.com > >>>>>>>> >> >> > > > >>>> > >wrote: > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > > The log says your mgmt server is > >>>>>>>> 192.168.233.10? But you > >>>>>>>> >> >> tried > >>>>>>>> >> >> > > to > >>>>>>>> >> >> > > > >>>> telnet > >>>>>>>> >> >> > > > >>>> > to > >>>>>>>> >> >> > > > >>>> > > 192.168.233.1? It might be enough to change > >>>>>>>> that in > >>>>>>>> >> >> > > > >>>> > > /etc/cloudstack/agent/agent.properties, but > you > >>>>>>>> may want > >>>>>>>> >> to > >>>>>>>> >> >> > edit > >>>>>>>> >> >> > > > the > >>>>>>>> >> >> > > > >>>> > config > >>>>>>>> >> >> > > > >>>> > > as well to tell it the real ms IP. > >>>>>>>> >> >> > > > >>>> > > On Sep 20, 2013 10:12 PM, "Mike Tutkowski" < > >>>>>>>> >> >> > > > >>>> mike.tutkow...@solidfire.com > >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > wrote: > >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > Here's what my /etc/network/interfaces file > >>>>>>>> looks > >>>>>>>> >> like, if > >>>>>>>> >> >> > > that > >>>>>>>> >> >> > > > >>>> is of > >>>>>>>> >> >> > > > >>>> > > > interest (the 192.168.233.0 network is the > >>>>>>>> NAT network > >>>>>>>> >> >> > VMware > >>>>>>>> >> >> > > > >>>> Fusion > >>>>>>>> >> >> > > > >>>> > set > >>>>>>>> >> >> > > > >>>> > > > up): > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > auto lo > >>>>>>>> >> >> > > > >>>> > > > iface lo inet loopback > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > auto eth0 > >>>>>>>> >> >> > > > >>>> > > > iface eth0 inet manual > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > auto cloudbr0 > >>>>>>>> >> >> > > > >>>> > > > iface cloudbr0 inet static > >>>>>>>> >> >> > > > >>>> > > > address 192.168.233.10 > >>>>>>>> >> >> > > > >>>> > > > netmask 255.255.255.0 > >>>>>>>> >> >> > > > >>>> > > > network 192.168.233.0 > >>>>>>>> >> >> > > > >>>> > > > broadcast 192.168.233.255 > >>>>>>>> >> >> > > > >>>> > > > dns-nameservers 8.8.8.8 > >>>>>>>> >> >> > > > >>>> > > > bridge_ports eth0 > >>>>>>>> >> >> > > > >>>> > > > bridge_fd 5 > >>>>>>>> >> >> > > > >>>> > > > bridge_stp off > >>>>>>>> >> >> > > > >>>> > > > bridge_maxwait 1 > >>>>>>>> >> >> > > > >>>> > > > post-up route add default gw > >>>>>>>> 192.168.233.2 metric 1 > >>>>>>>> >> >> > > > >>>> > > > pre-down route del default gw > >>>>>>>> 192.168.233.2 > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > On Fri, Sep 20, 2013 at 10:08 PM, Mike > >>>>>>>> Tutkowski < > >>>>>>>> >> >> > > > >>>> > > > mike.tutkow...@solidfire.com> wrote: > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > You appear to be correct. This is from > the > >>>>>>>> MS log > >>>>>>>> >> >> (below). > >>>>>>>> >> >> > > > >>>> Discovery > >>>>>>>> >> >> > > > >>>> > > > timed > >>>>>>>> >> >> > > > >>>> > > > > out. > >>>>>>>> >> >> > > > >>>> > > > > > >>>>>>>> >> >> > > > >>>> > > > > I'm not sure why this would be. My > network > >>>>>>>> settings > >>>>>>>> >> >> > > shouldn't > >>>>>>>> >> >> > > > >>>> have > >>>>>>>> >> >> > > > >>>> > > > changed > >>>>>>>> >> >> > > > >>>> > > > > since the last time I tried this. > >>>>>>>> >> >> > > > >>>> > > > > > >>>>>>>> >> >> > > > >>>> > > > > I am able to ping the KVM host from the > MS > >>>>>>>> host and > >>>>>>>> >> vice > >>>>>>>> >> >> > > > versa. > >>>>>>>> >> >> > > > >>>> > > > > > >>>>>>>> >> >> > > > >>>> > > > > I'm even able to manually kick off a VM > on > >>>>>>>> the KVM > >>>>>>>> >> host > >>>>>>>> >> >> > and > >>>>>>>> >> >> > > > >>>> ping from > >>>>>>>> >> >> > > > >>>> > > it > >>>>>>>> >> >> > > > >>>> > > > > to the MS host. > >>>>>>>> >> >> > > > >>>> > > > > > >>>>>>>> >> >> > > > >>>> > > > > I am using NAT from my Mac OS X host > (also > >>>>>>>> running > >>>>>>>> >> the > >>>>>>>> >> >> CS > >>>>>>>> >> >> > > MS) > >>>>>>>> >> >> > > > >>>> to the > >>>>>>>> >> >> > > > >>>> > VM > >>>>>>>> >> >> > > > >>>> > > > > running KVM (VMware Fusion). > >>>>>>>> >> >> > > > >>>> > > > > > >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,141 DEBUG > >>>>>>>> >> >> > > > >>>> [c.c.h.k.d.LibvirtServerDiscoverer] > >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3 > :ctx-6b28dc48) > >>>>>>>> Timeout, > >>>>>>>> >> to > >>>>>>>> >> >> > wait > >>>>>>>> >> >> > > > for > >>>>>>>> >> >> > > > >>>> the > >>>>>>>> >> >> > > > >>>> > > host > >>>>>>>> >> >> > > > >>>> > > > > connecting to mgt svr, assuming it is > failed > >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,144 WARN > >>>>>>>> >> >> [c.c.r.ResourceManagerImpl] > >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3 > :ctx-6b28dc48) > >>>>>>>> Unable to > >>>>>>>> >> >> find > >>>>>>>> >> >> > > the > >>>>>>>> >> >> > > > >>>> server > >>>>>>>> >> >> > > > >>>> > > > > resources at http://192.168.233.10 > >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,145 INFO > >>>>>>>> >> >> > [c.c.u.e.CSExceptionErrorCode] > >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3 > :ctx-6b28dc48) > >>>>>>>> Could not > >>>>>>>> >> >> find > >>>>>>>> >> >> > > > >>>> exception: > >>>>>>>> >> >> > > > >>>> > > > > com.cloud.exception.DiscoveryException in > >>>>>>>> error code > >>>>>>>> >> >> list > >>>>>>>> >> >> > > for > >>>>>>>> >> >> > > > >>>> > > exceptions > >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,147 WARN > >>>>>>>> >> [o.a.c.a.c.a.h.AddHostCmd] > >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3 > :ctx-6b28dc48) > >>>>>>>> Exception: > >>>>>>>> >> >> > > > >>>> > > > > com.cloud.exception.DiscoveryException: > >>>>>>>> Unable to add > >>>>>>>> >> >> the > >>>>>>>> >> >> > > host > >>>>>>>> >> >> > > > >>>> > > > > at > >>>>>>>> >> >> > > > >>>> > > > > > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > >>>>>>>> >> >> > >>>>>>>> >> > >>>>>>>> > com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:778) > >>>>>>>> >> >> > > > >>>> > > > > > >>>>>>>> >> >> > > > >>>> > > > > I do seem to be able to telnet in from my > >>>>>>>> KVM host to > >>>>>>>> >> >> the > >>>>>>>> >> >> > MS > >>>>>>>> >> >> > > > >>>> host's > >>>>>>>> >> >> > > > >>>> > > 8250 > >>>>>>>> >> >> > > > >>>> > > > > port: > >>>>>>>> >> >> > > > >>>> > > > > > >>>>>>>> >> >> > > > >>>> > > > > mtutkowski@ubuntu:~$ telnet > 192.168.233.1 > >>>>>>>> 8250 > >>>>>>>> >> >> > > > >>>> > > > > Trying 192.168.233.1... > >>>>>>>> >> >> > > > >>>> > > > > Connected to 192.168.233.1. > >>>>>>>> >> >> > > > >>>> > > > > Escape character is '^]'. > >>>>>>>> >> >> > > > >>>> > > > > > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > -- > >>>>>>>> >> >> > > > >>>> > > > *Mike Tutkowski* > >>>>>>>> >> >> > > > >>>> > > > *Senior CloudStack Developer, SolidFire > Inc.* > >>>>>>>> >> >> > > > >>>> > > > e: mike.tutkow...@solidfire.com > >>>>>>>> >> >> > > > >>>> > > > o: 303.746.7302 > >>>>>>>> >> >> > > > >>>> > > > Advancing the way the world uses the > >>>>>>>> >> >> > > > >>>> > > > cloud< > >>>>>>>> >> http://solidfire.com/solution/overview/?video=play> > >>>>>>>> >> >> > > > >>>> > > > *™* > >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > -- > >>>>>>>> >> >> > > > >>>> > *Mike Tutkowski* > >>>>>>>> >> >> > > > >>>> > *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>> >> >> > > > >>>> > e: mike.tutkow...@solidfire.com > >>>>>>>> >> >> > > > >>>> > o: 303.746.7302 > >>>>>>>> >> >> > > > >>>> > Advancing the way the world uses the > >>>>>>>> >> >> > > > >>>> > cloud< > >>>>>>>> http://solidfire.com/solution/overview/?video=play> > >>>>>>>> >> >> > > > >>>> > *™* > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >>> -- > >>>>>>>> >> >> > > > >>> *Mike Tutkowski* > >>>>>>>> >> >> > > > >>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>> >> >> > > > >>> e: mike.tutkow...@solidfire.com > >>>>>>>> >> >> > > > >>> o: 303.746.7302 > >>>>>>>> >> >> > > > >>> Advancing the way the world uses the cloud< > >>>>>>>> >> >> > > > http://solidfire.com/solution/overview/?video=play> > >>>>>>>> >> >> > > > >>> *™* > >>>>>>>> >> >> > > > >>> > >>>>>>>> >> >> > > > >> > >>>>>>>> >> >> > > > >> > >>>>>>>> >> >> > > > >> > >>>>>>>> >> >> > > > >> -- > >>>>>>>> >> >> > > > >> *Mike Tutkowski* > >>>>>>>> >> >> > > > >> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>> >> >> > > > >> e: mike.tutkow...@solidfire.com > >>>>>>>> >> >> > > > >> o: 303.746.7302 > >>>>>>>> >> >> > > > >> Advancing the way the world uses the cloud< > >>>>>>>> >> >> > > > http://solidfire.com/solution/overview/?video=play> > >>>>>>>> >> >> > > > >> *™* > >>>>>>>> >> >> > > > >> > >>>>>>>> >> >> > > > > > >>>>>>>> >> >> > > > > > >>>>>>>> >> >> > > > > > >>>>>>>> >> >> > > > > -- > >>>>>>>> >> >> > > > > *Mike Tutkowski* > >>>>>>>> >> >> > > > > *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>> >> >> > > > > e: mike.tutkow...@solidfire.com > >>>>>>>> >> >> > > > > o: 303.746.7302 > >>>>>>>> >> >> > > > > Advancing the way the world uses the cloud< > >>>>>>>> >> >> > > > http://solidfire.com/solution/overview/?video=play> > >>>>>>>> >> >> > > > > *™* > >>>>>>>> >> >> > > > > > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > -- > >>>>>>>> >> >> > > > *Mike Tutkowski* > >>>>>>>> >> >> > > > *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>> >> >> > > > e: mike.tutkow...@solidfire.com > >>>>>>>> >> >> > > > o: 303.746.7302 > >>>>>>>> >> >> > > > Advancing the way the world uses the > >>>>>>>> >> >> > > > cloud< > http://solidfire.com/solution/overview/?video=play > >>>>>>>> > > >>>>>>>> >> >> > > > *™* > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > >>>>>>>> >> >> > > >>>>>>>> >> >> > > >>>>>>>> >> >> > -- > >>>>>>>> >> >> > *Mike Tutkowski* > >>>>>>>> >> >> > *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>> >> >> > e: mike.tutkow...@solidfire.com > >>>>>>>> >> >> > o: 303.746.7302 > >>>>>>>> >> >> > Advancing the way the world uses the > >>>>>>>> >> >> > cloud<http://solidfire.com/solution/overview/?video=play> > >>>>>>>> >> >> > *™* > >>>>>>>> >> >> > > >>>>>>>> >> >> > >>>>>>>> >> > > >>>>>>>> >> > > >>>>>>>> >> > > >>>>>>>> >> > -- > >>>>>>>> >> > *Mike Tutkowski* > >>>>>>>> >> > *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>> >> > e: mike.tutkow...@solidfire.com > >>>>>>>> >> > o: 303.746.7302 > >>>>>>>> >> > Advancing the way the world uses the > >>>>>>>> >> > cloud<http://solidfire.com/solution/overview/?video=play> > >>>>>>>> >> > *™* > >>>>>>>> >> > >>>>>>>> > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> > -- > >>>>>>>> > *Mike Tutkowski* > >>>>>>>> > *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>>> > e: mike.tutkow...@solidfire.com > >>>>>>>> > o: 303.746.7302 > >>>>>>>> > Advancing the way the world uses the > >>>>>>>> > cloud<http://solidfire.com/solution/overview/?video=play> > >>>>>>>> > *™* > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> *Mike Tutkowski* > >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>>> e: mike.tutkow...@solidfire.com > >>>>>>> o: 303.746.7302 > >>>>>>> Advancing the way the world uses the cloud< > http://solidfire.com/solution/overview/?video=play> > >>>>>>> *™* > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> *Mike Tutkowski* > >>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>>> e: mike.tutkow...@solidfire.com > >>>>>> o: 303.746.7302 > >>>>>> Advancing the way the world uses the cloud< > http://solidfire.com/solution/overview/?video=play> > >>>>>> *™* > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> *Mike Tutkowski* > >>>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>>> e: mike.tutkow...@solidfire.com > >>>>> o: 303.746.7302 > >>>>> Advancing the way the world uses the cloud< > http://solidfire.com/solution/overview/?video=play> > >>>>> *™* > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> *Mike Tutkowski* > >>>> *Senior CloudStack Developer, SolidFire Inc.* > >>>> e: mike.tutkow...@solidfire.com > >>>> o: 303.746.7302 > >>>> Advancing the way the world uses the cloud< > http://solidfire.com/solution/overview/?video=play> > >>>> *™* > >>>> > >>> > >>> > >>> > >>> -- > >>> *Mike Tutkowski* > >>> *Senior CloudStack Developer, SolidFire Inc.* > >>> e: mike.tutkow...@solidfire.com > >>> o: 303.746.7302 > >>> Advancing the way the world uses the cloud< > http://solidfire.com/solution/overview/?video=play> > >>> *™* > >>> > >> > >> > >> > >> -- > >> *Mike Tutkowski* > >> *Senior CloudStack Developer, SolidFire Inc.* > >> e: mike.tutkow...@solidfire.com > >> o: 303.746.7302 > >> Advancing the way the world uses the cloud< > http://solidfire.com/solution/overview/?video=play> > >> *™* > >> > > > > > > > > -- > > *Mike Tutkowski* > > *Senior CloudStack Developer, SolidFire Inc.* > > e: mike.tutkow...@solidfire.com > > o: 303.746.7302 > > Advancing the way the world uses the > > cloud<http://solidfire.com/solution/overview/?video=play> > > *™* > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play> *™*