Nope, not running. That's just your grep process. It would look like: root 24429 24428 1 14:25 ? 00:00:08 jsvc.exec -cp /usr/share/java/commons-daemon.jar:/usr/share/java/jna.jar:/usr/share/cloudstack-agent/lib/activation-1.1.jar:/usr/share/cloudstack-agent/lib/antisamy-1.4.3.jar:/usr/share/cloudstack-agent/lib/aopalliance-1.0.jar:/usr/share/cloudstack-agent/lib/apache-log4j-extras-1.1.jar:/usr/share/cloudstack-agent/lib/aspectjrt-1.7.
Your agent log should tell you why it failed to start if you set it in debug and try to start... or maybe cloudstack-agent.out if it doesn't get far enough (say it's missing a class or something and can't start). On Mon, Sep 23, 2013 at 2:33 PM, Mike Tutkowski <mike.tutkow...@solidfire.com> wrote: > Looks like it's running, though: > > mtutkowski@ubuntu:~$ ps -ef | grep jsvc > 1000 7097 7013 0 14:32 pts/1 00:00:00 grep --color=auto jsvc > > > > On Mon, Sep 23, 2013 at 2:31 PM, Mike Tutkowski < > mike.tutkow...@solidfire.com> wrote: > >> Hey Marcus, >> >> Maybe you could give me a better idea of what the "flow" is when adding a >> KVM host. >> >> It looks like we SSH into the potential KVM host and execute a startup >> script (giving it necessary info about the cloud and the management server >> it should talk to). >> >> After this, is the Java VM started? >> >> After a reboot, I assume the JVM is started automatically? >> >> How do you debug your KVM-side Java code? >> >> Been looking through the logs and nothing obvious sticks out. I will have >> another look. >> >> Thanks >> >> >> On Mon, Sep 23, 2013 at 2:15 PM, Mike Tutkowski < >> mike.tutkow...@solidfire.com> wrote: >> >>> Hey Marcus, >>> >>> I've been investigating my issue with not being able to add a KVM host to >>> CS. >>> >>> For what it's worth, this comes back successful: >>> >>> SSHCmdHelper.sshExecuteCmd(sshConnection, "cloudstack-setup-agent " + >>> parameters, 3); >>> >>> This is what the command looks like: >>> >>> cloudstack-setup-agent -m 192.168.233.1 -z 1 -p 1 -c 1 -g >>> 6b4aa1c2-2ac9-3c60-aabe-704aed40c684 -a --pubNic=cloudbr0 --prvNic=cloudbr0 >>> --guestNic=cloudbr0 >>> >>> The problem is this method in LibvirtServerDiscoverer never finds a >>> matching host in the DB: >>> >>> waitForHostConnect(long dcId, long podId, long clusterId, String guid) >>> >>> I assume once the KVM host is up and running that it's supposed to call >>> into the CS MS so the DB can be updated as such? >>> >>> If so, the problem must be on the KVM side. >>> >>> I did run this again (from the KVM host) to see if the connection was in >>> place: >>> >>> mtutkowski@ubuntu:~$ telnet 192.168.233.1 8250 >>> >>> Trying 192.168.233.1... >>> >>> Connected to 192.168.233.1. >>> >>> Escape character is '^]'. >>> So that looks good. >>> >>> I turned on more info in the debug log, but nothing obvious jumps out as >>> of yet. >>> >>> If you have any thoughts on this, please shoot them my way. :) >>> >>> Thanks! >>> >>> >>> On Sun, Sep 22, 2013 at 12:11 AM, Mike Tutkowski < >>> mike.tutkow...@solidfire.com> wrote: >>> >>>> First step is for me to get this working for KVM, though. :) >>>> >>>> Once I do that, I can perhaps make modifications to the storage >>>> framework and hypervisor plug-ins to refactor the logic and such. >>>> >>>> >>>> On Sun, Sep 22, 2013 at 12:09 AM, Mike Tutkowski < >>>> mike.tutkow...@solidfire.com> wrote: >>>> >>>>> Same would work for KVM. >>>>> >>>>> If CreateCommand and DestroyCommand were called at the appropriate >>>>> times by the storage framework, I could move my connect and disconnect >>>>> logic out of the attach/detach logic. >>>>> >>>>> >>>>> On Sun, Sep 22, 2013 at 12:08 AM, Mike Tutkowski < >>>>> mike.tutkow...@solidfire.com> wrote: >>>>> >>>>>> Conversely, if the storage framework called the DestroyCommand for >>>>>> managed storage after the DetachCommand, then I could have had my remove >>>>>> SR/datastore logic placed in the DestroyCommand handling rather than in >>>>>> the >>>>>> DetachCommand handling. >>>>>> >>>>>> >>>>>> On Sun, Sep 22, 2013 at 12:06 AM, Mike Tutkowski < >>>>>> mike.tutkow...@solidfire.com> wrote: >>>>>> >>>>>>> Edison's plug-in calls the CreateCommand. Mine does not. >>>>>>> >>>>>>> The initial approach that was discussed during 4.2 was for me to >>>>>>> modify the attach/detach logic only in the XenServer and VMware >>>>>>> hypervisor >>>>>>> plug-ins. >>>>>>> >>>>>>> Now that I think about it more, though, I kind of would have liked to >>>>>>> have the storage framework send a CreateCommand to the hypervisor before >>>>>>> sending the AttachCommand if the storage in question was managed. >>>>>>> >>>>>>> Then I could have created my SR/datastore in the CreateCommand and >>>>>>> the AttachCommand would have had the SR/datastore that it was always >>>>>>> expecting (and I wouldn't have had to create the SR/datastore in the >>>>>>> AttachCommand). >>>>>>> >>>>>>> >>>>>>> On Sat, Sep 21, 2013 at 11:56 PM, Marcus Sorensen < >>>>>>> shadow...@gmail.com> wrote: >>>>>>> >>>>>>>> Yeah, I think it probably is as well, but I figured you'd be in a >>>>>>>> better position to tell. >>>>>>>> >>>>>>>> I see that copyAsync is unsupported in your current 4.2 driver, does >>>>>>>> that mean that there's no template support? Or is it some other call >>>>>>>> that does templating now? I'm still getting up to speed on all of the >>>>>>>> 4.2 changes. I was just looking at CreateCommand in >>>>>>>> LibvirtComputingResource, since that's the only place >>>>>>>> createPhysicalDisk is called, and it occurred to me that >>>>>>>> CreateCommand >>>>>>>> might be skipped altogether when utilizing storage plugins. >>>>>>>> >>>>>>>> On Sat, Sep 21, 2013 at 11:38 PM, Mike Tutkowski >>>>>>>> <mike.tutkow...@solidfire.com> wrote: >>>>>>>> > That's an interesting comment, Marcus. >>>>>>>> > >>>>>>>> > It was my intent that it should work with any CloudStack "managed" >>>>>>>> storage >>>>>>>> > that uses an iSCSI target. Even though I'm using CHAP, I wrote the >>>>>>>> code so >>>>>>>> > CHAP didn't have to be used. >>>>>>>> > >>>>>>>> > As I'm doing my testing, I can try to think about whether it is >>>>>>>> generic >>>>>>>> > enough to keep those names or not. >>>>>>>> > >>>>>>>> > My expectation is that it is generic enough. >>>>>>>> > >>>>>>>> > >>>>>>>> > On Sat, Sep 21, 2013 at 11:32 PM, Marcus Sorensen < >>>>>>>> shadow...@gmail.com>wrote: >>>>>>>> > >>>>>>>> >> I added a comment to your diff. In general I think it looks good, >>>>>>>> >> though I obviously can't vouch for whether or not it will work. >>>>>>>> One >>>>>>>> >> thing I do have reservations about is the adaptor/pool naming. If >>>>>>>> you >>>>>>>> >> think the code is generic enough that it will work for anyone who >>>>>>>> does >>>>>>>> >> an iscsi LUN-per-volume plugin, then it's OK, but if there's >>>>>>>> anything >>>>>>>> >> about it that's specific to YOUR iscsi target or how it likes to >>>>>>>> be >>>>>>>> >> treated then I'd say that they should be named something less >>>>>>>> generic >>>>>>>> >> than iScsiAdmStorage. >>>>>>>> >> >>>>>>>> >> On Sat, Sep 21, 2013 at 7:23 PM, Mike Tutkowski >>>>>>>> >> <mike.tutkow...@solidfire.com> wrote: >>>>>>>> >> > Great - thanks! >>>>>>>> >> > >>>>>>>> >> > Just to give you an overview of what my code does (for when you >>>>>>>> get a >>>>>>>> >> > chance to review it): >>>>>>>> >> > >>>>>>>> >> > SolidFireHostListener is registered in >>>>>>>> SolidfirePrimaryDataStoreProvider. >>>>>>>> >> > Its hostConnect method is invoked when a host connects with the >>>>>>>> CS MS. If >>>>>>>> >> > the host is running KVM, the listener sends a >>>>>>>> ModifyStoragePoolCommand to >>>>>>>> >> > the host. This logic was based off of DefaultHostListener. >>>>>>>> >> > >>>>>>>> >> > The handling of ModifyStoragePoolCommand is unchanged. It >>>>>>>> invokes >>>>>>>> >> > createStoragePool on the KVMStoragePoolManager. The >>>>>>>> KVMStoragePoolManager >>>>>>>> >> > asks for an adaptor and finds my new one: >>>>>>>> iScsiAdmStorageAdaptor (which >>>>>>>> >> was >>>>>>>> >> > registered in the constructor for KVMStoragePoolManager under >>>>>>>> the key of >>>>>>>> >> > StoragePoolType.Iscsi.toString()). >>>>>>>> >> > >>>>>>>> >> > iScsiAdmStorageAdaptor.createStoragePool just makes an instance >>>>>>>> of >>>>>>>> >> > iScsiAdmStoragePool, adds it to a map, and returns the pointer >>>>>>>> to the >>>>>>>> >> > iScsiAdmStoragePool object. The key of the map is the UUID of >>>>>>>> the storage >>>>>>>> >> > pool. >>>>>>>> >> > >>>>>>>> >> > When a volume is attached, createPhysicalDisk is invoked for >>>>>>>> managed >>>>>>>> >> > storage rather than getPhysicalDisk. createPhysicalDisk uses >>>>>>>> iscsiadm to >>>>>>>> >> > establish the iSCSI connection to the volume on the SAN and a >>>>>>>> >> > KVMPhysicalDisk is returned to be used in the attach logic that >>>>>>>> follows. >>>>>>>> >> > >>>>>>>> >> > When a volume is detached, getPhysicalDisk is invoked with the >>>>>>>> IQN of the >>>>>>>> >> > volume if the storage pool in question is managed storage. >>>>>>>> Otherwise, the >>>>>>>> >> > normal vol.getPath() is used. >>>>>>>> iScsiAdmStorageAdaptor.getPhysicalDisk just >>>>>>>> >> > returns a new instance of KVMPhysicalDisk to be used in the >>>>>>>> detach logic. >>>>>>>> >> > >>>>>>>> >> > Once the volume has been detached, >>>>>>>> iScsiAdmStoragePool.deletePhysicalDisk >>>>>>>> >> > is invoked if the storage pool is managed. deletePhysicalDisk >>>>>>>> removes the >>>>>>>> >> > iSCSI connection to the volume using iscsiadm. >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > On Sat, Sep 21, 2013 at 5:46 PM, Marcus Sorensen < >>>>>>>> shadow...@gmail.com >>>>>>>> >> >wrote: >>>>>>>> >> > >>>>>>>> >> >> Its the log4j properties file in /etc/cloudstack/agent change >>>>>>>> all INFO >>>>>>>> >> to >>>>>>>> >> >> DEBUG. I imagine the agent just isn't starting, you can tail >>>>>>>> the log >>>>>>>> >> when >>>>>>>> >> >> you try to start the service, or maybe it will spit something >>>>>>>> out into >>>>>>>> >> one >>>>>>>> >> >> of the other files in /var/log/cloudstack/agent >>>>>>>> >> >> On Sep 21, 2013 5:19 PM, "Mike Tutkowski" < >>>>>>>> mike.tutkow...@solidfire.com >>>>>>>> >> > >>>>>>>> >> >> wrote: >>>>>>>> >> >> >>>>>>>> >> >> > This is how I've been trying to query for the status of the >>>>>>>> service (I >>>>>>>> >> >> > assume it could be started this way, as well, by changing >>>>>>>> "status" to >>>>>>>> >> >> > "start" or "restart"?): >>>>>>>> >> >> > >>>>>>>> >> >> > mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo >>>>>>>> /usr/sbin/service >>>>>>>> >> >> > cloudstack-agent status >>>>>>>> >> >> > >>>>>>>> >> >> > I get this back: >>>>>>>> >> >> > >>>>>>>> >> >> > Failed to execute: * could not access PID file for >>>>>>>> cloudstack-agent >>>>>>>> >> >> > >>>>>>>> >> >> > I've made a bunch of code changes recently, though, so I >>>>>>>> think I'm >>>>>>>> >> going >>>>>>>> >> >> to >>>>>>>> >> >> > rebuild and redeploy everything. >>>>>>>> >> >> > >>>>>>>> >> >> > The debug info sounds helpful. Where can I set enable.debug? >>>>>>>> >> >> > >>>>>>>> >> >> > Thanks, Marcus! >>>>>>>> >> >> > >>>>>>>> >> >> > >>>>>>>> >> >> > On Sat, Sep 21, 2013 at 4:11 PM, Marcus Sorensen < >>>>>>>> shadow...@gmail.com >>>>>>>> >> >> > >wrote: >>>>>>>> >> >> > >>>>>>>> >> >> > > OK, will check it out in the next few days. As mentioned, >>>>>>>> you can >>>>>>>> >> set >>>>>>>> >> >> up >>>>>>>> >> >> > > your Ubuntu vm as the management server as well if all >>>>>>>> else fails. >>>>>>>> >> If >>>>>>>> >> >> > you >>>>>>>> >> >> > > can get to the mgmt server on 8250 from the KVM host, then >>>>>>>> you need >>>>>>>> >> to >>>>>>>> >> >> > > enable.debug on the agent. It won't run without >>>>>>>> complaining loudly >>>>>>>> >> if >>>>>>>> >> >> it >>>>>>>> >> >> > > can't get to the mgmt server, and I didn't see that in >>>>>>>> your agent >>>>>>>> >> log, >>>>>>>> >> >> so >>>>>>>> >> >> > > perhaps its not running. I assume you know how to >>>>>>>> stop/start the >>>>>>>> >> agent >>>>>>>> >> >> on >>>>>>>> >> >> > > KVM via 'service cloud stacks agent'. >>>>>>>> >> >> > > On Sep 21, 2013 3:02 PM, "Mike Tutkowski" < >>>>>>>> >> >> mike.tutkow...@solidfire.com> >>>>>>>> >> >> > > wrote: >>>>>>>> >> >> > > >>>>>>>> >> >> > > > Hey Marcus, >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > I haven't yet been able to test my new code, but I >>>>>>>> thought you >>>>>>>> >> would >>>>>>>> >> >> > be a >>>>>>>> >> >> > > > good person to ask to review it: >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > >>>>>>>> >> >> > >>>>>>>> >> >> >>>>>>>> >> >>>>>>>> https://github.com/mike-tutkowski/incubator-cloudstack/commit/ea74b312a8a36801994500407fd54f0cdda55e37 >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > All it is supposed to do is attach and detach a data >>>>>>>> disk (that >>>>>>>> >> has >>>>>>>> >> >> > > > guaranteed IOPS) with KVM as the hypervisor. The data >>>>>>>> disk >>>>>>>> >> happens to >>>>>>>> >> >> > be >>>>>>>> >> >> > > > from SolidFire-backed storage - where we have a 1:1 >>>>>>>> mapping >>>>>>>> >> between a >>>>>>>> >> >> > > > CloudStack volume and a data disk. >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > There is no support for hypervisor snapshots or stuff >>>>>>>> like that >>>>>>>> >> >> > (likely a >>>>>>>> >> >> > > > future release)...just attaching and detaching a data >>>>>>>> disk in 4.3. >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > Thanks! >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > On Fri, Sep 20, 2013 at 11:05 PM, Mike Tutkowski < >>>>>>>> >> >> > > > mike.tutkow...@solidfire.com> wrote: >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > > When I re-deployed the DEBs, I didn't remove >>>>>>>> cloudstack-agent >>>>>>>> >> >> first. >>>>>>>> >> >> > > > Would >>>>>>>> >> >> > > > > that be a problem? I just did a sudo apt-get install >>>>>>>> >> >> > cloudstack-agent. >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > On Fri, Sep 20, 2013 at 11:03 PM, Mike Tutkowski < >>>>>>>> >> >> > > > > mike.tutkow...@solidfire.com> wrote: >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > >> I get the same error running the command manually: >>>>>>>> >> >> > > > >> >>>>>>>> >> >> > > > >> mtutkowski@ubuntu:/etc/cloudstack/agent$ sudo >>>>>>>> >> /usr/sbin/service >>>>>>>> >> >> > > > >> cloudstack-agent status >>>>>>>> >> >> > > > >> * could not access PID file for cloudstack-agent >>>>>>>> >> >> > > > >> >>>>>>>> >> >> > > > >> >>>>>>>> >> >> > > > >> On Fri, Sep 20, 2013 at 11:00 PM, Mike Tutkowski < >>>>>>>> >> >> > > > >> mike.tutkow...@solidfire.com> wrote: >>>>>>>> >> >> > > > >> >>>>>>>> >> >> > > > >>> agent.log looks OK to me: >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,010 INFO >>>>>>>> [cloud.agent.AgentShell] >>>>>>>> >> >> (main:null) >>>>>>>> >> >> > > > Agent >>>>>>>> >> >> > > > >>> started >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,011 INFO >>>>>>>> [cloud.agent.AgentShell] >>>>>>>> >> >> (main:null) >>>>>>>> >> >> > > > >>> Implementation Version is 4.3.0-SNAPSHOT >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,015 INFO >>>>>>>> [cloud.agent.AgentShell] >>>>>>>> >> >> (main:null) >>>>>>>> >> >> > > > >>> agent.properties found at >>>>>>>> >> /etc/cloudstack/agent/agent.properties >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,023 INFO >>>>>>>> [cloud.agent.AgentShell] >>>>>>>> >> >> (main:null) >>>>>>>> >> >> > > > >>> Defaulting to using properties file for storage >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,024 INFO >>>>>>>> [cloud.agent.AgentShell] >>>>>>>> >> >> (main:null) >>>>>>>> >> >> > > > >>> Defaulting to the constant time backoff algorithm >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,029 INFO [cloud.utils.LogUtils] >>>>>>>> >> (main:null) >>>>>>>> >> >> > > log4j >>>>>>>> >> >> > > > >>> configuration found at >>>>>>>> /etc/cloudstack/agent/log4j-cloud.xml >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,163 INFO [cloud.agent.Agent] >>>>>>>> (main:null) >>>>>>>> >> id >>>>>>>> >> >> > is 3 >>>>>>>> >> >> > > > >>> 2013-09-20 19:35:39,197 INFO >>>>>>>> >> >> > > > >>> [resource.virtualnetwork.VirtualRoutingResource] >>>>>>>> (main:null) >>>>>>>> >> >> > > > >>> VirtualRoutingResource _scriptDir to use: >>>>>>>> >> >> scripts/network/domr/kvm >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >>> However, I wasn't aware that setup.log was >>>>>>>> important. This >>>>>>>> >> seems >>>>>>>> >> >> to >>>>>>>> >> >> > > be >>>>>>>> >> >> > > > a >>>>>>>> >> >> > > > >>> problem, but I'm not sure what it might indicate: >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service >>>>>>>> cloudstack-agent >>>>>>>> >> status >>>>>>>> >> >> > > > >>> DEBUG:root:Failed to execute: * could not access PID >>>>>>>> file for >>>>>>>> >> >> > > > >>> cloudstack-agent >>>>>>>> >> >> > > > >>> DEBUG:root:execute:sudo /usr/sbin/service >>>>>>>> cloudstack-agent >>>>>>>> >> start >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >>> On Fri, Sep 20, 2013 at 10:53 PM, Marcus Sorensen < >>>>>>>> >> >> > > shadow...@gmail.com >>>>>>>> >> >> > > > >wrote: >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >>>> Sorry, I saw that in the log, I thought it was the >>>>>>>> agent log >>>>>>>> >> for >>>>>>>> >> >> > > some >>>>>>>> >> >> > > > >>>> reason. Is the agent started? That might be the >>>>>>>> place to >>>>>>>> >> look. >>>>>>>> >> >> > There >>>>>>>> >> >> > > > is >>>>>>>> >> >> > > > >>>> an >>>>>>>> >> >> > > > >>>> agent log for the agent and one for the setup when >>>>>>>> it adds >>>>>>>> >> the >>>>>>>> >> >> > host, >>>>>>>> >> >> > > > >>>> both >>>>>>>> >> >> > > > >>>> in /var/log >>>>>>>> >> >> > > > >>>> On Sep 20, 2013 10:42 PM, "Mike Tutkowski" < >>>>>>>> >> >> > > > >>>> mike.tutkow...@solidfire.com> >>>>>>>> >> >> > > > >>>> wrote: >>>>>>>> >> >> > > > >>>> >>>>>>>> >> >> > > > >>>> > Is it saying that the MS is at the IP address or >>>>>>>> the KVM >>>>>>>> >> host? >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > The KVM host is at 192.168.233.10. >>>>>>>> >> >> > > > >>>> > The MS host is at 192.168.233.1. >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > I see this for my host Global Settings parameter: >>>>>>>> >> >> > > > >>>> > hostThe ip address of management >>>>>>>> server192.168.233.1 >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > /etc/cloudstack/agent/agent.properties has a >>>>>>>> >> >> host=192.168.233.1 >>>>>>>> >> >> > > > value. >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > On Fri, Sep 20, 2013 at 10:21 PM, Marcus Sorensen >>>>>>>> < >>>>>>>> >> >> > > > >>>> shadow...@gmail.com >>>>>>>> >> >> > > > >>>> > >wrote: >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > > The log says your mgmt server is >>>>>>>> 192.168.233.10? But you >>>>>>>> >> >> tried >>>>>>>> >> >> > > to >>>>>>>> >> >> > > > >>>> telnet >>>>>>>> >> >> > > > >>>> > to >>>>>>>> >> >> > > > >>>> > > 192.168.233.1? It might be enough to change >>>>>>>> that in >>>>>>>> >> >> > > > >>>> > > /etc/cloudstack/agent/agent.properties, but you >>>>>>>> may want >>>>>>>> >> to >>>>>>>> >> >> > edit >>>>>>>> >> >> > > > the >>>>>>>> >> >> > > > >>>> > config >>>>>>>> >> >> > > > >>>> > > as well to tell it the real ms IP. >>>>>>>> >> >> > > > >>>> > > On Sep 20, 2013 10:12 PM, "Mike Tutkowski" < >>>>>>>> >> >> > > > >>>> mike.tutkow...@solidfire.com >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > > wrote: >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > > > Here's what my /etc/network/interfaces file >>>>>>>> looks >>>>>>>> >> like, if >>>>>>>> >> >> > > that >>>>>>>> >> >> > > > >>>> is of >>>>>>>> >> >> > > > >>>> > > > interest (the 192.168.233.0 network is the >>>>>>>> NAT network >>>>>>>> >> >> > VMware >>>>>>>> >> >> > > > >>>> Fusion >>>>>>>> >> >> > > > >>>> > set >>>>>>>> >> >> > > > >>>> > > > up): >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > auto lo >>>>>>>> >> >> > > > >>>> > > > iface lo inet loopback >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > auto eth0 >>>>>>>> >> >> > > > >>>> > > > iface eth0 inet manual >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > auto cloudbr0 >>>>>>>> >> >> > > > >>>> > > > iface cloudbr0 inet static >>>>>>>> >> >> > > > >>>> > > > address 192.168.233.10 >>>>>>>> >> >> > > > >>>> > > > netmask 255.255.255.0 >>>>>>>> >> >> > > > >>>> > > > network 192.168.233.0 >>>>>>>> >> >> > > > >>>> > > > broadcast 192.168.233.255 >>>>>>>> >> >> > > > >>>> > > > dns-nameservers 8.8.8.8 >>>>>>>> >> >> > > > >>>> > > > bridge_ports eth0 >>>>>>>> >> >> > > > >>>> > > > bridge_fd 5 >>>>>>>> >> >> > > > >>>> > > > bridge_stp off >>>>>>>> >> >> > > > >>>> > > > bridge_maxwait 1 >>>>>>>> >> >> > > > >>>> > > > post-up route add default gw >>>>>>>> 192.168.233.2 metric 1 >>>>>>>> >> >> > > > >>>> > > > pre-down route del default gw >>>>>>>> 192.168.233.2 >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > On Fri, Sep 20, 2013 at 10:08 PM, Mike >>>>>>>> Tutkowski < >>>>>>>> >> >> > > > >>>> > > > mike.tutkow...@solidfire.com> wrote: >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > > You appear to be correct. This is from the >>>>>>>> MS log >>>>>>>> >> >> (below). >>>>>>>> >> >> > > > >>>> Discovery >>>>>>>> >> >> > > > >>>> > > > timed >>>>>>>> >> >> > > > >>>> > > > > out. >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > I'm not sure why this would be. My network >>>>>>>> settings >>>>>>>> >> >> > > shouldn't >>>>>>>> >> >> > > > >>>> have >>>>>>>> >> >> > > > >>>> > > > changed >>>>>>>> >> >> > > > >>>> > > > > since the last time I tried this. >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > I am able to ping the KVM host from the MS >>>>>>>> host and >>>>>>>> >> vice >>>>>>>> >> >> > > > versa. >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > I'm even able to manually kick off a VM on >>>>>>>> the KVM >>>>>>>> >> host >>>>>>>> >> >> > and >>>>>>>> >> >> > > > >>>> ping from >>>>>>>> >> >> > > > >>>> > > it >>>>>>>> >> >> > > > >>>> > > > > to the MS host. >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > I am using NAT from my Mac OS X host (also >>>>>>>> running >>>>>>>> >> the >>>>>>>> >> >> CS >>>>>>>> >> >> > > MS) >>>>>>>> >> >> > > > >>>> to the >>>>>>>> >> >> > > > >>>> > VM >>>>>>>> >> >> > > > >>>> > > > > running KVM (VMware Fusion). >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,141 DEBUG >>>>>>>> >> >> > > > >>>> [c.c.h.k.d.LibvirtServerDiscoverer] >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48) >>>>>>>> Timeout, >>>>>>>> >> to >>>>>>>> >> >> > wait >>>>>>>> >> >> > > > for >>>>>>>> >> >> > > > >>>> the >>>>>>>> >> >> > > > >>>> > > host >>>>>>>> >> >> > > > >>>> > > > > connecting to mgt svr, assuming it is failed >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,144 WARN >>>>>>>> >> >> [c.c.r.ResourceManagerImpl] >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48) >>>>>>>> Unable to >>>>>>>> >> >> find >>>>>>>> >> >> > > the >>>>>>>> >> >> > > > >>>> server >>>>>>>> >> >> > > > >>>> > > > > resources at http://192.168.233.10 >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,145 INFO >>>>>>>> >> >> > [c.c.u.e.CSExceptionErrorCode] >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48) >>>>>>>> Could not >>>>>>>> >> >> find >>>>>>>> >> >> > > > >>>> exception: >>>>>>>> >> >> > > > >>>> > > > > com.cloud.exception.DiscoveryException in >>>>>>>> error code >>>>>>>> >> >> list >>>>>>>> >> >> > > for >>>>>>>> >> >> > > > >>>> > > exceptions >>>>>>>> >> >> > > > >>>> > > > > 2013-09-20 19:40:40,147 WARN >>>>>>>> >> [o.a.c.a.c.a.h.AddHostCmd] >>>>>>>> >> >> > > > >>>> > > > > (613487991@qtp-1659933291-3:ctx-6b28dc48) >>>>>>>> Exception: >>>>>>>> >> >> > > > >>>> > > > > com.cloud.exception.DiscoveryException: >>>>>>>> Unable to add >>>>>>>> >> >> the >>>>>>>> >> >> > > host >>>>>>>> >> >> > > > >>>> > > > > at >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> >>>>>>>> >> >> > > > >>>>>>>> >> >> > > >>>>>>>> >> >> > >>>>>>>> >> >> >>>>>>>> >> >>>>>>>> com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:778) >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > I do seem to be able to telnet in from my >>>>>>>> KVM host to >>>>>>>> >> >> the >>>>>>>> >> >> > MS >>>>>>>> >> >> > > > >>>> host's >>>>>>>> >> >> > > > >>>> > > 8250 >>>>>>>> >> >> > > > >>>> > > > > port: >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > > mtutkowski@ubuntu:~$ telnet 192.168.233.1 >>>>>>>> 8250 >>>>>>>> >> >> > > > >>>> > > > > Trying 192.168.233.1... >>>>>>>> >> >> > > > >>>> > > > > Connected to 192.168.233.1. >>>>>>>> >> >> > > > >>>> > > > > Escape character is '^]'. >>>>>>>> >> >> > > > >>>> > > > > >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > > -- >>>>>>>> >> >> > > > >>>> > > > *Mike Tutkowski* >>>>>>>> >> >> > > > >>>> > > > *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> >> >> > > > >>>> > > > e: mike.tutkow...@solidfire.com >>>>>>>> >> >> > > > >>>> > > > o: 303.746.7302 >>>>>>>> >> >> > > > >>>> > > > Advancing the way the world uses the >>>>>>>> >> >> > > > >>>> > > > cloud< >>>>>>>> >> http://solidfire.com/solution/overview/?video=play> >>>>>>>> >> >> > > > >>>> > > > *™* >>>>>>>> >> >> > > > >>>> > > > >>>>>>>> >> >> > > > >>>> > > >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> > -- >>>>>>>> >> >> > > > >>>> > *Mike Tutkowski* >>>>>>>> >> >> > > > >>>> > *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> >> >> > > > >>>> > e: mike.tutkow...@solidfire.com >>>>>>>> >> >> > > > >>>> > o: 303.746.7302 >>>>>>>> >> >> > > > >>>> > Advancing the way the world uses the >>>>>>>> >> >> > > > >>>> > cloud< >>>>>>>> http://solidfire.com/solution/overview/?video=play> >>>>>>>> >> >> > > > >>>> > *™* >>>>>>>> >> >> > > > >>>> > >>>>>>>> >> >> > > > >>>> >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >>> -- >>>>>>>> >> >> > > > >>> *Mike Tutkowski* >>>>>>>> >> >> > > > >>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> >> >> > > > >>> e: mike.tutkow...@solidfire.com >>>>>>>> >> >> > > > >>> o: 303.746.7302 >>>>>>>> >> >> > > > >>> Advancing the way the world uses the cloud< >>>>>>>> >> >> > > > http://solidfire.com/solution/overview/?video=play> >>>>>>>> >> >> > > > >>> *™* >>>>>>>> >> >> > > > >>> >>>>>>>> >> >> > > > >> >>>>>>>> >> >> > > > >> >>>>>>>> >> >> > > > >> >>>>>>>> >> >> > > > >> -- >>>>>>>> >> >> > > > >> *Mike Tutkowski* >>>>>>>> >> >> > > > >> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> >> >> > > > >> e: mike.tutkow...@solidfire.com >>>>>>>> >> >> > > > >> o: 303.746.7302 >>>>>>>> >> >> > > > >> Advancing the way the world uses the cloud< >>>>>>>> >> >> > > > http://solidfire.com/solution/overview/?video=play> >>>>>>>> >> >> > > > >> *™* >>>>>>>> >> >> > > > >> >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > > -- >>>>>>>> >> >> > > > > *Mike Tutkowski* >>>>>>>> >> >> > > > > *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> >> >> > > > > e: mike.tutkow...@solidfire.com >>>>>>>> >> >> > > > > o: 303.746.7302 >>>>>>>> >> >> > > > > Advancing the way the world uses the cloud< >>>>>>>> >> >> > > > http://solidfire.com/solution/overview/?video=play> >>>>>>>> >> >> > > > > *™* >>>>>>>> >> >> > > > > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > >>>>>>>> >> >> > > > -- >>>>>>>> >> >> > > > *Mike Tutkowski* >>>>>>>> >> >> > > > *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> >> >> > > > e: mike.tutkow...@solidfire.com >>>>>>>> >> >> > > > o: 303.746.7302 >>>>>>>> >> >> > > > Advancing the way the world uses the >>>>>>>> >> >> > > > cloud<http://solidfire.com/solution/overview/?video=play >>>>>>>> > >>>>>>>> >> >> > > > *™* >>>>>>>> >> >> > > > >>>>>>>> >> >> > > >>>>>>>> >> >> > >>>>>>>> >> >> > >>>>>>>> >> >> > >>>>>>>> >> >> > -- >>>>>>>> >> >> > *Mike Tutkowski* >>>>>>>> >> >> > *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> >> >> > e: mike.tutkow...@solidfire.com >>>>>>>> >> >> > o: 303.746.7302 >>>>>>>> >> >> > Advancing the way the world uses the >>>>>>>> >> >> > cloud<http://solidfire.com/solution/overview/?video=play> >>>>>>>> >> >> > *™* >>>>>>>> >> >> > >>>>>>>> >> >> >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > -- >>>>>>>> >> > *Mike Tutkowski* >>>>>>>> >> > *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> >> > e: mike.tutkow...@solidfire.com >>>>>>>> >> > o: 303.746.7302 >>>>>>>> >> > Advancing the way the world uses the >>>>>>>> >> > cloud<http://solidfire.com/solution/overview/?video=play> >>>>>>>> >> > *™* >>>>>>>> >> >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > -- >>>>>>>> > *Mike Tutkowski* >>>>>>>> > *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> > e: mike.tutkow...@solidfire.com >>>>>>>> > o: 303.746.7302 >>>>>>>> > Advancing the way the world uses the >>>>>>>> > cloud<http://solidfire.com/solution/overview/?video=play> >>>>>>>> > *™* >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Mike Tutkowski* >>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>> e: mike.tutkow...@solidfire.com >>>>>>> o: 303.746.7302 >>>>>>> Advancing the way the world uses the >>>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>>>> *™* >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Mike Tutkowski* >>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>> e: mike.tutkow...@solidfire.com >>>>>> o: 303.746.7302 >>>>>> Advancing the way the world uses the >>>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>>> *™* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> *Mike Tutkowski* >>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>> e: mike.tutkow...@solidfire.com >>>>> o: 303.746.7302 >>>>> Advancing the way the world uses the >>>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>>> *™* >>>>> >>>> >>>> >>>> >>>> -- >>>> *Mike Tutkowski* >>>> *Senior CloudStack Developer, SolidFire Inc.* >>>> e: mike.tutkow...@solidfire.com >>>> o: 303.746.7302 >>>> Advancing the way the world uses the >>>> cloud<http://solidfire.com/solution/overview/?video=play> >>>> *™* >>>> >>> >>> >>> >>> -- >>> *Mike Tutkowski* >>> *Senior CloudStack Developer, SolidFire Inc.* >>> e: mike.tutkow...@solidfire.com >>> o: 303.746.7302 >>> Advancing the way the world uses the >>> cloud<http://solidfire.com/solution/overview/?video=play> >>> *™* >>> >> >> >> >> -- >> *Mike Tutkowski* >> *Senior CloudStack Developer, SolidFire Inc.* >> e: mike.tutkow...@solidfire.com >> o: 303.746.7302 >> Advancing the way the world uses the >> cloud<http://solidfire.com/solution/overview/?video=play> >> *™* >> > > > > -- > *Mike Tutkowski* > *Senior CloudStack Developer, SolidFire Inc.* > e: mike.tutkow...@solidfire.com > o: 303.746.7302 > Advancing the way the world uses the > cloud<http://solidfire.com/solution/overview/?video=play> > *™*