Re: Missing support for LXC on CS 4.2

[Old, Curtis]

So I build it from source?  I'M not a developer (lowly SA type) in the
source I downloaded the file nonossComponentContext.xml doesn't exists

[root@lab2-mgt-cloud-1 apache-cloudstack-4.2.0-src]# find . -name
nonossComponentContext.xml
[root@lab2-mgt-cloud-1 apache-cloudstack-4.2.0-src]#

So do I have to git clone the repo and build from there?

Dazed and confusedŠ..





On 10/23/13 11:42 AM, "Francois Gaudreault" 
wrote:

>This is how I solved this:
>https://issues.apache.org/jira/browse/CLOUDSTACK-4849
>
>Francois
>
>On 10/23/2013, 11:40 AM, Old, Curtis wrote:
>> I installed CS 4.2 from the repo.  I went to install an LXC Cluser and
>>got this:
>>
>> Could not find corresponding resource manager for LXC
>>
>> A little digging on the web and found and there is no plugin for LXC in
>>the rpm:
>>
>> [root@lab2-mgt-cloud-1 lib]# rpm -ql cloudstack-management |grep
>>hyperviso
>> 
>>/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-
>>hypervisor-baremetal-4.2.0.jar
>> 
>>/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-
>>hypervisor-kvm-4.2.0.jar
>> 
>>/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-
>>hypervisor-ovm-4.2.0.jar
>> 
>>/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-
>>hypervisor-ucs-4.2.0.jar
>> 
>>/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-
>>hypervisor-vmware-4.2.0.jar
>> 
>>/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-
>>hypervisor-xen-4.2.0.jar
>>
>> I downloaded the source from:
>>
>> 
>>http://mirror.symnds.com/software/Apache/cloudstack/releases/4.2.0/apache
>>-cloudstack-4.2.0-src.tar.bz2
>>
>> Doesn't seem to be in there either, so what's the deal?  What is needed
>>to get a CS Cluster to use LXC?
>>
>>
>>
>
>
>-- 
>Francois Gaudreault
>Architecte de Solution Cloud | Cloud Solutions Architect
>fgaudrea...@cloudops.com
>514-629-6775
>- - -
>CloudOps
>420 rue Guy
>Montréal QC  H3J 1S6
>www.cloudops.com
>@CloudOps_
>

Missing support for LXC on CS 4.2

[Old, Curtis]

I installed CS 4.2 from the repo.  I went to install an LXC Cluser and got this:

Could not find corresponding resource manager for LXC

A little digging on the web and found and there is no plugin for LXC in the rpm:

[root@lab2-mgt-cloud-1 lib]# rpm -ql cloudstack-management |grep hyperviso
/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-hypervisor-baremetal-4.2.0.jar
/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-hypervisor-kvm-4.2.0.jar
/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-hypervisor-ovm-4.2.0.jar
/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-hypervisor-ucs-4.2.0.jar
/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-hypervisor-vmware-4.2.0.jar
/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/cloud-plugin-hypervisor-xen-4.2.0.jar

I downloaded the source from:

http://mirror.symnds.com/software/Apache/cloudstack/releases/4.2.0/apache-cloudstack-4.2.0-src.tar.bz2

Doesn't seem to be in there either, so what's the deal?  What is needed to get 
a CS Cluster to use LXC?

Re: Cloudstack 4.0 after power|cooling loss

[Old, Curtis]

Also noticed Console Proxy VM stuck in staring state (runs on KVM node)

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 10/3/13 9:26 AM, "Old, Curtis"  wrote:

>Guys,
>
>   My lab took a hit this morning the air handler for my lab crapped out,
>temp was over a 100 in the lab.  Servers powering off or bouncing.
>Cloudstack 4.0 is backup, but out of the 10 compute nodes (5 KVM, 5
>Xenserver) the Xenserver nodes keep going into Alert or Connecting state.
>Is there anything specific I should try?
>
>Curtis Old 
>
>>
>

Cloudstack 4.0 after power|cooling loss

[Old, Curtis]

Guys,

My lab took a hit this morning the air handler for my lab crapped out,
temp was over a 100 in the lab.  Servers powering off or bouncing.
Cloudstack 4.0 is backup, but out of the 10 compute nodes (5 KVM, 5
Xenserver) the Xenserver nodes keep going into Alert or Connecting state.
Is there anything specific I should try?

Curtis Old 

>

4.2.0 Install guide, section 4.4.2 RPM Repo

[Old, Curtis]

Has anyone thought of updating the section 4.4.2 on the yum repo from:


[cloudstack]
name=cloudstack
baseurl=http://cloudstack.apt-get.eu/rhel/4.1/
enabled=1
gpgcheck=0

To

[cloudstack]
name=cloudstack
baseurl=http://cloudstack.apt-get.eu/rhel/4.2/
enabled=1
gpgcheck=0


After I changed it I see:

[root@lab2-mgt-cloud-1 ~]# yum list cloud*
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
 * base: mirror.nexcess.net
 * extras: mirror.es.its.nyu.edu
 * updates: mirror.trouble-free.net
cloudstack  
 | 1.1 kB 00:00
cloudstack/primary  
 | 2.6 kB 00:00
cloudstack  
7/7
Available Packages
cloudstack-agent.x86_64   4.2.0-1.el6   
  cloudstack
cloudstack-awsapi.x86_64  4.2.0-1.el6   
  cloudstack
cloudstack-baremetal-agent.x86_64 4.2.0-1.el6   
  cloudstack
cloudstack-cli.x86_64 4.2.0-1.el6   
  cloudstack
cloudstack-common.x86_64  4.2.0-1.el6   
  cloudstack
cloudstack-management.x86_64  4.2.0-1.el6   
  cloudstack
cloudstack-usage.x86_64   4.2.0-1.el6   
  cloudstack

Just a thought, and sorry if this has been brought up already :-)

Curtis Old
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / 
curtis@neustar.biz  / 
www.neustar.biz

The information contained in this e-mail message is intended only for the use 
of the recipient(s) named above and may contain confidential and/or privileged 
information. If you are not the intended recipient you have received this 
e-mail message in error and any review, dissemination, distribution, or copying 
of this message is strictly prohibited. If you have received this communication 
in error, please notify us immediately and delete the original message.

Re: What about adding Linux Containers to Cloudstack

[Old, Curtis]

I have Docker and OpenVZ setup in my lab, been focusing on OpenVZ right
now, will look into docker soon.  I see ways to integrate with Jenkins,
puppet|chef for a Continuous Deployment env.  Standing up containers using
the UI via Cloudstack would be a great tool for my users/team having only
a single interface to provide the resources they need.  Cloudstack has
KVM, Xen, VMware, OVM, and Bare Metal clusters, lets get a Container
cluster ;-)

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 9/19/13 2:23 PM, "Nux!"  wrote:

>On 19.09.2013 14:50, Old, Curtis wrote:
>> Have been looking in to containers as an alternative to KVM/Xen
>> virtual machines, has anyone thought about adding support to manage
>> hosts running OpenVZ or LXC via CloudStack?
>> 
>> Curtis Old
>
>+1 !
>
>Over 60% of my VMs could easily be LXC containers and there is strong
>trend towards containers (Docker has just been integrated with
>Openshift, openstack supports has lxc & docker drivers etc).
>
>-- 
>Sent from the Delta quadrant using Borg technology!
>
>Nux!
>www.nux.ro

Re: What about adding Linux Containers to Cloudstack

[Old, Curtis]

Cool, I am about to setup a new Cloudstack environment in my other lab,
I'll look into it, Thanks!!!

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 9/19/13 3:51 PM, "Sebastien Goasguen"  wrote:

>
>On Sep 19, 2013, at 2:23 PM, Nux!  wrote:
>
>> On 19.09.2013 14:50, Old, Curtis wrote:
>>> Have been looking in to containers as an alternative to KVM/Xen
>>> virtual machines, has anyone thought about adding support to manage
>>> hosts running OpenVZ or LXC via CloudStack?
>>> Curtis Old
>> 
>> +1 !
>> 
>> Over 60% of my VMs could easily be LXC containers and there is strong
>>trend towards containers (Docker has just been integrated with
>>Openshift, openstack supports has lxc & docker drivers etc).
>> 
>
>I believe it's there, if not 4.2 it's in master
>
>
>> -- 
>> Sent from the Delta quadrant using Borg technology!
>> 
>> Nux!
>> www.nux.ro
>

What about adding Linux Containers to Cloudstack

[Old, Curtis]

Have been looking in to containers as an alternative to KVM/Xen virtual 
machines, has anyone thought about adding support to manage hosts running 
OpenVZ or LXC via CloudStack?

Curtis Old

Re: Best Doc for upgrade path from 4.0.0 to 4.1.1

[Old, Curtis]

Thx!

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 8/15/13 3:49 PM, "Kirk Kosinski"  wrote:

>Hi, Curtis.  The upgrade procedure is in the release notes.
>
>http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.1.1/html-singl
>e/Release_Notes/index.html
>
>Best regards,
>Kirk
>
>On 08/15/2013 12:39 PM, Old, Curtis wrote:
>> I was looking over the install guide, unless I missed it I didn't see
>>the Upgrade option from 4.0.0 to 4.1 any good docs out there
>> 
>> Curtis Old
>> 
>> 

Best Doc for upgrade path from 4.0.0 to 4.1.1

[Old, Curtis]

I was looking over the install guide, unless I missed it I didn't see the 
Upgrade option from 4.0.0 to 4.1 any good docs out there

Curtis Old

Re: Question after a VM is deployed can I Š..

[Old, Curtis]

Dang I have to upgrade first :-( she's been working so well I didn't want
to touch it lol

Thanks!

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 8/15/13 10:40 AM, "Venkata SwamyBabu Budumuru"
 wrote:

>There is an event framework added in 4.1 release which you can leverage
>and subscribe for events you are interested in and then have your script
>or framework program DNS / other stuff based on that.
>
>Here is the link for the same :
>http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.1.0/html/Admin
>_
>Guide/events.html#event-framework
>
>On 15/08/13 7:09 PM, "Old, Curtis"  wrote:
>
>>Trigger an event or post processing from the management server?  What I
>>am interested in doing is that after a user provisions a VM in CS I want
>>to be able to dynamically do certain tasks.  Possibly a DNS updates to my
>>external bind server, run knife to add to chef, add to Nagios etc.
>>
>>Sorry if it has already been covered.
>>
>>Cheers
>>
>>Curtis Old
>>
>

Question after a VM is deployed can I Š..

[Old, Curtis]

Trigger an event or post processing from the management server?  What I am 
interested in doing is that after a user provisions a VM in CS I want to be 
able to dynamically do certain tasks.  Possibly a DNS updates to my external 
bind server, run knife to add to chef, add to Nagios etc.

Sorry if it has already been covered.

Cheers

Curtis Old

Re: Update resolve.conf search string

[Old, Curtis]

No problem I can write a script and make sure the config is the way I want
it, run it every 10 min or something to ensure the config is cool

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 8/1/13 6:52 PM, "Chiradeep Vittal"  wrote:

>Yeah you would have to use automation outside of CloudStack for this
>
>On 8/1/13 3:14 PM, "Old, Curtis"  wrote:
>
>>Basic, yes dnsmasq on the system vm would set that up, but how do you
>>make
>>it persistent?  If I have to re-do the router/dns/dhcp system vm, am I
>>always going to have to go in and edit config file?
>>
>>Curtis Old 
>>Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>46000 Center Oak Plaza Sterling, VA 20166
>>Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
>>/ www.neustar.biz <http://www.neustar.biz/>
>>
>>
>>
>>The information contained in this e-mail message is intended only for the
>>use of the recipient(s) named above and may contain confidential and/or
>>privileged information. If you are not the intended recipient you have
>>received this e-mail message in error and any review, dissemination,
>>distribution, or copying of this message is strictly prohibited. If you
>>have received this communication in error, please notify us immediately
>>and delete the original message.
>>
>>
>>
>>
>>
>>On 8/1/13 5:56 PM, "Chiradeep Vittal" 
>>wrote:
>>
>>>Is this for Advanced or basic?
>>>I think on shared networks and basic zone, the DHCP server returns the
>>>DNS
>>>server configured for the zone.
>>>
>>>On 8/1/13 1:34 PM, "Old, Curtis"  wrote:
>>>
>>>>Is there way to persistently change the search entry for the
>>>>/etc/resolv.conf on cloudstack clients?  How do I ensure that the
>>>>system
>>>>router vm takes care of this?
>>>>
>>>>Curtis Old
>>>>Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>>>46000 Center Oak Plaza Sterling, VA 20166
>>>>Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>>>curtis@neustar.biz<mailto:curtis@neustar.biz>  /
>>>>www.neustar.biz<http://www.neustar.biz/>
>>>>
>>>>The information contained in this e-mail message is intended only for
>>>>the
>>>>use of the recipient(s) named above and may contain confidential and/or
>>>>privileged information. If you are not the intended recipient you have
>>>>received this e-mail message in error and any review, dissemination,
>>>>distribution, or copying of this message is strictly prohibited. If you
>>>>have received this communication in error, please notify us immediately
>>>>and delete the original message.
>>>
>>
>

Re: Update resolve.conf search string

[Old, Curtis]

Basic, yes dnsmasq on the system vm would set that up, but how do you make
it persistent?  If I have to re-do the router/dns/dhcp system vm, am I
always going to have to go in and edit config file?

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 8/1/13 5:56 PM, "Chiradeep Vittal"  wrote:

>Is this for Advanced or basic?
>I think on shared networks and basic zone, the DHCP server returns the DNS
>server configured for the zone.
>
>On 8/1/13 1:34 PM, "Old, Curtis"  wrote:
>
>>Is there way to persistently change the search entry for the
>>/etc/resolv.conf on cloudstack clients?  How do I ensure that the system
>>router vm takes care of this?
>>
>>Curtis Old
>>Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>46000 Center Oak Plaza Sterling, VA 20166
>>Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>curtis@neustar.biz<mailto:curtis@neustar.biz>  /
>>www.neustar.biz<http://www.neustar.biz/>
>>
>>The information contained in this e-mail message is intended only for the
>>use of the recipient(s) named above and may contain confidential and/or
>>privileged information. If you are not the intended recipient you have
>>received this e-mail message in error and any review, dissemination,
>>distribution, or copying of this message is strictly prohibited. If you
>>have received this communication in error, please notify us immediately
>>and delete the original message.
>

Update resolve.conf search string

[Old, Curtis]

Is there way to persistently change the search entry for the /etc/resolv.conf 
on cloudstack clients?  How do I ensure that the system router vm takes care of 
this?

Curtis Old
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / 
curtis@neustar.biz  / 
www.neustar.biz

The information contained in this e-mail message is intended only for the use 
of the recipient(s) named above and may contain confidential and/or privileged 
information. If you are not the intended recipient you have received this 
e-mail message in error and any review, dissemination, distribution, or copying 
of this message is strictly prohibited. If you have received this communication 
in error, please notify us immediately and delete the original message.

Xenserver add instance from ISO Ubuntu 12.04 fails but...

[Old, Curtis]

If I add a centOS instance from an ISO on Xenserver it works.  I have used the 
same Ubuntu 12.04 ISO to create an instance on KVM any ideas?

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Ok no worries, first thing in the AM will update after :-)

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/30/13 10:29 AM, "Geoff Higginbottom"
 wrote:

>Unfortunately you have to re-install XenServer before re-adding back into
>CloudStack
>
>Regards
>
>Geoff Higginbottom
>
>D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581
>
>geoff.higginbot...@shapeblue.com
>
>
>-Original Message-
>From: Old, Curtis [mailto:curtis@neustar.biz]
>Sent: 30 May 2013 15:27
>To: Ahmad Emneina
>Cc: users@cloudstack.apache.org
>Subject: Re: Xenserver hypervisior will not reconnect after reboot
>
>Not yet was hoping to avoid that
>
>Curtis Old
>Neustar, Inc. / Neustarlabs / Senior Research Engineer
>46000 Center Oak Plaza Sterling, VA 20166
>Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
>/ www.neustar.biz <http://www.neustar.biz/>
>
>
>
>The information contained in this e-mail message is intended only for the
>use of the recipient(s) named above and may contain confidential and/or
>privileged information. If you are not the intended recipient you have
>received this e-mail message in error and any review, dissemination,
>distribution, or copying of this message is strictly prohibited. If you
>have received this communication in error, please notify us immediately
>and delete the original message.
>
>
>
>
>
>On 5/30/13 10:30 AM, "Ahmad Emneina"  wrote:
>
>>Did you reinstall the hypervisor before reconnecting it back to the pool?
>>
>>Ahmad
>>
>>On May 30, 2013, at 4:10 AM, "Old, Curtis" 
>>wrote:
>>
>>> Ok both are in an "Alert" state.  So I'M kinda at a lost for what to
>>>try
>>> next.
>>>
>>>
>>> On 5/30/13 6:55 AM, "Old, Curtis"  wrote:
>>>
>>>> Ok totally weird stuff now.
>>>>
>>>> After trying the previous steps, both Xenserver nodes failed to
>>>>reconnect
>>>> to the cluster, so I decided to remove and re-add them.  I performed
>>>>the
>>>> following:
>>>>
>>>>  1.  Unmanaged the cluster
>>>>  2.  Removed the 2 nodes
>>>>  3.  Managed the cluster, the 3 remaining nodes went into an Up state.
>>>>  4.  Add 1 of the Xenserver nodes, and was going to wait to add the
>>>>last
>>>> one, if the 1st added successfully.
>>>>
>>>> The result I see both of the 2 nodes trying to connect (one I didn't
>>>>add
>>>> yet), waiting to see if the complete or fail.
>>>>
>>>>
>>>>
>>>> On 5/29/13 8:39 AM, "Old, Curtis"  wrote:
>>>>
>>>>> Same status after an hour.  Should I delete them and re-add them or
>>>>>??
>>>>>
>>>>>
>>>>>
>>>>> On 5/29/13 7:30 AM, "Old, Curtis"  wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> Well one lab-cloud-9 is in an "Alert" State the other, lab-cloud-10
>>>>>>is
>>>>>> "Disconnected", and after a refresh on the UI lab-cloud-9 is back to
>>>>>> "Connecting"
>>>>>>
>>>>>>
>>>>>> Suggestions?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 5/29/13 7:17 AM, "Old, Curtis"  wrote:
>>>>>>
>>>>>>> Interesting the UUID's in the database where different then the
>>>>>>> Xenserver
>>>>>>> UUID's (there where 2 Xenservers hosts that had inode problems) But
>>>>>>> then
>>>>>>> again the other servers all had different UUID's as well .
>>>>>>>

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Not yet was hoping to avoid that

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/30/13 10:30 AM, "Ahmad Emneina"  wrote:

>Did you reinstall the hypervisor before reconnecting it back to the pool?
>
>Ahmad
>
>On May 30, 2013, at 4:10 AM, "Old, Curtis"  wrote:
>
>> Ok both are in an "Alert" state.  So I'M kinda at a lost for what to try
>> next.
>> 
>> 
>> On 5/30/13 6:55 AM, "Old, Curtis"  wrote:
>> 
>>> Ok totally weird stuff now.
>>> 
>>> After trying the previous steps, both Xenserver nodes failed to
>>>reconnect
>>> to the cluster, so I decided to remove and re-add them.  I performed
>>>the
>>> following:
>>> 
>>>  1.  Unmanaged the cluster
>>>  2.  Removed the 2 nodes
>>>  3.  Managed the cluster, the 3 remaining nodes went into an Up state.
>>>  4.  Add 1 of the Xenserver nodes, and was going to wait to add the
>>>last
>>> one, if the 1st added successfully.
>>> 
>>> The result I see both of the 2 nodes trying to connect (one I didn't
>>>add
>>> yet), waiting to see if the complete or fail.
>>> 
>>> 
>>> 
>>> On 5/29/13 8:39 AM, "Old, Curtis"  wrote:
>>> 
>>>> Same status after an hour.  Should I delete them and re-add them or ??
>>>> 
>>>> 
>>>> 
>>>> On 5/29/13 7:30 AM, "Old, Curtis"  wrote:
>>>> 
>>>>> 
>>>>> 
>>>>> Well one lab-cloud-9 is in an "Alert" State the other, lab-cloud-10
>>>>>is
>>>>> "Disconnected", and after a refresh on the UI lab-cloud-9 is back to
>>>>> "Connecting"
>>>>> 
>>>>> 
>>>>> Suggestions?
>>>>> 
>>>>> 
>>>>> 
>>>>> On 5/29/13 7:17 AM, "Old, Curtis"  wrote:
>>>>> 
>>>>>> Interesting the UUID's in the database where different then the
>>>>>> Xenserver
>>>>>> UUID's (there where 2 Xenservers hosts that had inode problems) But
>>>>>> then
>>>>>> again the other servers all had different UUID's as well .
>>>>>> 
>>>>>> Mysql DB:
>>>>>> 
>>>>>>  id: 21
>>>>>>name: lab-cloud-10.neustarlab.biz
>>>>>>uuid: 5c65a56a-5a71-44e8-b4ae-89bf81ef4d24
>>>>>> 
>>>>>> And on the Host:
>>>>>> 
>>>>>> 
>>>>>> uuid=dde1a6d7-8708-451f-a4a8-a9c5be79f57e
>>>>>> 
>>>>>> It has been 20 to 25 min and one lab-cloud-9 its state is
>>>>>>"Connecting"
>>>>>> lab-cloud-10 is in "Disconnected" and the other 3 hosts
>>>>>>(lab-cloud-6,
>>>>>> 7,
>>>>>> 8) are "Up"
>>>>>> 
>>>>>> Wait a bit longer?
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 5/28/13 6:37 PM, "Kirk Kosinski"  wrote:
>>>>>> 
>>>>>>> Before hacking the db, I suggest trying one more thing.
>>>>>>> 
>>>>>>> 1. In the CS UI, unmanage the cluster with the affected host.
>>>>>>> 2. From the CLI of a host in the XS pool, clear the host tags of
>>>>>>>the
>>>>>>> affected host.
>>>>>>> xe host-param-clear param-name=tags uuid=
>>>>>>> 3. In the CS UI, manage the cluster with the affected host.
>>>>>>> 
>>>>>>> This usually solves

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Ok both are in an "Alert" state.  So I'M kinda at a lost for what to try
next.


On 5/30/13 6:55 AM, "Old, Curtis"  wrote:

>Ok totally weird stuff now.
>
>After trying the previous steps, both Xenserver nodes failed to reconnect
>to the cluster, so I decided to remove and re-add them.  I performed the
>following:
>
>   1.  Unmanaged the cluster
>   2.  Removed the 2 nodes
>   3.  Managed the cluster, the 3 remaining nodes went into an Up state.
>   4.  Add 1 of the Xenserver nodes, and was going to wait to add the last
>one, if the 1st added successfully.
>
>The result I see both of the 2 nodes trying to connect (one I didn't add
>yet), waiting to see if the complete or fail.
>
>
>
>On 5/29/13 8:39 AM, "Old, Curtis"  wrote:
>
>>Same status after an hour.  Should I delete them and re-add them or ??
>>
>>
>>
>>On 5/29/13 7:30 AM, "Old, Curtis"  wrote:
>>
>>>
>>>
>>>Well one lab-cloud-9 is in an "Alert" State the other, lab-cloud-10 is
>>>"Disconnected", and after a refresh on the UI lab-cloud-9 is back to
>>>"Connecting"
>>>
>>>
>>>Suggestions?
>>>
>>>
>>>
>>>On 5/29/13 7:17 AM, "Old, Curtis"  wrote:
>>>
>>>>Interesting the UUID's in the database where different then the
>>>>Xenserver
>>>>UUID's (there where 2 Xenservers hosts that had inode problems) But
>>>>then
>>>>again the other servers all had different UUID's as well .
>>>>
>>>>Mysql DB:
>>>>
>>>>   id: 21
>>>> name: lab-cloud-10.neustarlab.biz
>>>> uuid: 5c65a56a-5a71-44e8-b4ae-89bf81ef4d24
>>>>
>>>>And on the Host:
>>>>
>>>>
>>>>uuid=dde1a6d7-8708-451f-a4a8-a9c5be79f57e
>>>>
>>>>It has been 20 to 25 min and one lab-cloud-9 its state is "Connecting"
>>>>lab-cloud-10 is in "Disconnected" and the other 3 hosts (lab-cloud-6,
>>>>7,
>>>>8) are "Up"
>>>>
>>>>Wait a bit longer?
>>>>
>>>>
>>>>
>>>>On 5/28/13 6:37 PM, "Kirk Kosinski"  wrote:
>>>>
>>>>>Before hacking the db, I suggest trying one more thing.
>>>>>
>>>>>1. In the CS UI, unmanage the cluster with the affected host.
>>>>>2. From the CLI of a host in the XS pool, clear the host tags of the
>>>>>affected host.
>>>>>xe host-param-clear param-name=tags uuid=
>>>>>3. In the CS UI, manage the cluster with the affected host.
>>>>>
>>>>>This usually solves this type of problem (it might take a couple
>>>>>minutes).  Clearing host tags won't affect running VMs (if any).
>>>>>
>>>>>Best regards,
>>>>>Kirk
>>>>>
>>>>>On 05/28/2013 03:03 PM, Ahmad Emneina wrote:
>>>>>> first, I'd query to make sure its the right host, judging from the
>>>>>>logs
>>>>>>it
>>>>>> has an id of 20... execute the following
>>>>>> backup command from the cli that can access the db server:
>>>>>> 
>>>>>> mysqldump  --databases cloud > cloud.sql
>>>>>> 
>>>>>> Then start operating...
>>>>>> 
>>>>>> select * from host where id=20\G (hit the enter key)
>>>>>> 
>>>>>> if this is infact the misbehaving host, execute
>>>>>> 
>>>>>> update host set status='Up' where id=20; (again hit enter)
>>>>>> 
>>>>>> then try to launch a vm on that host (in the deployVirtualMachine
>>>>>>api
>>>>>> command you can/should be able to specify the exact host)
>>>>>> 
>>>>>> 
>>>>>> On Tue, May 28, 2013 at 2:46 PM, Old, Curtis
>>>>>>
>>>>>>wrote:
>>>>>> 
>>>>>>> Oh god where do I find that sucker in the DB?  Sorry I'M an admin
>>>>>>>type
>>>>>>> that isn't a mysql guru or Ji
>>>>>>>
>>>>>>> Curtis Old
>>>>>>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>>>>>> 46000 Center O

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Ok totally weird stuff now.

After trying the previous steps, both Xenserver nodes failed to reconnect
to the cluster, so I decided to remove and re-add them.  I performed the
following:

   1.  Unmanaged the cluster
   2.  Removed the 2 nodes
   3.  Managed the cluster, the 3 remaining nodes went into an Up state.
   4.  Add 1 of the Xenserver nodes, and was going to wait to add the last
one, if the 1st added successfully.

The result I see both of the 2 nodes trying to connect (one I didn't add
yet), waiting to see if the complete or fail.



On 5/29/13 8:39 AM, "Old, Curtis"  wrote:

>Same status after an hour.  Should I delete them and re-add them or ??
>
>
>
>On 5/29/13 7:30 AM, "Old, Curtis"  wrote:
>
>>
>>
>>Well one lab-cloud-9 is in an "Alert" State the other, lab-cloud-10 is
>>"Disconnected", and after a refresh on the UI lab-cloud-9 is back to
>>"Connecting"
>>
>>
>>Suggestions?
>>
>>
>>
>>On 5/29/13 7:17 AM, "Old, Curtis"  wrote:
>>
>>>Interesting the UUID's in the database where different then the
>>>Xenserver
>>>UUID's (there where 2 Xenservers hosts that had inode problems) But then
>>>again the other servers all had different UUID's as well .
>>>
>>>Mysql DB:
>>>
>>>   id: 21
>>> name: lab-cloud-10.neustarlab.biz
>>> uuid: 5c65a56a-5a71-44e8-b4ae-89bf81ef4d24
>>>
>>>And on the Host:
>>>
>>>
>>>uuid=dde1a6d7-8708-451f-a4a8-a9c5be79f57e
>>>
>>>It has been 20 to 25 min and one lab-cloud-9 its state is "Connecting"
>>>lab-cloud-10 is in "Disconnected" and the other 3 hosts (lab-cloud-6, 7,
>>>8) are "Up"
>>>
>>>Wait a bit longer?
>>>
>>>
>>>
>>>On 5/28/13 6:37 PM, "Kirk Kosinski"  wrote:
>>>
>>>>Before hacking the db, I suggest trying one more thing.
>>>>
>>>>1. In the CS UI, unmanage the cluster with the affected host.
>>>>2. From the CLI of a host in the XS pool, clear the host tags of the
>>>>affected host.
>>>>xe host-param-clear param-name=tags uuid=
>>>>3. In the CS UI, manage the cluster with the affected host.
>>>>
>>>>This usually solves this type of problem (it might take a couple
>>>>minutes).  Clearing host tags won't affect running VMs (if any).
>>>>
>>>>Best regards,
>>>>Kirk
>>>>
>>>>On 05/28/2013 03:03 PM, Ahmad Emneina wrote:
>>>>> first, I'd query to make sure its the right host, judging from the
>>>>>logs
>>>>>it
>>>>> has an id of 20... execute the following
>>>>> backup command from the cli that can access the db server:
>>>>> 
>>>>> mysqldump  --databases cloud > cloud.sql
>>>>> 
>>>>> Then start operating...
>>>>> 
>>>>> select * from host where id=20\G (hit the enter key)
>>>>> 
>>>>> if this is infact the misbehaving host, execute
>>>>> 
>>>>> update host set status='Up' where id=20; (again hit enter)
>>>>> 
>>>>> then try to launch a vm on that host (in the deployVirtualMachine api
>>>>> command you can/should be able to specify the exact host)
>>>>> 
>>>>> 
>>>>> On Tue, May 28, 2013 at 2:46 PM, Old, Curtis 
>>>>>wrote:
>>>>> 
>>>>>> Oh god where do I find that sucker in the DB?  Sorry I'M an admin
>>>>>>type
>>>>>> that isn't a mysql guru or Ji
>>>>>>
>>>>>> Curtis Old
>>>>>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>>>>> 46000 Center Oak Plaza Sterling, VA 20166
>>>>>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>>>>>curtis@neustar.biz
>>>>>> / www.neustar.biz <http://www.neustar.biz/>
>>>>>> 
>>>>>>
>>>>>>
>>>>>> The information contained in this e-mail message is intended only
>>>>>>for
>>>>>>the
>>>>>> use of the recipient(s) named above and may contain confidential
>>>>>>and/or
>>>>>> privileged information. If

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Same status after an hour.  Should I delete them and re-add them or ??



On 5/29/13 7:30 AM, "Old, Curtis"  wrote:

>
>
>Well one lab-cloud-9 is in an "Alert" State the other, lab-cloud-10 is
>"Disconnected", and after a refresh on the UI lab-cloud-9 is back to
>"Connecting"
>
>
>Suggestions?
>
>
>
>On 5/29/13 7:17 AM, "Old, Curtis"  wrote:
>
>>Interesting the UUID's in the database where different then the Xenserver
>>UUID's (there where 2 Xenservers hosts that had inode problems) But then
>>again the other servers all had different UUID's as well .
>>
>>Mysql DB:
>>
>>   id: 21
>> name: lab-cloud-10.neustarlab.biz
>> uuid: 5c65a56a-5a71-44e8-b4ae-89bf81ef4d24
>>
>>And on the Host:
>>
>>
>>uuid=dde1a6d7-8708-451f-a4a8-a9c5be79f57e
>>
>>It has been 20 to 25 min and one lab-cloud-9 its state is "Connecting"
>>lab-cloud-10 is in "Disconnected" and the other 3 hosts (lab-cloud-6, 7,
>>8) are "Up"
>>
>>Wait a bit longer?
>>
>>
>>
>>On 5/28/13 6:37 PM, "Kirk Kosinski"  wrote:
>>
>>>Before hacking the db, I suggest trying one more thing.
>>>
>>>1. In the CS UI, unmanage the cluster with the affected host.
>>>2. From the CLI of a host in the XS pool, clear the host tags of the
>>>affected host.
>>>xe host-param-clear param-name=tags uuid=
>>>3. In the CS UI, manage the cluster with the affected host.
>>>
>>>This usually solves this type of problem (it might take a couple
>>>minutes).  Clearing host tags won't affect running VMs (if any).
>>>
>>>Best regards,
>>>Kirk
>>>
>>>On 05/28/2013 03:03 PM, Ahmad Emneina wrote:
>>>> first, I'd query to make sure its the right host, judging from the
>>>>logs
>>>>it
>>>> has an id of 20... execute the following
>>>> backup command from the cli that can access the db server:
>>>> 
>>>> mysqldump  --databases cloud > cloud.sql
>>>> 
>>>> Then start operating...
>>>> 
>>>> select * from host where id=20\G (hit the enter key)
>>>> 
>>>> if this is infact the misbehaving host, execute
>>>> 
>>>> update host set status='Up' where id=20; (again hit enter)
>>>> 
>>>> then try to launch a vm on that host (in the deployVirtualMachine api
>>>> command you can/should be able to specify the exact host)
>>>> 
>>>> 
>>>> On Tue, May 28, 2013 at 2:46 PM, Old, Curtis 
>>>>wrote:
>>>> 
>>>>> Oh god where do I find that sucker in the DB?  Sorry I'M an admin
>>>>>type
>>>>> that isn't a mysql guru or Ji
>>>>>
>>>>> Curtis Old
>>>>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>>>> 46000 Center Oak Plaza Sterling, VA 20166
>>>>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>>>>curtis@neustar.biz
>>>>> / www.neustar.biz <http://www.neustar.biz/>
>>>>> ________
>>>>>
>>>>>
>>>>> The information contained in this e-mail message is intended only for
>>>>>the
>>>>> use of the recipient(s) named above and may contain confidential
>>>>>and/or
>>>>> privileged information. If you are not the intended recipient you
>>>>>have
>>>>> received this e-mail message in error and any review, dissemination,
>>>>> distribution, or copying of this message is strictly prohibited. If
>>>>>you
>>>>> have received this communication in error, please notify us
>>>>>immediately
>>>>> and delete the original message.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 5/28/13 5:43 PM, "Ahmad Emneina"  wrote:
>>>>>
>>>>>> oh this looks like a bug. A potential workaround for it is marking
>>>>>>the
>>>>>> host
>>>>>> as Up in the cloudstack db. I'd make a backup first then modify the
>>>>>>db
>>>>>> entry.
>>>>>>
>>>>>>
>>>>>> On Tue, May 28, 2013 at

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Well one lab-cloud-9 is in an "Alert" State the other, lab-cloud-10 is
"Disconnected", and after a refresh on the UI lab-cloud-9 is back to
"Connecting"


Suggestions?



On 5/29/13 7:17 AM, "Old, Curtis"  wrote:

>Interesting the UUID's in the database where different then the Xenserver
>UUID's (there where 2 Xenservers hosts that had inode problems) But then
>again the other servers all had different UUID's as well .
>
>Mysql DB:
>
>   id: 21
> name: lab-cloud-10.neustarlab.biz
> uuid: 5c65a56a-5a71-44e8-b4ae-89bf81ef4d24
>
>And on the Host:
>
>
>uuid=dde1a6d7-8708-451f-a4a8-a9c5be79f57e
>
>It has been 20 to 25 min and one lab-cloud-9 its state is "Connecting"
>lab-cloud-10 is in "Disconnected" and the other 3 hosts (lab-cloud-6, 7,
>8) are "Up"
>
>Wait a bit longer?
>
>
>
>On 5/28/13 6:37 PM, "Kirk Kosinski"  wrote:
>
>>Before hacking the db, I suggest trying one more thing.
>>
>>1. In the CS UI, unmanage the cluster with the affected host.
>>2. From the CLI of a host in the XS pool, clear the host tags of the
>>affected host.
>>xe host-param-clear param-name=tags uuid=
>>3. In the CS UI, manage the cluster with the affected host.
>>
>>This usually solves this type of problem (it might take a couple
>>minutes).  Clearing host tags won't affect running VMs (if any).
>>
>>Best regards,
>>Kirk
>>
>>On 05/28/2013 03:03 PM, Ahmad Emneina wrote:
>>> first, I'd query to make sure its the right host, judging from the logs
>>>it
>>> has an id of 20... execute the following
>>> backup command from the cli that can access the db server:
>>> 
>>> mysqldump  --databases cloud > cloud.sql
>>> 
>>> Then start operating...
>>> 
>>> select * from host where id=20\G (hit the enter key)
>>> 
>>> if this is infact the misbehaving host, execute
>>> 
>>> update host set status='Up' where id=20; (again hit enter)
>>> 
>>> then try to launch a vm on that host (in the deployVirtualMachine api
>>> command you can/should be able to specify the exact host)
>>> 
>>> 
>>> On Tue, May 28, 2013 at 2:46 PM, Old, Curtis 
>>>wrote:
>>> 
>>>> Oh god where do I find that sucker in the DB?  Sorry I'M an admin type
>>>> that isn't a mysql guru or Ji
>>>>
>>>> Curtis Old
>>>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>>> 46000 Center Oak Plaza Sterling, VA 20166
>>>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>>>curtis@neustar.biz
>>>> / www.neustar.biz <http://www.neustar.biz/>
>>>> 
>>>>
>>>>
>>>> The information contained in this e-mail message is intended only for
>>>>the
>>>> use of the recipient(s) named above and may contain confidential
>>>>and/or
>>>> privileged information. If you are not the intended recipient you have
>>>> received this e-mail message in error and any review, dissemination,
>>>> distribution, or copying of this message is strictly prohibited. If
>>>>you
>>>> have received this communication in error, please notify us
>>>>immediately
>>>> and delete the original message.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 5/28/13 5:43 PM, "Ahmad Emneina"  wrote:
>>>>
>>>>> oh this looks like a bug. A potential workaround for it is marking
>>>>>the
>>>>> host
>>>>> as Up in the cloudstack db. I'd make a backup first then modify the
>>>>>db
>>>>> entry.
>>>>>
>>>>>
>>>>> On Tue, May 28, 2013 at 2:36 PM, Old, Curtis 
>>>>> wrote:
>>>>>
>>>>>>   Tailed last 3000 lines right after trying a forced reconnect:
>>>>>> http://paste.cloudstack.org/hTOm/
>>>>>>
>>>>>>  Listed as Management log N*,
>>>>>>
>>>>>>   Author Curtis Old
>>>>>>
>>>>>>
>>>>>>   Can we see the latest logs? paste.cloudstack.org
>>>>>>
>>>>>>
>>>>>> On Tue, May 28, 2013 at 2:15 PM, Old, Curtis
>>>>>

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Interesting the UUID's in the database where different then the Xenserver
UUID's (there where 2 Xenservers hosts that had inode problems) But then
again the other servers all had different UUID's as well .

Mysql DB:

   id: 21
 name: lab-cloud-10.neustarlab.biz
 uuid: 5c65a56a-5a71-44e8-b4ae-89bf81ef4d24

And on the Host:


uuid=dde1a6d7-8708-451f-a4a8-a9c5be79f57e

It has been 20 to 25 min and one lab-cloud-9 its state is "Connecting"
lab-cloud-10 is in "Disconnected" and the other 3 hosts (lab-cloud-6, 7,
8) are "Up"

Wait a bit longer?



On 5/28/13 6:37 PM, "Kirk Kosinski"  wrote:

>Before hacking the db, I suggest trying one more thing.
>
>1. In the CS UI, unmanage the cluster with the affected host.
>2. From the CLI of a host in the XS pool, clear the host tags of the
>affected host.
>xe host-param-clear param-name=tags uuid=
>3. In the CS UI, manage the cluster with the affected host.
>
>This usually solves this type of problem (it might take a couple
>minutes).  Clearing host tags won't affect running VMs (if any).
>
>Best regards,
>Kirk
>
>On 05/28/2013 03:03 PM, Ahmad Emneina wrote:
>> first, I'd query to make sure its the right host, judging from the logs
>>it
>> has an id of 20... execute the following
>> backup command from the cli that can access the db server:
>> 
>> mysqldump  --databases cloud > cloud.sql
>> 
>> Then start operating...
>> 
>> select * from host where id=20\G (hit the enter key)
>> 
>> if this is infact the misbehaving host, execute
>> 
>> update host set status='Up' where id=20; (again hit enter)
>> 
>> then try to launch a vm on that host (in the deployVirtualMachine api
>> command you can/should be able to specify the exact host)
>> 
>> 
>> On Tue, May 28, 2013 at 2:46 PM, Old, Curtis 
>>wrote:
>> 
>>> Oh god where do I find that sucker in the DB?  Sorry I'M an admin type
>>> that isn't a mysql guru or Ji
>>>
>>> Curtis Old
>>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>> 46000 Center Oak Plaza Sterling, VA 20166
>>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>>curtis@neustar.biz
>>> / www.neustar.biz <http://www.neustar.biz/>
>>> 
>>>
>>>
>>> The information contained in this e-mail message is intended only for
>>>the
>>> use of the recipient(s) named above and may contain confidential and/or
>>> privileged information. If you are not the intended recipient you have
>>> received this e-mail message in error and any review, dissemination,
>>> distribution, or copying of this message is strictly prohibited. If you
>>> have received this communication in error, please notify us immediately
>>> and delete the original message.
>>>
>>>
>>>
>>>
>>>
>>> On 5/28/13 5:43 PM, "Ahmad Emneina"  wrote:
>>>
>>>> oh this looks like a bug. A potential workaround for it is marking the
>>>> host
>>>> as Up in the cloudstack db. I'd make a backup first then modify the db
>>>> entry.
>>>>
>>>>
>>>> On Tue, May 28, 2013 at 2:36 PM, Old, Curtis 
>>>> wrote:
>>>>
>>>>>   Tailed last 3000 lines right after trying a forced reconnect:
>>>>> http://paste.cloudstack.org/hTOm/
>>>>>
>>>>>  Listed as Management log N*,
>>>>>
>>>>>   Author Curtis Old
>>>>>
>>>>>
>>>>>   Can we see the latest logs? paste.cloudstack.org
>>>>>
>>>>>
>>>>> On Tue, May 28, 2013 at 2:15 PM, Old, Curtis
>>>>> wrote:
>>>>>
>>>>>> Tried that already (like 2 times)
>>>>>>
>>>>>> Curtis Old
>>>>>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>>>>> 46000 Center Oak Plaza Sterling, VA 20166
>>>>>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>>>>> curtis@neustar.biz
>>>>>> / www.neustar.biz <http://www.neustar.biz/>
>>>>>> 
>>>>>>
>>>>>>
>>>>>> The information contained in this e-mail message is intended only
>>>>>>for
>>>>>> the
>>>>>> use of the r

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Oh god where do I find that sucker in the DB?  Sorry I'M an admin type
that isn't a mysql guru or Ji

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/28/13 5:43 PM, "Ahmad Emneina"  wrote:

>oh this looks like a bug. A potential workaround for it is marking the
>host
>as Up in the cloudstack db. I'd make a backup first then modify the db
>entry.
>
>
>On Tue, May 28, 2013 at 2:36 PM, Old, Curtis 
>wrote:
>
>>   Tailed last 3000 lines right after trying a forced reconnect:
>> http://paste.cloudstack.org/hTOm/
>>
>>  Listed as Management log N*,
>>
>>   Author Curtis Old
>>
>>
>>   Can we see the latest logs? paste.cloudstack.org
>>
>>
>> On Tue, May 28, 2013 at 2:15 PM, Old, Curtis
>>wrote:
>>
>>> Tried that already (like 2 times)
>>>
>>> Curtis Old
>>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>> 46000 Center Oak Plaza Sterling, VA 20166
>>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>>curtis@neustar.biz
>>> / www.neustar.biz <http://www.neustar.biz/>
>>> 
>>>
>>>
>>> The information contained in this e-mail message is intended only for
>>>the
>>> use of the recipient(s) named above and may contain confidential and/or
>>> privileged information. If you are not the intended recipient you have
>>> received this e-mail message in error and any review, dissemination,
>>> distribution, or copying of this message is strictly prohibited. If you
>>> have received this communication in error, please notify us immediately
>>> and delete the original message.
>>>
>>>
>>>
>>>
>>>
>>>  On 5/28/13 5:12 PM, "Ahmad Emneina"  wrote:
>>>
>>> >how about a reboot of the host, then force reconnect.
>>> >
>>> >
>>> >On Tue, May 28, 2013 at 2:05 PM, Old, Curtis 
>>> >wrote:
>>> >
>>> >> Bummer no goodŠ...
>>> >>
>>> >> Curtis Old
>>> >> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>> >> 46000 Center Oak Plaza Sterling, VA 20166
>>> >> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>> curtis@neustar.biz
>>> >> / www.neustar.biz <http://www.neustar.biz/>
>>> >> 
>>> >>
>>> >>
>>> >> The information contained in this e-mail message is intended only
>>>for
>>> >>the
>>> >> use of the recipient(s) named above and may contain confidential
>>>and/or
>>> >> privileged information. If you are not the intended recipient you
>>>have
>>> >> received this e-mail message in error and any review, dissemination,
>>> >> distribution, or copying of this message is strictly prohibited. If
>>>you
>>> >> have received this communication in error, please notify us
>>>immediately
>>> >> and delete the original message.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On 5/28/13 5:01 PM, "Ahmad Emneina"  wrote:
>>> >>
>>> >> >dont do that :D do a toolstack restart on the xenserver host, and a
>>> >>force
>>> >> >reconnect from cloudstack. see if that makes them play nice.
>>> >> >
>>> >> >
>>> >> >On Tue, May 28, 2013 at 1:58 PM, Old, Curtis
>>>
>>> >> >wrote:
>>> >> >
>>> >> >>   Well the hypervisior looks fine from a Xenserver prospective
>>>but
>>> >> >> CloudStack won't "reconnect" it.  I thought about deleting it in
>>>CS
>>&

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Tailed last 3000 lines right after trying a forced reconnect:  
http://paste.cloudstack.org/hTOm/

Listed as Management log N*,

 Author Curtis Old


Can we see the latest logs? paste.cloudstack.org<http://paste.cloudstack.org>


On Tue, May 28, 2013 at 2:15 PM, Old, Curtis 
mailto:curtis@neustar.biz>> wrote:
Tried that already (like 2 times)

Curtis Old
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / 
curtis@neustar.biz<mailto:curtis@neustar.biz>
/ www.neustar.biz<http://www.neustar.biz> <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/28/13 5:12 PM, "Ahmad Emneina" 
mailto:aemne...@gmail.com>> wrote:

>how about a reboot of the host, then force reconnect.
>
>
>On Tue, May 28, 2013 at 2:05 PM, Old, Curtis 
>mailto:curtis@neustar.biz>>
>wrote:
>
>> Bummer no goodŠ...
>>
>> Curtis Old
>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>> 46000 Center Oak Plaza Sterling, VA 20166
>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 / 
>> curtis@neustar.biz<mailto:curtis@neustar.biz>
>> / www.neustar.biz<http://www.neustar.biz> <http://www.neustar.biz/>
>> 
>>
>>
>> The information contained in this e-mail message is intended only for
>>the
>> use of the recipient(s) named above and may contain confidential and/or
>> privileged information. If you are not the intended recipient you have
>> received this e-mail message in error and any review, dissemination,
>> distribution, or copying of this message is strictly prohibited. If you
>> have received this communication in error, please notify us immediately
>> and delete the original message.
>>
>>
>>
>>
>>
>> On 5/28/13 5:01 PM, "Ahmad Emneina" 
>> mailto:aemne...@gmail.com>> wrote:
>>
>> >dont do that :D do a toolstack restart on the xenserver host, and a
>>force
>> >reconnect from cloudstack. see if that makes them play nice.
>> >
>> >
>> >On Tue, May 28, 2013 at 1:58 PM, Old, Curtis 
>> >mailto:curtis@neustar.biz>>
>> >wrote:
>> >
>> >>   Well the hypervisior looks fine from a Xenserver prospective but
>> >> CloudStack won't "reconnect" it.  I thought about deleting it in CS
>>and
>> >> re-adding it, but that bugs me lol
>> >>
>> >>  *Curtis Old**
>> >> **Neustar, Inc. / Neustarlabs / **Senior Research Engineer*
>> >>
>> >> *46000 Center Oak Plaza Sterling, VA 20166**
>> >> **Office:** **+1.571.434.5384** **Mobile: **+1.540.532.2230** **/**
>>**
>> >> curtis@neustar.biz<mailto:curtis@neustar.biz>**  / 
>> >> **www.neustar.biz<http://www.neustar.biz>** *
>> >>  *
>> >> --
>> >> *
>> >>
>> >> *The information contained in this e-mail message is intended only
>>for
>> >> the use of the recipient(s) named above and may contain confidential
>> >>and/or
>> >> privileged information. If you are not the intended recipient you
>>have
>> >> received this e-mail message in error and any review, dissemination,
>> >> distribution, or copying of this message is strictly prohibited. If
>>you
>> >> have received this communication in error, please notify us
>>immediately
>> >>and
>> >> delete the original message.*
>> >>
>> >>   From: Ahmad Emneina mailto:aemne...@gmail.com>>
>> >> Reply-To: "aemne...@gmail.com<mailto:aemne...@gmail.com>" 
>> >> mailto:aemne...@gmail.com>>
>> >> Date: Tuesday, May 28, 2013 4:56 PM
>> >> To: "Old, Curtis" mailto:curtis@neustar.biz>>
>> >> Subject: Re: Xenserver hypervisior will not reconnect after reboot
>> >>
>> >>   are you good to go again, after cleaning up some space?
>> >>
>>

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Tried that already (like 2 times)

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/28/13 5:12 PM, "Ahmad Emneina"  wrote:

>how about a reboot of the host, then force reconnect.
>
>
>On Tue, May 28, 2013 at 2:05 PM, Old, Curtis 
>wrote:
>
>> Bummer no goodŠ...
>>
>> Curtis Old
>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>> 46000 Center Oak Plaza Sterling, VA 20166
>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
>> / www.neustar.biz <http://www.neustar.biz/>
>> 
>>
>>
>> The information contained in this e-mail message is intended only for
>>the
>> use of the recipient(s) named above and may contain confidential and/or
>> privileged information. If you are not the intended recipient you have
>> received this e-mail message in error and any review, dissemination,
>> distribution, or copying of this message is strictly prohibited. If you
>> have received this communication in error, please notify us immediately
>> and delete the original message.
>>
>>
>>
>>
>>
>> On 5/28/13 5:01 PM, "Ahmad Emneina"  wrote:
>>
>> >dont do that :D do a toolstack restart on the xenserver host, and a
>>force
>> >reconnect from cloudstack. see if that makes them play nice.
>> >
>> >
>> >On Tue, May 28, 2013 at 1:58 PM, Old, Curtis 
>> >wrote:
>> >
>> >>   Well the hypervisior looks fine from a Xenserver prospective but
>> >> CloudStack won't "reconnect" it.  I thought about deleting it in CS
>>and
>> >> re-adding it, but that bugs me lol
>> >>
>> >>  *Curtis Old**
>> >> **Neustar, Inc. / Neustarlabs / **Senior Research Engineer*
>> >>
>> >> *46000 Center Oak Plaza Sterling, VA 20166**
>> >> **Office:** **+1.571.434.5384** **Mobile: **+1.540.532.2230** **/**
>>**
>> >> curtis@neustar.biz**  / **www.neustar.biz** *
>> >>  *
>> >> --
>> >> *
>> >>
>> >> *The information contained in this e-mail message is intended only
>>for
>> >> the use of the recipient(s) named above and may contain confidential
>> >>and/or
>> >> privileged information. If you are not the intended recipient you
>>have
>> >> received this e-mail message in error and any review, dissemination,
>> >> distribution, or copying of this message is strictly prohibited. If
>>you
>> >> have received this communication in error, please notify us
>>immediately
>> >>and
>> >> delete the original message.*
>> >>
>> >>   From: Ahmad Emneina 
>> >> Reply-To: "aemne...@gmail.com" 
>> >> Date: Tuesday, May 28, 2013 4:56 PM
>> >> To: "Old, Curtis" 
>> >> Subject: Re: Xenserver hypervisior will not reconnect after reboot
>> >>
>> >>   are you good to go again, after cleaning up some space?
>> >>
>> >>
>> >> On Tue, May 28, 2013 at 1:24 PM, Old, Curtis
>> >>wrote:
>> >>
>> >>> Sorry the log entries are from the xenserver log
>> >>>
>> >>> Curtis Old
>> >>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>> >>> 46000 Center Oak Plaza Sterling, VA 20166
>> >>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>> >>>curtis@neustar.biz
>> >>> / www.neustar.biz <http://www.neustar.biz/>
>> >>> 
>> >>>
>> >>>
>> >>> The information contained in this e-mail message is intended only
>>for
>> >>>the
>> >>> use of the recipient(s) named above and may contain confidentia

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Bummer no goodŠ...

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/28/13 5:01 PM, "Ahmad Emneina"  wrote:

>dont do that :D do a toolstack restart on the xenserver host, and a force
>reconnect from cloudstack. see if that makes them play nice.
>
>
>On Tue, May 28, 2013 at 1:58 PM, Old, Curtis 
>wrote:
>
>>   Well the hypervisior looks fine from a Xenserver prospective but
>> CloudStack won't "reconnect" it.  I thought about deleting it in CS and
>> re-adding it, but that bugs me lol
>>
>>  *Curtis Old**
>> **Neustar, Inc. / Neustarlabs / **Senior Research Engineer*
>>
>> *46000 Center Oak Plaza Sterling, VA 20166**
>> **Office:** **+1.571.434.5384** **Mobile: **+1.540.532.2230** **/** **
>> curtis@neustar.biz**  / **www.neustar.biz** *
>>  *
>> --
>> *
>>
>> *The information contained in this e-mail message is intended only for
>> the use of the recipient(s) named above and may contain confidential
>>and/or
>> privileged information. If you are not the intended recipient you have
>> received this e-mail message in error and any review, dissemination,
>> distribution, or copying of this message is strictly prohibited. If you
>> have received this communication in error, please notify us immediately
>>and
>> delete the original message.*
>>
>>   From: Ahmad Emneina 
>> Reply-To: "aemne...@gmail.com" 
>> Date: Tuesday, May 28, 2013 4:56 PM
>> To: "Old, Curtis" 
>> Subject: Re: Xenserver hypervisior will not reconnect after reboot
>>
>>   are you good to go again, after cleaning up some space?
>>
>>
>> On Tue, May 28, 2013 at 1:24 PM, Old, Curtis
>>wrote:
>>
>>> Sorry the log entries are from the xenserver log
>>>
>>> Curtis Old
>>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>> 46000 Center Oak Plaza Sterling, VA 20166
>>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>>curtis@neustar.biz
>>> / www.neustar.biz <http://www.neustar.biz/>
>>> 
>>>
>>>
>>> The information contained in this e-mail message is intended only for
>>>the
>>> use of the recipient(s) named above and may contain confidential and/or
>>> privileged information. If you are not the intended recipient you have
>>> received this e-mail message in error and any review, dissemination,
>>> distribution, or copying of this message is strictly prohibited. If you
>>> have received this communication in error, please notify us immediately
>>> and delete the original message.
>>>
>>>
>>>
>>>
>>>
>>>  On 5/28/13 4:20 PM, "Old, Curtis"  wrote:
>>>
>>> >The in innodes did fill, so after I cleaned them up I rebooted and
>>>then
>>> >the host was singing and dancing again in XenCenter, and primary
>>>storage
>>> >was mounted.  What from the management log, if we can narrow it down?
>>> >When I try the forced reconnect only this shows up:
>>> >
>>> >[20130528T20:17:39.118Z|debug|lab-cloud-9|8622
>>>inet_rpc||http_critical]
>>> >Connection terminated
>>> >[20130528T20:17:51.019Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>>> >D:4de2ede3bcba|xapi] stunnel pid: 23045 (cached = false) connected to
>>> >10.31.105.158:443
>>> >[20130528T20:17:51.019Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>>> >D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None
>>>s_pid=23063
>>> >[20130528T20:18:21.089Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>>> >D:4de2ede3bcba|xapi] stunnel pid: 23063 (cached = false) connected to
>>> >10.31.105.158:443
>>> >[20130528T20:18:21.089Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>>> >D:4de2ede3

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Well the hypervisior looks fine from a Xenserver prospective but CloudStack 
won't "reconnect" it.  I thought about deleting it in CS and re-adding it, but 
that bugs me lol

Curtis Old
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / 
curtis@neustar.biz<mailto:curtis@neustar.biz>  / 
www.neustar.biz<http://www.neustar.biz/>

The information contained in this e-mail message is intended only for the use 
of the recipient(s) named above and may contain confidential and/or privileged 
information. If you are not the intended recipient you have received this 
e-mail message in error and any review, dissemination, distribution, or copying 
of this message is strictly prohibited. If you have received this communication 
in error, please notify us immediately and delete the original message.

From: Ahmad Emneina mailto:aemne...@gmail.com>>
Reply-To: "aemne...@gmail.com<mailto:aemne...@gmail.com>" 
mailto:aemne...@gmail.com>>
Date: Tuesday, May 28, 2013 4:56 PM
To: "Old, Curtis" mailto:curtis@neustar.biz>>
Subject: Re: Xenserver hypervisior will not reconnect after reboot

are you good to go again, after cleaning up some space?


On Tue, May 28, 2013 at 1:24 PM, Old, Curtis 
mailto:curtis@neustar.biz>> wrote:
Sorry the log entries are from the xenserver log

Curtis Old
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / 
curtis@neustar.biz<mailto:curtis@neustar.biz>
/ www.neustar.biz<http://www.neustar.biz> <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/28/13 4:20 PM, "Old, Curtis" 
mailto:curtis@neustar.biz>> wrote:

>The in innodes did fill, so after I cleaned them up I rebooted and then
>the host was singing and dancing again in XenCenter, and primary storage
>was mounted.  What from the management log, if we can narrow it down?
>When I try the forced reconnect only this shows up:
>
>[20130528T20:17:39.118Z|debug|lab-cloud-9|8622 inet_rpc||http_critical]
>Connection terminated
>[20130528T20:17:51.019Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] stunnel pid: 23045 (cached = false) connected to
>10.31.105.158:443<http://10.31.105.158:443>
>[20130528T20:17:51.019Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23063
>[20130528T20:18:21.089Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] stunnel pid: 23063 (cached = false) connected to
>10.31.105.158:443<http://10.31.105.158:443>
>[20130528T20:18:21.089Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23063
>[20130528T20:18:39.124Z|debug|lab-cloud-9|8623 inet_rpc||http_critical]
>Connection terminated
>[20130528T20:18:51.159Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] stunnel pid: 23068 (cached = false) connected to
>10.31.105.158:443<http://10.31.105.158:443>
>[20130528T20:18:51.159Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23068
>[20130528T20:19:21.229Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] stunnel pid: 23072 (cached = false) connected to
>10.31.105.158:443<http://10.31.105.158:443>
>[20130528T20:19:21.229Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23072
>
>
>
>
>
>Curtis Old
>Neustar, Inc. / Neustarlabs / Senior Research Engineer
>46000 Center Oak Plaza Sterling, VA 20166
>Office: +1.571.434.5384 Mobile: +1.540.532.2230 / 
>curtis@neustar.biz<mailto:curtis@neustar.biz>
>/ www.neustar.biz<http://www.neustar.biz> <http://www.neustar.biz/>
>
>
>
>The information contained in this e-mail message is intended only for the
>use of the recipient(s) named above and may contain confidential and/or
>privileged information. If you are not the intended recipient you have
>received this e-mail message in error and any review, dissemination,
>distribution, or copying of this m

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Sorry the log entries are from the xenserver log

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/28/13 4:20 PM, "Old, Curtis"  wrote:

>The in innodes did fill, so after I cleaned them up I rebooted and then
>the host was singing and dancing again in XenCenter, and primary storage
>was mounted.  What from the management log, if we can narrow it down?
>When I try the forced reconnect only this shows up:
>
>[20130528T20:17:39.118Z|debug|lab-cloud-9|8622 inet_rpc||http_critical]
>Connection terminated
>[20130528T20:17:51.019Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] stunnel pid: 23045 (cached = false) connected to
>10.31.105.158:443
>[20130528T20:17:51.019Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23063
>[20130528T20:18:21.089Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] stunnel pid: 23063 (cached = false) connected to
>10.31.105.158:443
>[20130528T20:18:21.089Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23063
>[20130528T20:18:39.124Z|debug|lab-cloud-9|8623 inet_rpc||http_critical]
>Connection terminated
>[20130528T20:18:51.159Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] stunnel pid: 23068 (cached = false) connected to
>10.31.105.158:443
>[20130528T20:18:51.159Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23068
>[20130528T20:19:21.229Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] stunnel pid: 23072 (cached = false) connected to
>10.31.105.158:443
>[20130528T20:19:21.229Z| info|lab-cloud-9|28 heartbeat|Heartbeat
>D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23072
>
>
>
>
>
>Curtis Old 
>Neustar, Inc. / Neustarlabs / Senior Research Engineer
>46000 Center Oak Plaza Sterling, VA 20166
>Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
>/ www.neustar.biz <http://www.neustar.biz/>
>
>
>
>The information contained in this e-mail message is intended only for the
>use of the recipient(s) named above and may contain confidential and/or
>privileged information. If you are not the intended recipient you have
>received this e-mail message in error and any review, dissemination,
>distribution, or copying of this message is strictly prohibited. If you
>have received this communication in error, please notify us immediately
>and delete the original message.
>
>
>
>
>
>On 5/28/13 4:08 PM, "Ahmad Emneina"  wrote:
>
>>pretty odd... how does the disk space look on the xenserver host? we'd
>>need
>>more from the management log, as well as the xensource.log off the
>>hypervisor.
>>
>>
>>On Tue, May 28, 2013 at 9:21 AM, Old, Curtis 
>>wrote:
>>
>>>
>>> Tried the force reconnect from CS 4.0.0 UI and I get "Command Failed
>>>due
>>> to internal server Error" any ideas?
>>>
>>> management-server.log
>>>
>>> 2013-05-24 10:33:00,974 DEBUG [cloud.async.AsyncJobManagerImpl]
>>> (catalina-exec-7:null) submit async job-230, details: AsyncJobVO
>>>{id:230,
>>> userId: 2, accountId: 2, sessionKey: null, instanceType: Host,
>>>instanceId:
>>> 20, cmd: com.cloud.api.commands.ReconnectHostCmd, cmdOriginator: null,
>>> cmdInfo:
>>> 
>>>{"response":"json","id":"346bcc09-e56b-4b27-b269-29252bc2a653","sessionk
>>>e
>>>y":"k/+GoOS0g1IOaSr5bKA2BWg1yD4\u003d","ctxUserId":"2","_":"136940598077
>>>8
>>>","ctxAccountId":"2","ctxStartEventId":"1514"},
>>> cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
>>> processStatus: 0, resultCode: 0, result: null, initMsid:

Re: Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

The in innodes did fill, so after I cleaned them up I rebooted and then
the host was singing and dancing again in XenCenter, and primary storage
was mounted.  What from the management log, if we can narrow it down?
When I try the forced reconnect only this shows up:

[20130528T20:17:39.118Z|debug|lab-cloud-9|8622 inet_rpc||http_critical]
Connection terminated
[20130528T20:17:51.019Z| info|lab-cloud-9|28 heartbeat|Heartbeat
D:4de2ede3bcba|xapi] stunnel pid: 23045 (cached = false) connected to
10.31.105.158:443
[20130528T20:17:51.019Z| info|lab-cloud-9|28 heartbeat|Heartbeat
D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23063
[20130528T20:18:21.089Z| info|lab-cloud-9|28 heartbeat|Heartbeat
D:4de2ede3bcba|xapi] stunnel pid: 23063 (cached = false) connected to
10.31.105.158:443
[20130528T20:18:21.089Z| info|lab-cloud-9|28 heartbeat|Heartbeat
D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23063
[20130528T20:18:39.124Z|debug|lab-cloud-9|8623 inet_rpc||http_critical]
Connection terminated
[20130528T20:18:51.159Z| info|lab-cloud-9|28 heartbeat|Heartbeat
D:4de2ede3bcba|xapi] stunnel pid: 23068 (cached = false) connected to
10.31.105.158:443
[20130528T20:18:51.159Z| info|lab-cloud-9|28 heartbeat|Heartbeat
D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23068
[20130528T20:19:21.229Z| info|lab-cloud-9|28 heartbeat|Heartbeat
D:4de2ede3bcba|xapi] stunnel pid: 23072 (cached = false) connected to
10.31.105.158:443
[20130528T20:19:21.229Z| info|lab-cloud-9|28 heartbeat|Heartbeat
D:4de2ede3bcba|xapi] with_recorded_stunnelpid task_opt=None s_pid=23072





Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/28/13 4:08 PM, "Ahmad Emneina"  wrote:

>pretty odd... how does the disk space look on the xenserver host? we'd
>need
>more from the management log, as well as the xensource.log off the
>hypervisor.
>
>
>On Tue, May 28, 2013 at 9:21 AM, Old, Curtis 
>wrote:
>
>>
>> Tried the force reconnect from CS 4.0.0 UI and I get "Command Failed due
>> to internal server Error" any ideas?
>>
>> management-server.log
>>
>> 2013-05-24 10:33:00,974 DEBUG [cloud.async.AsyncJobManagerImpl]
>> (catalina-exec-7:null) submit async job-230, details: AsyncJobVO
>>{id:230,
>> userId: 2, accountId: 2, sessionKey: null, instanceType: Host,
>>instanceId:
>> 20, cmd: com.cloud.api.commands.ReconnectHostCmd, cmdOriginator: null,
>> cmdInfo:
>> 
>>{"response":"json","id":"346bcc09-e56b-4b27-b269-29252bc2a653","sessionke
>>y":"k/+GoOS0g1IOaSr5bKA2BWg1yD4\u003d","ctxUserId":"2","_":"1369405980778
>>","ctxAccountId":"2","ctxStartEventId":"1514"},
>> cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
>> processStatus: 0, resultCode: 0, result: null, initMsid: 112939138816,
>> completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
>> 2013-05-24 10:33:00,978 DEBUG [cloud.async.AsyncJobManagerImpl]
>> (Job-Executor-16:job-230) Executing
>>com.cloud.api.commands.ReconnectHostCmd
>> for job-230
>> 2013-05-24 10:33:00,985 INFO  [agent.manager.AgentManagerImpl]
>> (Job-Executor-16:job-230) Unable to disconnect host because it is not in
>> the correct state: host=20; Status=Disconnected
>> 2013-05-24 10:33:00,986 WARN  [api.commands.ReconnectHostCmd]
>> (Job-Executor-16:job-230) Exception:
>> com.cloud.api.ServerApiException
>> at
>> 
>>com.cloud.api.commands.ReconnectHostCmd.execute(ReconnectHostCmd.java:108
>>)
>> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
>> at 
>>com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
>> at 
>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> 
>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExe

Xenserver hypervisior will not reconnect after reboot

[Old, Curtis]

Tried the force reconnect from CS 4.0.0 UI and I get "Command Failed due to 
internal server Error" any ideas?

management-server.log

2013-05-24 10:33:00,974 DEBUG [cloud.async.AsyncJobManagerImpl] 
(catalina-exec-7:null) submit async job-230, details: AsyncJobVO {id:230, 
userId: 2, accountId: 2, sessionKey: null, instanceType: Host, instanceId: 20, 
cmd: com.cloud.api.commands.ReconnectHostCmd, cmdOriginator: null, cmdInfo: 
{"response":"json","id":"346bcc09-e56b-4b27-b269-29252bc2a653","sessionkey":"k/+GoOS0g1IOaSr5bKA2BWg1yD4\u003d","ctxUserId":"2","_":"1369405980778","ctxAccountId":"2","ctxStartEventId":"1514"},
 cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, 
processStatus: 0, resultCode: 0, result: null, initMsid: 112939138816, 
completeMsid: null, lastUpdated: null, lastPolled: null, created: null}
2013-05-24 10:33:00,978 DEBUG [cloud.async.AsyncJobManagerImpl] 
(Job-Executor-16:job-230) Executing com.cloud.api.commands.ReconnectHostCmd for 
job-230
2013-05-24 10:33:00,985 INFO  [agent.manager.AgentManagerImpl] 
(Job-Executor-16:job-230) Unable to disconnect host because it is not in the 
correct state: host=20; Status=Disconnected
2013-05-24 10:33:00,986 WARN  [api.commands.ReconnectHostCmd] 
(Job-Executor-16:job-230) Exception:
com.cloud.api.ServerApiException
at com.cloud.api.commands.ReconnectHostCmd.execute(ReconnectHostCmd.java:108)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-05-24 10:33:00,986 WARN  [cloud.api.ApiDispatcher] 
(Job-Executor-16:job-230) class com.cloud.api.ServerApiException : null
2013-05-24 10:33:00,986 DEBUG [cloud.async.AsyncJobManagerImpl] 
(Job-Executor-16:job-230) Complete async job-230, jobStatus: 2, resultCode: 
530, result: Error Code: 534 Error text: null
2013-05-24 10:33:05,998 DEBUG [cloud.async.AsyncJobManagerImpl] 
(catalina-exec-4:null) Async job-230 completed

catalina.out

INFO  [agent.manager.AgentManagerImpl] (Job-Executor-16:job-230) Unable to 
disconnect host because it is not in the correct state: host=20; 
Status=Disconnected
WARN  [api.commands.ReconnectHostCmd] (Job-Executor-16:job-230) Exception:
com.cloud.api.ServerApiException
at 
com.cloud.api.commands.ReconnectHostCmd.execute(ReconnectHostCmd.java:108)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
at 
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
WARN  [cloud.api.ApiDispatcher] (Job-Executor-16:job-230) class 
com.cloud.api.ServerApiException : null

Re: secstorage.service.offering in GS set to service offering ID Webui fails to start

[Old, Curtis]

Yes I made the change, restarted MS, and then the Webui would not come up,
got a Java exception error:

ERROR [utils.component.ComponentLocator] (main:) Unable to load
configuration for management-server from components.xml
java.lang.NumberFormatException: For input string:
"a4ba0546-7d18-4f74-8af6-2ffb8e730ec3"





On 5/24/13 1:29 AM, "Nitin Mehta"  wrote:

>From the code it looks that it should work. Did you restart your MS after
>this change ? You might also need to destroy the SSVM. Can you give it a
>try ? 
>
>On 24/05/13 1:41 AM, "Ahmad Emneina"  wrote:
>
>>Good question, we'll have to defer to someone who has used that
>>feature...
>>
>>
>>On Thu, May 23, 2013 at 12:59 PM, Old, Curtis
>>wrote:
>>
>>> Thanks!!!  That worked, so is there a valid value?  I tried one of my
>>> service offering ID's that was a miserable failure lol
>>>
>>> Curtis Old
>>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>> 46000 Center Oak Plaza Sterling, VA 20166
>>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>>curtis@neustar.biz
>>> / www.neustar.biz <http://www.neustar.biz/>
>>> 
>>>
>>>
>>> The information contained in this e-mail message is intended only for
>>>the
>>> use of the recipient(s) named above and may contain confidential and/or
>>> privileged information. If you are not the intended recipient you have
>>> received this e-mail message in error and any review, dissemination,
>>> distribution, or copying of this message is strictly prohibited. If you
>>> have received this communication in error, please notify us immediately
>>> and delete the original message.
>>>
>>>
>>>
>>>
>>>
>>> On 5/23/13 3:27 PM, "Ahmad Emneina"  wrote:
>>>
>>> >:D sorry, its in the configuration table.
>>> >
>>> >update configuration set value=null where name =
>>> >'secstorage.service.offering';
>>> >
>>> >
>>> >On Thu, May 23, 2013 at 12:19 PM, Old, Curtis
>>> >wrote:
>>> >
>>> >> Yeah if I knew where in the DB I would, I've been looking for hours
>>>lol
>>> >>
>>> >> Curtis Old
>>> >> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>>> >> 46000 Center Oak Plaza Sterling, VA 20166
>>> >> Office: +1.571.434.5384 Mobile: +1.540.532.2230 /
>>> curtis@neustar.biz
>>> >> / www.neustar.biz <http://www.neustar.biz/>
>>> >> 
>>> >>
>>> >>
>>> >> The information contained in this e-mail message is intended only
>>>for
>>> >>the
>>> >> use of the recipient(s) named above and may contain confidential
>>>and/or
>>> >> privileged information. If you are not the intended recipient you
>>>have
>>> >> received this e-mail message in error and any review, dissemination,
>>> >> distribution, or copying of this message is strictly prohibited. If
>>>you
>>> >> have received this communication in error, please notify us
>>>immediately
>>> >> and delete the original message.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> On 5/23/13 3:16 PM, "Ahmad Emneina"  wrote:
>>> >>
>>> >> >set it back to null in the db, and see if that helps. make sure to
>>> >>backup
>>> >> >your db before making any modifications.
>>> >> >
>>> >> >
>>> >> >On Thu, May 23, 2013 at 12:11 PM, Old, Curtis
>>> >> >wrote:
>>> >> >
>>> >> >>
>>> >> >> Does anyone know how to fix this in the database or where ever?
>>> >>Running
>>> >> >> 4.0.0
>>> >> >>
>>> >> >>
>>> >>
>>> >>
>>>
>>>
>

secstorage.service.offering in GS set to service offering ID Webui fails to start

[Old, Curtis]

Does anyone know how to fix this in the database or where ever?  Running 4.0.0

Re: secstorage.service.offering in GS set to service offering ID Webui fails to start

[Old, Curtis]

Thanks!!!  That worked, so is there a valid value?  I tried one of my
service offering ID's that was a miserable failure lol

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/23/13 3:27 PM, "Ahmad Emneina"  wrote:

>:D sorry, its in the configuration table.
>
>update configuration set value=null where name =
>'secstorage.service.offering';
>
>
>On Thu, May 23, 2013 at 12:19 PM, Old, Curtis
>wrote:
>
>> Yeah if I knew where in the DB I would, I've been looking for hours lol
>>
>> Curtis Old
>> Neustar, Inc. / Neustarlabs / Senior Research Engineer
>> 46000 Center Oak Plaza Sterling, VA 20166
>> Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
>> / www.neustar.biz <http://www.neustar.biz/>
>> 
>>
>>
>> The information contained in this e-mail message is intended only for
>>the
>> use of the recipient(s) named above and may contain confidential and/or
>> privileged information. If you are not the intended recipient you have
>> received this e-mail message in error and any review, dissemination,
>> distribution, or copying of this message is strictly prohibited. If you
>> have received this communication in error, please notify us immediately
>> and delete the original message.
>>
>>
>>
>>
>>
>> On 5/23/13 3:16 PM, "Ahmad Emneina"  wrote:
>>
>> >set it back to null in the db, and see if that helps. make sure to
>>backup
>> >your db before making any modifications.
>> >
>> >
>> >On Thu, May 23, 2013 at 12:11 PM, Old, Curtis
>> >wrote:
>> >
>> >>
>> >> Does anyone know how to fix this in the database or where ever?
>>Running
>> >> 4.0.0
>> >>
>> >>
>>
>>

Re: secstorage.service.offering in GS set to service offering ID Webui fails to start

[Old, Curtis]

Yeah if I knew where in the DB I would, I've been looking for hours lol

Curtis Old 
Neustar, Inc. / Neustarlabs / Senior Research Engineer
46000 Center Oak Plaza Sterling, VA 20166
Office: +1.571.434.5384 Mobile: +1.540.532.2230 / curtis@neustar.biz
/ www.neustar.biz <http://www.neustar.biz/>



The information contained in this e-mail message is intended only for the
use of the recipient(s) named above and may contain confidential and/or
privileged information. If you are not the intended recipient you have
received this e-mail message in error and any review, dissemination,
distribution, or copying of this message is strictly prohibited. If you
have received this communication in error, please notify us immediately
and delete the original message.





On 5/23/13 3:16 PM, "Ahmad Emneina"  wrote:

>set it back to null in the db, and see if that helps. make sure to backup
>your db before making any modifications.
>
>
>On Thu, May 23, 2013 at 12:11 PM, Old, Curtis
>wrote:
>
>>
>> Does anyone know how to fix this in the database or where ever?  Running
>> 4.0.0
>>
>>

secstorage.service.offering in GS set to service offering ID Webui fails to start

[Old, Curtis]

Does anyone know how to fix this in the database or where ever?  Running 4.0.0