4.2.1: CLVM to CLVM migrate storage not successfull
Hi devs,
I think I have found a bug affecting a primary storage migration if
the source and destination volume are on a CLVM storage.
So far I could follow the trail at :
- secondary storage is mounted on the source host with a new template id
- cp is issued to copy /dev/volgrpoup/uuid CLVM LV to NFS mount fom
the sec storage
- sec storage is mounted on the destination host
- a LV is created on the destination CLVM
- qemu-img convert is launched to migrate sec storage disk to CLVM lv
but it fails with
014-05-28 19:54:07,421 ERROR [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-2:null) Failed to convert
/mnt/b3a9edfb-2ce7-3c96-ae77-78131bffaa4f/99bddb1b-13aa-4559-aac2-080d5c9576fc.qcow2
to /dev/vg_vmem-003a74_vdi5/3811e2e5-9ef8-4825-9b84-7f5c6fdf87fd the
error was: qemu-img: Could not open
'/mnt/b3a9edfb-2ce7-3c96-ae77-78131bffaa4f/99bddb1b-13aa-4559-aac2-080d5c9576fc.qcow2'qemu-img:
Could not open
'/mnt/b3a9edfb-2ce7-3c96-ae77-78131bffaa4f/99bddb1b-13aa-4559-aac2-080d5c9576fc.qcow2'
At this point I have no clue what might be wrong. All storage is
available and has enough space available. Help would be good, I can
also create a bug if it turns out.
When I do the same convert from CLVM to NFS volume on the same hosts,
all operation are complete, with the exception of another bug that RAW
volumes get converted to qcow2 and windows VMs suddenly can't boot
anymore so I have to convert those back to RAW mode.
Thanks for any help,
Bjoern
Cloustack log :
2014-05-28 19:38:46,952 DEBUG [cloud.async.AsyncJobManagerImpl]
(catalina-exec-1:null) submit async job-3613 = [
68bcafc5-9db8-4de9-bcee-e174d6982ab8 ], details: AsyncJobVO {id:3613,
userId: 3, accountId: 3, sessionKey: null, instanceType: None,
instanceId: null, cmd:
org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd,
cmdOriginator: null, cmdInfo:
{"response":"json","sessionkey":"zG/kiTfpBAgaaNNmCz7tjQRJ7qs\u003d","virtualmachineid":"7d2ae1c9-f306-4050-b53c-deec39d90f8a","cmdEventType":"VM.MIGRATE","ctxUserId":"3","storageid":"f451646b-9e04-42ab-9787-41e4dd5e90c6","httpmethod":"GET","_":"1401331288946","projectid":"a98b95ee-1eba-44f1-a714-00254ec068c5","ctxAccountId":"3","ctxStartEventId":"14750"},
cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0,
processStatus: 0, resultCode: 0, result: null, initMsid: 110493003717,
completeMsid: null, lastUpdated: null, lastPolled: null, created:
null}
2014-05-28 19:38:46,953 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-65:job-3613 = [ 68bcafc5-9db8-4de9-bcee-e174d6982ab8 ])
Executing org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd for
job-3613 = [ 68bcafc5-9db8-4de9-bcee-e174d6982ab8 ]
2014-05-28 19:38:46,983 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-65:job-3613 = [ 68bcafc5-9db8-4de9-bcee-e174d6982ab8 ])
VM state transitted from :Stopped to Migrating with event:
StorageMigrationRequestedvm's original host id: 7 new host id: null
host id before state transition: null
2014-05-28 19:38:47,006 DEBUG
[storage.motion.AncientDataMotionStrategy] (Job-Executor-65:job-3613 =
[ 68bcafc5-9db8-4de9-bcee-e174d6982ab8 ]) copyAsync inspecting src
type VOLUME copyAsync inspecting dest type VOLUME
2014-05-28 19:38:47,009 DEBUG
[cache.allocator.StorageCacheRandomAllocator]
(Job-Executor-65:job-3613 = [ 68bcafc5-9db8-4de9-bcee-e174d6982ab8 ])
Can't find staging storage in zone: 1
2014-05-28 19:38:47,035 DEBUG [agent.transport.Request]
(Job-Executor-65:job-3613 = [ 68bcafc5-9db8-4de9-bcee-e174d6982ab8 ])
Seq 35-134884895: Sending { Cmd , MgmtId: 110493003717, via: 35, Ver:
v1, Flags: 100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"59c7ca2e-dbd1-4d09-a9a2-8c4e28f773f8","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"da7702e3-9629-4a5d-908d-d0d31eb9ba18","id":11,"poolType":"CLVM","host":"localhost","path":"/vg_vmem-003a74_vdi2","port":0}},"name":"ROOT-106","size":42949672960,"path":"c31934cc-bc60-460f-9a36-d7c9b6eb466b","volumeId":163,"vmName":"i-7-106-VM","accountId":7,"format":"QCOW2","id":163,"hypervisorType":"KVM"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"59c7ca2e-dbd1-4d09-a9a2-8c4e28f773f8","volumeType":"ROOT","dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://xx/vol/cloud_sec_1","_role":"Image"}},"name":"ROOT-106","size":42949672960,"path":"volumes/7/163","volumeId":163,"vmName":"i-7-106-VM","accountId":7,"format":"QCOW2","id":163,"hypervisorType":"KVM"}},"executeInSequence":true,"wait":10800}}]
}
2014-05-28 19:54:06,689 DEBUG [agent.transport.Request]
(Job-Executor-65:job-3613 = [ 68bcafc5-9db8-4de9-bcee-e174d6982ab8 ])
Seq 35-134884895: Received: { Ans: , MgmtId: 110493003717, via: 35,
Ver: v1, Flags: 110, { CopyCmdAnswer } }
2014-05-28 19:54:06,706 DEBUG [agent.transport.Request]
(Job-Executor-65:job-3613 = [ 68bcafc5-9db8-4de9-bcee-e174d6982ab8 ])
Seq 45-2103117064: Sending { Cmd , MgmtId: 1104
Re: NIC / VLAN ISSUE ON KVM HOST
Well one NIC is already a problem and I would encourage you to use VLANs on top of a NIC or bonding interface like I did: === ifcfg-eth0 === DEVICE=eth0 TYPE=Ethernet BOOTPROTO=none ONBOOT=yes BRIDGE=cloudbr0 === ifcfg-eth0.2578 === DEVICE=eth0.2578 VLAN=yes TYPE=Ethernet BOOTPROTO=none ONBOOT=yes BRIDGE=cloud-storage === ifcfg-eth0.2610 === DEVICE=eth0.2610 VLAN=yes TYPE=Ethernet BOOTPROTO=none ONBOOT=yes BRIDGE=cloud-mgmt === ifcfg-cloudbr0 === DEVICE=cloudbr0 TYPE=Ethernet BOOTPROTO=none ONBOOT=yes IPADDR=Management Hypervisor IP NETMASK=xxx GATEWAY=xxx DNS1=xxx DNS2=xxx DOMAIN=xxx === ifcfg-cloud-mgmt === DEVICE=cloud-mgmt TYPE=Bridge ONBOOT=yes BOOTPROTO=none DELAY=0 === ifcfg-cloud-storage === DEVICE=cloud-storage TYPE=Bridge ONBOOT=yes BOOTPROTO=none DELAY=0 eth0.2610 and eth0.2578 are VLANIDs I created for the storage and mgmt network only. As guest network I just specified cloudbr0 as KVM label and put a LAN with a VLANID ontop as defined in CS guest LANs. CS will create additional bridged ifcfg-cloudVirBR automatically. Thanks, Bjoern
Re: cloudstack installation
Hello Carlos, /usr/lib64/cloud/common/scripts/storage/secondary/cloud-install-sys-tmpl or /mnt/secondary is missing. Other then that, Cloudstack is a IaaS solution, so you can provision automated resources (GUI or API) and also some sort of billing capabilities are included. At least cloudstack is collecting the usage data. But I haven't looked into that. There are also a lot of other IaaS solutions out there but I found cloudstack better documented and somehow more mature compared to other projects. They look sometimes still like unfinished software. Bjoern On 3/30/2013 8:22 PM, Carlos Cruz wrote: I'm trying to install CloudStack to test when I enter this command as per the instructions: /usr/lib64/cloud/common/scripts/storage/secondary/cloud-install-sys-tmplt -m /mnt/secondary -u http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2 -h xenserver -s -F I get the error message no such file or directory Can anyone tell me what I'm doing wrong. A little off topic maybe someone can share some of their insight as to if I'm on the right track, the reason I'm evaluating CloudStack is because I want to host my own application for resale as a subscription. I'm contemplating provisioning each client with their own VM. Is CloudStack a good solution for what I'm trying to do? Thanks Carlos 2
Re: VLAN traffic not seen by System Router
Well it's quite difficult to debug your issue. I can tell only what helped me in such cases : 1. Get the vnet interfaces from your VM (virsh dumpxml s-xx-VM) 2. Get bridge infos using brctl show 3. Log into the system VM from the hypervisor as root using ssh -i .ssh/id_rsa.cloud r...@169.254.xx.xx -p 3922 ( The IP is the link local IP) 4. Check each interface if you can ping gateways Bye, Bjoern On 04/03/2013 03:14 AM, Valery Fongang wrote: Hi, I have the following setup: - 1 Physical Host with single NIC on Centos 6.3+KVM - 1 Virtual Cloud Management Server on a separate Physical Host from my KVM Host - All 2 Physical servers plug into the same switch a NETGEAR GS748T - My KVM host is connected on Port 1 where I have untagged VLAN trunk. - My guest network VLAN range is set to 200-300. - System VMs and Instances can be created with no issue. The Centos Template has been downloaded and I am able to spin VMs with it. THE ISSUE: - My system routers are not able to communicate over their public IPs. - I am not able to ping my system Router Public IP from a guest VM however I am able to ping all the system VMs from guest VMs. - My guest VMs are unable to reach the Internet. With the NETGEAR switch I have I am not able to trunk a range of VLANs but I make sure I manually add any VLAN used by Cloudstack into the switch and add port 1 where my host is connected to that VLAN. I have played around with my traffic labels and /etc/cloud/agent/agent.properties but I am still unable to get this working. Any idea how to get my guest VMs to access the public network. Thanks,
Re: FW: Auto-scaling
Important question for me is, does it work with the built in LB ? I guess I saw only support for Netscaler, which would be a disappointment for me.. Bjoern On 6/15/2013 8:19 AM, Farid BinBeshr wrote: Does auto-scaling work with basic networking? Regards,Farid Binbeshr
F5 inline mode
Hi Guys, who's using the F5 load balancer in the new inline mode ? Does is need basic or advanced network ? Thanks, Bjoern
Re: F5 inline mode
Thanks, Venkata That's what I thought. How to you do security on advanced mode since security groups are not available ? Bjoern On 8/3/2013 10:38 PM, Venkata SwamyBabu Budumuru wrote: F5 is not supported in basic. Scenario that you mentioned here can only be done in advanced network. Sent from Samsung tablet Bjoern Teipel wrote: Hi Guys, who's using the F5 load balancer in the new inline mode ? Does is need basic or advanced network ? Thanks, Bjoern
LXC
A question came up if LXC might be useful for cloudstack and I personally agree that's a pretty interesting product for all those internet guys like me who runs their hardware to the extreme. I came today across an interesting podcast (http://www.theregister.co.uk/2013/08/01/speaking_in_tech_episode_69/) which covers also the announcement of Openstack and AWS API and they mentioned LXC support's being developed on. More info at http://tech.gilt.com/post/41713823209/adding-lxc-support-to-cloudstack So I hope we'll getting a new toy pretty soon ... Regards, Bjoern
System VM Template : wrong path for downloading in 4.2
How can I fix the doc ? There is a wrong path at http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Installation_Guide/management-server-install-flow.html#prepare-system-vm-template Stating the path /usr/lib64/cloud/common/scripts/storage/secondary/cloud-install-sys-tmplt but it should be /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt Not a bid deal for me but for other users it might be Bjoern
Re: System VM Template : wrong path for downloading in 4.2
Hi Travis, I'm running centos 6.4, the documentation doesn't mention anything regarding a difference between Centos and ubuntu. And honestly there shouldn't be a difference.. What is the correct path of a KVM system VM template ? Doub't that it is acton-systemvm-02062012.qcow2.bz2 again which was used in a earlier version Bjoern On 10/09/2013 07:11 AM, Adrian Lewis wrote: Centos is wrong - I've used Centos 6.4 and the path needs to be: /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-t mplt instead of: /usr/lib64/cloud/common/scripts/storage/secondary/cloud-install-sys-tmplt as documented Not sure if the download URL is correct either. I've seen other posts suggesting it should be (in the case of Xen as an example): http://download.cloud.com/templates/4.2/systemvmtemplate-2013-07-12-master -xen.vhd.bz2 instead of: http://download.cloud.com/templates/acton/acton-systemvm-02062012.vhd.bz2 as documented. Is this URL change correct? Not tried it with the old one and still setting up so not sure if I should actually be using the one I have used. The obvious question would be "well does it work?" but unfortunately I've come up against other issues which prevent me from testing! -Original Message- From: Travis Graham [mailto:tgra...@tgraham.us] Sent: 09 October 2013 05:22 To: users@cloudstack.apache.org Subject: Re: System VM Template : wrong path for downloading in 4.2 Both are referenced in that doc, the first is for CentOS and the second is for Ubuntu. Which OS are you using? Travis On Oct 9, 2013, at 12:15 AM, Bjoern Teipel wrote: How can I fix the doc ? There is a wrong path at http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.2.0/html/Insta llation_Guide/management-server-install-flow.html#prepare-system-vm-templa te Stating the path /usr/lib64/cloud/common/scripts/storage/secondary/cloud-install-sys-tmplt but it should be /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-t mplt Not a bid deal for me but for other users it might be Bjoern
Creating advanced network
Hi Guys, I wanted to setup a advanced zone with security groups and saw this exceptions in the log while I was configuring the networks. What does that mean, what are the limitations from a advanced zone + SG ? ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-23 = [ 7c7e4264-721d-448b-8a75-b68ffeb52d56 ]) Unexpected exception while executing org.apache.cloudstack.api.command.admin.network.UpdatePhysicalNetworkCmd com.cloud.exception.InvalidParameterValueException: Can't add vnet range to the physical network in the zone that supports Advanced network, Security Group enabled: true at com.cloud.network.NetworkServiceImpl.updatePhysicalNetwork(NetworkServiceImpl.java:2527) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at org.apache.cloudstack.api.command.admin.network.UpdatePhysicalNetworkCmd.execute(UpdatePhysicalNetworkCmd.java:98) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Also I tagged the cloudstack storage network (specified Vlan ID in the wizard) but I see the interfaces are bound to cloudbr0 oppose to brxxx-VLANID Does the storage network need to be untagged ? Also is it true the guest network is the public network ? That confuses me a little bit since I configured a internal IP range but now I can't see how/where to configure the external/public IP. Thanks in advance, Bjoern
Re: Creating advanced network
Who can help me here ? Right now the biggest issue for me are the last questions. On 10/14/2013 01:03 AM, Bjoern Teipel wrote: Hi Guys, I wanted to setup a advanced zone with security groups and saw this exceptions in the log while I was configuring the networks. What does that mean, what are the limitations from a advanced zone + SG ? ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-23:job-23 = [ 7c7e4264-721d-448b-8a75-b68ffeb52d56 ]) Unexpected exception while executing org.apache.cloudstack.api.command.admin.network.UpdatePhysicalNetworkCmd com.cloud.exception.InvalidParameterValueException: Can't add vnet range to the physical network in the zone that supports Advanced network, Security Group enabled: true at com.cloud.network.NetworkServiceImpl.updatePhysicalNetwork(NetworkServiceImpl.java:2527) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at org.apache.cloudstack.api.command.admin.network.UpdatePhysicalNetworkCmd.execute(UpdatePhysicalNetworkCmd.java:98) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Also I tagged the cloudstack storage network (specified Vlan ID in the wizard) but I see the interfaces are bound to cloudbr0 oppose to brxxx-VLANID Does the storage network need to be untagged ? Also is it true the guest network is the public network ? That confuses me a little bit since I configured a internal IP range but now I can't see how/where to configure the external/public IP. Thanks in advance, Bjoern
Traffic Type : public gone when using SG and advanced network
The traffic type "public" is gone when using SG and advanced network in 4.2 How I'm supposed to use the cloudstack loadbalancing (VR) in this constellation ? Bjoern
Re: Creating advanced network
Wow, all user@cloudstack mails got catched in my spam filter, so sorry for the late response. After tinkering the whole day I gave up using a tagged VLAN for the storage traffic, seems not to work. It ignores the VID and doesn't create the VLAN on the hypervisor. I added the vlan to the hypervisor now and bound cloudbr1 to it and using it untagged in cloudstack. Finally all is up. :-) Now I was looking how to use a load balancer like the internal cloudstack one or even the F5 and it seems it's not supported. No cloudstack support for internal LB (the VR one) or F5 ? Really !!! According to the advanced network and security groups specification ( https://cwiki.apache.org/confluence/display/CLOUDSTACK/Isolation+based+on+Security+Groups+in+Advance+zone) AddF5LoadBalancerCmd api commands will just fail in SG enabled zone. That's just a joke. I'm really close to end the cloudstack adventure and move on with open stack. Having a shared network with SG and loadbalancer is not really a uncommon solution Thanks, Bjoern On 10/14/2013 11:09 AM, motty cruz wrote: Hello Bjoern, I'm not an expert with Cloudstack, but I will share my limited knowledge. Guest traffic This is the network traffic generated by the communication between the guest VMs. This traffic flows over the guest network and it can be shared or isolated. I have Cloudstack 4.1 installed configured with Advance networking, my hypervisor has two network interfaces on for private (management/storage) one for public (public/guest) network private interface eth1 bridge to cloudbr1 public interface eth0 bridge to cloudbr0 guest vlan eth0.100 (10.1.1.0/24 CIDR) my setup is partially working, I can create instances but can't ping my virtual router, I'm in the process as well, Thanks, On Mon, Oct 14, 2013 at 10:45 AM, Bjoern Teipel < bjoern.tei...@internetbrands.com> wrote: Who can help me here ? Right now the biggest issue for me are the last questions. On 10/14/2013 01:03 AM, Bjoern Teipel wrote: Hi Guys, I wanted to setup a advanced zone with security groups and saw this exceptions in the log while I was configuring the networks. What does that mean, what are the limitations from a advanced zone + SG ? ERROR [cloud.async.**AsyncJobManagerImpl] (Job-Executor-23:job-23 = [ 7c7e4264-721d-448b-8a75-**b68ffeb52d56 ]) Unexpected exception while executing org.apache.cloudstack.api.**command.admin.network.** UpdatePhysicalNetworkCmd com.cloud.exception.**InvalidParameterValueException**: Can't add vnet range to the physical network in the zone that supports Advanced network, Security Group enabled: true at com.cloud.network.**NetworkServiceImpl.** updatePhysicalNetwork(**NetworkServiceImpl.java:2527) at com.cloud.utils.component.**ComponentInstantiationPostProc** essor$InterceptorDispatcher.**intercept(**ComponentInstantiationPostProc* *essor.java:125) at org.apache.cloudstack.api.**command.admin.network.** UpdatePhysicalNetworkCmd.**execute(**UpdatePhysicalNetworkCmd.java:**98) at com.cloud.api.ApiDispatcher.**dispatch(ApiDispatcher.java:** 158) at com.cloud.async.**AsyncJobManagerImpl$1.run(** AsyncJobManagerImpl.java:531) at java.util.concurrent.**Executors$RunnableAdapter.** call(Executors.java:471) at java.util.concurrent.**FutureTask$Sync.innerRun(** FutureTask.java:334) at java.util.concurrent.**FutureTask.run(FutureTask.**java:166) at java.util.concurrent.**ThreadPoolExecutor.runWorker(** ThreadPoolExecutor.java:1146) at java.util.concurrent.**ThreadPoolExecutor$Worker.run(** ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.**java:679) Also I tagged the cloudstack storage network (specified Vlan ID in the wizard) but I see the interfaces are bound to cloudbr0 oppose to brxxx-VLANID Does the storage network need to be untagged ? Also is it true the guest network is the public network ? That confuses me a little bit since I configured a internal IP range but now I can't see how/where to configure the external/public IP. Thanks in advance, Bjoern
Re: Creating advanced network
Murali, That would be great if you're right. But I'm now in a dead lock: Adding new network offering including LB: 2013-10-15 23:34:50,920 WARN [network.element.VirtualRouterElement] (catalina-exec-19:null) Virtual router can't enable services [Dns Dhcp UserData Lb ] without source NAT service 2013-10-15 23:34:50,924 ERROR [cloud.api.ApiServer] (catalina-exec-19:null) unhandled exception executing api command: createNetworkOffering com.cloud.exception.UnsupportedServiceException: Provider VirtualRouter doesn't support services combination: [Dns, Dhcp, UserData, Lb] That forces me to add source nat, but once I want add a guest network in the zone I get the opposite error. I can't mix SG + sourceNat 013-10-15 23:46:30,896 INFO [cloud.api.ApiServer] (catalina-exec-22:null) Service SourceNat is not allowed in security group enabled zone So no internal lb ? Thanks, Bjoern On 10/15/2013 11:28 PM, Murali Reddy wrote: On 16/10/13 7:17 AM, "Bjoern Teipel" wrote: Wow, all user@cloudstack mails got catched in my spam filter, so sorry for the late response. After tinkering the whole day I gave up using a tagged VLAN for the storage traffic, seems not to work. It ignores the VID and doesn't create the VLAN on the hypervisor. I added the vlan to the hypervisor now and bound cloudbr1 to it and using it untagged in cloudstack. Finally all is up. :-) Now I was looking how to use a load balancer like the internal cloudstack one or even the F5 and it seems it's not supported. No cloudstack support for internal LB (the VR one) or F5 ? Really !!! According to the advanced network and security groups specification ( https://cwiki.apache.org/confluence/display/CLOUDSTACK/Isolation+based+on+ Security+Groups+in+Advance+zone) AddF5LoadBalancerCmd api commands will just fail in SG enabled zone. That's just a joke. 4.1 did not support PF/NAT/LB services in shared network. From 4.2, all network services are supported in shared network with or without SG so you should be able to use F5/VR/Netscaler for LB. I'm really close to end the cloudstack adventure and move on with open stack. Having a shared network with SG and loadbalancer is not really a uncommon solution
Re: Creating advanced network
Hi Murali,
I saw your git commits. I want to compile now your changes into
our source code. Do i need just the one for 4.2 or also the
master commits:
Commit
4d07493a5e6e13462b80ba09c3535fa4af0ebdc7 in branch
refs/heads/4.2 from Murali Reddy
ASF
subversion and git services added a comment - Today 06:18
Commit
df3b09944968718111d9b6b29d4c7f5a5cfaf630 in branch
refs/heads/master from Murali Reddy
ASF
subversion and git services added a comment - Today 14:45
Commit
df3b09944968718111d9b6b29d4c7f5a5cfaf630 in branch
refs/heads/ui-restyle from Murali Reddy
Thanks,
Bjoern
On 10/16/2013 2:35 AM, Murali Reddy wrote:
On 16/10/13 12:23 PM, "Bjoern Teipel"
wrote:
Murali,
That would be great if you're right. But I'm now in a dead lock:
Adding new network offering including LB:
2013-10-15 23:34:50,920 WARN [network.element.VirtualRouterElement]
(catalina-exec-19:null) Virtual router can't enable services [Dns Dhcp
UserData Lb ] without source NAT service
2013-10-15 23:34:50,924 ERROR [cloud.api.ApiServer]
(catalina-exec-19:null) unhandled exception executing api command:
createNetworkOffering
com.cloud.exception.UnsupportedServiceException: Provider VirtualRouter
doesn't support services combination: [Dns, Dhcp, UserData, Lb]
That forces me to add source nat, but once I want add a guest network in
the zone I get the opposite error. I can't mix SG + sourceNat
013-10-15 23:46:30,896 INFO [cloud.api.ApiServer]
(catalina-exec-22:null) Service SourceNat is not allowed in security
group enabled zone
First issue is know issue (CLOUDSTACK-4717) is getting addressed in 4.2.1.
Not sure why source NAT should not be allowed in SG network. Sorry, this
is indeed a dead lock situation. It does not look like you can use LB with
in shared network with SG in advanced zone.
So no internal lb ?
Thanks,
Bjoern
On 10/15/2013 11:28 PM, Murali Reddy wrote:
On 16/10/13 7:17 AM, "Bjoern Teipel"
wrote:
Wow, all user@cloudstack mails got catched in my spam filter, so sorry
for the late response.
After tinkering the whole day I gave up using a tagged VLAN for the
storage traffic, seems not to work. It ignores the VID and doesn't
create the VLAN on the hypervisor.
I added the vlan to the hypervisor now and bound cloudbr1 to it and
using it untagged in cloudstack.
Finally all is up. :-)
Now I was looking how to use a load balancer like the internal
cloudstack one or even the F5 and it seems it's not supported.
No cloudstack support for internal LB (the VR one) or F5 ? Really !!!
According to the advanced network and security groups specification (
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Isolation+based+o
n+
Security+Groups+in+Advance+zone)
AddF5LoadBalancerCmd api commands will just fail in SG enabled zone.
That's just a joke.
4.1 did not support PF/NAT/LB services in shared network. From 4.2, all
network services are supported in shared network with or without SG so
you
should be able to use F5/VR/Netscaler for LB.
I'm really close to end the cloudstack adventure and move on with open
stack.
Having a shared network with SG and loadbalancer is not really a
uncommon solution
4.2.0: KVM snapshorts are not working
b_id, snapshot_store_ref.install_path, snapshot_store_ref.update_count, snapshot_store_ref.updated, snapshot_store_ref.state, snapshot_store_ref.ref_cnt, snapshot_store_ref.volume_id FROM snapshot_store_ref WHERE snapshot_store_ref.snapshot_id = 1 AND snapshot_store_ref.store_id = 1 AND snapshot_store_ref.store_role = 'Image' ORDER BY RAND() LIMIT 1 at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280) at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:138) at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:264) at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1013) ... 16 more 2013-10-15 21:45:59,437 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-2:job-40 = [ 81b303ad-acb2-483f-9571-d09064d1d086 ]) Complete async job-40 = [ 81b303ad-acb2-483f-9571-d09064d1d086 ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to create snapshot due to an internal error creating snapshot for volume 5 SQL Content : mysql> SELECT snapshot_store_ref.id, snapshot_store_ref.store_id, snapshot_store_ref.store_role, snapshot_store_ref.snapshot_id, snapshot_store_ref.created, snapshot_store_ref.last_updated, snapshot_store_ref.size, snapshot_store_ref.physical_size, snapshot_store_ref.parent_snapshot_id, snapshot_store_ref.job_id, snapshot_store_ref.install_path, snapshot_store_ref.update_count, snapshot_store_ref.updated, snapshot_store_ref.state, snapshot_store_ref.ref_cnt, snapshot_store_ref.volume_id FROM snapshot_store_ref WHERE snapshot_store_ref.snapshot_id = 1 AND snapshot_store_ref.store_id = 1 AND snapshot_store_ref.store_role = 'Image' ORDER BY RAND() LIMIT 1\G *** 1. row *** id: 2 store_id: 1 store_role: Image snapshot_id: 1 created: 2013-10-16 04:45:16 last_updated: NULL size: 0 physical_size: 0 parent_snapshot_id: 0 job_id: NULL install_path: snapshots/4/5 update_count: 1 updated: 2013-10-16 04:45:16 state: Creating ref_cnt: 0 volume_id: 5 mysql> SELECT alert.id, alert.type, alert.cluster_id, alert.pod_id, alert.data_center_id, alert.subject, alert.sent_count, alert.created, alert.last_sent, alert.resolved, alert.uuid, alert.archived FROM alert WHERE alert.type = 22 AND alert.data_center_id = 0 AND alert.archived = 0 AND alert.pod_id = 0 ORDER BY alert.created DESC LIMIT 0, 1\G Empty set (0.00 sec) Actually I can see a qcow2 snapshot file has been created. Not sure why command fails qemu-img info d1595fc0-8220-4d4f-99b9-d02759262bce image: d1595fc0-8220-4d4f-99b9-d02759262bce file format: qcow2 virtual size: 2.0G (2097152000 bytes) disk size: 122M cluster_size: 65536 backing file: /mnt/46ae9df0-3e29-3535-9acc-506dafa3f52c/d07390a2-edf9-4c6c-839c-2e03a4d3cd5c -- Bjoern Teipel System Operation Manager UNIX Internet Brands 909 N. Sepulveda Blvd., 11th Floor El Segundo, CA 90245 P: (310) 280- 5287 bjoern.tei...@internetbrands.com
4.2.1-snapshot : com.cloud.exception.InsufficientServerCapacityException:
Hi,
my cstk deployment stopped working and I can't figure out why, because
all capacity related metrics are ok.
2013-10-21 13:17:51,412 INFO [user.vm.DeployVMCmd]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
com.cloud.exception.InsufficientServerCapacityException: Unable to
create a deployment for
VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]Scope=interface
com.cloud.dc.DataCenter; id=1
2013-10-21 13:17:51,412 INFO [user.vm.DeployVMCmd]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
Unable to create a deployment for
VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]
com.cloud.exception.InsufficientServerCapacityException: Unable to
create a deployment for
VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841)
at
com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577)
at
org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:237)
at
org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3404)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2964)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2950)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-10-21 13:17:51,413 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
Complete async job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ],
jobStatus: 2
, resultCode: 530, result: Error Code: 533 Error text: Unable to create
a deployment for VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]
but this is what I saw before this error:
2013-10-21 13:17:50,617 DEBUG [cloud.server.StatsCollector]
(StatsCollector-1:null) StorageCollector is running...
2013-10-21 13:17:50,620 INFO [storage.endpoint.DefaultEndPointSelector]
(StatsCollector-1:null) No running ssvm is found, so command will be
sent to LocalHostEndPoint
2013-10-21 13:17:50,674 DEBUG [agent.transport.Request]
(StatsCollector-1:null) Seq 1-531890650: Received: { Ans: , MgmtId:
110493122496, via: 1, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2013-10-21 13:17:50,946 DEBUG [agent.transport.Request]
(StatsCollector-1:null) Seq 1-531890651: Received: { Ans: , MgmtId:
110493122496, via: 1, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2013-10-21 13:17:51,142 DEBUG [agent.transport.Request]
(AgentManager-Handler-15:null) Seq 1-531890647: Processing: { Ans: ,
MgmtId: 110493122496, via: 1, Ver: v1, Flags: 110,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"3000605a-b528-400c-8ff2-a894c20b0705","id":0,"format":"QCOW2","accountId":0,"hvm":false}},"result":true,"wait":0}}]
}
2013-10-21 13:17:51,142 DEBUG [agent.transport.Request]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ]) Seq
1-531890647: Received: { Ans: , MgmtId: 110493122496, via: 1, Ver: v1,
Flags: 110, { CopyCmdAnswer } }
2013-10-21 13:17:51,142 DEBUG [agent.manager.AgentAttache]
(AgentManager-Handler-15:null) Seq 1-531890647: No more commands found
2013-10-21 13:17:51,143 DEBUG [image.store.TemplateObject]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
failed to process event and answer
com.cloud.utils.exception.CloudRuntimeException: DB Exception on:
com.mysql.jdbc.JDBC4PreparedStatement@5cdcbeb5: SELECT
template_spool_ref.id, template_spool_ref.pool_id,
template_spool_ref.template_id, template_spool_ref.created,
template_spool_ref.last_updated, template_spool_ref.download_pct,
template_spool_ref.download_state, template_spool_ref.local_path,
template_spool_ref.error_str, template_spool_ref.job_id,
template_spool_ref.install_path, template_spool_ref.template_size,
template_spool_ref.marked_for_gc, template_spool_ref.update_coun
Re: Bridge issues after upgrade to 4.2
Didn't the script /usr/bin/cloudstack-agent-upgrade run while you updated the cloudstack-agent ? Bjoern On 10/21/2013 6:20 AM, Andrija Panic wrote: HI, I have done upgrade from 4.0.0 to 4.2, and it all went fine, until I stoped and tried to start VMs (Vm belongs to an account, which has it's on VLAN dedicated for private network)... and got errors about not being able to joint eth1.500 to breth1-500 because it is already joined to cloudVirBr500 (original bridge that I manualy created during original CS 4.0.0 installation, inside /etc/sysconfig/network-scripts/). So if I use brctl and remove interface from old bridhe and join it to new bridgem, then all VMs DO have internet acces, and all fine. But if I change/reconfigure OS, that is, renam cloudVirBr500 to breth1-500 (on host OS, /etc/sysconfig/network-scripts/) than CloudStack agent give me alert that it could not find public interface and does not start. Then I edit the CS agent conf file, and define new bridge "breth1-xxx" for public network, and agent starts fine, but with alert state, and constantly disconnecting from CS MGMT host... Does anybody have good explanation on how to behave with this cloud bridge naming convention now in 4.2 ? BTW: using advanced networking, eth0-->cloudBr0, eth1-->cloudVirBr500, this is configured via /etc/sysconfig/network-scripts/. eth0 (management, storage, guest,,,) eth1 (guest, public) Maybe I should NOT define cloudVirBr500 at all ? Thanks,
KVM and openvswitch
Is somebody using cloustack >= 4.1 and openvswitch ? I configured everything according the cloudstack wiki and openswitch seems to work fine but the management server refuses to add the host with: 2013-10-22 23:30:52,742 DEBUG [agent.manager.AgentManagerImpl] (AgentConnectTaskPool-1320:null) Failed to handle host connection: com.cloud.exception.ConnectionException: Incorrect Network setup on agent, Reinitialize agent after network names are setup, details :*Can not find network: cloudbr0* But it appears to work fine in ovs: sudo ovs-vsctl show 15e04687-6948-4866-922a-031e5d86672e Bridge "cloud0" Port "cloud0" Interface "cloud0" type: internal Bridge "cloudbr1" Port "bond0.501" Interface "bond0.501" Port "cloudbr1" Interface "cloudbr1" type: internal Bridge "cloudbr0" Port "bond0" Interface "bond0" Port "cloudbr0" Interface "cloudbr0" type: internal ovs_version: "1.9.0" Any ideas ? I'm using a 4.2.1 snapshot build btw. Thanks, Bjoern
Re: 4.2.1-snapshot : com.cloud.exception.InsufficientServerCapacityException:
Yes I had and I changed the delay to 1h.
I also got link communication failures and the secondary storage was
stuck during this time so I decided to rebuild the zone and change the
NFS storage from Gluster to a standard NFS server removing the iptables
rules etc. But I'm still rebuilding it and testing right now KVM with
openvswitch though ...
Bjoern
On 10/22/2013 11:54 PM, Whyspirit wrote:
Hello. Do you have any VMs that needs to be expunged. If it is the
case, please reduce number of them by changing parameters in
cloudstack UI (reduce expunge.delay and expunge. Interval).
Envoyé avec AquaMail pour Android
http://www.aqua-mail.com
Le 21 octobre 2013 22:24:46 Bjoern Teipel
a écrit :
Hi,
my cstk deployment stopped working and I can't figure out why,
because all capacity related metrics are ok.
2013-10-21 13:17:51,412 INFO [user.vm.DeployVMCmd]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
com.cloud.exception.InsufficientServerCapacityException: Unable to
create a deployment for
VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]Scope=interface
com.cloud.dc.DataCenter; id=1
2013-10-21 13:17:51,412 INFO [user.vm.DeployVMCmd]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
Unable to create a deployment for
VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]
com.cloud.exception.InsufficientServerCapacityException: Unable to
create a deployment for
VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841)
at
com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577)
at
org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:237)
at
org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3404)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2964)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2950)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-10-21 13:17:51,413 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
Complete async job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ],
jobStatus: 2
, resultCode: 530, result: Error Code: 533 Error text: Unable to
create a deployment for VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]
but this is what I saw before this error:
2013-10-21 13:17:50,617 DEBUG [cloud.server.StatsCollector]
(StatsCollector-1:null) StorageCollector is running...
2013-10-21 13:17:50,620 INFO
[storage.endpoint.DefaultEndPointSelector] (StatsCollector-1:null) No
running ssvm is found, so command will be sent to LocalHostEndPoint
2013-10-21 13:17:50,674 DEBUG [agent.transport.Request]
(StatsCollector-1:null) Seq 1-531890650: Received: { Ans: , MgmtId:
110493122496, via: 1, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2013-10-21 13:17:50,946 DEBUG [agent.transport.Request]
(StatsCollector-1:null) Seq 1-531890651: Received: { Ans: , MgmtId:
110493122496, via: 1, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2013-10-21 13:17:51,142 DEBUG [agent.transport.Request]
(AgentManager-Handler-15:null) Seq 1-531890647: Processing: { Ans: ,
MgmtId: 110493122496, via: 1, Ver: v1, Flags: 110,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"newData":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"3000605a-b528-400c-8ff2-a894c20b0705","id":0,"format":"QCOW2","accountId":0,"hvm":false}},"result":true,"wait":0}}]
}
2013-10-21 13:17:51,142 DEBUG [agent.transport.Request]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
Seq 1-531890647: Received: { Ans: , MgmtId: 110493122496, via: 1,
Ver: v1, Flags: 110,
Re: KVM and openvswitch
Guess I can narrow it down in the code at
LibvirtComputingResource.java 1036:
File[] interfaces = f.listFiles(); >>> my array is basically
empty.. <<<
for (int i = 0; i < interfaces.length; i++) {
String fname = interfaces[i].getName();
s_logger.debug("matchPifFileInDirectory: file name
'"+fname+"'");
if (fname.startsWith("eth") || fname.startsWith("bond")
|| fname.startsWith("vlan") || fname.startsWith("em")
|| fname.matches("^p\\d+p\\d+.*")) {
return fname;
}
}
My brif directory is empty so I'm not sure if I have a configuration
issue in openvswitch.
Going to check this now.
Bjoern
On 10/22/2013 11:36 PM, Bjoern Teipel wrote:
Is somebody using cloustack >= 4.1 and openvswitch ?
I configured everything according the cloudstack wiki and openswitch
seems to work fine but the management server refuses to add the host with:
2013-10-22 23:30:52,742 DEBUG [agent.manager.AgentManagerImpl]
(AgentConnectTaskPool-1320:null) Failed to handle host connection:
com.cloud.exception.ConnectionException: Incorrect Network setup on
agent, Reinitialize agent after network names are setup, details :*Can
not find network: cloudbr0*
But it appears to work fine in ovs:
sudo ovs-vsctl show
15e04687-6948-4866-922a-031e5d86672e
Bridge "cloud0"
Port "cloud0"
Interface "cloud0"
type: internal
Bridge "cloudbr1"
Port "bond0.501"
Interface "bond0.501"
Port "cloudbr1"
Interface "cloudbr1"
type: internal
Bridge "cloudbr0"
Port "bond0"
Interface "bond0"
Port "cloudbr0"
Interface "cloudbr0"
type: internal
ovs_version: "1.9.0"
Any ideas ? I'm using a 4.2.1 snapshot build btw.
Thanks,
Bjoern
Re: KVM and openvswitch
Yes that's what I followed exactly. My OS is centos 6.4 and brctl bridge namebridge idSTP enabledinterfaces cloud0/sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory .ba2d3a409d48no cloudbr0/sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory .00219ba07885no cloudbr1/sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory .00219ba07885no But the directories are there so it doesn't make a whole lot of sense /sys/class/net/cloud0/bridge [bteipel@hq-kvmhv-002 bridge]$ ll total 0 -rw-r--r-- 1 root root 4096 Oct 23 00:32 ageing_time -r--r--r-- 1 root root 4096 Oct 23 00:32 bridge_id -rw-r--r-- 1 root root 4096 Oct 23 00:32 forward_delay -r--r--r-- 1 root root 4096 Oct 23 00:32 gc_timer -rw-r--r-- 1 root root 4096 Oct 23 09:23 group_addr -rw-r--r-- 1 root root 4096 Oct 23 00:32 hello_time -r--r--r-- 1 root root 4096 Oct 23 00:32 hello_timer -rw-r--r-- 1 root root 4096 Oct 23 00:32 max_age -rw-r--r-- 1 root root 4096 Oct 23 09:23 priority -r--r--r-- 1 root root 4096 Oct 23 00:32 root_id -r--r--r-- 1 root root 4096 Oct 23 00:32 root_path_cost -r--r--r-- 1 root root 4096 Oct 23 00:32 root_port -rw-r--r-- 1 root root 4096 Oct 23 00:32 stp_state -r--r--r-- 1 root root 4096 Oct 23 00:32 tcn_timer -r--r--r-- 1 root root 4096 Oct 23 00:32 topology_change -r--r--r-- 1 root root 4096 Oct 23 00:32 topology_change_detected -r--r--r-- 1 root root 4096 Oct 23 00:32 topology_change_timer On 10/22/2013 11:42 PM, Nguyen Anh Tu wrote: 2013/10/23 Bjoern Teipel Is somebody using cloustack >= 4.1 and openvswitch ? Hi, take a look here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+with+OpenVSwitch
Re: KVM and openvswitch
Yes I did. On 10/23/2013 09:27 AM, Francois Gaudreault wrote: Did you blacklist the linux bridge module? You have to blacklist it. FG On 10/23/2013, 12:24 PM, Bjoern Teipel wrote: Yes that's what I followed exactly. My OS is centos 6.4 and brctl bridge namebridge idSTP enabledinterfaces cloud0/sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory /sys/class/net/cloud0/bridge: No such file or directory .ba2d3a409d48no cloudbr0/sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory /sys/class/net/cloudbr0/bridge: No such file or directory .00219ba07885no cloudbr1/sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory /sys/class/net/cloudbr1/bridge: No such file or directory .00219ba07885no But the directories are there so it doesn't make a whole lot of sense /sys/class/net/cloud0/bridge [bteipel@hq-kvmhv-002 bridge]$ ll total 0 -rw-r--r-- 1 root root 4096 Oct 23 00:32 ageing_time -r--r--r-- 1 root root 4096 Oct 23 00:32 bridge_id -rw-r--r-- 1 root root 4096 Oct 23 00:32 forward_delay -r--r--r-- 1 root root 4096 Oct 23 00:32 gc_timer -rw-r--r-- 1 root root 4096 Oct 23 09:23 group_addr -rw-r--r-- 1 root root 4096 Oct 23 00:32 hello_time -r--r--r-- 1 root root 4096 Oct 23 00:32 hello_timer -rw-r--r-- 1 root root 4096 Oct 23 00:32 max_age -rw-r--r-- 1 root root 4096 Oct 23 09:23 priority -r--r--r-- 1 root root 4096 Oct 23 00:32 root_id -r--r--r-- 1 root root 4096 Oct 23 00:32 root_path_cost -r--r--r-- 1 root root 4096 Oct 23 00:32 root_port -rw-r--r-- 1 root root 4096 Oct 23 00:32 stp_state -r--r--r-- 1 root root 4096 Oct 23 00:32 tcn_timer -r--r--r-- 1 root root 4096 Oct 23 00:32 topology_change -r--r--r-- 1 root root 4096 Oct 23 00:32 topology_change_detected -r--r--r-- 1 root root 4096 Oct 23 00:32 topology_change_timer On 10/22/2013 11:42 PM, Nguyen Anh Tu wrote: 2013/10/23 Bjoern Teipel Is somebody using cloustack >= 4.1 and openvswitch ? Hi, take a look here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+with+OpenVSwitch -- Bjoern Teipel System Operation Manager UNIX Internet Brands 909 N. Sepulveda Blvd., 11th Floor El Segundo, CA 90245 P: (310) 280- 5287 bjoern.tei...@internetbrands.com
Re: KVM and openvswitch
Yup all done, did it multiple time now even with a clean host.
As I said while I was looking into the code, the issue might just that
my brif directory is empty at
/sys/class/net//brif I'm not sure if the open vswitch 1.0.9
version has a bug...
LibvirtComputingResource.java 1036:
(f is an file object from this directory above)
File[] interfaces = f.listFiles(); >>> my array is basically
empty.. <<<
for (int i = 0; i < interfaces.length; i++) {
String fname = interfaces[i].getName();
s_logger.debug("matchPifFileInDirectory: file name
'"+fname+"'");
if (fname.startsWith("eth") || fname.startsWith("bond")
|| fname.startsWith("vlan") || fname.startsWith("em")
|| fname.matches("^p\\d+p\\d+.*")) {
return fname;
}
}
My brif directory is empty so I'm not sure if I have a configuration
issue in openvswitch.
Going to check this now.
On 10/23/2013 10:36 AM, Francois Gaudreault wrote:
So brctl is likely to return a mess.
How did you configure the agent? Did you put :
libvirt.vif.driver=com.cloud.hypervisor.kvm.resource.OvsVifDriver
and
network.bridge.type=openvswitch
in agent.properties?
Francois
On 10/23/2013, 1:24 PM, Bjoern Teipel wrote:
Yes I did.
On 10/23/2013 09:27 AM, Francois Gaudreault wrote:
Did you blacklist the linux bridge module? You have to blacklist it.
FG
On 10/23/2013, 12:24 PM, Bjoern Teipel wrote:
Yes that's what I followed exactly.
My OS is centos 6.4 and brctl
bridge namebridge idSTP enabledinterfaces
cloud0/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
/sys/class/net/cloud0/bridge: No such file or directory
.ba2d3a409d48no
cloudbr0/sys/class/net/cloudbr0/bridge: No such file or
directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
/sys/class/net/cloudbr0/bridge: No such file or directory
.00219ba07885no
cloudbr1/sys/class/net/cloudbr1/bridge: No such file or
directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
/sys/class/net/cloudbr1/bridge: No such file or directory
.00219ba07885no
But the directories are there so it doesn't make a whole lot of sense
/sys/class/net/cloud0/bridge
[bteipel@hq-kvmhv-002 bridge]$ ll
total 0
-rw-r--r-- 1 root root 4096 Oct 23 00:32 ageing_time
-r--r--r-- 1 root root 4096 Oct 23 00:32 bridge_id
-rw-r--r-- 1 root root 4096 Oct 23 00:32 forward_delay
-r--r--r-- 1 root root 4096 Oct 23 00:32 gc_timer
-rw-r--r-- 1 root root 4096 Oct 23 09:23 group_addr
-rw-r--r-- 1 root root 4096 Oct 23 00:32 hello_time
-r--r--r-- 1 root root 4096 Oct 23 00:32 hello_timer
-rw-r--r-- 1 root root 4096 Oct 23 00:32 max_age
-rw-r--r-- 1 root root 4096 Oct 23 09:23 priority
-r--r--r-- 1 root root 4096 Oct 23 00:32 root_id
-r--r--r-- 1 root root 4096 Oct 23 00:32 root_path_cost
-r--r--r-- 1 root root 4096 Oct 23 00:32 root_port
-rw-r--r-- 1 root root 4096 Oct 23 00:32 stp_state
-r--r--r-- 1 root root 4096 Oct 23 00:32 tcn_timer
-r--r--r-- 1 root root 4096 Oct 23 00:32 topology_change
-r--r--r-- 1 root root 4096 Oct 23 00:32 topology_change_detected
-r--r--r-- 1 root root 4096 Oct 23 00:32 topology_change_timer
On 10/22/2013 11:42 PM,
Re: Bridge issues after upgrade to 4.2
The bash skript renames iptables rules and the brdge names. The problem might be it couldn't execute the ip link set XXX down command and renaming it afterwards. You can execute the script again, see what it reports. Bjoern On 10/22/2013 12:16 AM, Andrija Panic wrote: HI, I did not see it run, but I did run it manualy. Still the problem is that the physical interface eth1 is plugged in in cloudVrBr500, and it could not be renamed (this is bridge configured via ifcfg scripts, not sure that is possible to be renamed by script...) Also, does this /usr/bin/cloudstack-agent-**upgrade script, update the cloudstack database to set new label for netowrks... not sure what is the purpose of the script (I inspected source code) except renaming already running bridges..at least some number of it... On 22 October 2013 06:13, Bjoern Teipel wrote: Didn't the script /usr/bin/cloudstack-agent-**upgrade run while you updated the cloudstack-agent ? Bjoern On 10/21/2013 6:20 AM, Andrija Panic wrote: HI, I have done upgrade from 4.0.0 to 4.2, and it all went fine, until I stoped and tried to start VMs (Vm belongs to an account, which has it's on VLAN dedicated for private network)... and got errors about not being able to joint eth1.500 to breth1-500 because it is already joined to cloudVirBr500 (original bridge that I manualy created during original CS 4.0.0 installation, inside /etc/sysconfig/network-**scripts/). So if I use brctl and remove interface from old bridhe and join it to new bridgem, then all VMs DO have internet acces, and all fine. But if I change/reconfigure OS, that is, renam cloudVirBr500 to breth1-500 (on host OS, /etc/sysconfig/network-**scripts/) than CloudStack agent give me alert that it could not find public interface and does not start. Then I edit the CS agent conf file, and define new bridge "breth1-xxx" for public network, and agent starts fine, but with alert state, and constantly disconnecting from CS MGMT host... Does anybody have good explanation on how to behave with this cloud bridge naming convention now in 4.2 ? BTW: using advanced networking, eth0-->cloudBr0, eth1-->cloudVirBr500, this is configured via /etc/sysconfig/network-**scripts/. eth0 (management, storage, guest,,,) eth1 (guest, public) Maybe I should NOT define cloudVirBr500 at all ? Thanks, -- Bjoern Teipel System Operation Manager UNIX Internet Brands 909 N. Sepulveda Blvd., 11th Floor El Segundo, CA 90245 P: (310) 280- 5287 bjoern.tei...@internetbrands.com
Re: 4.2.1-snapshot : com.cloud.exception.InsufficientServerCapacityException:
Damn, I guess I found the issue.
The JDBC error bothered me that much that I looked into my DB config and
then I knew what happened.
The wait_timeout is set to 30 seconds, so during the write of the qcow
snapshot file to the secondary storage, the mysql session was terminated.
After the copy was done, cloudstack tried to select/update tables on a
non existing session anymore.
I removed now this parameter and wait_timeout is set to 8h, the standard
value. So at least all backups should work within this period.
Probably the db/or snapshot code in cloudstack should be changed a
little bit to check for a connection before writing to the DB.
Or even better it should just close the connection and let the copy
finish and then reopen a new connection. On large volumes the copy could
take significant amount of time and I don't want to deal with
wait_timeout all time
Bjoern
On 10/23/2013 12:04 AM, Bjoern Teipel wrote:
Yes I had and I changed the delay to 1h.
I also got link communication failures and the secondary storage was
stuck during this time so I decided to rebuild the zone and change the
NFS storage from Gluster to a standard NFS server removing the
iptables rules etc. But I'm still rebuilding it and testing right now
KVM with openvswitch though ...
Bjoern
On 10/22/2013 11:54 PM, Whyspirit wrote:
Hello. Do you have any VMs that needs to be expunged. If it is the
case, please reduce number of them by changing parameters in
cloudstack UI (reduce expunge.delay and expunge. Interval).
Envoyé avec AquaMail pour Android
http://www.aqua-mail.com
Le 21 octobre 2013 22:24:46 Bjoern Teipel
a écrit :
Hi,
my cstk deployment stopped working and I can't figure out why,
because all capacity related metrics are ok.
2013-10-21 13:17:51,412 INFO [user.vm.DeployVMCmd]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
com.cloud.exception.InsufficientServerCapacityException: Unable to
create a deployment for
VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]Scope=interface
com.cloud.dc.DataCenter; id=1
2013-10-21 13:17:51,412 INFO [user.vm.DeployVMCmd]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
Unable to create a deployment for
VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]
com.cloud.exception.InsufficientServerCapacityException: Unable to
create a deployment for
VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]Scope=interface
com.cloud.dc.DataCenter; id=1
at
com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841)
at
com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:577)
at
org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:237)
at
org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3404)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2964)
at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2950)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
at
com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2013-10-21 13:17:51,413 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-8:job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ])
Complete async job-126 = [ fed97281-7904-427c-b450-80fb32210cbc ],
jobStatus: 2
, resultCode: 530, result: Error Code: 533 Error text: Unable to
create a deployment for VM[User|d0bc4ffd-7850-4503-a62b-482895b4b08a]
but this is what I saw before this error:
2013-10-21 13:17:50,617 DEBUG [cloud.server.StatsCollector]
(StatsCollector-1:null) StorageCollector is running...
2013-10-21 13:17:50,620 INFO
[storage.endpoint.DefaultEndPointSelector] (StatsCollector-1:null)
No running ssvm is found, so command will be sent to LocalHostEndPoint
2013-10-21 13:17:50,674 DEBUG [agent.transport.Request]
(StatsCollector-1:null) Seq 1-531890650: Received: { Ans: , MgmtId:
110493122496, via: 1, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2013-10-21 13:
Re: Can't compile CloudStack-4.2:
I had issues with similar jars and here my quick fix: cd $HOME/.m2/repository/org/apache/axis2/axis2-mtompolicy/1.5.4/ && \ wget http://repo1.maven.org/maven2/org/apache/axis2/axis2-mtompolicy/1.5.4/axis2-mtompolicy-1.5.4.jar -O axis2-mtompolicy-1.5.4.jar cd $HOME/.m2/repository/org/apache/ws/commons/axiom/axiom-dom/1.2.10 && \ wget http://repo1.maven.org/maven2/org/apache/ws/commons/axiom/axiom-dom/1.2.10/axiom-dom-1.2.10.jar -O axiom-dom-1.2.10.jar cd $HOME/.m2/repository/org/opensaml/opensaml1/1.1 && \ wget http://repo1.maven.org/maven2/org/opensaml/opensaml/1.1/opensaml-1.1.jar -O opensaml1-1.1.jar cd $HOME/.m2/repository/commons-lang/commons-lang/2.3 && \ wget http://repo1.maven.org/maven2/commons-lang/commons-lang/2.3/commons-lang-2.3.jar -O commons-lang-2.3.jar cd $HOME/.m2/repository/bouncycastle/bcprov-jdk14/140 && \ wget http://repo1.maven.org/maven2/bouncycastle/bcprov-jdk14/140/bcprov-jdk14-140.jar -O bcprov-jdk14-140.jar On 10/29/2013 02:10 AM, Vladimir Melnik wrote: Dear colleagues, I'm using 2.2.14 and it holds about a hundred of instances). I decided to try 4.2.0 in the testing lab, but I encountered a problem while I was trying to build RPMs. That's what I've got: [INFO] Apache CloudStack Console Proxy ... SUCCESS [37.729s] [INFO] Apache CloudStack Client UI ... SUCCESS [30.354s] [INFO] Apache CloudStack Console Proxy Plugin SUCCESS [0.840s] [INFO] Apache CloudStack AWS API Bridge .. FAILURE [49.932s] [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 12:16.096s [INFO] Finished at: Sat Oct 26 14:13:44 EEST 2013 [INFO] Final Memory: 55M/246M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project cloud-awsapi: Compilation failure: Compilation failure: [ERROR] error: error reading /root/.m2/repository/org/apache/axis2/mex/1.5.4/mex-1.5.4-impl.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/org/apache/axis2/axis2-mtompolicy/1.5.4/axis2-mtompolic y-1.5.4.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/org/apache/ws/commons/axiom/axiom-dom/1.2.10/axiom-dom- 1.2.10.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/org/opensaml/opensaml1/1.1/opensaml1-1.1.jar; error in opening zip file [ERROR] error: error reading /root/.m2/repository/commons-lang/commons-lang/2.3/commons-lang-2.3.jar; error in opening zip file As I can see the problem is in downloading these JAR-files: Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/axis2/mex/1.5.4/ mex-1.5.4-impl.jar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/axis2/axis2-mtom policy/1.5.4/axis2-mtompolicy-1.5.4.jar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/axis2/addressing /1.5.4/addressing-1.5.4.mar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/apache/ws/commons/axiom /axiom-dom/1.2.10/axiom-dom-1.2.10.jar Downloading: http://shibboleth.internet2.edu/downloads/maven2/org/opensaml/opensaml1/1.1/ opensaml1-1.1.jar I have several [WARNING]s (the builder said that "Checksum validation failed") exactly for these files and, as I can see, these files aren't there anymore. Honestly, Java is such a dark forest for me, so, please, can anyone advice how to fix these links? Thank you very much! -- Bjoern Teipel System Operation Manager UNIX Internet Brands 909 N. Sepulveda Blvd., 11th Floor El Segundo, CA 90245 P: (310) 280- 5287 bjoern.tei...@internetbrands.com
Re: After First Time Cloudstack
Don't count or you get sleepy On 11/01/2013 10:36 AM, sebgoa wrote: On Nov 1, 2013, at 6:30 PM, Vahric Muhtaryan wrote: Overview is we are all vmware admins were nurseling :D :P Now we are fighting :D Hehheheheheh Happy good weekends to all See you next week :D Vahric Muhtaryan Enjoy the week end, don't dream about clouds :) -- Bjoern Teipel System Operation Manager UNIX Internet Brands 909 N. Sepulveda Blvd., 11th Floor El Segundo, CA 90245 P: (310) 280- 5287 bjoern.tei...@internetbrands.com
Add a node to an existing clvm cluster
I'm trying to join a new node into an existing 5 node CLVM cluster but I
just can't get it to work.
When ever I add a new node (I put into the cluster.conf and reloaded with
cman_tool version -r -S) I end up with situations like the new node wants
to gain the quorum and starts to fence the existing pool master and appears
to generate some sort of split cluster. Does it work at all? corosync and
dlm do not know about the recently added node, at least when I look in the
current members list ?
New Node
==
Node Sts Inc Joined Name
1 X 0hv-b1clcy1
2 X 0hv-b1flcy1
3 X 0hv-b1fmcy1
4 X 0hv-b1dmcy1
5 X 0hv-b1fkcy1
6 M 80 2014-01-07 21:37:42 hv-b1dkcy1 <--- host added
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [TOTEM ] The network
interface [10.14.18.77] is now up.
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [QUORUM] Using quorum
provider quorum_cman
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [SERV ] Service engine
loaded: corosync cluster quorum service v0.1
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [CMAN ] CMAN 3.0.12.1 (built
Sep 3 2013 09:17:34) started
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [SERV ] Service engine
loaded: corosync CMAN membership service 2.90
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [SERV ] Service engine
loaded: openais checkpoint service B.01.01
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [SERV ] Service engine
loaded: corosync extended virtual synchrony service
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [SERV ] Service engine
loaded: corosync configuration service
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [SERV ] Service engine
loaded: corosync cluster closed process group service v1.01
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [SERV ] Service engine
loaded: corosync cluster config database access v1.01
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [SERV ] Service engine
loaded: corosync profile loading service
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [QUORUM] Using quorum
provider quorum_cman
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [SERV ] Service engine
loaded: corosync cluster quorum service v0.1
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [MAIN ] Compatibility mode
set to whitetank. Using V1 and V2 of the synchronization engine.
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [TOTEM ] adding new UDPU
member {10.14.18.65}
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [TOTEM ] adding new UDPU
member {10.14.18.67}
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [TOTEM ] adding new UDPU
member {10.14.18.68}
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [TOTEM ] adding new UDPU
member {10.14.18.70}
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [TOTEM ] adding new UDPU
member {10.14.18.66}
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [TOTEM ] adding new UDPU
member {10.14.18.77}
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [TOTEM ] A processor joined
or left the membership and a new membership was formed.
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [CMAN ] quorum regained,
resuming activity
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [QUORUM] This node is within
the primary component and will provide service.
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [QUORUM] Members[1]: 6
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [QUORUM] Members[1]: 6
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [CPG ] chosen downlist:
sender r(0) ip(10.14.18.77) ; members(old:0 left:0)
Jan 7 21:37:42 hv-b1dkcy1 corosync[12564]: [MAIN ] Completed service
synchronization, ready to provide service.
Jan 7 21:37:46 hv-b1dkcy1 fenced[12620]: fenced 3.0.12.1 started
Jan 7 21:37:46 hv-b1dkcy1 dlm_controld[12643]: dlm_controld 3.0.12.1
started
Jan 7 21:37:47 hv-b1dkcy1 gfs_controld[12695]: gfs_controld 3.0.12.1
started
Jan 7 21:37:54 hv-b1dkcy1 fenced[12620]: fencing node hv-b1clcy1
sudo -i corosync-objctl |grep member
totem.interface.member.memberaddr=hv-b1clcy1
totem.interface.member.memberaddr=hv-b1fmcy1
totem.interface.member.memberaddr=hv-b1dmcy1
totem.interface.member.memberaddr=hv-b1fkcy1
totem.interface.member.memberaddr=hv-b1flcy1
totem.interface.member.memberaddr=hv-b1dkcy1
runtime.totem.pg.mrp.srp.members.6.ip=r(0) ip(10.14.18.77)
runtime.totem.pg.mrp.srp.members.6.join_count=1
runtime.totem.pg.mrp.srp.members.6.status=joined
Existing Node
=
member 6 has not been added to the quorum list :
Jan 7 21:36:28 hv-b1clcy1 corosync[7769]: [QUORUM] Members[4]: 1 2 3 5
Jan 7 21:37:54 hv-b1clcy1 corosync[7769]: [TOTEM ] A processor joined or
left the membership and a new membership was formed.
Jan 7 21:37:54 hv-b1clcy1 corosync[7769]: [CPG ] chosen downlist:
sender r(0) ip(10.14.18.65) ; members(old:4 left:0)
Node Sts Inc Joined Name
1 M 4468 2013-12-10 14:33:27 hv-b1clcy1
2 M
Windows >2008 and virtio network
Hi, I just noticed that Windows guests, even with 2012 profile will be provisioned with e1000 network driver opposed to virtio Is it in some way possible to prevent and use virtio for all windows 2008 and higher guests ? Thanks, Bjoern
Urgent : Network stuck in implementing state
Hi guys, I had trouble while I was creating a new VLAN and now I can't delete it anymore, because it's stuck in implementing state. I guess that happened after the VR did not came up and I restarted the management server so I could delete the VR and all addresses. If I would spin up a guest, the VR and the guest would come up but I sill want to get rid of it since I have an error in the network service offering. After I cleaned all up though the GUI I can only see an external IP for the old VR is still associated with the VLAN but can't be deleted (I don't get it offered) In the logs I found this and the NPE matches the table content : 2014-01-26 13:53:02,857 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) Network id=212 is destroyed successfully, cleaning up corresponding resources now. 2014-01-26 13:53:02,871 DEBUG [network.guru.DirectNetworkGuru] (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) Releasing ip 10.16.48.1 of placeholder nic Nic[189-null-null-10.16.48.1] 2014-01-26 13:53:02,872 DEBUG [db.Transaction.Transaction] (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) Rolling back the transaction: Time = 14 Name = -AsyncJobManagerImpl$1.run:494-Exec utors$RunnableAdapter.call:471-FutureTask.run:262-ThreadPoolExecutor.runWorker:1145-ThreadPoolExecutor$Worker.run:615-Thread.run:744; called by -Transaction.rollback:897-Transaction.removeUpTo:840-Transaction.cl ose:664-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-NetworkManagerImpl.destroyNetwork:3144-ComponentInstantiationPostProcessor$Intercep torDispatcher.intercept:125-NetworkServiceImpl.deleteNetwork:1767-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:125-DeleteNetworkCmd.execute:70-ApiDispatcher.dispatch:158-AsyncJobManagerImp l$1.run:531 2014-01-26 13:53:02,877 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) Unexpected exception while executing org.apache.cloudstack.api.command.user.ne twork.DeleteNetworkCmd java.lang.NullPointerException at com.cloud.network.guru.DirectNetworkGuru.trash(DirectNetworkGuru.java:311) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.network.NetworkManagerImpl.destroyNetwork(NetworkManagerImpl.java:3144) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.network.NetworkServiceImpl.deleteNetwork(NetworkServiceImpl.java:1767) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd.execute(DeleteNetworkCmd.java:70) at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2014-01-26 13:53:02,880 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) Complete async job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: null 2014-01-26 13:53:02,888 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) Sync queue (6) is currently empty 2014-01-26 13:53:02,889 WARN [cloud.async.AsyncJobManagerImpl] (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) Unable to unregister active job [ 1127 ] = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24 020 ] from JMX monitoring mysql> select * from nics where id = 189\G *** 1. row *** id: 189 uuid: 623d0af3-7ca5-4700-89a2-26252abdc054 instance_id: NULL mac_address: NULL ip4_address: 10.16.48.1 netmask: NULL gateway: NULL ip_type: NULL broadcast_uri: NULL network_id: 212 mode: NULL state: Reserved strategy: PlaceHolder reserver_name: NULL reservation_id: NULL device_id: 0 update_time: 2014-01-24 16:22:56 isolation_uri: NULL ip6_address: NULL default_nic: 0 vm_type: DomainRouter created: 2014-01-25 00:22:56 removed: NULL ip6_gateway: NULL ip6_cidr: NULL secondary_ip: 0 display_nic: 1 1
Re: Urgent : Network stuck in implementing state
Thanks, I got one step further and the NPE is not coming up but I guess I need to change the overall network state :? 014-01-30 15:23:16,159 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-67:job-1218 = [ 7de934e0-2b42-476b-83da-cf03109331fa ]) Network is not implemented: Ntwk[212|Guest|16] 2014-01-30 15:23:16,160 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-67:job-1218 = [ 7de934e0-2b42-476b-83da-cf03109331fa ]) Network is not not in the correct state to be destroyed: Implementing 2014-01-30 15:23:16,164 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-67:job-1218 = [ 7de934e0-2b42-476b-83da-cf03109331fa ]) Complete async job-1218 = [ 7de934e0-2b42-476b-83da-cf03109331fa ], jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to delete network On Mon, Jan 27, 2014 at 10:57 PM, Sanjeev Neelarapu < sanjeev.neelar...@citrix.com> wrote: > Hi Bjoern, > > Since the nic's instance_id is NULL you can set the removed field with > some timestamp (e.g. now()). Also check the external IP address state in > user_ip_address table. If this ip address state is in allocated set it to > NULL and try delete network again. > > -Sanjeev > > -Original Message- > From: Bjoern Teipel [mailto:bjoern.tei...@gmail.com] > Sent: Monday, January 27, 2014 12:44 PM > To: users; d...@cloudstack.apache.org > Subject: Urgent : Network stuck in implementing state > > Hi guys, > > I had trouble while I was creating a new VLAN and now I can't delete it > anymore, because it's stuck in implementing state. > I guess that happened after the VR did not came up and I restarted the > management server so I could delete the VR and all addresses. > > If I would spin up a guest, the VR and the guest would come up but I sill > want to get rid of it since I have an error in the network service offering. > After I cleaned all up though the GUI I can only see an external IP for > the old VR is still associated with the VLAN but can't be deleted (I don't > get it offered) > > In the logs I found this and the NPE matches the table content : > > > > 2014-01-26 13:53:02,857 DEBUG [cloud.network.NetworkManagerImpl] > (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) > Network id=212 is destroyed successfully, cleaning up corresponding > resources now. > 2014-01-26 13:53:02,871 DEBUG [network.guru.DirectNetworkGuru] > (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) > Releasing ip 10.16.48.1 of placeholder nic Nic[189-null-null-10.16.48.1] > 2014-01-26 13:53:02,872 DEBUG [db.Transaction.Transaction] > (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) > Rolling back the transaction: Time = 14 Name = > -AsyncJobManagerImpl$1.run:494-Exec > > utors$RunnableAdapter.call:471-FutureTask.run:262-ThreadPoolExecutor.runWorker:1145-ThreadPoolExecutor$Worker.run:615-Thread.run:744; > called by > -Transaction.rollback:897-Transaction.removeUpTo:840-Transaction.cl > > ose:664-TransactionContextBuilder.interceptException:63-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:133-NetworkManagerImpl.destroyNetwork:3144-ComponentInstantiationPostProcessor$Intercep > > torDispatcher.intercept:125-NetworkServiceImpl.deleteNetwork:1767-ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept:125-DeleteNetworkCmd.execute:70-ApiDispatcher.dispatch:158-AsyncJobManagerImp > l$1.run:531 > 2014-01-26 13:53:02,877 ERROR [cloud.async.AsyncJobManagerImpl] > (Job-Executor-2:job-1127 = [ f90b84ab-0d4a-45ce-9cd1-d73b63e24020 ]) > Unexpected exception while executing > org.apache.cloudstack.api.command.user.ne > twork.DeleteNetworkCmd > java.lang.NullPointerException > at > com.cloud.network.guru.DirectNetworkGuru.trash(DirectNetworkGuru.java:311) > at > > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > > com.cloud.network.NetworkManagerImpl.destroyNetwork(NetworkManagerImpl.java:3144) > at > > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > > com.cloud.network.NetworkServiceImpl.deleteNetwork(NetworkServiceImpl.java:1767) > at > > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > > org.apache.cloudstack.api.command.user.network.DeleteNetworkCmd.execute(DeleteNetworkCmd.java:70) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.ja
KVM storage questions
Hi admins, do you want to share what kind of storage you use in connection for CSTK 4.2.1 ? I guess most of you is using NFS but myself is forced to use CLVM since I have a iscsi flash based storage system available. This is also the reason why I contact you, I personally find CLVM clunky, error prone and dead lock heavy. Who is using CLVM in production and what kind of cluster config do you use ? I'm still trying to find out if I don't switch to NFS to avoid all upcoming CLVM and ldm locking issues. Maybe some of you are using OCFS, which I used few years ago with limited success. Thanks Bjoern
Re: Help with SSVM routes
If you can't ping 10.0.0.1, the internal network then most likely the physical network connection or the bridge/vswitch has an issue on the hypervisor. Wrong traffic label assigned or if you use VLANs then port mode or tagging could be wrong. What hypervisor do you use anyway ? Bjoern On Thu, Mar 27, 2014 at 10:28 AM, Matthew Midgett wrote: > Yes the whole 10.0.0.0/16 is behind our firewall and has Internet access. > > > > On 03/27/2014 01:22 PM, Erik Weber wrote: >> >> Does your management network have internet access? CS routes internal dns >> over mgmt network and if your mgmt network doesnt have internet access >> you're basically without dns if you specify the same dns servers (Looks >> like you did). >> >> Erik >> 27. mars 2014 17:57 skrev "Matthew Midgett" >> følgende: >> >>> I am having trouble getting ssvm to connect to the management lan over >>> the >>> private ip. Also I can't get it connect to the outside world. I can ping >>> the public gateway from ssvm and I can ping the public ip of the ssvm >>> from >>> an external address. Something I noticed is that I can't ping anything on >>> the management lan from the ssvm. >>> >>> BTW my primary storage is on fiber so its really not across the >>> management >>> lan. My secondary storage is on the management lan. >>> >>> Management lan / Storage Network >>> >>> 10.0.0.0/16 >>> >>> Guest Lan >>> >>> 10.1.0.0/16 >>> >>> Public >>> 216.249.111.0/24 >>> >>> root@s-34191-VM:~# /usr/local/cloud/systemvm/./ssvm-check.sh >>> >>> First DNS server is 8.8.8.8 >>> PING 8.8.8.8 (8.8.8.8): 56 data bytes >>> --- 8.8.8.8 ping statistics --- >>> 2 packets transmitted, 0 packets received, 100% packet loss >>> WARNING: cannot ping DNS server >>> route follows >>> Kernel IP routing table >>> Destination Gateway Genmask Flags Metric Ref Use >>> Iface >>> 0.0.0.0 216.249.111.1 0.0.0.0 UG0 00 eth2 >>> 8.8.4.4 10.0.0.1255.255.255.255 UGH 0 00 eth1 >>> 8.8.8.8 10.0.0.1255.255.255.255 UGH 0 00 eth1 >>> 10.0.0.010.0.0.1255.255.255.0 UG0 00 eth1 >>> 10.0.0.00.0.0.0 255.255.0.0 U 0 00 eth1 >>> 10.0.0.00.0.0.0 255.255.0.0 U 0 00 eth3 >>> 169.254.0.0 0.0.0.0 255.255.0.0 U 0 00 eth0 >>> 216.249.111.0 0.0.0.0 255.255.255.0 U 0 00 eth2 >>> >>> ERROR: DNS not resolving download.cloud.com >>> resolv.conf follows >>> nameserver 8.8.8.8 >>> nameserver 8.8.4.4 >>> nameserver 8.8.8.8 >>> nameserver 8.8.4.4 >>> >>> root@s-34191-VM:~# ifconfig >>> eth0 Link encap:Ethernet HWaddr 0e:00:a9:fe:00:86 >>>inet addr:169.254.0.134 Bcast:169.254.255.255 >>> Mask:255.255.0.0 >>>inet6 addr: fe80::c00:a9ff:fefe:86/64 Scope:Link >>>UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>RX packets:997 errors:0 dropped:0 overruns:0 frame:0 >>>TX packets:763 errors:0 dropped:0 overruns:0 carrier:0 >>>collisions:0 txqueuelen:1000 >>>RX bytes:86924 (84.8 KiB) TX bytes:103694 (101.2 KiB) >>> >>> eth1 Link encap:Ethernet HWaddr 06:d8:72:00:5a:34 >>>inet addr:10.0.90.53 Bcast:10.0.255.255 Mask:255.255.0.0 >>>inet6 addr: fe80::4d8:72ff:fe00:5a34/64 Scope:Link >>>UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>RX packets:3174 errors:0 dropped:0 overruns:0 frame:0 >>>TX packets:698 errors:0 dropped:0 overruns:0 carrier:0 >>>collisions:0 txqueuelen:1000 >>>RX bytes:223025 (217.7 KiB) TX bytes:49040 (47.8 KiB) >>> >>> eth2 Link encap:Ethernet HWaddr 06:59:b4:00:fe:fe >>>inet addr:216.249.111.2 Bcast:216.249.111.255 >>> Mask:255.255.255.0 >>>inet6 addr: fe80::459:b4ff:fe00:fefe/64 Scope:Link >>>UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>RX packets:9223 errors:0 dropped:10 overruns:0 frame:0 >>>TX packets:288 errors:0 dropped:0 overruns:0 carrier:0 >>>collisions:0 txqueuelen:1000 >>>RX bytes:614771 (600.3 KiB) TX bytes:25160 (24.5 KiB) >>> >>> eth3 Link encap:Ethernet HWaddr 06:96:be:00:18:d2 >>>inet addr:10.0.24.211 Bcast:10.0.255.255 Mask:255.255.0.0 >>>inet6 addr: fe80::496:beff:fe00:18d2/64 Scope:Link >>>UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>RX packets:3051 errors:0 dropped:0 overruns:0 frame:0 >>>TX packets:3 errors:0 dropped:0 overruns:0 carrier:0 >>>collisions:0 txqueuelen:1000 >>>RX bytes:215627 (210.5 KiB) TX bytes:318 (318.0 B) >>> >>> loLink encap:Local Loopback >>>inet addr:127.0.0.1 Mask:255.0.0.0 >>>inet6 addr: ::1/128 Scope:Host >>>
