Donal - nice blog. One more thing you might want to add is that for these shell scripts you can uncomment the set -x option in the scripts and run them yourself. SmLog sometimes might not narrow down the issue but has the info about the script path and the arguments. Once you uncomment the set -x and run it yourself you can see the verbose output of the run. Or better just uncomment the set -x option in the script and rerun the entire operation. You should see a verbose output :)
Thanks, -Nitin On 23/08/13 2:56 PM, "Amit Das" <amit....@cloudbyte.com> wrote: >Thanks Donal for the site. It explains the issue pretty well. > >I had already gone through the SMLog by observing some already raised jira >tickets. However, did not understand much then. > >Now i can see some *difference *between the logs on your blog vs. the ones >in my XenServer. > >[28437] 2013-08-23 07:37:28.703267 #### VMOPS enter > copy_vhd_from_secondarystorage #### >[28437] 2013-08-23 07:37:28.703390 ['bash', '/opt/xensource/bin/* >copy_vhd_from_secondarystorage.sh*', '10.10.171.155:* >/export/secondary/template/tmpl/1/1/*', >'b45c8c85-4e55-983c-cb16-4c2f2356951c', >'cloud-ebed83d7-afde-4470-b867-312bad7c21c7'] >[28437] 2013-08-23 07:37:28.826465 SUCCESS >[28437] 2013-08-23 07:37:28.826691 #### VMOPS exit > copy_vhd_from_secondarystorage #### >[28463] 2013-08-23 07:37:29.787555 #### VMOPS enter >kill_copy_process >#### >[28463] 2013-08-23 07:37:29.787680 ['bash', >'/opt/xensource/bin/kill_copy_process.sh', ''] >[28463] 2013-08-23 07:37:29.800439 SUCCESS >[28463] 2013-08-23 07:37:29.800565 #### VMOPS exit kill_copy_process >#### > >The difference that I find here is the .vhd template is never mentioned in >the copy_vhd_from_secondarystorage.sh >Not sure if logic has changed. > > >Regards, >Amit >*CloudByte Inc.* <http://www.cloudbyte.com/> > > >On Fri, Aug 23, 2013 at 1:59 PM, Donal Lafferty ><donal.laffe...@citrix.com>wrote: > >> HI Amit, >> >> Still having problems? >> >> Probably time to have a look at the logs on XenServer. There's a guide >> >>http://dlafferty.blogspot.co.uk/2013/08/using-cloudstacks-log-files-xense >>rver.html >> >> DL >> >> > -----Original Message----- >> > From: Amit Das [mailto:amit....@cloudbyte.com] >> > Sent: 23 August 2013 05:52 >> > To: dev@cloudstack.apache.org; aemne...@gmail.com >> > Subject: Re: System VMs not running >> > >> > Yes. The step was missed. >> > >> > However, i tried out the steps on 'vhd_util' later & restarted nfs, >> rpcbind & >> > management services. >> > >> > Do i need to reset the DB as well ? >> > >> > Is there any work around for this ? >> > >> > Regards, >> > Amit >> > *CloudByte Inc.* <http://www.cloudbyte.com/> >> > >> > >> > On Fri, Aug 23, 2013 at 9:56 AM, Ahmad Emneina <aemne...@gmail.com> >> > wrote: >> > >> > > this usually indicates that you missed the step where you prep >> > > vhd-util on the hypervisor. >> > > >> > > >> > > On Thu, Aug 22, 2013 at 9:18 PM, Amit Das <amit....@cloudbyte.com> >> > wrote: >> > > >> > > > The step worked fine after seeding system VM into secondary >>storage >> > > > location. >> > > > >> > > > However, i see the next error: >> > > > >> > > > WARN [xen.resource.CitrixResourceBase] (DirectAgent-446:) Catch >> > > Exception >> > > > com.cloud.utils.exception.CloudRuntimeException on >> > > > host:21318624-cfd9-412d-aeb4-b1418424f287 for template: nfs:// >> > > > 10.10.171.155/export/secondary/template/tmpl/1/1/ due to >> > > > com.cloud.utils.exception.CloudRuntimeException: *can not create >>vdi >> > > > in >> > > sr >> > > > * >> > > > b45c8c85-4e55-983c-cb16-4c2f2356951c >> > > > com.cloud.utils.exception.CloudRuntimeException: can not create >>vdi >> > > > in sr b45c8c85-4e55-983c-cb16-4c2f2356951c >> > > > at >> > > > >> > > > >> > > >> > com.cloud.hypervisor.xen.resource.CitrixResourceBase.copy_vhd_from_sec >> > > ondarystorage(CitrixResourceBase.java:2707) >> > > > >> > > > ... >> > > > >> > > > 2013-08-23 09:46:12,815 DEBUG [agent.transport.Request] >> > > (secstorage-1:null) >> > > > Seq 12-90517578: Sending { Cmd , MgmtId: 268749071320359, via: >>12, >> Ver: >> > > > v1, Flags: 100111, [{"storage.*PrimaryStorageDownloadCommand* >> > > > >> > > > >> > > >>":{"localPath":"/mnt/15370e0e-c8ae-32bb-94e1-74f1fad0cce3","poolUuid": >> > > >>"15370e0e-c8ae-32bb-94e1-74f1fad0cce3","poolId":211,"primaryPool":{"id >> > > >>":211,"uuid":"15370e0e-c8ae-32bb-94e1-74f1fad0cce3","host":"10.10.171. >> > > >>155","path":"/export/primary","port":2049,"type":"NetworkFilesystem"}, >> > > "secondaryStorageUrl":"nfs:// >> > > > 10.10.171.155/export/secondary","primaryStorageUrl":"nfs:// >> > > > 10.10.171.155/export/primary","url":"nfs:// >> > > > 10.10.171.155/export/secondary/template/tmpl/1/1/ >> > > > ","format":"VHD","accountId":1,"name":"routing-1","wait":10800}}] >> > > > } >> > > > >> > > > ... >> > > > >> > > > 2013-08-23 09:46:15,077 DEBUG [agent.transport.Request] >> > > > (DirectAgent-72:null) Seq 12-90517578: Processing: { Ans: , >>MgmtId: >> > > > 268749071320359, via: 12, Ver: v1, Flags: 110, [{"storage.* >> > > > >> > > > >> > > >>PrimaryStorageDownloadAnswer*":{"templateSize":0,"result":false,"detai >> > > ls":"Catch >> > > > Exception com.cloud.utils.exception.CloudRuntimeException on >> > > > host:21318624-cfd9-412d-aeb4-b1418424f287 for template: nfs:// >> > > > 10.10.171.155/export/secondary/template/tmpl/1/1/ due to >> > > > com.cloud.utils.exception.CloudRuntimeException: can not create >>vdi >> > > > in sr b45c8c85-4e55-983c-cb16-4c2f2356951c","wait":0}}] } >> > > > >> > > > Have i missed any other step for configuring primary & secondary >> > storage. >> > > > >> > > > Regards, >> > > > Amit >> > > > *CloudByte Inc.* <http://www.cloudbyte.com/> >> > > > >> > > > >> > > > On Thu, Aug 22, 2013 at 5:24 PM, Harikrishna Patnala < >> > > > harikrishna.patn...@citrix.com> wrote: >> > > > >> > > > > Please follow the link to seed the system template >> > > > > >> > > > >> > > http://cloudstack.apache.org/docs/en- >> > US/Apache_CloudStack/4.1.1/html/I >> > > >>nstallation_Guide/management-server-install-flow.html#prepare-system- >> > v >> > > m-template >> > > > > >> > > > > -Harikrishna >> > > > > On 22-Aug-2013, at 5:08 PM, Amit Das <amit....@cloudbyte.com> >> > > > > wrote: >> > > > > >> > > > > > Hi, >> > > > > > >> > > > > > I have not seeded the secondary storage with system vm >>template. >> > > > > > >> > > > > > The directory ../template/tmpl/1/1 is not present. >> > > > > > >> > > > > > >> > > > > > Regards, >> > > > > > Amit >> > > > > > *CloudByte Inc.* <http://www.cloudbyte.com/> >> > > > > > >> > > > > > >> > > > > > On Thu, Aug 22, 2013 at 4:58 PM, Devdeep Singh < >> > > > devdeep.si...@citrix.com >> > > > > >wrote: >> > > > > > >> > > > > >> Did you seed the secondary storage with the systemvm template >> > > > > >> before starting the setup? If not, can you check if under >>your >> > > > > >> secondary >> > > > > storage >> > > > > >> the ../template/tmpl/1/1 directory is present? >> > > > > >> >> > > > > >> Regards, >> > > > > >> Devdeep >> > > > > >> >> > > > > >>> -----Original Message----- >> > > > > >>> From: Amit Das [mailto:amit....@cloudbyte.com] >> > > > > >>> Sent: Thursday, August 22, 2013 4:50 PM >> > > > > >>> To: dev@cloudstack.apache.org >> > > > > >>> Subject: System VMs not running >> > > > > >>> >> > > > > >>> Hi folks, >> > > > > >>> >> > > > > >>> I am running CloudStack 4.1.1. & running the NFS share on >>the >> > > > > >>> same >> > > > box >> > > > > as >> > > > > >>> CloudStack management server (CentOS 6). The hypervisor is a >> > > separate >> > > > > Xen >> > > > > >>> server. >> > > > > >>> >> > > > > >>> I am not able to get the system vms in a running state. >> > > > > >>> We had not downloaded the vhd-util initially but it has been >> > > > downloaded >> > > > > >> now. >> > > > > >>> The management server has been restarted & appropriate >> > > > > >>> rpcbind, nfs services started. The configuration was redone >> from >> > scratch. >> > > However, >> > > > > we >> > > > > >> are >> > > > > >>> stuck with below error always. >> > > > > >>> >> > > > > >>> com.cloud.utils.exception.CloudRuntimeException: can't mount >> > > > > >>> 10.10.171.155:/export/secondary/template/tmpl/1/1 >> > > > > >>> to /var/run/cloud_mount/2cc7d1a3-130b-4ae9-9381-5f8ec6bf751a >> > > > at >> > > > > >>> com.cloud.hypervisor.xen.resource.CitrixResourceBase. >> > > > > >>> copy_vhd_from_secondarystorage(CitrixResourceBase.java:2707) >> > > > > >>> >> > > > > >>> .. >> > > > > >>> 2013-08-22 16:30:52,866 DEBUG [agent.transport.Request] >> > > (secstorage- >> > > > > >>> 1:null) Seq 12-749863191: Executing: { Cmd , MgmtId: >> > > > 268749071320359, >> > > > > >>> via: 12, >> > > > > >>> Ver: v1, Flags: 100111, >> > > > > >>> [{"storage.*PrimaryStorageDownloadCommand* >> > > > > >>> ":{"localPath":"/mnt/15370e0e-c8ae-32bb-94e1- >> > > > > >>> 74f1fad0cce3","poolUuid":"15370e0e-c8ae-32bb-94e1- >> > > > > >>> >> > > > > >> > > >>74f1fad0cce3","poolId":211,"primaryPool":{"id":211,"uuid":"15370e0e-c8 >> > > ae- >> > > > > >>> 32bb-94e1- >> > > > > >>> >> > > > > >> >> > > > > >> > > > >> > > >>74f1fad0cce3","host":"10.10.171.155","path":"/export/primary","port":2 >> > > 049, >> > > > > >>> "type":"NetworkFilesystem"},"secondaryStorageUrl":"nfs:// >> > > > > >>> 10.10.171.155/export/secondary","primaryStorageUrl":"nfs:// >> > > > > >>> 10.10.171.155/export/primary","url":"nfs:// >> > > > > >>> 10.10.171.155/export/secondary/template/tmpl/1/1/ >> > > > > ","format":"VHD","acco >> > > > > >>> untId":1,"name":"routing-1","wait":10800}}] >> > > > > >>> } >> > > > > >>> >> > > > > >>> .. >> > > > > >>> 2013-08-22 16:30:55,128 DEBUG [agent.transport.Request] >> > > > > >>> (DirectAgent-244:null) Seq 12-749863191: Processing: { >>Ans: , >> > > > MgmtId: >> > > > > >>> 268749071320359, via: 12, Ver: v1, Flags: 110, [{"storage.* >> > > > > >>> >> > > > > >> >> > > > > >> > > > >> > > >> > >>PrimaryStorageDownloadAnswer*":{"templateSize":0,"result":false,"details" >> > :" >> > > > > >>> Catch >> > > > > >>> Exception com.cloud.utils.exception.CloudRuntimeException on >> > > > > >>> host:21318624-cfd9-412d-aeb4-b1418424f287 for template: >>nfs:// >> > > > > >>> 10.10.171.155/export/secondary/template/tmpl/1/1/ due to >> > > > > >>> com.cloud.utils.exception.CloudRuntimeException: can't mount >> > > > > >>> 10.10.171.155:/export/secondary/template/tmpl/1/1 >> > > > > >>> to /var/run/cloud_mount/60e04e29-8064-4a3f-90c2- >> > > > > >>> b1e639b9b05a","wait":0}}] } >> > > > > >>> >> > > > > >>> Regards, >> > > > > >>> Amit >> > > > > >>> *CloudByte Inc.* <http://www.cloudbyte.com/> >> > > > > >> >> > > > > >> > > > > >> > > > >> > > >>
