Check the ACL on your NFS share – remember your SSVM is using an IP address in the management range of your zone to access NFS, so we sometimes see issues where hypervisors are on the ACL, but not the management IP range.
Regards, Dag Sonstebo Cloud Architect ShapeBlue On 09/02/2018, 11:15, "Swastik Mittal" <mittal.swas...@gmail.com> wrote: Hey, I checked accessing nfs server from my ssvm. I tried the following root@s-1-VM:~# rpcinfo -p program vers proto port service 100000 4 tcp 111 portmapper 100000 3 tcp 111 portmapper 100000 2 tcp 111 portmapper 100000 4 udp 111 portmapper 100000 3 udp 111 portmapper 100000 2 udp 111 portmapper 100024 1 udp 58883 status 100024 1 tcp 48454 status I have only portmapper running. Thats it. No nfs ports found. root@s-1-VM:~# mount -t nfs above gives no result. I manually mounted ssvm to /export/secondary of secondary storage nfs server. But again all the above commands give the same result. root@s-1-VM:~# df -k Filesystem 1K-blocks Used Available Use% Mounted on rootfs 472036 160666 286999 36% / udev 10240 0 10240 0% /dev tmpfs 50896 256 50640 1% /run /dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d 472036 160666 286999 36% / tmpfs 5120 0 5120 0% /run/lock tmpfs 211960 0 211960 0% /run/shm /dev/vda1 74367 22706 47821 33% /boot /dev/vda6 93207 5654 82741 7% /home /dev/vda8 188403 6091 172584 4% /opt /dev/vda11 93207 5679 82716 7% /tmp /dev/vda7 768016 565696 163308 78% /usr /dev/vda9 575500 235564 310704 44% /var /dev/vda10 188403 10441 168234 6% /var/log 10.1.0.77:/export/secondary 708019200 12345344 659686400 2% /mnt/SecStorage/15b578c2-972f-3178-ac74-acc61e05ec25 10.1.0.77:/export/secondary 708019200 12345344 659686400 2% /mnt/SecStorage/15b578c2-972f-3178-ac74-acc61e05ec25 root@s-1-VM:~# (10.1.0.77 - management server and also secondary and primary storage server) I checked cloud.out and it states "no credentials for host found". I cannot ssh into management from ssvm. I tried adding management server (also my storage server) in known_host of ssvm but it din't work. @Dag I guess you were right about SSVM not able to access Secondary Storage. I tried mounting SSVM manually to Secondary storage server but it din't work. Any idea? cheers Swastik On Fri, Feb 9, 2018 at 11:56 AM, Swastik Mittal <mittal.swas...@gmail.com> wrote: > Hey, > > I took a lookat my NFS storage. > > root@MY-PC:~# mount -t nfs > 10.1.0.77:/export/primary on /mnt/9ebaae44-dc8f-33d5-8fb6-90c193e69a22 > type nfs (rw,vers=4,addr=10.1.0.77,clientaddr=10.1.0.77) > 10.1.0.77:/export/secondary on /mnt/secondary type nfs > (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,addr=10.1.0.77) > 10.1.0.77:/export/primary on /mnt/primary type nfs > (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,addr=10.1.0.77) > root@MY-PC:~# df -h > Filesystem Size Used Avail Use% Mounted on > /dev/sda4 676G 12G 630G 2% / > none 4.0K 0 4.0K 0% /sys/fs/cgroup > udev 7.8G 4.0K 7.8G 1% /dev > tmpfs 1.6G 1.9M 1.6G 1% /run > none 5.0M 0 5.0M 0% /run/lock > none 7.9G 208K 7.9G 1% /run/shm > none 100M 48K 100M 1% /run/user > 10.1.0.77:/export/primary 676G 12G 630G 2% > /mnt/9ebaae44-dc8f-33d5-8fb6-90c193e69a22 > 10.1.0.77:/export/primary 676G 12G 630G 2% /mnt/primary > > My secondary storage is mounted as /mnt/secondary but it does not show as > the diskspace available like the primary memory. > > Cheers > Swastik > > > On Fri, Feb 9, 2018 at 9:23 AM, Swastik Mittal <mittal.swas...@gmail.com> > wrote: > >> Hey Dag, >> >> On running ssvm-check I got : >> >> root@s-1-VM:/usr/local/cloud/systemvm# ./ssvm-check.sh >> ================================================ >> First DNS server is 8.8.8.8 >> PING 8.8.8.8 (8.8.8.8): 48 data bytes >> --- 8.8.8.8 ping statistics --- >> 2 packets transmitted, 0 packets received, 100% packet loss >> WARNING: cannot ping DNS server >> route follows >> Kernel IP routing table >> Destination Gateway Genmask Flags Metric Ref Use >> Iface >> 0.0.0.0 10.1.0.2 0.0.0.0 UG 0 0 0 >> eth2 >> 8.8.8.8 10.1.0.2 255.255.255.255 UGH 0 0 0 >> eth1 >> 10.1.0.0 0.0.0.0 255.255.255.0 U 0 0 0 >> eth1 >> 10.1.0.0 0.0.0.0 255.255.255.0 U 0 0 0 >> eth2 >> 169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 >> eth0 >> 192.168.1.1 10.1.0.2 255.255.255.255 UGH 0 0 0 >> eth1 >> ================================================ >> Good: DNS resolves download.cloud.com >> ================================================ >> nfs is currently mounted >> Mount point is /mnt/SecStorage/15b578c2-972f-3178-ac74-acc61e05ec25 >> Good: Can write to mount point >> ================================================ >> Management server is 10.1.0.77. Checking connectivity. >> Good: Can connect to management server port 8250 >> ================================================ >> Good: Java process is running >> ================================================ >> Tests Complete. Look for ERROR or WARNING above. >> root@s-1-VM:/usr/local/cloud/systemvm# >> >> It doesn't show any connection to secondary storage though. I has >> connection to management server and my secondary storage currently is on >> the same server. >> >> On Thu, Feb 8, 2018 at 10:44 PM, Dag Sonstebo <dag.sonst...@shapeblue.com >> > wrote: >> >>> Swastik, >>> >>> Are you confident your SSVM can write to your secondary storage? >>> >>> Can you try to run /usr/local/cloud/systemvm/ssvm-check.sh and see what >>> this comes back with? >>> >>> Regards, >>> Dag Sonstebo >>> Cloud Architect >>> ShapeBlue >>> >>> On 08/02/2018, 11:06, "Swastik Mittal" <mittal.swas...@gmail.com> wrote: >>> >>> Hey Dag, >>> >>> As I start the downloading of iso I get a broken pipe error on my >>> terminal >>> logging http local file server details. But I checked the cloud.log >>> and it >>> showed processing and downloadStatus="NOT_DOWNLOADED". >>> Downloadpath="/mnt/SecStorage" but NO ERROR untill after waiting for >>> a >>> while I myself turned off the http server. Till that time even the >>> iso >>> status din't show anything as no error string was generated and when >>> I >>> turned off the server it displayed partial http get request not >>> supported. >>> I don't think downloading from a local server would take time. I did >>> allow >>> it around 20 minutes. I think that broken pipe is connection to http >>> server >>> being suddenly stopped due to something. Not able to figure out >>> how:). >>> >>> The download path shows /mnt/SecStorage which is in SSVM. >>> >>> >>> Regards >>> Swastik >>> >>> On Thu, Feb 8, 2018 at 2:59 PM, Dag Sonstebo < >>> dag.sonst...@shapeblue.com> >>> wrote: >>> >>> > Hi Swastik, >>> > >>> > Don’t worry about this – you were just doing a test where you were >>> trying >>> > to wget to the local SSVM filesystem as a test. The SSVM is just a >>> small VM >>> > with not a lot of disk space – so you are just checking >>> connectivity here, >>> > you should delete the partially downloaded file thereafter. >>> > >>> > When you do the template download from CloudStack the SSVM will >>> write >>> > straight to secondary storage, not to it’s own filesystem. >>> > >>> > Regards, >>> > Dag Sonstebo >>> > Cloud Architect >>> > ShapeBlue >>> > >>> > On 08/02/2018, 08:41, "Swastik Mittal" <mittal.swas...@gmail.com> >>> wrote: >>> > >>> > Hey >>> > >>> > A correction, I do see secondary storage and it shows only 1% >>> utilized >>> > so >>> > my secondary storage does have enough memory.The issue seems >>> to be >>> > mostly >>> > because of SSVM memory. >>> > >>> > Regards >>> > Swastik >>> > >>> > On Thu, Feb 8, 2018 at 1:20 PM, Swastik Mittal < >>> > mittal.swas...@gmail.com> >>> > wrote: >>> > >>> > > Hey Dag and Glenn, >>> > > >>> > > On entering the local http server url, UI gives status that >>> only >>> > ports 80, >>> > > 8080 and 443 are supported . >>> > > So as my management server runs on 8080 and 443 would be for >>> ssl so I >>> > > hosted my iso on port 80. >>> > > On logging into my ssvm and using wget to download it >>> downloads upto >>> > 35% >>> > > and then stops it displaying not enough space on my >>> terminal. So >>> > definetly >>> > > now there is no issue of credentials. And the broken pipe >>> error on >>> > > intiating download from local http from UI might be becuase >>> I am >>> > running >>> > > short on space in my SSVM. >>> > > >>> > > But isn't the iso stored in secondary storage which is the >>> management >>> > > server itself? My secondary storage is up as shown in >>> infrastructure >>> > and >>> > > even my system vm template was downloaded in /mnt/secondary. >>> But my >>> > > dashboard does not show secondary storage infact shows >>> primary >>> > storage >>> > > having 1 TB of memory which is the complete memory of the >>> system. Do >>> > I need >>> > > to increase the SSVM storage and if yes how do I do that? >>> > > >>> > > Regards >>> > > Swastik >>> > > >>> > > On Wed, Feb 7, 2018 at 8:43 PM, Swastik Mittal < >>> > mittal.swas...@gmail.com> >>> > > wrote: >>> > > >>> > >> Hey, >>> > >> >>> > >> Thanks Dag and Glen for reply. >>> > >> >>> > >> I'll try changing the port and check. >>> > >> >>> > >> Regards >>> > >> Swastik >>> > >> >>> > >> >>> > >> <https://mailtrack.io/> Sent with Mailtrack >>> > >> <https://chrome.google.com/webstore/detail/mailtrack-for- >>> > gmail-inbox/ndnaehgpjlnokgebbaldlmgkapkpjkkb?utm_source=gmail&utm_ >>> > medium=signature&utm_campaign=signaturevirality> >>> > >> >>> > >> On Wed, Feb 7, 2018 at 5:57 PM, Glenn Wagner < >>> > glenn.wag...@shapeblue.com> >>> > >> wrote: >>> > >> >>> > >>> Hi >>> > >>> >>> > >>> As Dag pointed out (the errors) when using the simpleHTTP >>> server >>> > its >>> > >>> better to use either port 8080 or 8000 so you don't have >>> to setup >>> > SSL on >>> > >>> port 443 >>> > >>> >>> > >>> Example >>> > >>> python -m SimpleHTTPServer 8080 >>> > >>> >>> > >>> Now try the wget or Curl from the SSVM to download the ISO >>> use >>> > http not >>> > >>> https >>> > >>> >>> > >>> Regards >>> > >>> Glenn >>> > >>> >>> > >>> >>> > >>> glenn.wag...@shapeblue.com >>> > >>> www.shapeblue.com >>> > >>> Winter Suite, 1st Floor, The Avenues, Drama Street, >>> Somerset West, >>> > Cape >>> > >>> Town 7129South Africa >>> > >>> @shapeblue >>> > >>> >>> > >>> >>> > >>> >>> > >>> >>> > >>> -----Original Message----- >>> > >>> From: Dag Sonstebo [mailto:dag.sonst...@shapeblue.com] >>> > >>> Sent: Wednesday, 07 February 2018 2:13 PM >>> > >>> To: users@cloudstack.apache.org >>> > >>> Subject: Re: iso not downloading from local path >>> > >>> >>> > >>> Hi Swastik, >>> > >>> >>> > >>> Moving discussion to this thread. >>> > >>> >>> > >>> The simple test here is to log in to the SSVM from console >>> or over >>> > ssh >>> > >>> and do a “wget” or “curl” against your download URL and >>> see if >>> > this starts. >>> > >>> >>> > >>> I think your issue is with either your simpleHTTP server >>> asking for >>> > >>> credentials, or your proxy configuration asking for >>> credentials – >>> > “No >>> > >>> credentials configured for host <local file server ip>” >>> seems >>> > fairly >>> > >>> conclusive. >>> > >>> >>> > >>> >>> > >>> Regards, >>> > >>> Dag Sonstebo >>> > >>> Cloud Architect >>> > >>> ShapeBlue >>> > >>> >>> > >>> On 07/02/2018, 06:36, "Swastik Mittal" < >>> mittal.swas...@gmail.com> >>> > wrote: >>> > >>> >>> > >>> I checked my cloud.log in my SSVM. After the URL >>> request it >>> > mentions >>> > >>> >>> > >>> [storage.template.HttpTemplateDownloader] >>> > >>> (agentRequest-Handler-3:null) No >>> > >>> credentials configured for host <local file server ip> >>> > >>> >>> > >>> On Wed, Feb 7, 2018 at 11:32 AM, Swastik Mittal < >>> > >>> mittal.swas...@gmail.com> >>> > >>> wrote: >>> > >>> >>> > >>> > Hey, >>> > >>> > >>> > >>> > I do not have internet on management server and host >>> so to >>> > upload >>> > >>> an iso I >>> > >>> > set secstorage.allowed.internal.sites to my CIDR. I >>> used >>> > >>> > >>> > >>> > $ python -m SimpleHTTPServer 443 >>> > >>> > >>> > >>> > to host my directory on http server where I had kept >>> my >>> > downloaded >>> > >>> iso. By >>> > >>> > manually visiting the local http server I am able to >>> > download the >>> > >>> file. But >>> > >>> > on mentioning the same url in registering the iso it >>> shows >>> > >>> registered >>> > >>> > successfully but does not get downloaded. >>> > >>> > >>> > >>> > In the zone section in the iso it mentions not >>> ready. I >>> > receive a >>> > >>> broken >>> > >>> > pipe error at the initial stage in my local file >>> server log >>> > but >>> > >>> then it >>> > >>> > again shows processing request. Even the iso status >>> in >>> > management >>> > >>> shows >>> > >>> > nothing and when I stop the local file server the >>> status of >>> > iso >>> > >>> shows 0% >>> > >>> > downloaded and then partial get request cannot be >>> served. I >>> > believe >>> > >>> > management server keeps on pinging the local file >>> server but >>> > the >>> > >>> connection >>> > >>> > is not getting established. >>> > >>> > >>> > >>> > I even refreshed my server again and again and also >>> did wait >>> > for a >>> > >>> long >>> > >>> > enough time to see if the process is slow. logging >>> into >>> > system VM >>> > >>> and >>> > >>> > running ./run.sh does not show any error and cloud >>> services >>> > are >>> > >>> running >>> > >>> > fine. I am pretty sure about the CIDR I have >>> mentioned in >>> > >>> > secstorage.allowed.internal.sites. Any idea? >>> > >>> > >>> > >>> > Regards >>> > >>> > Swastik >>> > >>> > >>> > >>> >>> > >>> >>> > >>> >>> > >>> dag.sonst...@shapeblue.com >>> > >>> www.shapeblue.com >>> > >>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK >>> @shapeblue >>> > >>> >>> > >>> >>> > >>> >>> > >>> >>> > >> >>> > > >>> > >>> > >>> > >>> > dag.sonst...@shapeblue.com >>> > www.shapeblue.com >>> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK >>> > @shapeblue >>> > >>> > >>> > >>> > >>> >>> >>> >>> dag.sonst...@shapeblue.com >>> www.shapeblue.com >>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK >>> @shapeblue >>> >>> >>> >>> >> > dag.sonst...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue