Just a little update on this:
The problem doesn't seem to have anything to do with "managed" storage code
that was introduced back in March.
The dsName is derived from the UUID of the datastore. This leads to a null
being returned from diskInfoBuilder.getDiskInfoByBackingFileBaseName, which
subsequently leads to a null being returned from this method (and a
NullPointerException being thrown).
I plan to take a look at the history of this file in gitk.
private VirtualMachineDiskInfo
getMatchingExistingDisk(VirtualMachineDiskInfoBuilder diskInfoBuilder,
DiskTO vol) {
if (diskInfoBuilder != null) {
VolumeObjectTO volume = (VolumeObjectTO)vol.getData();
String dsName = volume.getDataStore().getUuid().replace("-", ""
);
Map<String, String> details = vol.getDetails();
boolean isManaged = details != null &&
Boolean.parseBoolean(details.get(DiskTO.MANAGED));
VirtualMachineDiskInfo diskInfo =
diskInfoBuilder.getDiskInfoByBackingFileBaseName(isManaged ? new
DatastoreFile(volume.getPath()).getFileBaseName() : volume.getPath(),
dsName);
On Thu, Sep 18, 2014 at 4:34 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:
> I think I found the magic. :)
>
> sudo mount -t nfs 192.168.129.46:/export/secondary /mnt/sec
>
> If I mount /export/secondary to /mnt/sec (I did it in a bit of a
> roundabout way here), then the management server is happy.
>
> I think this is only VMware "thing" for the management server.
>
> Does anyone know if we document this?
>
> Thanks!
>
> On Thu, Sep 18, 2014 at 4:18 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> The instructions even say the following:
>>
>> If your secondary storage mount point is not named /mnt/secondary,
>> substitute your own mount point name.
>>
>> On Thu, Sep 18, 2014 at 4:15 PM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com> wrote:
>>
>>> Yeah, I've been through the instructions a couple times and don't see
>>> anything that indicates why it's having trouble with /mnt/sec (it says
>>> /mnt/sec instead of /mnt/secondary in the log messages).
>>>
>>> On Thu, Sep 18, 2014 at 4:01 PM, Ritu Sabharwal <rsabh...@brocade.com>
>>> wrote:
>>>
>>>> I have an external NFS server and it is mounted to /mnt/secondary on my
>>>> CS server so seeding command I used is:
>>>>
>>>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>>>> -m /mnt/secondary -u
>>>> http://cloudstack.apt-get.eu/systemvm/4.4/systemvm64template-4.4.0-6-vmware.ova
>>>> -h vmware -F
>>>>
>>>> I guess, if the secondary storage is local, then /export/secondary
>>>> should be fine.
>>>>
>>>> Regards,
>>>> Ritu S.
>>>> -----Original Message-----
>>>> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>>>> Sent: Thursday, September 18, 2014 2:34 PM
>>>> To: dev@cloudstack.apache.org
>>>> Cc: ilya musayev; Koushik Das; Sateesh Chodapuneedi; Adip Shetty
>>>> Subject: Re: [VMWARE SETUP ERROR] Error seeing in brining up System VMs
>>>> with Vmware setup with CS server
>>>>
>>>> The instructions to seed secondary storage for VMware look like this:
>>>>
>>>> /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
>>>> \ -m /mnt/secondary \ -u
>>>> http://cloudstack.apt-get.eu/systemvm/4.4/systemvm64template-4.4.0-6-vmware.ova
>>>> \
>>>> -h vmware \
>>>> -s <optional-management-server-secret-key> \ -F
>>>>
>>>> For -m, I have /export/secondary (not /mnt/secondary). I assume that's
>>>> correct because that is, in fact, where my secondary storage system
>>>> template is located for XenServer and that seems to work just fine.
>>>>
>>>>
>>>> On Thu, Sep 18, 2014 at 3:20 PM, Mike Tutkowski <
>>>> mike.tutkow...@solidfire.com> wrote:
>>>>
>>>> > I'm not getting far enough along in the process to repro (or not) the
>>>> > problem.
>>>> >
>>>> > I do have a /mnt/sec folder (and I even made it 777).
>>>> >
>>>> > Any thoughts on this?
>>>> >
>>>> > WARN [c.c.s.r.VmwareStorageProcessor] (DirectAgent-46:ctx-fdd23ec9
>>>> > 192.168.129.71, job-95/job-123, cmd: CopyCommand) Exception: tar
>>>> > --no-same-owner -xf /mnt/sec/template/tmpl/1/8//routing-8.ova
>>>> > java.io.IOException: Cannot run program "tar" (in directory
>>>> > "/mnt/sec/template/tmpl/1/8"): error=2, No such file or directory
>>>> > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
>>>> > at com.cloud.utils.script.Script.execute(Script.java:195)
>>>> > at com.cloud.utils.script.Script.execute(Script.java:163)
>>>> > at com.cloud.storage.res
>>>> >
>>>> > On Wed, Sep 17, 2014 at 2:53 PM, Mike Tutkowski <
>>>> > mike.tutkow...@solidfire.com> wrote:
>>>> >
>>>> >> You're right, Marcus. Now that I think about it, asserts are ignored
>>>> >> by default.
>>>> >>
>>>> >> On Wed, Sep 17, 2014 at 1:59 PM, Marcus <shadow...@gmail.com> wrote:
>>>> >>
>>>> >>> Aren't asserts ignored by default? I think we had an issue with
>>>> >>> asserts in the past, and I believe at the time the default config
>>>> >>> had asserts disabled.
>>>> >>>
>>>> >>>
>>>> >>> On Wed, Sep 17, 2014 at 1:35 PM, Ritu Sabharwal
>>>> >>> <rsabh...@brocade.com>
>>>> >>> wrote:
>>>> >>>
>>>> >>> > Hi Mike,
>>>> >>> >
>>>> >>> > I updated the master just now and tried the setup. I see the same
>>>> >>> error on
>>>> >>> > same line number.
>>>> >>> >
>>>> >>> > Thanks & Regards,
>>>> >>> > Ritu S.
>>>> >>> >
>>>> >>> > -----Original Message-----
>>>> >>> > From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
>>>> >>> > Sent: Tuesday, September 16, 2014 8:32 PM
>>>> >>> > To: dev@cloudstack.apache.org
>>>> >>> > Cc: ilya musayev; Koushik Das; Sateesh Chodapuneedi; Adip Shetty
>>>> >>> > Subject: Re: [VMWARE SETUP ERROR] Error seeing in brining up
>>>> >>> > System VMs with Vmware setup with CS server
>>>> >>> >
>>>> >>> > Can you tell us what commit SHA you are running under?
>>>> >>> >
>>>> >>> > I just updated master and line 2094 in VmwareResource is the
>>>> following:
>>>> >>> >
>>>> >>> > String[] diskChain = diskInfo.getDiskChain();
>>>> >>> >
>>>> >>> > The only candidate for a NullPointerException there is diskInfo;
>>>> >>> however,
>>>> >>> > the previous line would have caught this as it looks like this:
>>>> >>> >
>>>> >>> > assert (diskInfo != null);
>>>> >>> >
>>>> >>> > That being the case, I'm curious what commit SHA you saw this on?
>>>> >>> >
>>>> >>> > Thanks!
>>>> >>> >
>>>> >>> > On Tue, Sep 16, 2014 at 5:22 PM, Ritu Sabharwal
>>>> >>> > <rsabh...@brocade.com>
>>>> >>> > wrote:
>>>> >>> >
>>>> >>> > > Hi Sateesh, Koushik,
>>>> >>> > >
>>>> >>> > > I am setting up Vmware Cluster with CS (master) and seeing
>>>> errors.
>>>> >>> > > ilya was helping me for this setup and figured out this error.
>>>> >>> > >
>>>> >>> > > The System VMs(Secondary Stogare VM and Console Proxy VM) are
>>>> >>> > > not coming up properly. The system vms are reconfiguring and
>>>> >>> > > destroying again and again in cycle.
>>>> >>> > >
>>>> >>> > > I have used Vmware Vsphere 5.1 SDK for building CS.
>>>> >>> > >
>>>> >>> > > The error logs give this error:
>>>> >>> > >
>>>> >>> > > 2014-09-16 16:11:01,619 WARN [c.c.h.v.r.VmwareResource]
>>>> >>> > > (DirectAgent-8:ctx-8909215e 10.24.41.149, job-54/job-68, cmd:
>>>> >>> > > StartCommand) StartCommand failed due to Exception:
>>>> >>> > > java.lang.NullPointerException
>>>> >>> > > Message: null
>>>> >>> > >
>>>> >>> > > java.lang.NullPointerException
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> com.cloud.hypervisor.vmware.resource.VmwareResource.postDiskConfigBe
>>>> >>> foreStart(VmwareResource.java:2094)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareRe
>>>> >>> source.java:1685)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(V
>>>> >>> mwareResource.java:448)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectA
>>>> >>> gentAttache.java:294)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(M
>>>> >>> anagedContextRunnable.java:49)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.c
>>>> >>> all(DefaultManagedContext.java:56)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.cal
>>>> >>> lWithContext(DefaultManagedContext.java:103)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.run
>>>> >>> WithContext(DefaultManagedContext.java:53)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(Man
>>>> >>> agedContextRunnable.java:46)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:4
>>>> >>> 71)
>>>> >>> > > at
>>>> java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
>>>> >>> .access$201(ScheduledThreadPoolExecutor.java:178)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
>>>> >>> .run(ScheduledThreadPoolExecutor.java:292)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
>>>> >>> .java:1145)
>>>> >>> > > at
>>>> >>> > >
>>>> >>> >
>>>> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecuto
>>>> >>> r.java:615)
>>>> >>> > > at java.lang.Thread.run(Thread.java:744)
>>>> >>> > >
>>>> >>> > > Please help to resolve this issue.
>>>> >>> > >
>>>> >>> > > Thanks & Regards,
>>>> >>> > > Ritu S.
>>>> >>> > >
>>>> >>> > >
>>>> >>> >
>>>> >>> >
>>>> >>> > --
>>>> >>> > *Mike Tutkowski*
>>>> >>> > *Senior CloudStack Developer, SolidFire Inc.*
>>>> >>> > e: mike.tutkow...@solidfire.com
>>>> >>> > o: 303.746.7302
>>>> >>> > Advancing the way the world uses the cloud
>>>> >>> > <http://solidfire.com/solution/overview/?video=play>*™*
>>>> >>> >
>>>> >>>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> *Mike Tutkowski*
>>>> >> *Senior CloudStack Developer, SolidFire Inc.*
>>>> >> e: mike.tutkow...@solidfire.com
>>>> >> o: 303.746.7302
>>>> >> Advancing the way the world uses the cloud
>>>> >> <http://solidfire.com/solution/overview/?video=play>*™*
>>>> >>
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > *Mike Tutkowski*
>>>> > *Senior CloudStack Developer, SolidFire Inc.*
>>>> > e: mike.tutkow...@solidfire.com
>>>> > o: 303.746.7302
>>>> > Advancing the way the world uses the cloud
>>>> > <http://solidfire.com/solution/overview/?video=play>*™*
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> *Mike Tutkowski*
>>>> *Senior CloudStack Developer, SolidFire Inc.*
>>>> e: mike.tutkow...@solidfire.com
>>>> o: 303.746.7302
>>>> Advancing the way the world uses the cloud
>>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>>
>>>
>>>
>>>
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkow...@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the cloud
>>> <http://solidfire.com/solution/overview/?video=play>*™*
>>>
>>
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the cloud
>> <http://solidfire.com/solution/overview/?video=play>*™*
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud
> <http://solidfire.com/solution/overview/?video=play>*™*
>
--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
<http://solidfire.com/solution/overview/?video=play>*™*
