Thanks Jeffrey. I was aware of the limitation and how image randomization
works in Vista and higher so disabling ASLR was the first thing I tried.
Unfortunately, disabling ASLR didn't fix the problem. The DLL gets relocated
with or without ASLR enabled. The VC9-generated DLL does not have this
problem. It always loads at the correct base address.
Since the FIPS canister fails its self-test (before the OpenSSL DLL is ever
generated with it), I suspect the problem is more significant than the DLL not
residing at the correct address. I know the DLL will fail if it doesn't reside
at the correct address but I suspect the relocation is a symptom of a larger
issue.
Statically linking with the OpenSSL FIPS library reports the exact same error
("Incorrect fingerprint") as when linking to the DLL.
I would be happy to try any additional suggestions.
Thanks,Grant
> Date: Mon, 18 Oct 2010 09:54:18 -0400
> From: [email protected]
> To: [email protected]
> Subject: Re: FIPS Module 1.2 build with Visual Studio 2010 fails self-tests
>
> On 10/17/2010 4:36 PM, Dr. Stephen Henson wrote:
> > On Sun, Oct 17, 2010, [email protected] wrote:
> >
> >> Ugh. This is worse than I thought. It's *intermittently* failing like
> >> that. After a few more minutes, I tried it again, and got the expected
> >> output.
> >>
> >> Is there some way to specify a base address during the creation of the
> >> DLL,
> >> after the fipscanister is created? Would that invalidate it?
> >>
> >> The default appears to be 0x00d60000, and it works when loaded there.
> >>
> >
> > You can't modify the 1.2 module build process in any way but you can specify
> > an alternate base address when you link against a newer version of OpenSSL
> > such as 0.9.8o.
> >
> > One way to get more information would be to dump the fingerprinted data in
> > the
> > FIPS_incore_fingerprint() function along with the addresses when it works
> > and
> > when it fails. Then see if the addresses and/or the dumped binary data have
> > changed.
>
> On Windows it is not possible to require that a DLL be loaded at a
> specific address in memory within a process. The base address is simply
> a recommendation and if correct will result in the library loading
> process being faster than if it is not. Any fingerprinting of a
> library needs to be performed by computing the memory offsets compared
> to the base address and using those.
>
> Microsoft Vista, Server 2008, Win7 and Server 2008-R2 all support enable
> by default Address space layout randomization (ASLR). Visual Studio
> 2010 is the first version of Windows development tools to turn ASLR on
> by default for EXEs and DLLs. To disable, use /DYNAMICBASE:NO when
> linking. (Or disable the "Randomized Base Address property in Visual
> Studio.)
>
> Jeffrey Altman
> Secure Endpoints, Inc.
>
>
>
