On Wed, 1 Dec 2010, Andy Polyakov via RT wrote:
(I said:)
> > I thought that unicows had to be linked as the first library, so I
> > manually added -lunicows at the beginning of the LIBRARIES variable in
> > apps/Makefile, after Configure had completed.
>
> ...
>
> >> else /*if (GetLastError()==ERROR_NO_UNICODE_TRANSLATION)*/
> >>
> >> I'm not suggesting this a fix, just trying to understand what happens
> >> exactly.
> >
> > This seems to make the problems go away. With this commented out, the
> > application doesn't crash. Linking with libunicows does not appear
> > necessary. For openssl.exe version -a, see the attached file
> > "version.good".
>
> I don't quite understand what happens if you do *not* uncomment and do
> *not* link with unicows? As mentioned I'm not comfortable with removing
> the above mentioned if clause, I'd rather adjust it. The fact that it
> works is proof enough that it's actually MultiByteToWideChar that
> returned 1004 and not wfopen. So could you test
>
> else if ((ret=GetLastError())==ERROR_NO_UNICODE_TRANSLATION
> || ret==ERROR_INVALID_FLAGS)
>
> You'd have to add 'DWORD ret;' declaration after #ifdef _WIN32 in the
> beginning of function. A.
I used this as a jumping point to investigate and can share some
information about the build on Win98 with mingw. I am also attaching a
patch for you to look at. It turns out that despite the logs I submitted
to you, unicows wasn't being used. A "strings openssl.exe" revealed no
"unicows" string in the executable. When I instead added "-lunicows"
to the beginning of the $lflags variable for mingw's config-string in
Configure, "unicows.dll" began to appear in the executable file. I am not
sure why this is.
I am not a programmer, so I put in various printf statements in the code
to see what was happening.
The ERROR_INVALID_FLAGS appears when compiling without unicows.
Looking at the documentation for MultiByteToWideChar function on MSDN,
MB_ERR_INVALID_CHARS is only supported on Windows 2000 with SP4 or later
systems, with earlier systems only accepting a flag of "0" when using
CP_UTF8. I couldn't find documentation for it, but when I compiled
with unicows, however, there was no longer an error when using the
MB_ERR_INVALID_CHARS flag. Your fallback to fopen from _wfopen didn't
work because mingw issued an errno of EBADF (Bad file descriptor)
instead of ENOENT. If I allowed either error, then the fallback worked.
Attached is a patch which allows errors of either ENOENT or EBADF to
trigger fopen when the wide function failed. I also put in a section to
use a flag of "0" instead of MB_ERR_INVALID_CHARS when the latter led to
ERROR_INVALID_FLAGS. With the patch, the openssl.exe file runs without
crashing, but on my system I get the EBADF error from the wide function
and it uses fopen regardless of whether unicows is used or not in the
compilation. With unicows, it uses fopen from the section with flag of
MB_ERR_INVALID_CHARS and without unicows it uses fopen from the section
I put in with flag "0".
I don't have a test case to see if the multibyte conversion code works
on Win98, since just running "openssl version -a" causes EBADF and the
fallback to fopen for opening the configuration file.
I will be away at a conference and unable to do any testing for the next
week or so, but if you have things you want me to check, I will try to
do it when I get back.
The output from "openssl version -a" with the patch applied is attached
as file "openssl.version".
Let me know if you want other data from my logs or need me to do any
tests. Thanks for all your help so far.
Doug
--
Doug Kaufman
Internet: [email protected]
--- crypto/bio/bss_file.c.ori 2010-12-01 18:34:48.000000000 -0800
+++ crypto/bio/bss_file.c 2010-12-01 20:48:34.000000000 -0800
@@ -123,6 +123,7 @@
#if defined(_WIN32) && defined(CP_UTF8)
int sz, len_0 = (int)strlen(filename)+1;
+ DWORD ret1;
/*
* Basically there are three cases to cover: a) filename is
@@ -146,15 +147,33 @@
filename,len_0,wfilename,sz) &&
MultiByteToWideChar(CP_UTF8,0,mode,strlen(mode)+1,
wmode,sizeof(wmode)/sizeof(wmode[0])) &&
- (file=_wfopen(wfilename,wmode))==NULL && errno==ENOENT
+ (file=_wfopen(wfilename,wmode))==NULL &&
+ (errno==ENOENT || errno==EBADF)
) /* UTF-8 decode succeeded, but no file, filename
* could still have been locale-ized... */
file = fopen(filename,mode);
}
- else if (GetLastError()==ERROR_NO_UNICODE_TRANSLATION)
+ else if ((ret1=GetLastError())==ERROR_INVALID_FLAGS)
+ {
+ if ((sz=MultiByteToWideChar(CP_UTF8,0,
+ filename,len_0,NULL,0))>0)
{
- file = fopen(filename,mode);
+ WCHAR wmode[8];
+ WCHAR *wfilename = _alloca(sz*sizeof(WCHAR));
+
+ if (MultiByteToWideChar(CP_UTF8,0,
+ filename,len_0,wfilename,sz) &&
+ MultiByteToWideChar(CP_UTF8,0,mode,strlen(mode)+1,
+ wmode,sizeof(wmode)/sizeof(wmode[0])) &&
+ (file=_wfopen(wfilename,wmode))==NULL &&
+ (errno==ENOENT || errno==EBADF)
+ ) /* UTF-8 decode succeeded, but no file, filename
+ * could still have been locale-ized... */
+ file = fopen(filename,mode);
}
+ }
+ else if (ret1==ERROR_NO_UNICODE_TRANSLATION)
+ file = fopen(filename,mode);
#else
file=fopen(filename,mode);
#endif
OpenSSL 1.1.0-dev xx XXX xxxx
built on: Wed Dec 1 20:51:08 PST 2010
platform: mingw
options: bn(64,32) rc4(4x,int) des(ptr,risc1,16,long) blowfish(idx)
compiler: gcc -DOPENSSL_THREADS -D_MT -DDSO_WIN32 -mno-cygwin -DL_ENDIAN
-DWIN32_LEAN_AND_MEAN -fomit-frame-pointer -O3 -march=i486 -Wall
-DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT
-DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM
-DWHIRLPOOL_ASM -DGHASH_ASM
OPENSSLDIR: "d:/cygwin/mingw/ssl"
