Hi all,
I have quite a bit of experience in building and deploying both static and dll-based programs, so I figured I'd throw in my $0.02 :) It seems to me that the crux of this particular part of the argument is here (from one of Dana's previous mails): "Because a DLL built by linking to a static CRT will have its own CRT state, it is not recommended to link statically to the CRT in a DLL unless the consequences of this are specifically desired and understood." Note that it says "...to link statically to the CRT _in a DLL_ unless...". Yes, linking the CRT statically into a DLL that is meant for redistribution is a bad idea. HOWEVER, that doesn't mean that static linking per se is bad. Linking a whole program statically (including the CRT) is a convenient way to make self-contained executables that can be distributed by just copying one binary around, that will not interfere with or depend on system-installed runtimes and that are portable across a large range of OS's, without having to wonder whether that OS has the right version of the CRT baked-in. So, the original question, to have a static build option of HDF, is quite reasonable IMO (I have made my own Visual Studio projects in the past to do just that). The argument that static builds might be misused when they are linked into a DLL which will then go on and be used by programs that use a different version of the CRT doesn't seem like a very convincing argument to me, since (a) when doing builds with dependency chains this complicated, one should be building everything with the same version of the compiler anyway, and (b) there are many scenarios (scenarios that happen more than the dangerous one, IMO) where static building is perfectly safe. So, the following assertion "Personally, I'm not keen on making it easy for people to statically link to the CRT since it's a documented bad idea" is, I'd say, not quite true; it's only not recommended when doing so *in a DLL*, *and* when that DLL is then used by programs that use different versions of the CRT. Now, as to whether dev time (which is basically money) should be spend on supporting this, I have no opinion about; that is a trade-off that I am in no position to make. Making one's own VS project files and building against that is trivial anyway, and people relying on HDF heavily should always do that, IMO, and it's easy to add static build targets there. But I don't think that technical objections to static linking should tip the scale in the direction of 'don't support'. cheers, roel ________________________________ From: Hdf-forum [mailto:[email protected]] On Behalf Of Dana Robinson Sent: Monday, June 10, 2013 10:11 To: HDF Users Discussion List Subject: Re: [Hdf-forum] [netcdfgroup] Make the Cmake Windows build staticplease ! Hi Pedro, Answers interspersed below... On Sun, Jun 9, 2013 at 11:26 PM, Pedro Vicente <[email protected]<mailto:[email protected]>> wrote: Hi Dana My original thread title "Make the CMake Windows build static" is a bit misleading, sorry. HDF5, in the NCO and netCDF cases, is only one of several input libraries. You can think of both HDF5 and netCDF in 2 ways 1) A library that will be used as *input* for a program (EXE file) that any HDF/netCDF user wants to build, his/her own program. In this case he will need your LIB (in case he wants to compile) or the DLL (if he just wants to run the EXE) 2) A final program destination already done (EXE) , like h5dump or ncdump, that can be either statically built (no DLL) or dynamically built (need DLL) For NCO, only case 2) exists. NCO is just the final program, not an input library to anyone. We only distribute EXE files, not LIB files, like you do. If I am taking the HDF5 binaries you provide, and combine with other several other library binaries, and even other libraries source code , in a combination of several LIB, DLL, and EXE files, dynamic is the way to go, because of the reasons you mention. Note: just the binaries. Not the whatever way I have to obtain them, in this case, a Cmake compiler flag. But, on the other hand, if I take all the source code only from these "libraries", and combine then into an EXE file statically build, then I can consider all those "libraries" not libraries any more, just pieces of source files that will produce an EXE. One note, though, and here I would like you to correct me, if I'm wrong. I can separate all these individual "libraries", like HDF5, into an intermediate input file, a LIB file, that I use *only* for my compilation into an EXE, *not* to distribute to anyone. Take the simplest case of 2 source files a.c that I compile into a.LIB main.c that I compile into main.EXE with a link input of a.LIB If I compile both of these Visual Studio projects with the same static flag, then, there is no DLL dependency, and I think the CRT issues you mention do not happen. Do you agree with this? No. a.lib and main.exe will have separate CRT states. You will not be able to malloc in a.c and free in main.exe without risking heap corruption in your exe. I have attached a helpful set of slides that explains this. Note that what you described will "work" in the sense that the OS won't complain. It's not very well documented, but it appears that the Win32 underpinnings are perfectly happy (this really isn't that surprising since everything belongs to the same process and the Win32 layer doesn't care about what's going on above it). It's just the bookkeeping information in the CRTs that gets messed up and this may or may not manifest itself as an error in your application. At any rate, it's not correct behavior. This is not what happens in Linux and other Unix-like operating systems. The standard C library is a central part of the Linux OS but Windows uses the Win32 API exclusively. The bizarre linking issues on Windows seem to be due to this non-central nature of the library. This is also why you have to install the CRT on Windows, whereas on Linux it's just there. The executive summary is that you should dynamically link ALL libraries that expose CRT objects via the library API. This obviously includes the Microsoft CRTs and, not so obviously, the HDF5 library. Also, just to be clear, it is totally ok to write to memory allocated by a different CRT. That doesn't affect internal CRT heap tracking. It's just unsafe to resize or free the memory unless that CRT provided it. When I started building Windows software in the 90's with the Visual Studio from that time I was using the DLL option. Why? It seemed like a good idea. Dozens of programs done, thousands of compiler builds, why not just make them share the system DLL? Many of this software I kept both the source and the final EXE, stored in a backup drive. But I forgot to store also the DLL, just because I never cared about them, I was not distributing software to anyone, just doing my college things. Fast forward 20 years, I want to run the EXE from that program. what happens? DLL whatever year 1993 is missing, oh my , I forgot to backup it. And now where can I find it?, nowhere :-) The funny thing is that the EXE is still *binary* compatible regarding the Windows year 1993 version and Windows 7 today. I just cannot run it because I don't have the DLL. If I *had* used the static option, now, today, I could run that EXE. So, that's all what I was trying to say with my comments, "people, don't use DLLs, they are bad" Totally different case as compared to distributing HDF5 as a library. CMake is only an automatic generator of Visual Studio projects. Like it is now, I can go to your generated projects and change the compiler flags. But I thought it would be a valuable feature for the community to have this choice already built directly into the Cmake step, like Ward did for netCDF, and John also did, I think, in the original setup, but you took off the final distribution. Is that right, John? Because I think the option -enable-static/shared, is just a given thing with the system, just a matter of adding a one liner to the script ? So, regarding your comment "Personally, I'm not keen on making it easy for people to statically link to the CRT since it's a documented bad idea" I don't think you should be limiting our choices. I am just asking to let us do your own choices in the way we compile our software. As for supporting static CRT linking in the HDF5 library, the problem we face is support. If we supply it, people will use it, especially when it's the "easy" thing, and then complain when it doesn't work for them. We don't put it in CMake for the same reason you don't store knives with the pointy ends up or allow smoking at the gas station. Given what I've demonstrated in the attached slides, you can probably see why I feel this way. I'm not 100% opposed, I'm just strongly opposed, and I'd want such an option flagged so harshly that people would feel personally insulted when they selected it. It's not enough to just document it, because almost nobody reads documentation. I understand that this sucks. Using standard C on Microsoft platforms has been that way for a very long time. There's no C99 support, no standard place to put libraries (the system directory does not count), you have to rely on the PATH environment variable to find dlls and tools, POSIX support barely exists, etc. AND then you have the CRT linking issue on top of it. It's a complete nightmare, really. Also, if I'm saying anything incorrect here, I would really appreciate being corrected. I'd be especially keen on seeing any authoritative sources that contradict what I've said. I've spent a lot of time digging around for information on this topic and MS doesn't make it easy. Cheers, Dana
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
