Re: [Bf-committers] External libs builder for Windows [v2013.01.18]
On Sat, Jan 19, 2013 at 9:59 AM, Brecht Van Lommel
brechtvanlom...@pandora.be wrote:
Great work! So have you been able to build Blender with these
libraries, does it need build system changes too or do things get
installed in the same directory structure as before?
I was able to build blender (VS2008/32-bit) with the following libs
replaced with ones I built by script:
boost
freetype
jpeg
lapack
lcms
llvm
openal
opencollada
opencolorio
openexr
openimageio
png
pthreads
python
samplerate
sdl
tiff
zlib
I had to do some more tweaks with the scripts to fix the include names
in some cases and copying some libs to legacy names, and roll boost
back to 1.49 (while updating libs one-by-one since old boost name
implib's existed -- but now I can try 1.52 again). I'll try to upload
an updated lib-builder zip soon. I also had to make a few temp mods to
the cmake files since I split out some of the bundled libs into their
own package:
-
Index: blender/source/creator/CMakeLists.txt
===
--- blender/source/creator/CMakeLists.txt (revision 54059)
+++ blender/source/creator/CMakeLists.txt (working copy)
@@ -599,7 +599,7 @@
install(
FILES
${LIBDIR}/openal/lib/OpenAL32.dll
- ${LIBDIR}/openal/lib/wrap_oal.dll
+# ${LIBDIR}/openal/lib/wrap_oal.dll
DESTINATION ${TARGETDIR}
)
endif()
Index: blender/CMakeLists.txt
===
--- blender/CMakeLists.txt (revision 54059)
+++ blender/CMakeLists.txt (working copy)
@@ -903,6 +903,7 @@
set(OPENAL ${LIBDIR}/openal)
set(OPENAL_INCLUDE_DIR ${OPENAL}/include)
set(OPENAL_LIBRARY wrap_oal)
+set(OPENAL_LIBRARY OpenAL32)
set(OPENAL_LIBPATH ${OPENAL}/lib)
endif()
@@ -1070,7 +1071,9 @@
if(WITH_IMAGE_OPENEXR)
set_lib_path(OPENEXR openexr)
set(OPENEXR_INCLUDE_DIR ${OPENEXR}/include)
+set(OPENEXR_INCLUDE_DIR ${OPENEXR_INCLUDE_DIR} ${LIBDIR}/ilmbase/include)
set(OPENEXR_INCLUDE_DIRS ${OPENEXR}/include/OpenEXR)
+set(OPENEXR_INCLUDE_DIRS ${OPENEXR_INCLUDE_DIRS}
${LIBDIR}/ilmbase/include/OpenEXR)
set(OPENEXR_LIBPATH ${OPENEXR}/lib)
set(OPENEXR_LIBRARIES
${OPENEXR_LIBPATH}/Iex.lib
@@ -1079,6 +1082,13 @@
${OPENEXR_LIBPATH}/Imath.lib
${OPENEXR_LIBPATH}/IlmThread.lib
)
+set(OPENEXR_LIBRARIES
+ ${LIBDIR}/ilmbase/lib/Iex.lib
+ ${LIBDIR}/ilmbase/lib/Half.lib
+ ${OPENEXR_LIBPATH}/IlmImf.lib
+ ${LIBDIR}/ilmbase/lib/Imath.lib
+ ${LIBDIR}/ilmbase/lib/IlmThread.lib
+)
endif()
if(WITH_IMAGE_TIFF)
@@ -1164,6 +1174,9 @@
set(OPENCOLORIO_LIBRARIES OpenColorIO)
set(OPENCOLORIO_LIBPATH ${LIBDIR}/opencolorio/lib)
set(OPENCOLORIO_DEFINITIONS)
+set(OPENCOLORIO_LIBRARIES ${OPENCOLORIO_LIBRARIES} optimized tinyxml
debug tinyxml_d)
+set(OPENCOLORIO_LIBRARIES ${OPENCOLORIO_LIBRARIES} optimized
libyaml-cppmt debug libyaml-cppmtd)
+set(OPENCOLORIO_LIBPATH ${OPENCOLORIO_LIBPATH} ${LIBDIR}/tinyxml/lib
${LIBDIR}/yaml/lib)
endif()
-
Those changes could be done with something more permanent by adding
WITH_TINYXML, WITH_YAML, WITH_ILMBASE, and maybe WITH_OPENAL_SOFT (the
soft version doesn't need/have wrap_oal anymore) which would allow old
or new libs to be used during a [good 'old known libs vs iffy new
sparkly libs] transition phase.
I can build/link and start blender fine, but I didn't do any real
testing to exercise each lib. Also, since I built the soft version of
openal, it comes up with this message on start:
AL lib: (EE) UpdateDeviceParams: Failed to set 44100hz, got 48000hz instead
I saw some other references to this online (with linux where I guess
openal-soft is the system version). I'm not sure if it is a serious
error. I also saw mention of another openal problem where someone's
code worked fine until they switched to the soft version and had to
correct their code -- this might imply the soft lib is less forgiving
of API usage errors than the old one, if there are also such broken
usages in the blender code.
But on the plus side, some of the 32-bit vs 64-bit lib names would be
simplified as these scripts always build the same names (no more
special case for libz on win64). I also build a debug version of each
lib (which only exist for some svn ones now), so debugging of errors
that happen while calling these libs might be easier to trace.
-Chad
Re: [Bf-committers] External libs builder for Windows [v2013.01.18]
Great work! So have you been able to build Blender with these libraries, does it need build system changes too or do things get installed in the same directory structure as before? So let's see how we can get this stuff in svn. Can we get some feedback from Windows platform maintainers, do you all agree with the general idea here? Does anyone want to review the scripts and setups here? The question then is also still what kind of directory structure we want to use for windows libraries. In my opinion we should just have a lib/windows-arch-vsxxx directory for each combination we support. This gives a minimal and simple download for users. The include directories would be duplicated, but that's also an advantage if not all platforms get updated to a new library version at the same time, otherwise the build might break. Thanks, Brecht. On Sat, Jan 19, 2013 at 6:47 AM, Chad Fraleigh ch...@triularity.org wrote: After some more work, a few breaks to avoid insanity fighting with some of the VS and cmake configs, and a lot of hacking of the various build scripts/projects (the patches it has to do is a mess!), I got farther. Now I'm able to build all the following libs under VS2008/32-bit, VS2010/32-bit, VS2010+SDK7.1/32-bit, and VS2010+SDK7.1/64-bit: boost freetype ilmbase jpeg lcms llvm openal opencollada opencolorio openexr openimageio png pthreads python samplerate sdl tiff tinyxml yaml zlib zlibwapi I'll try to install the 6.1 SDK soon and see if it also works for VS2008 x64. Unfortunately when I reinstalled my VS's, VS2012 decided to manifest a bug where it doesn't install any of the C header files (thanks M$!).. so I can't do any more testing on them at the moment. The builder setup (with and without source tar/zip distros): http://www.triularity.org/download/blender/lib-builder.zip (186M) http://www.triularity.org/download/blender/lib-builder-nosrc.zip (4.5M) The results of my compiles (the VS10+SDK7.1 x32 was omitted due to size and being mostly redundant if someone just needs the 2010 binaries). http://www.triularity.org/download/blender/install-vs9.0_x32_sdk6.0.zip (217M) ** http://www.triularity.org/download/blender/install-vs10.0_x32_sdk7.0.zip (214M) http://www.triularity.org/download/blender/install-vs10.0_x64_sdk7.1.zip (254M) ** install-vs9.0_x32_sdk6.0.zip is still uploading as of composing this message, so give it half an hour or so if you want that one. The logs for each of those builds (generated by the buildall.bat script) for reference: http://www.triularity.org/download/blender/logs-vs9.0_x32_sdk6.0.zip (275K) http://www.triularity.org/download/blender/logs-vs10.0_x32_sdk7.0.zip (471K) http://www.triularity.org/download/blender/logs-vs10.0_x32_sdk7.1.zip (472K) http://www.triularity.org/download/blender/logs-vs10.0_x64_sdk7.1.zip (498K) All the old install-*.zip's have been deleted (to free up space). -Chad ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] External libs builder for Windows [v2013.01.18]
On Sat, Jan 19, 2013 at 9:59 AM, Brecht Van Lommel brechtvanlom...@pandora.be wrote: Great work! So have you been able to build Blender with these libraries, does it need build system changes too or do things get installed in the same directory structure as before? I haven't tried them with blender itself yet. Right now I'm still trying to verify that others can repeat the builds without the same compile errors that were experienced on my earlier script releases. I suppose I could copy some into my normal blender lib/ directory and try a few. It wasn't until I did openimageio that I realized that boost was still compiling 32-bit mode (even though I was giving it a 64-bit compiler environment). Right now I've gone back to building only the static libs (where it easily let me) to reduce static/dynamic conflicts. But I was wondering if eventually using the dynamic ones for most in blender would be of value? I see it has some dll's in the blender install, but not many. Since not much would be sharing the libs (really only moving stuff from blender.exe to dll's), it may just be extra work with no real benefit. Maybe the thumbhandler could share some (why _can't_ its code be in the normal build area again??) So let's see how we can get this stuff in svn. Can we get some feedback from Windows platform maintainers, do you all agree with the general idea here? Does anyone want to review the scripts and setups here? Just a note -- the makefile is still ugly. Eventually I would like to move the section for each lib to it's own .mk file with the base one including them all, or each being a standalone make (and including a .mk for common parts), rather than the monolithic thing it is now. The question then is also still what kind of directory structure we want to use for windows libraries. In my opinion we should just have a lib/windows-arch-vsxxx directory for each combination we support. This gives a minimal and simple download for users. The include directories would be duplicated, but that's also an advantage if not all platforms get updated to a new library version at the same time, otherwise the build might break. That's what I had in mind also.. since there is already include duplication with the x32-vs-x64 and mingw, it wouldn't be a major change (and the cmake config for blender itself would be simplified -- just configure the base lib path). There would be a few binary duplicates for the few packages that won't compile under VS (due to no vs/cmake config, C99 support, or just being fortran *grin*). I might try to write cmake files for the simple cases (but then someone would need to tweak them with each new lib version). The 'jack' and 'sndfile' libs are the only ones I've found [so far] with predictable sources of precompiled ones for both 32-bit and 64-bit for the VS-no-go list. Also I include a VERSION.txt file (and LICENSE.txt when I can find one in the source) in the install directory for each.. one for license compliance and the other since it is sometimes hard to tell what version a lib is (sometimes in the lib name, sometimes in a header, other times haven't a clue). I appended the .txt part since it might be easier for windows users to open [via double-click] (not counting it running notepad and the file having unix newlines). If these files are added for the other platforms it would be nice to make the names the same (be it with or without the extension). Oh, and the VERSION[.txt] file is always installed last -- so it can be used as a generic is this lib completely here test. Also, I don't know what is done now but I think a library manifest (just a txt file would due) that lists all the external libs bundled with blender and their versions to be included with each release so if there are any concerns about security issues with those libs, one can also look at the list. This of course can be pulled from the VERSION.txt files during the blender build. Speaking of mingw.. eventually I would like to have support for that in the builder (not the same makefile since this is nmake specific), but another makefile or script that does those environments (and maybe osx and/or android). It could use any common resources like the source zip/tar distros, and maybe the patches where applicable. -Chad On Sat, Jan 19, 2013 at 6:47 AM, Chad Fraleigh ch...@triularity.org wrote: After some more work, a few breaks to avoid insanity fighting with some of the VS and cmake configs, and a lot of hacking of the various build scripts/projects (the patches it has to do is a mess!), I got farther. Now I'm able to build all the following libs under VS2008/32-bit, VS2010/32-bit, VS2010+SDK7.1/32-bit, and VS2010+SDK7.1/64-bit: boost freetype ilmbase jpeg lcms llvm openal opencollada opencolorio openexr openimageio png pthreads python samplerate sdl
Re: [Bf-committers] External libs builder for Windows [v2013.01.18]
Hi, What does that mean for the current x32/x64 MSVC 2008 libs? (VC 2008 is what we use for official releases). Will they stay the same, but if we have to upgrade a library we can use your script? I don't really see the benefit here (yet). It's not the same situation as on Linux imo. We don't update libs that often on Windows and if I have to rebuild one lib or two in weeks/months I rather do it manually with full control, rather than with a script (which sounds like a big hack if I get you right). Regarding separate repositories (no shared headers, no shared libs between different vc versions) is a really good thing though and I highly support that. Regards, Thomas Am 19.01.2013 06:47, schrieb Chad Fraleigh: After some more work, a few breaks to avoid insanity fighting with some of the VS and cmake configs, and a lot of hacking of the various build scripts/projects (the patches it has to do is a mess!), I got farther. Now I'm able to build all the following libs under VS2008/32-bit, VS2010/32-bit, VS2010+SDK7.1/32-bit, and VS2010+SDK7.1/64-bit: boost freetype ilmbase jpeg lcms llvm openal opencollada opencolorio openexr openimageio png pthreads python samplerate sdl tiff tinyxml yaml zlib zlibwapi I'll try to install the 6.1 SDK soon and see if it also works for VS2008 x64. Unfortunately when I reinstalled my VS's, VS2012 decided to manifest a bug where it doesn't install any of the C header files (thanks M$!).. so I can't do any more testing on them at the moment. The builder setup (with and without source tar/zip distros): http://www.triularity.org/download/blender/lib-builder.zip (186M) http://www.triularity.org/download/blender/lib-builder-nosrc.zip (4.5M) The results of my compiles (the VS10+SDK7.1 x32 was omitted due to size and being mostly redundant if someone just needs the 2010 binaries). http://www.triularity.org/download/blender/install-vs9.0_x32_sdk6.0.zip (217M) ** http://www.triularity.org/download/blender/install-vs10.0_x32_sdk7.0.zip (214M) http://www.triularity.org/download/blender/install-vs10.0_x64_sdk7.1.zip (254M) ** install-vs9.0_x32_sdk6.0.zip is still uploading as of composing this message, so give it half an hour or so if you want that one. The logs for each of those builds (generated by the buildall.bat script) for reference: http://www.triularity.org/download/blender/logs-vs9.0_x32_sdk6.0.zip (275K) http://www.triularity.org/download/blender/logs-vs10.0_x32_sdk7.0.zip (471K) http://www.triularity.org/download/blender/logs-vs10.0_x32_sdk7.1.zip (472K) http://www.triularity.org/download/blender/logs-vs10.0_x64_sdk7.1.zip (498K) All the old install-*.zip's have been deleted (to free up space). -Chad ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers -- Thomas Dinges Blender Developer, Artist and Musician www.dingto.org ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
Re: [Bf-committers] External libs builder for Windows [v2013.01.18]
On Sat, Jan 19, 2013 at 2:21 PM, Thomas Dinges blen...@dingto.org wrote: Hi, What does that mean for the current x32/x64 MSVC 2008 libs? (VC 2008 is what we use for official releases). Will they stay the same, but if we have to upgrade a library we can use your script? They could be.. at least when updated (no reason to necessarily go replace them all with recompiled ones that are equivalent just for the sake of doing so). I don't really see the benefit here (yet). It's not the same situation as on Linux imo. We don't update libs that often on Windows and if I have to rebuild one lib or two in weeks/months I rather do it manually with full control, rather than with a script (which sounds like a big hack if I get you right). The problem with manual is it is not as reliable. For example for some of the [svn-ed] libs there is a readme file that described how to build it, which was useful.. on the other hand some things were obviously missing because when I followed only those instructions I got errors in some cases without doing more work. With a script everything is effectively documented (maybe not user friendly docs, but everything is there). And there is no need to even know how to use the Visual Studio GUI (just install it and bring up a VS configured prompt). The hacks that need to be done will still be there, only they would have to all be done manually if not scripted (either by editing files or changing various project config options). It also forces that there is a build for VS2010 (and eventually VS2012) for those developers that use those and things, like having no python 3.3 for VS2010 for awhile, shouldn't happen (just because the one compiling that lib manually only did it for VS2008). In some cases (too bad not most) changing versions of a lib is just dropping in the new tar/zip for the source and changing the version tag in the makefile. In fact I remembered recently seeing a security advisory for freetype prior 2.4.11 (the blender win32 svn binary is still 2.4.4), so I _just_ downloaded the 2.4.11 tar (2.4.10 is what they had when I started this builder), changed the version line in the makefile and compiled and installed (in the builder install tree) 2008/32, 2010/32, 2010/64 with one nmake run in each VS command prompt. If done manually it would have required extracting the compressed tar (maybe not obvious for some windows only developers), creating a new x64 platform (as it only comes with Win32) in Visual Studio, selecting the correct configuration (from the multiple choices in freetype), compile, then find and copy all the include, lib, and license files to the right place and ideally create a version file).. and this is if the developer already has experience with this lib's specific build needs. NOTE to anyone that didn't read between the lines in the previous paragraph: freetype*.lib needs to be updated for security fixes!!! (via scripted or manual) At minimum it has these problems: 2.4.11 http://portaudit.FreeBSD.org/1ae613c3-5728-11e2-9483-14dae938ec40.html 2.4.9 http://portaudit.FreeBSD.org/462e2d6c-8017-11e1-a571-bcaec565249c.html 2.4.7 http://portaudit.FreeBSD.org/54075e39-04ac-11e1-a94e-bcaec565249c.html 2.4.6 http://portaudit.FreeBSD.org/5d374b01-c3ee-11e0-8aa5-485d60cb5385.html freetype/freetype.h will have its version, e.g.: #define FREETYPE_MAJOR 2 #define FREETYPE_MINOR 4 #define FREETYPE_PATCH 4-- Ugg 11 ! In the end the scripts are for the same reasons that linux has build scripts for their package source distros. Yes, those linux packages could all be compiled manually by the various authors/venders and then distributed in binary packages for use by the masses.. and in the cases where the few (well, few relative to the entire user base) want to compile something themselves, then it is up to them to reinvent the compile wheel. Maybe blender doesn't have as many needed libs, but it is the same principle. Soo.. there's my 2 cents (give or take $5). =) -Chad Regards, Thomas Am 19.01.2013 06:47, schrieb Chad Fraleigh: After some more work, a few breaks to avoid insanity fighting with some of the VS and cmake configs, and a lot of hacking of the various build scripts/projects (the patches it has to do is a mess!), I got farther. Now I'm able to build all the following libs under VS2008/32-bit, VS2010/32-bit, VS2010+SDK7.1/32-bit, and VS2010+SDK7.1/64-bit: boost freetype ilmbase jpeg lcms llvm openal opencollada opencolorio openexr openimageio png pthreads python samplerate sdl tiff tinyxml yaml zlib zlibwapi I'll try to install the 6.1 SDK soon and see if it also works for VS2008 x64. Unfortunately when I reinstalled my VS's, VS2012 decided to manifest a bug where it doesn't install any of the C header files (thanks M$!).. so I can't do any more testing on them at the
[Bf-committers] External libs builder for Windows [v2013.01.18]
After some more work, a few breaks to avoid insanity fighting with some of the VS and cmake configs, and a lot of hacking of the various build scripts/projects (the patches it has to do is a mess!), I got farther. Now I'm able to build all the following libs under VS2008/32-bit, VS2010/32-bit, VS2010+SDK7.1/32-bit, and VS2010+SDK7.1/64-bit: boost freetype ilmbase jpeg lcms llvm openal opencollada opencolorio openexr openimageio png pthreads python samplerate sdl tiff tinyxml yaml zlib zlibwapi I'll try to install the 6.1 SDK soon and see if it also works for VS2008 x64. Unfortunately when I reinstalled my VS's, VS2012 decided to manifest a bug where it doesn't install any of the C header files (thanks M$!).. so I can't do any more testing on them at the moment. The builder setup (with and without source tar/zip distros): http://www.triularity.org/download/blender/lib-builder.zip (186M) http://www.triularity.org/download/blender/lib-builder-nosrc.zip (4.5M) The results of my compiles (the VS10+SDK7.1 x32 was omitted due to size and being mostly redundant if someone just needs the 2010 binaries). http://www.triularity.org/download/blender/install-vs9.0_x32_sdk6.0.zip (217M) ** http://www.triularity.org/download/blender/install-vs10.0_x32_sdk7.0.zip (214M) http://www.triularity.org/download/blender/install-vs10.0_x64_sdk7.1.zip (254M) ** install-vs9.0_x32_sdk6.0.zip is still uploading as of composing this message, so give it half an hour or so if you want that one. The logs for each of those builds (generated by the buildall.bat script) for reference: http://www.triularity.org/download/blender/logs-vs9.0_x32_sdk6.0.zip (275K) http://www.triularity.org/download/blender/logs-vs10.0_x32_sdk7.0.zip (471K) http://www.triularity.org/download/blender/logs-vs10.0_x32_sdk7.1.zip (472K) http://www.triularity.org/download/blender/logs-vs10.0_x64_sdk7.1.zip (498K) All the old install-*.zip's have been deleted (to free up space). -Chad ___ Bf-committers mailing list Bf-committers@blender.org http://lists.blender.org/mailman/listinfo/bf-committers
