[hugin-ptx] Re: [OSX] Hugin 32bit vs. 64 bit and the way to go forward
Bruno Postle schrieb: On Mon 19-Apr-2010 at 14:17 -0700, Battle wrote: So where are my bottle necks? The optimization process in hugin proper in the 32 bit build seems slow. On smaller projects of 10 to 20 images the optimization tab can take almost as long as actually stitching the project. I don't know whether a 64 bit version of the Hugin UI would address this or if this is an issue with another sub program. Since I don't know where the optimization is taking place or if it is multithreaded or otherwise optimized for multiple core machines I think its not possible for me to answer whether a 64 bit build of Hugin is useful. Yes for big projects the optimisation is a bottleneck as it is single-threaded. As far as I know there is no plan to make this multi-threaded, or even if it is possible. The best way to speed up the optimisation is to exploit the problem structure when doing the optimisation. Currently a general purpose nonlinear least square minimiser is used in libpano This is very inefficient for large problems where the variables are only sparsely connected. A sparse levenberg marquardt algorithm would work wonders there. There is a fast implementation for 3D reconstruction problems that could be adapted to optimize panoramas http://www.ics.forth.gr/~lourakis/sba/. However, it is too inflexible (all pictures can only have the same number of parameters, no linking of parameters possible) for use within hugin or libpano. ciao Pablo -- You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx
Re: [hugin-ptx] Re: [OSX] Hugin 32bit vs. 64 bit and the way to go forward
On Mon 19-Apr-2010 at 14:17 -0700, Battle wrote: So where are my bottle necks? The optimization process in hugin proper in the 32 bit build seems slow. On smaller projects of 10 to 20 images the optimization tab can take almost as long as actually stitching the project. I don't know whether a 64 bit version of the Hugin UI would address this or if this is an issue with another sub program. Since I don't know where the optimization is taking place or if it is multithreaded or otherwise optimized for multiple core machines I think its not possible for me to answer whether a 64 bit build of Hugin is useful. Yes for big projects the optimisation is a bottleneck as it is single-threaded. As far as I know there is no plan to make this multi-threaded, or even if it is possible. -- Bruno -- You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx
Re: [hugin-ptx] Re: [OSX] Hugin 32bit vs. 64 bit and the way to go forward
Hi Battle and others, I mailed Battle in an off-list mail, and now repeated in < http://old.nabble.com/enblend-out-of-memory-error-td28250757.html>, that Ingemar Bergmark also made openmp enabled enblend/enfuse builds. He did this for (Snow)Leopard ppc/i386/x86_64. So no ppc64 (G5) builds. Harry 2010/4/20 Harry van der Wolf > Hi Battle and others, > > Threading is a long standing wish for Hugin on several areas and might > become one of the GSOC 2010 projects, that's all I can say about it. > > Yesterday evening I built the following stand-alone enblend/enfuse > versions: > - (Snow)Leopard Universal 32bit OpenMP enabled. > - (Snow)Leopard Universal 64bit OpenMP enabled. > - (Snow)Leopard Universal 64bit. > > I already had the 32bit dynamically compiled (Snow)Leopard OpenMP enabled > in the bundle and I already had the 32bit Tiger standalone build without > openmp, as Tiger doesn't support OpenMP. It doesn't make sense to make a > (Snow)Leopard non-openmp enabled version as that would be the same as the > Tiger version. > > I'm now thinking of 2 options: > - All these enblend/enfuse versions will be delivered with the future hugin > builds so that everyone can pick his/her favourite version or "hardware > bound" version. (another 25MB boost of the bundle dmg image). > - Deliver the hugin bundle with 32bit openmp enabled and tiger > enblend/enfuse versions, like 5102. And a separate enblend/enfuse package > like I did in the past. > > This week I will (try to) build a 64bit Hugin version. We could at least > see what it does in (Snow)Leopard. > I say try, as it is at least 1½ years ago that I built the last 64bit > version. > Note also that some part of it will remain 32bit as wxmac (for the gui) has > problems on 64bit, at least it had "some time ago" (something else that > needs analysis). I do not know yet whether the optimize step will be 32bit > or 64bit. I think that's one of the steps compiled into the "big"hugin > itself and would therfor be 32bit as well. Nona will be 64bit as it is a > stand-alone part of Hugin. > > > Harry > > > > 2010/4/19 Battle > > HI Harry, >> Likewise, let me say thanks for making the OSX builds. >> >> Here is my experience on an early 2009 Mac Pro Quad xeon (Nehalem) >> 2.93 MHz intel 24GB RAM 10.6 Snow Leopard. This machine will hyper >> thread so it acts like 8 cores. I realize this puts me on the high end >> of the hugin user base in terms of hardware. None the less... >> >> I downloaded enblend-openmp and enfuse-openmp from Source Forge in 64 >> bit versions (part of enblend 4.0 download), moved the folder to my >> applications folder, and set preferences in Hugin to run the openmp >> versions as alternate enblend and enfuse versions. I'm using your 32 >> bit build svn 5102 to do this. This works great for enblend. I >> haven't tried enfuse. In fact I just built a pano resulting in a >> 3.2GB uncompressed tiff that enblend used up about 12GB real and 12GB >> virtual memory use out of the 24GB real memory on the machine >> according to Activity Monitor. So this enblend 64 bit build clearly >> resolves the enblend out of memory error that I wrote about in another >> post. Also nona takes good advantage of the multiple cores/threads >> on this machine. Nona processed just under 100 12MP jpg images to >> tiff in about 7 minutes or so. That's just under 4.5 seconds an image >> on average reading from one drive and writing to another. Not bad. >> Enblend-openmp 64 bit ran and hugin wrote the final output in about 9 >> minutes, for a total of under 17 minutes overall. Again not bad. >> >> So where are my bottle necks? The optimization process in hugin >> proper in the 32 bit build seems slow. On smaller projects of 10 to >> 20 images the optimization tab can take almost as long as actually >> stitching the project. I don't know whether a 64 bit version of the >> Hugin UI would address this or if this is an issue with another sub >> program. Since I don't know where the optimization is taking place or >> if it is multithreaded or otherwise optimized for multiple core >> machines I think its not possible for me to answer whether a 64 bit >> build of Hugin is useful. I can't find anything in activity monitor >> but a couple of threads in Hugin and an increase in the CPU time. So >> it looks like the optimization is single threaded and only uses one >> cpu. But this is the place where I'd get the most improvement at the >> moment, and it seems to me where increasing capability of the the >> hardware is taking us.Maybe someone else in the community can >> respond to how the optimization works, what the possibilities are, and >> whether 64 bit processing would actually help. >> >> I note that from your site http://panorama.dyndns.org/macoverview.php >> that over 80% of your downloads are intel core 2 or better which will >> run Leopard. My oldest machine at this point is a intel core 2 duo >> running Tiger. I'm a bit too lazy to update it Leopard j
Re: [hugin-ptx] Re: [OSX] Hugin 32bit vs. 64 bit and the way to go forward
Hi Battle and others, Threading is a long standing wish for Hugin on several areas and might become one of the GSOC 2010 projects, that's all I can say about it. Yesterday evening I built the following stand-alone enblend/enfuse versions: - (Snow)Leopard Universal 32bit OpenMP enabled. - (Snow)Leopard Universal 64bit OpenMP enabled. - (Snow)Leopard Universal 64bit. I already had the 32bit dynamically compiled (Snow)Leopard OpenMP enabled in the bundle and I already had the 32bit Tiger standalone build without openmp, as Tiger doesn't support OpenMP. It doesn't make sense to make a (Snow)Leopard non-openmp enabled version as that would be the same as the Tiger version. I'm now thinking of 2 options: - All these enblend/enfuse versions will be delivered with the future hugin builds so that everyone can pick his/her favourite version or "hardware bound" version. (another 25MB boost of the bundle dmg image). - Deliver the hugin bundle with 32bit openmp enabled and tiger enblend/enfuse versions, like 5102. And a separate enblend/enfuse package like I did in the past. This week I will (try to) build a 64bit Hugin version. We could at least see what it does in (Snow)Leopard. I say try, as it is at least 1½ years ago that I built the last 64bit version. Note also that some part of it will remain 32bit as wxmac (for the gui) has problems on 64bit, at least it had "some time ago" (something else that needs analysis). I do not know yet whether the optimize step will be 32bit or 64bit. I think that's one of the steps compiled into the "big"hugin itself and would therfor be 32bit as well. Nona will be 64bit as it is a stand-alone part of Hugin. Harry 2010/4/19 Battle > HI Harry, > Likewise, let me say thanks for making the OSX builds. > > Here is my experience on an early 2009 Mac Pro Quad xeon (Nehalem) > 2.93 MHz intel 24GB RAM 10.6 Snow Leopard. This machine will hyper > thread so it acts like 8 cores. I realize this puts me on the high end > of the hugin user base in terms of hardware. None the less... > > I downloaded enblend-openmp and enfuse-openmp from Source Forge in 64 > bit versions (part of enblend 4.0 download), moved the folder to my > applications folder, and set preferences in Hugin to run the openmp > versions as alternate enblend and enfuse versions. I'm using your 32 > bit build svn 5102 to do this. This works great for enblend. I > haven't tried enfuse. In fact I just built a pano resulting in a > 3.2GB uncompressed tiff that enblend used up about 12GB real and 12GB > virtual memory use out of the 24GB real memory on the machine > according to Activity Monitor. So this enblend 64 bit build clearly > resolves the enblend out of memory error that I wrote about in another > post. Also nona takes good advantage of the multiple cores/threads > on this machine. Nona processed just under 100 12MP jpg images to > tiff in about 7 minutes or so. That's just under 4.5 seconds an image > on average reading from one drive and writing to another. Not bad. > Enblend-openmp 64 bit ran and hugin wrote the final output in about 9 > minutes, for a total of under 17 minutes overall. Again not bad. > > So where are my bottle necks? The optimization process in hugin > proper in the 32 bit build seems slow. On smaller projects of 10 to > 20 images the optimization tab can take almost as long as actually > stitching the project. I don't know whether a 64 bit version of the > Hugin UI would address this or if this is an issue with another sub > program. Since I don't know where the optimization is taking place or > if it is multithreaded or otherwise optimized for multiple core > machines I think its not possible for me to answer whether a 64 bit > build of Hugin is useful. I can't find anything in activity monitor > but a couple of threads in Hugin and an increase in the CPU time. So > it looks like the optimization is single threaded and only uses one > cpu. But this is the place where I'd get the most improvement at the > moment, and it seems to me where increasing capability of the the > hardware is taking us.Maybe someone else in the community can > respond to how the optimization works, what the possibilities are, and > whether 64 bit processing would actually help. > > I note that from your site http://panorama.dyndns.org/macoverview.php > that over 80% of your downloads are intel core 2 or better which will > run Leopard. My oldest machine at this point is a intel core 2 duo > running Tiger. I'm a bit too lazy to update it Leopard just because > its a home machine that is only used for email, internet, and > generating light correspondence mostly by other family members. But > if I needed to run hugin on it and could get the the productivity > gains of 64 bit enblend I wouldn't hesitate for a moment to go to > Leopard. > > Thanks again for your OSX builds. > Battle > > > On Apr 17, 5:53 am, Harry van der Wolf wrote: > > Hi mac users, > > > > (A long mail). > > > > I have not
[hugin-ptx] Re: [OSX] Hugin 32bit vs. 64 bit and the way to go forward
HI Harry, Likewise, let me say thanks for making the OSX builds. Here is my experience on an early 2009 Mac Pro Quad xeon (Nehalem) 2.93 MHz intel 24GB RAM 10.6 Snow Leopard. This machine will hyper thread so it acts like 8 cores. I realize this puts me on the high end of the hugin user base in terms of hardware. None the less... I downloaded enblend-openmp and enfuse-openmp from Source Forge in 64 bit versions (part of enblend 4.0 download), moved the folder to my applications folder, and set preferences in Hugin to run the openmp versions as alternate enblend and enfuse versions. I'm using your 32 bit build svn 5102 to do this. This works great for enblend. I haven't tried enfuse. In fact I just built a pano resulting in a 3.2GB uncompressed tiff that enblend used up about 12GB real and 12GB virtual memory use out of the 24GB real memory on the machine according to Activity Monitor. So this enblend 64 bit build clearly resolves the enblend out of memory error that I wrote about in another post. Also nona takes good advantage of the multiple cores/threads on this machine. Nona processed just under 100 12MP jpg images to tiff in about 7 minutes or so. That's just under 4.5 seconds an image on average reading from one drive and writing to another. Not bad. Enblend-openmp 64 bit ran and hugin wrote the final output in about 9 minutes, for a total of under 17 minutes overall. Again not bad. So where are my bottle necks? The optimization process in hugin proper in the 32 bit build seems slow. On smaller projects of 10 to 20 images the optimization tab can take almost as long as actually stitching the project. I don't know whether a 64 bit version of the Hugin UI would address this or if this is an issue with another sub program. Since I don't know where the optimization is taking place or if it is multithreaded or otherwise optimized for multiple core machines I think its not possible for me to answer whether a 64 bit build of Hugin is useful. I can't find anything in activity monitor but a couple of threads in Hugin and an increase in the CPU time. So it looks like the optimization is single threaded and only uses one cpu. But this is the place where I'd get the most improvement at the moment, and it seems to me where increasing capability of the the hardware is taking us.Maybe someone else in the community can respond to how the optimization works, what the possibilities are, and whether 64 bit processing would actually help. I note that from your site http://panorama.dyndns.org/macoverview.php that over 80% of your downloads are intel core 2 or better which will run Leopard. My oldest machine at this point is a intel core 2 duo running Tiger. I'm a bit too lazy to update it Leopard just because its a home machine that is only used for email, internet, and generating light correspondence mostly by other family members. But if I needed to run hugin on it and could get the the productivity gains of 64 bit enblend I wouldn't hesitate for a moment to go to Leopard. Thanks again for your OSX builds. Battle On Apr 17, 5:53 am, Harry van der Wolf wrote: > Hi mac users, > > (A long mail). > > I have not made up my mind what to do next with future builds, maybe you > have ideas for me. Please do not consider this as a wish list for your > special requirements. I'm trying to make up my mind and I hope you can help > me with that. > > Some remarks to what bothers me: > - Currently I don't want to build an integrated 4 architecture 32bit/64bit > (ppc/i386/ppc64/x86_64) Hugin bundle as it will pump up the bundle to 200+ > MB as it will mean that every library/binary has 4 architectures inside. > Besides it's a lot of extra work and especially the 64bit libraries sometime > require extra tweaking. Next to that the enblend/enfuse w.r.t. "openmp" will > remain a pain in these builds (next remark). Also startup times will > increase. > - I don't want to maintain a 32bit bundle for Tiger on ppc/intel mono-core, > a 32bit enblend/enfuse openmp enabled build for G4/intel monocore on Leopard > and a 64bit enblend/enfuse openmp enabled bundle for (Snow)Leopard on > G5/Intel duo/quad/8-core. Note that hugin is dynamically linked which means > that all binaries/libraries use the same libraries, which means that these > libraries need to be all in the same architectures with the same build > options, which means that I would need 3 huge separated build environments: > No way. > - Building 64bit for G5 (ppc64) is becoming harder compared to building 64 > bit on intel (x86_64), as G5 will slowly disappear and newer libs with newer > options no longer support it. > - Building a 64bit bundle brings hardly any benefits on (Snow)Leopard, no > matter what Apple "markets" about 64bit SnowLeopard(*), (1,2,3,4,5). > - Only enblend, enfuse and to some extent nona will profit from a 64bit > build w.r.t. speed and memory address space (despite my previous remark). > - I'm thinking as well of building a triple
Re: [hugin-ptx] Re: [OSX] Hugin 32bit vs. 64 bit and the way to go forward
Hello Martin, Thanks for your reply. 2010/4/18 Martin Middelhoek > Hallo Harry, > > As a Mac programmer myself I feel I must respectfully disagree with > your remarks on the 32bits kernel in SL. The 4GB memory limit only > exists for the kernel itself and its extensions, not for the programs > running on it. This kernel limit is not considered a problem > (certainly not by Apple) as the kernel has no serious data space > requirement anyway. For now the only reason for running the 64bit > kernel would be a small increase in speed. > > A 64bit user program can claim all the RAM in the system, which I > found out the hard way when a bug in my own program exhausted the full > 12GB in my Mac Pro and then tried to eat my HD to. Since my MP is the > old 1,1 it can only run the 32bit kernel. I also run Aperture3 which > has no problem using all the RAM. > > To start with: I'm definitely not a programmer (like you), at least not in C/C++. After your mail I started googling again, but this time not on general and dev (forum)discussions, but directly on applications like photoshop and aperture and so on. They all mention the fact that their new shiny 64bit versions can use the 64bit address space on SnowLeopard and duo core or better, and already tests are available to show speed comparisons and so on. So, it really helps to change your search criteria once in a while when searching on the same topic. > I agree with you that for Hugin itself there is probably not much to > be gained as it is mostly UI, and you would have to load a lot of > images to exceed the 4GB limit. Only the Gigapixel people might > disagree. Enblend&Enfuse are a different story. Speedwise don't expect > much, and personally I will only put the effort in for a factor of at > least 2. However, E&E do run out of memory on a regular basis, most > often when going to HDR on a full 360/180 panorama with, say, 5 > exposures. The new 32bit openmp enblend throws a memory error almost > every time, so I had to return to an earlier version. The memory > request shown in error message seems a bit excessive most of the time, > so I am glad that it can not be honoured in 32bit. Maybe it would be > better to return to a 64bit version of non-openml E&E until openml is > more stable. > > After having googled for (marketing info on ) 64 applications I started to build on a 64bit version (Despite my own words in my starting mail). I'm not succesfull in compiling libxmi correctly in 64bit pp64 yet (some of the slowly degrading support for pp64 I think) and I can't get an x86_64 E&E compiled with openmp. No problem to compile a 32bit version though. Some unsolved business there. I'm still thinking about all kinds of options without being too busy with it. > Xcode on SL can still target Tiger according to the release notes, > although I never tried it myself. So no reason there to postpone > upgrading your system to 10.6. I already heard from Skip Gaede, who is on 10.6, that simply using an older XCode version (not the Snow Leopard version) simply solves it. Hoi, Harry -- You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx
[hugin-ptx] Re: [OSX] Hugin 32bit vs. 64 bit and the way to go forward
Hallo Harry, let me start by thanking you for providing the Mac builds of this great program for us leaches ;-) As a Mac programmer myself I feel I must respectfully disagree with your remarks on the 32bits kernel in SL. The 4GB memory limit only exists for the kernel itself and its extensions, not for the programs running on it. This kernel limit is not considered a problem (certainly not by Apple) as the kernel has no serious data space requirement anyway. For now the only reason for running the 64bit kernel would be a small increase in speed. A 64bit user program can claim all the RAM in the system, which I found out the hard way when a bug in my own program exhausted the full 12GB in my Mac Pro and then tried to eat my HD to. Since my MP is the old 1,1 it can only run the 32bit kernel. I also run Aperture3 which has no problem using all the RAM. I agree with you that for Hugin itself there is probably not much to be gained as it is mostly UI, and you would have to load a lot of images to exceed the 4GB limit. Only the Gigapixel people might disagree. Enblend&Enfuse are a different story. Speedwise don't expect much, and personally I will only put the effort in for a factor of at least 2. However, E&E do run out of memory on a regular basis, most often when going to HDR on a full 360/180 panorama with, say, 5 exposures. The new 32bit openmp enblend throws a memory error almost every time, so I had to return to an earlier version. The memory request shown in error message seems a bit excessive most of the time, so I am glad that it can not be honoured in 32bit. Maybe it would be better to return to a 64bit version of non-openml E&E until openml is more stable. Xcode on SL can still target Tiger according to the release notes, although I never tried it myself. So no reason there to postpone upgrading your system to 10.6. However, I must admit that only a "clean" install of 10.6 on a fresh HD resulted in a stable system for me. I suspect that this was caused by all the leftovers from an earlier 10.4 to 10.5 migration. Oh well, a good reason for some spring cleaning. -- You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx
