[hugin-ptx] Re: tilt functions
2009/9/26 Brent > > > I also experimented with varying d,e (it seems that letting them vary > per-image doesn't help anything which is consistent with Pablo's > explanation), and jointly optimizing with some of the lens parameters > (a,b,c). In the latter case, the optimization usually fell apart > completely with RMS errors > 20. > > *Lens model* We really need to be clear about the definitions of the parameters, In my *guess *of how the lens model now works it seems quite logical that optimizing d &e as representing camera movement together with lens parameter must fail. This is also the case in the previous lens model as: d & e represents x,y shift of the sensor from the optical axis of the lens. That is, the senser is moved arouns, *not *the whole camera with the lens. If there is any lens distorsion( which depends on the radius from the optitical axis) Distorsion will affect images more and more as they get further away from the anchor image. I assume that his has happened in the XYZ model (correct pablo?) 1. TrX,TrY,TrZ represents a shift om camera position and thus *optical axis * of the actual *lens number* of that position 2. y, p ,r - have been *changed *to apply only to the actual optical axis of that particular lens lens number (Trx, Try) 3. a, b ,c lens distorsion parameters are applied in origo of the optical axis (Trx, Try) 4. d, e as before represents sensor shift X,Y from optical axis for each lens (Trx, Try) Interesting here is that there seems to be room for a "camera" definition in conjunction with a "lens" definition. In a linear panorama/mosaic Lens parameters a,b,c,d,e are same while camera position changes. (If my assumption on 2 above is wrong, Tix,Tiy and Tiz are needed at the same time) /Cheers O --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx -~--~~~~--~~--~--~---
[hugin-ptx] Re: tilt functions
Ahh, I misinterpreted what you meant by multi-step optimization. I can see how incrementally adding images can help out the minimization. And I see what you mean about higher-dimensional solution spaces having more local minima traps. Though minimizing a subset of the variables with the other ones set to the "wrong" values, can also result in going off into a bumpy region of the solution space where there are local minimum traps. But what's odd is that Hugin is sometimes able to get out of traps by reducing the number of variables being optimized after accepting the results of a previous run with more variables. In that case, if you're in a local minimum in the higher-dimensional space, and the error function is unchanged, then simply optimizing with less variables shouldn't be able to help. But it does seem to. On Sep 28, 10:08 pm, Pablo d'Angelo wrote: > Brent wrote: > > Pablo, > > Now that I think about it, there is something odd in the fact that > > doing a multi-step optimization by changing which variables are > > optimized works any better than a full optimization. > > Maybe my explanation in the previous email was a bit misleading. What I > meant is to start optimizing with optimizing two images (with good > connections etc.), then add the third, optimize again, add the fourth > etc. This is done by the incremental optimization in hugin. (Note that > here, control points in the non-optimized images are ignored). > > > That is, if the > > current solution is in a local minima with respect to 10 variables, > > for example, then it will still be in a local minima if fewer than the > > full number is optimized starting from that point. > > However, there might be other local minima, which can also be found by > the optimizer, as its way though the solution space is changed, and some > "shortcuts" might be blocked (such as modifying lens distortions and d,e > shift to make images fit better during the first iterations), might send > the iterative optimization into the wrong direction, from which it might > be unable to recover. Then the first local minima of will be different, > and thus later optimization with more parameters will also find another > solution. > > > If there's a way > > > out of trap when optimizing a subset of the variables, then that same > > way out is available when more variables are allowed to vary. > > Hmm, I don't think there are always ways of of local minimas, and adding > some new parameters to the optimisation do not nessecarily mean that the > previous local minima ceases to be one in the new parameter space. I agree with what you're saying here, but what I meant was the converse. Reducing the number of variables to be optimized does not help get out of a local minimum, but it may help get out of a shallow valley which is not a true local minimum. If that's what's happening sometimes, then it should be fixable with the convergence tolerances since it is not a true local minimum. > > ciao > Pablo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx -~--~~~~--~~--~--~---
[hugin-ptx] Re: tilt functions
Brent wrote: > Pablo, >Now that I think about it, there is something odd in the fact that > doing a multi-step optimization by changing which variables are > optimized works any better than a full optimization. Maybe my explanation in the previous email was a bit misleading. What I meant is to start optimizing with optimizing two images (with good connections etc.), then add the third, optimize again, add the fourth etc. This is done by the incremental optimization in hugin. (Note that here, control points in the non-optimized images are ignored). > That is, if the > current solution is in a local minima with respect to 10 variables, > for example, then it will still be in a local minima if fewer than the > full number is optimized starting from that point. However, there might be other local minima, which can also be found by the optimizer, as its way though the solution space is changed, and some "shortcuts" might be blocked (such as modifying lens distortions and d,e shift to make images fit better during the first iterations), might send the iterative optimization into the wrong direction, from which it might be unable to recover. Then the first local minima of will be different, and thus later optimization with more parameters will also find another solution. > If there's a way > out of trap when optimizing a subset of the variables, then that same > way out is available when more variables are allowed to vary. Hmm, I don't think there are always ways of of local minimas, and adding some new parameters to the optimisation do not nessecarily mean that the previous local minima ceases to be one in the new parameter space. ciao Pablo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx -~--~~~~--~~--~--~---
[hugin-ptx] Re: tilt functions
Pablo, Now that I think about it, there is something odd in the fact that doing a multi-step optimization by changing which variables are optimized works any better than a full optimization. That is, if the current solution is in a local minima with respect to 10 variables, for example, then it will still be in a local minima if fewer than the full number is optimized starting from that point. If there's a way out of trap when optimizing a subset of the variables, then that same way out is available when more variables are allowed to vary. So then, why does manually choosing some subsets of variables in Hugin seem to work better than just letting it optimize everything from the start? One possibility might be that there are very gently sloped valleys out of the local minima that are below the optimizer tolerance so it stops, whereas when there are less variables being optimized that slope looks relatively larger and above the tolerance. It might make sense to try some experiments on a dataset we know finds a good minima with multi-step optimization but not with one-shot and see if changing the convergence tolerances allows the one-shot to work. Brent On Sep 27, 11:58 am, Pablo d'Angelo wrote: > Hi Brent, > > Brent schrieb: > > > From a purely empirical point of view, it appears that there is some > > undesirable linkage between these parameters that makes the > > minimization process get trapped into local minima and have difficulty > > finding a solution in some cases. > > I have had the same problem, with both the tilt parameters (TiX,TiY,TiZ, > TiS) and the position parameters (TrX, TrY, TrZ). The problem that the > optimizer gets trapped in local minima isn't really something new, so we > have worked a bit around that with multi-step optimization, and > especially the incremental optimization in hugin and autooptimizer (cmd > line optimisation program, part of the hugin package). I found that with > adding image by image to the optimisation process, I didn't get really > bad random behaviour anymore, even when later "finetuning" the project > and doing reoptimisations, as the project is already close to a nice > minima (I hope). > > As for correlation between the parameters, I quite sure that d,e and > TrX, TrY are highly correlated when using rectilinear images, especially > if they are shot without large variation in yaw and pitch. > > In my experiements with the graffiti wall example posted by Bruno, I got > some problems when optimizing d,e (linked between all images), even if > the solution before was quite good, and his images contain shots from > many different angles. > > I haven't expected an high correlation between p and TrZ though, so > this is interesting and should be analyzed in more detail. It might just > be that the starting point in the solution space is very far from the > global optima. > > ciao > Pablo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx -~--~~~~--~~--~--~---
[hugin-ptx] Re: tilt functions
Hi Brent, Brent schrieb: > From a purely empirical point of view, it appears that there is some > undesirable linkage between these parameters that makes the > minimization process get trapped into local minima and have difficulty > finding a solution in some cases. I have had the same problem, with both the tilt parameters (TiX,TiY,TiZ, TiS) and the position parameters (TrX, TrY, TrZ). The problem that the optimizer gets trapped in local minima isn't really something new, so we have worked a bit around that with multi-step optimization, and especially the incremental optimization in hugin and autooptimizer (cmd line optimisation program, part of the hugin package). I found that with adding image by image to the optimisation process, I didn't get really bad random behaviour anymore, even when later "finetuning" the project and doing reoptimisations, as the project is already close to a nice minima (I hope). As for correlation between the parameters, I quite sure that d,e and TrX, TrY are highly correlated when using rectilinear images, especially if they are shot without large variation in yaw and pitch. In my experiements with the graffiti wall example posted by Bruno, I got some problems when optimizing d,e (linked between all images), even if the solution before was quite good, and his images contain shots from many different angles. I haven't expected an high correlation between p and TrZ though, so this is interesting and should be analyzed in more detail. It might just be that the starting point in the solution space is very far from the global optima. ciao Pablo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx -~--~~~~--~~--~--~---
[hugin-ptx] Re: tilt functions
> > The bottom line is that although I could eventually find a solution > which seems good, I felt that the sensitivity to small changes in the > optimizer's ability to find a good solution makes it difficult to > solve the system and also makes it hard to know when the solution is > good. It may just be that the solution space with these parameters > is quite bumpy and prone to local minima, but it may also be that the > parameter choice results in high correlations between some of the > parameters. > I fully agree. Although I don't quite understand the optimizer, I believe it does not discriminate on which parameters are more important. ANother problem is that a bad control point might be nicely optimized (small error) and increase error in better points. I would really like to be able to say: "I know this control is good, so minimize the error of this point at the expense of others that I am not so sure about". > Brent > > > On Sep 24, 11:39 am, Pablo d'Angelo wrote: >> Jim Watters schrieb: >> >> > Pablo d'Angelo wrote: >> >> > If I understand correctly both models will align an image so it is on >> > the same plane as others. >> >> After some more reading of the code, I'm quite sure that the tilt can >> only work well with rectilinear input images. If only that case is >> interesting, one doesn't even need angles, a simple 3x3 homography >> matrix transform is sufficient, and probably more stable than the tilt >> model. >> >> > The XYZ model calculates the position of the camera, this will cause >> > certain distortions to the image. XYZ have to be optimized with ypr. >> > The TxTyTzTs model calculates the actual distortion to align the image >> > after it has been placed correctly after optimizing ypr. >> >> I don't believe that ypr and the TxTyTzTs are independent. For the shift >> compensation, the TxTyTzTs model will also require simultanious >> optimisation of d and e. >> >> > The advantage to the XYZ model is it uses fewer overall parameters. >> >> > But when I try imagine manipulating images in my head using TxTyTzTs i >> > don't see how Tz is any different than r. And maybe Tz it is not >> > necessary. The advantage to the TxTyTs model is ypr can be optimized >> > first to get a good placement before distorting the image by optimizing >> > TxTyTs overall needing fewer control points. >> >> You still need Tx, Ty, Ts and d,e. This assumes that you do not need to >> touch y,p,r from a previous optimisation. However, I think y,p,r will >> try to compensate for the parallax, and they will thus be quite wrong, >> in an attempt to rescue the situation. I'm not sure that a later tilt >> estimation works satisfactory with the previously estimated y,p,r >> parameters. Maybe one can get away, if the camera hasn't been moved too far. >> >> My experience with the tilt model was not so good. I have prepared a >> nice testset with the aerial images posted earlier, and it works quite >> well with the XYZ mode. I'll give more details later. >> >> ciao >> Pablo > > > -- --dmg --- Daniel M. German http://turingmachine.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx -~--~~~~--~~--~--~---
[hugin-ptx] Re: tilt functions
>From a purely empirical point of view, it appears that there is some
undesirable linkage between these parameters that makes the
minimization process get trapped into local minima and have difficulty
finding a solution in some cases.
My basis for saying that is that I've been running some test cases of
photos of a plane taken from a constant at varying angles, starting
from overhead (from a height of 1.5m) and then moving in one direction
a fixed step size (about 0.3m). Each photo was centered on the same
point and the resulting pitch changed by about 10º per photo. ASC
and panomatic both found lots of control points easily and I pruned
out any bad ones. Optimizing the result turned out to be more
difficult than I expected with the relatively clean data. I was
optimizing y,p,r,TrX,TrY,TrZ for eight of the frames and the ninth
one, the overhead one, was left fixed with 0 values for these
parameters. What I saw was wildly different results in the output
after small changes. For example, PToptimizer would give an RMS error
of ~15 from one starting point and then I would delete one of the
control points (there were over 400) and then end up with a completely
different result with an error of ~5. Other small changes resulted in
results that approx. equal RMS error but very different solutions.
It seemed that there was a tradeoff occurring between the value of p
and the value of Tz. In some solutions p would vary considerably (0
to 58) and seem about right, with Tr{X,Y,Z} varying between 0 and
0.18. Another solution, with similar error, kept p between -7 and +7,
but Tr{X,Y,Z} varying from 0 to 3.6.
I also experimented with varying d,e (it seems that letting them vary
per-image doesn't help anything which is consistent with Pablo's
explanation), and jointly optimizing with some of the lens parameters
(a,b,c). In the latter case, the optimization usually fell apart
completely with RMS errors > 20.
The bottom line is that although I could eventually find a solution
which seems good, I felt that the sensitivity to small changes in the
optimizer's ability to find a good solution makes it difficult to
solve the system and also makes it hard to know when the solution is
good. It may just be that the solution space with these parameters
is quite bumpy and prone to local minima, but it may also be that the
parameter choice results in high correlations between some of the
parameters.
Brent
On Sep 24, 11:39 am, Pablo d'Angelo wrote:
> Jim Watters schrieb:
>
> > Pablo d'Angelo wrote:
>
> > If I understand correctly both models will align an image so it is on
> > the same plane as others.
>
> After some more reading of the code, I'm quite sure that the tilt can
> only work well with rectilinear input images. If only that case is
> interesting, one doesn't even need angles, a simple 3x3 homography
> matrix transform is sufficient, and probably more stable than the tilt
> model.
>
> > The XYZ model calculates the position of the camera, this will cause
> > certain distortions to the image. XYZ have to be optimized with ypr.
> > The TxTyTzTs model calculates the actual distortion to align the image
> > after it has been placed correctly after optimizing ypr.
>
> I don't believe that ypr and the TxTyTzTs are independent. For the shift
> compensation, the TxTyTzTs model will also require simultanious
> optimisation of d and e.
>
> > The advantage to the XYZ model is it uses fewer overall parameters.
>
> > But when I try imagine manipulating images in my head using TxTyTzTs i
> > don't see how Tz is any different than r. And maybe Tz it is not
> > necessary. The advantage to the TxTyTs model is ypr can be optimized
> > first to get a good placement before distorting the image by optimizing
> > TxTyTs overall needing fewer control points.
>
> You still need Tx, Ty, Ts and d,e. This assumes that you do not need to
> touch y,p,r from a previous optimisation. However, I think y,p,r will
> try to compensate for the parallax, and they will thus be quite wrong,
> in an attempt to rescue the situation. I'm not sure that a later tilt
> estimation works satisfactory with the previously estimated y,p,r
> parameters. Maybe one can get away, if the camera hasn't been moved too far.
>
> My experience with the tilt model was not so good. I have prepared a
> nice testset with the aerial images posted earlier, and it works quite
> well with the XYZ mode. I'll give more details later.
>
> ciao
> Pablo
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"hugin and other free panoramic software" group.
A list of frequently asked questions is available at:
http://wiki.panotools.org/Hugin_FAQ
To post to this group, send email to hugin-ptx@googlegroups.com
To unsubscribe from this group, send email to
hugin-ptx-unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/hugin-ptx
-~--~~~
[hugin-ptx] Re: tilt functions
Jim Watters schrieb: > Pablo d'Angelo wrote: >> > If I understand correctly both models will align an image so it is on > the same plane as others. After some more reading of the code, I'm quite sure that the tilt can only work well with rectilinear input images. If only that case is interesting, one doesn't even need angles, a simple 3x3 homography matrix transform is sufficient, and probably more stable than the tilt model. > The XYZ model calculates the position of the camera, this will cause > certain distortions to the image. XYZ have to be optimized with ypr. > The TxTyTzTs model calculates the actual distortion to align the image > after it has been placed correctly after optimizing ypr. I don't believe that ypr and the TxTyTzTs are independent. For the shift compensation, the TxTyTzTs model will also require simultanious optimisation of d and e. > The advantage to the XYZ model is it uses fewer overall parameters. > > But when I try imagine manipulating images in my head using TxTyTzTs i > don't see how Tz is any different than r. And maybe Tz it is not > necessary. The advantage to the TxTyTs model is ypr can be optimized > first to get a good placement before distorting the image by optimizing > TxTyTs overall needing fewer control points. You still need Tx, Ty, Ts and d,e. This assumes that you do not need to touch y,p,r from a previous optimisation. However, I think y,p,r will try to compensate for the parallax, and they will thus be quite wrong, in an attempt to rescue the situation. I'm not sure that a later tilt estimation works satisfactory with the previously estimated y,p,r parameters. Maybe one can get away, if the camera hasn't been moved too far. My experience with the tilt model was not so good. I have prepared a nice testset with the aerial images posted earlier, and it works quite well with the XYZ mode. I'll give more details later. ciao Pablo --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group. A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ To post to this group, send email to hugin-ptx@googlegroups.com To unsubscribe from this group, send email to hugin-ptx-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/hugin-ptx -~--~~~~--~~--~--~---
