Hi all, I have recently been playing around with this issue also. Has your hangout happened already? Is there a summary of conclusions planned or existing?
Thanks! On Tue, Aug 5, 2014 at 9:03 AM, Bradley Lowekamp <[email protected]> wrote: > Ali, > > Glad to hear the registration iteration graphs where effective. > > Yes, you are right you can initialize the parameters of the second stage > affine registration. Specifically the Center. However, it not readily > apparent ( no example, no filter/initializer ) how that center fixed > parameter should be initialized. And the "CenteredTransformInitializer" is > not the correct way. > > For a Composite transformed defined as ϕ=T0(T1(x)) where ϕ:Ωfixed→Ωmoving then > if C such that C∈Ωmoving is the correct center initialization for T1, > then the correct center initialization for T0 when in the composite > transform is C′=T−10(C). > > Cool pasting from the IPython Notebook with inline latex appears to have > worked :) > > While it is easy to do this with the MomentsCaluclator and manually doing > the transform inverse, I think to make it more readily apparent and easier > to use it should be available as a filter/initializer. > > There are several things that are done manually in the examples that I > would think should be updated to use the latest filters and initializers. > For example The BSplines are manually initialized instead of using the > BSplineTransform Initializer[1]. Additionally, conversion of BSpline to a > deformation field[2], and the the upsampling of the BSpine[3], though that > last one still looks like ITKv3 framework, so maybe it'll be updated to the > adaptors with v4. However I am after simple functions for these common > tasks for SimpleITK, so I have a bit different perspective then one who is > OK with implementing a 50 line solution in C++ and being OK with the > interface. > > We'll discuss these initialization issue further this afternoon. > > Brad > > [1] > https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/DeformableRegistration4.cxx#L215-L235 > [2] > https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/DeformableRegistration4.cxx#L401-L430 > [3] > https://github.com/InsightSoftwareConsortium/ITK/blob/master/Examples/RegistrationITKv4/DeformableRegistration6.cxx#L273-L330 > > On Aug 4, 2014, at 11:56 PM, Ghayoor, Ali <[email protected]> wrote: > > Hello Brad, > > It is an awesome way for demonstration, and thanks for verification of > my example. It seems that this email is sent on Saturday, but unfortunately > I got that this morning! And I should inspect the reason! > Anyway, over the weekend I ran almost the same experiment in ITK and got > close results to you. > > Also, today I tried to make the registration process work when Affine > Transform is used in a multi stage structure, and here is my suggestion: > > I think we do not need to bother ourselves to think about a new > optimizer or a new initializer for the composite transform since we can > initialize the fixed parameters of each transform individually at the > beginning of each stage. > > Remember how we run a BSpline registration stage after some linear > registrations. At the beginning of the BSpline stage, we should instantiate > a BSpline transform object and define its fixed parameters (grid size, > etc). Then, we pass this initialized transform to the registration filter > using “SetInitialTransform()”, yet we can initialize this stage by passing > a composite transform, resultant from the previous stages, to the > registration filter by “SetMovingInitialTransform()”. > > The same condition holds for all other transform types with fixed > unoptimizable parameters like the center of rotation in our Affine > transform case. Attachment file includes a multi stage registration example > that I have created for the new software guide. In this example, we use a > simple 2 stages registration process for a multi modal problem that moving > image is misaligned from the fixed image by a translation shift and > rotation. In registration process, a translation transform is followed by > an affine transform. Also, an initial transform is used at the beginning. > > If I do not set the center of the affine transform, the registration > fails as you have shown in your Ipython example. However, if at the > beginning of the second stage, I compute the geometrical center (or center > of the mass) of the fixed image, and pass that to the affine transform > using “SetCenter()” or “SetFixedParameters()”, the registration succeeds. > Please take a quick look in the attached file that also includes lots of > explanations. > > Therefore, we can follow the same procedure for all other transform > types that their fixed parameters are important in the optimization, but > those fixed parameters cannot be set explicitly inside a composite > transform. Please let me know with you think about this. > > Thank you, > Ali > > > From: Bradley Lowekamp <[email protected]> > Date: Saturday, August 2, 2014 at 6:48 PM > To: Ali Ghayoor <[email protected]> > Cc: Hans Johnson <[email protected]>, Insight Developers List < > [email protected]> > Subject: Re: [ITK-dev] [ITK] Optimizing composite transforms and center > of transform > > Hello Ali, > > I created a notebook to try to demonstrate the problem you are > describing: > > http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/blob/master/Importance%20of%20Center%20with%20Affine%20Registration.ipynb > > If the center of an Affine transformation isn't correctly initialized > then the optimization is not speedy or robust. I think it's best > illustrated with these two graphs: > > <unknown.png> > <unknown.png> > > Notice that with the Center being the origin, all 1000 iterations are > exhausted, and the metric is slowing decreasing. And this is with the > translation path being around the length to 10. If the spacing was bigger > or more offset from the origin the situation would be worse. > > I don't think this is a bug with the registration framework. I think new > initializers for composite transforms are needed, along with documentation > how the 2 good ways to initialize the registration methods transforms. > > > Brad > > On Aug 1, 2014, at 8:08 PM, Ghayoor, Ali <[email protected]> wrote: > > Brad, > > I have exactly the same issue. I wrote a simple registration program that > is run in two stages: > -Translation > -Affine > > My fixed and moving images are in 2D, and the moving image is created from > the fixed image by > rotation of 10 degree and translation of [13,17] in X and Y directions. > > The first stage does a translation registration and the result transform > is added to a composite transform > that is used as the initial moving transform of the second stage. > > After the registration the result transform of the second stage is also > added to the composite transform > to be used by the resampler. > > I have printed the composite transform to the screen: > > 2481: CompositeTransform (0x7fa359a18b00) > 2481: RTTI typeinfo: itk::CompositeTransform<double, 2u> > 2481: Reference Count: 2 > 2481: Modified Time: 20949 > 2481: Debug: Off > 2481: Object Name: > 2481: Observers: > 2481: none > 2481: Transforms in queue, from begin to end: > 2481: >>>>>>>>> > 2481: TranslationTransform (0x7fa359a199b0) > 2481: RTTI typeinfo: itk::TranslationTransform<double, 2u> > 2481: Reference Count: 7 > 2481: Modified Time: 1648 > 2481: Debug: Off > 2481: Object Name: > 2481: Observers: > 2481: none > 2481: Offset: [12.801, 15.8578] > 2481: >>>>>>>>> > 2481: AffineTransform (0x7fa359d095a0) > 2481: RTTI typeinfo: itk::AffineTransform<double, 2u> > 2481: Reference Count: 5 > 2481: Modified Time: 20946 > 2481: Debug: Off > 2481: Object Name: > 2481: Observers: > 2481: none > 2481: Matrix: > 2481: 1.01593 -0.0100051 > 2481: -0.00835669 1.01113 > 2481: Offset: [-0.358829, -0.290093] > 2481: Center: [0, 0] > 2481: Translation: [-0.358829, -0.290093] > 2481: Inverse: > 2481: 0.984397 0.00974056 > 2481: 0.00813575 0.989073 > 2481: Singular: 0 > 2481: End of MultiTransform. > 2481: <<<<<<<<<< > > As it can be seen, the translation transform provides a good > initialization since its offset (Offset: [12.801, 15.8578]) is close to the > shift parameters. > > However, the Affine transform fails to rotate the moving image since its > Matrix is almost identity. > Possible reason can be the improper Center of Affine transform that is > fixed to [0,0], and not at geometrical center or center of mass. > > To make sure that I have not made any crazy mistake in my example, I > simulated my example parameters in ANTs, and ANTs returned > the same results and failed to register the images!! > > ANTs command line <<<<<<<<<<< > > > PROGPATH=/scratch/ANTS/release-new/bin > > fi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainT1SliceBorder20.png > > mi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainProtonDensitySliceR10X13Y17.png > > time ${PROGPATH}/antsRegistration -d 2 \ > --float \ > --output [test1, warpedMoving.nii.gz] \ > --transform "Translation[16]" \ > --metric MI[${fi},${mi},1,32,None,1] \ > --convergence [100,1e-2,5] \ > --shrink-factors 3 \ > --smoothing-sigmas 2 \ > --use-histogram-matching 1 \ > \ > --transform "Affine[1]" \ > --metric MI[${fi},${mi},1,32] \ > --convergence [100x100,1e-2,2] \ > --shrink-factors 2x1 \ > --smoothing-sigmas 1x0 \ > --use-histogram-matching 1 > > <<<<<<<<<<<<<<<<<<<<<<<< > > Fixed and moving images can be loaded from the ITK data files. What do you > think about this example? Is it a bug in ITK or just the parameters are not > tuned finely? > > Thanks, > Ali > > ________________________________________ > From: Johnson, Hans J > Sent: Tuesday, July 29, 2014 6:19 PM > To: Ghayoor, Ali > Subject: FW: [ITK-dev] [ITK] Optimizing composite transforms and center of > transform > > Ali, > > You need to follow this discussion. > > Hans > > On 7/29/14, 6:16 PM, "Bradley Lowekamp" <[email protected]> wrote: > > Hello, > > Ok, to further explore this problem, and hopeful help other people too: I > created an IPython Notebook viewable here: > http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/bl > ob/master/Transform%20Composition%20And%20Order.ipynb > > It doesn't come through on the static page, but I added some nifty > interactive slider widgets to change the rotation parameter. Which is > really the point to enable understanding of the parameters for the > transform and how they should be best optimized. The notebook should work > with SimpleITK >= 0.8 and IPython Notebooks 2.1. I'll encourage every one > to try it out :) IPython Notebooks widget are very cool! > > EXPLANATION: > > The the order of the composite transform is that the new transforms are > applied first. > > BUT they map from the virtual domain ( fixed ) to the moving image. > Therefore if your first transform moves the center of mass to the origin > then the second added transform's space will be centered on the mass, > which is good for optimizing rotation and affine parameters. > > QUESTION: > > Therefore I am inclined to conclude that its best practices to map the > fixed and moving image to the virtual domain such that the center of mass > ( or some other "center feature" ) are at the origin. This would then the > scale, rotation and other affine parameters around the center, with out > having to used the "Center" parameter the transforms currently have. > > Is this what others are doing in their registration? For general best > practices should this be recommend? > > @Brian I know I have seen this multi-domain description many time, but I > think I may have just gotten it... Is the well describe in the literature > some place? I think this may be an important part add the software guide. > > Thanks, > Brad > > > > > On Jul 29, 2014, at 5:38 PM, Matt McCormick <[email protected]> > wrote: > > Hi Brad, > > Your assessment that the fixed center is important is correct. > > In most cases, a transform with a Center is the first transform in a > Composite transform. There not any issues here. However, if the > transform with a Center is further along the chain, then a > CenteredTransformInitializer that was applied before the registration > is started will not generate the desired result. In this case, we > would have to respond to an Event in the registration process and > estimated the center on the imaged after the other transforms have > been applied. It is more work, but it is a more unusual case. > > HTH, > Matt > > > > On Mon, Jul 28, 2014 at 10:11 PM, Bradley Lowekamp > <[email protected]> wrote: > > Helloo Nick! > > I am glad you got back to me. > > I suspect that I have spent more time this past week looking at the > ITK affine transforms than anyone else[1] this week. With that here is > my current understanding... > > 1) The CenterTransformInitializers estimates an initial "Center" and > an initial "Translation" parameter from either the geometry or the > first moments. From my problematic experiences, getting the initial > transform which is a "good" guess is been critical for the optimizer to > head to the correct solution. > > 2) The Center parameter of a transform can either be "Fixed" or an > optimizable parameter. The transforms with the "Centered" monkier are > the the ones with the Center parameter optimizable. This issue seems to > be what you were referring to in to message. > > In many ways the point I am trying to make is independent of #2. I > have observed that it's important to have the "Center" of an affine > transform ( or sub parameterization thereof ) at the center of the > object you are trying to register. That is the coordinate space of the > optimizable parameters' point 0 is near the center. This enables > rotation to easily rotate around the center, as well as scaling to > maintain the same relative center when "zooming". This issue is > independent of wether that center point can be optimized. > > For example, consider changing the origin of an image for alignment by > say 500, and compensating with just an initial translation and not > setting the "Center" parameter. If we are trying to optimize rotation > and translation, then the optimization path would be very difficult to > traverse with these inter-dependent parameters to force it to rotate > around the center of the object. This scenario may be more common in > microscopy then medical imaging, due to microscopy frequently having > multiple subjects across large images. I could write up a IPython > Notebook to illustrate the case. > > So that is my understanding of why using a fixed center is important. > I thought this may have been shared knowledge, but perhaps is not or is > incorrect... > > Now I am trying to understand how this interacts with all the > coordinate frames involved with the ITKv4 registration framework, and > the composite transforms. Specifically the composite transform apply > the transforms in "reverse order"[2]. As I understand that that means > that the newest transform get applied first. So that transform > parameters are being optimized in the images space not in the virtual > domains we are working towards. Therefore the center of our transforms > is still important as we composite transforms? > > I hope I explained that clearly, a white board may really be needed.... > > Thanks, > Brad > > > [1] > https://github.com/SimpleITK/SimpleITK/compare/master...9bc9bc8440e64d94 > 099d3519eb23bcc16c7cf015 > [2] http://www.itk.org/Doxygen/html/classitk_1_1CompositeTransform.html > > On Jul 28, 2014, at 8:57 PM, Nicholas Tustison <[email protected]> > wrote: > > Hi Brad, > > I apologize for the delayed response. I¹m still catching up on my > workload after moving to CA. We might want to talk to Luis as it > was my understanding that the ³center² component of the linear > transforms might be a carry-over from all the ³Centered² transforms, > e.g., > > > http://www.itk.org/Doxygen/html/classitk_1_1CenteredAffineTransform.htm > l > > which, if I remember correctly, Luis said were somewhat sub optimally > conceived but this was such a long time ago that I might be completely > misremembering. However, fixing the center for all transforms within > a composite transform is certainly not necessary within the specified > framework. Rather, the matrix and offset are updated with the > ³Center² > being updated implicitly. > > Nick > > > > On Jul 24, 2014, at 7:27 AM, Bradley Lowekamp > <[email protected]> wrote: > > Hello, > > There are a lot of transforms involved in the new registration > framework. > > I am trying to figure out what the implications for the fix > parameters of the transform "Center" ( ie center of rotation/scaling > ) are when combined with composite transforms and the > fixed/moving/registration coordinate frames... > > Based on how the transforms are composed it seems necessary to set > the fix "Center" for subsequent transformations. Additionally, I am > unsure how one would "improve" ( poorly defined? ) the center for a > subsequent transform ie Affine after a similarity... > > Are there some documented guidance or figures to help with this > issue? > Do we have a comprehensive diagram of these transforms? > > Thanks, > Brad > > > > _______________________________________________ > Powered by www.kitware.com > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Kitware offers ITK Training Courses, for more information visit: > http://kitware.com/products/protraining.php > > Please keep messages on-topic and check the ITK FAQ at: > http://www.itk.org/Wiki/ITK_FAQ > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/insight-developers > _______________________________________________ > Community mailing list > [email protected] > http://public.kitware.com/mailman/listinfo/community > > > _______________________________________________ > Powered by www.kitware.com > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Kitware offers ITK Training Courses, for more information visit: > http://kitware.com/products/protraining.php > > Please keep messages on-topic and check the ITK FAQ at: > http://www.itk.org/Wiki/ITK_FAQ > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/insight-developers > > > > > ________________________________ > Notice: This UI Health Care e-mail (including attachments) is covered by > the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is > confidential and may be legally privileged. If you are not the intended > recipient, you are hereby notified that any retention, dissemination, > distribution, or copying of this communication is strictly prohibited. > Please reply to the sender that you have received the message in error, > then delete it. Thank you. > ________________________________ > > > <unknown.png><unknown.png><MultiStageImageRegistration1.cxx> > > > > _______________________________________________ > Powered by www.kitware.com > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Kitware offers ITK Training Courses, for more information visit: > http://kitware.com/products/protraining.php > > Please keep messages on-topic and check the ITK FAQ at: > http://www.itk.org/Wiki/ITK_FAQ > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/insight-developers > >
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Kitware offers ITK Training Courses, for more information visit: http://kitware.com/products/protraining.php Please keep messages on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/insight-developers
