Hello Brad, It is an awesome way for demonstration, and thanks for verification of my example. It seems that this email is sent on Saturday, but unfortunately I got that this morning! And I should inspect the reason! Anyway, over the weekend I ran almost the same experiment in ITK and got close results to you.
Also, today I tried to make the registration process work when Affine Transform is used in a multi stage structure, and here is my suggestion: I think we do not need to bother ourselves to think about a new optimizer or a new initializer for the composite transform since we can initialize the fixed parameters of each transform individually at the beginning of each stage. Remember how we run a BSpline registration stage after some linear registrations. At the beginning of the BSpline stage, we should instantiate a BSpline transform object and define its fixed parameters (grid size, etc). Then, we pass this initialized transform to the registration filter using “SetInitialTransform()”, yet we can initialize this stage by passing a composite transform, resultant from the previous stages, to the registration filter by “SetMovingInitialTransform()”. The same condition holds for all other transform types with fixed unoptimizable parameters like the center of rotation in our Affine transform case. Attachment file includes a multi stage registration example that I have created for the new software guide. In this example, we use a simple 2 stages registration process for a multi modal problem that moving image is misaligned from the fixed image by a translation shift and rotation. In registration process, a translation transform is followed by an affine transform. Also, an initial transform is used at the beginning. If I do not set the center of the affine transform, the registration fails as you have shown in your Ipython example. However, if at the beginning of the second stage, I compute the geometrical center (or center of the mass) of the fixed image, and pass that to the affine transform using “SetCenter()” or “SetFixedParameters()”, the registration succeeds. Please take a quick look in the attached file that also includes lots of explanations. Therefore, we can follow the same procedure for all other transform types that their fixed parameters are important in the optimization, but those fixed parameters cannot be set explicitly inside a composite transform. Please let me know with you think about this. Thank you, Ali From: Bradley Lowekamp <[email protected]<mailto:[email protected]>> Date: Saturday, August 2, 2014 at 6:48 PM To: Ali Ghayoor <[email protected]<mailto:[email protected]>> Cc: Hans Johnson <[email protected]<mailto:[email protected]>>, Insight Developers List <[email protected]<mailto:[email protected]>> Subject: Re: [ITK-dev] [ITK] Optimizing composite transforms and center of transform Hello Ali, I created a notebook to try to demonstrate the problem you are describing: http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/blob/master/Importance%20of%20Center%20with%20Affine%20Registration.ipynb If the center of an Affine transformation isn't correctly initialized then the optimization is not speedy or robust. I think it's best illustrated with these two graphs: [unknown.png] [unknown.png] Notice that with the Center being the origin, all 1000 iterations are exhausted, and the metric is slowing decreasing. And this is with the translation path being around the length to 10. If the spacing was bigger or more offset from the origin the situation would be worse. I don't think this is a bug with the registration framework. I think new initializers for composite transforms are needed, along with documentation how the 2 good ways to initialize the registration methods transforms. Brad On Aug 1, 2014, at 8:08 PM, Ghayoor, Ali <[email protected]<mailto:[email protected]>> wrote: Brad, I have exactly the same issue. I wrote a simple registration program that is run in two stages: -Translation -Affine My fixed and moving images are in 2D, and the moving image is created from the fixed image by rotation of 10 degree and translation of [13,17] in X and Y directions. The first stage does a translation registration and the result transform is added to a composite transform that is used as the initial moving transform of the second stage. After the registration the result transform of the second stage is also added to the composite transform to be used by the resampler. I have printed the composite transform to the screen: 2481: CompositeTransform (0x7fa359a18b00) 2481: RTTI typeinfo: itk::CompositeTransform<double, 2u> 2481: Reference Count: 2 2481: Modified Time: 20949 2481: Debug: Off 2481: Object Name: 2481: Observers: 2481: none 2481: Transforms in queue, from begin to end: 2481: >>>>>>>>> 2481: TranslationTransform (0x7fa359a199b0) 2481: RTTI typeinfo: itk::TranslationTransform<double, 2u> 2481: Reference Count: 7 2481: Modified Time: 1648 2481: Debug: Off 2481: Object Name: 2481: Observers: 2481: none 2481: Offset: [12.801, 15.8578] 2481: >>>>>>>>> 2481: AffineTransform (0x7fa359d095a0) 2481: RTTI typeinfo: itk::AffineTransform<double, 2u> 2481: Reference Count: 5 2481: Modified Time: 20946 2481: Debug: Off 2481: Object Name: 2481: Observers: 2481: none 2481: Matrix: 2481: 1.01593 -0.0100051 2481: -0.00835669 1.01113 2481: Offset: [-0.358829, -0.290093] 2481: Center: [0, 0] 2481: Translation: [-0.358829, -0.290093] 2481: Inverse: 2481: 0.984397 0.00974056 2481: 0.00813575 0.989073 2481: Singular: 0 2481: End of MultiTransform. 2481: <<<<<<<<<< As it can be seen, the translation transform provides a good initialization since its offset (Offset: [12.801, 15.8578]) is close to the shift parameters. However, the Affine transform fails to rotate the moving image since its Matrix is almost identity. Possible reason can be the improper Center of Affine transform that is fixed to [0,0], and not at geometrical center or center of mass. To make sure that I have not made any crazy mistake in my example, I simulated my example parameters in ANTs, and ANTs returned the same results and failed to register the images!! ANTs command line <<<<<<<<<<< PROGPATH=/scratch/ANTS/release-new/bin fi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainT1SliceBorder20.png mi=/scratch/ITK-softwareGuide/release/ITKv4/Examples/Data/BrainProtonDensitySliceR10X13Y17.png time ${PROGPATH}/antsRegistration -d 2 \ --float \ --output [test1, warpedMoving.nii.gz] \ --transform "Translation[16]" \ --metric MI[${fi},${mi},1,32,None,1] \ --convergence [100,1e-2,5] \ --shrink-factors 3 \ --smoothing-sigmas 2 \ --use-histogram-matching 1 \ \ --transform "Affine[1]" \ --metric MI[${fi},${mi},1,32] \ --convergence [100x100,1e-2,2] \ --shrink-factors 2x1 \ --smoothing-sigmas 1x0 \ --use-histogram-matching 1 <<<<<<<<<<<<<<<<<<<<<<<< Fixed and moving images can be loaded from the ITK data files. What do you think about this example? Is it a bug in ITK or just the parameters are not tuned finely? Thanks, Ali ________________________________________ From: Johnson, Hans J Sent: Tuesday, July 29, 2014 6:19 PM To: Ghayoor, Ali Subject: FW: [ITK-dev] [ITK] Optimizing composite transforms and center of transform Ali, You need to follow this discussion. Hans On 7/29/14, 6:16 PM, "Bradley Lowekamp" <[email protected]<mailto:[email protected]>> wrote: Hello, Ok, to further explore this problem, and hopeful help other people too: I created an IPython Notebook viewable here: http://nbviewer.ipython.org/github/blowekamp/SimpleITK-Notebook-Answers/bl ob/master/Transform%20Composition%20And%20Order.ipynb It doesn't come through on the static page, but I added some nifty interactive slider widgets to change the rotation parameter. Which is really the point to enable understanding of the parameters for the transform and how they should be best optimized. The notebook should work with SimpleITK >= 0.8 and IPython Notebooks 2.1. I'll encourage every one to try it out :) IPython Notebooks widget are very cool! EXPLANATION: The the order of the composite transform is that the new transforms are applied first. BUT they map from the virtual domain ( fixed ) to the moving image. Therefore if your first transform moves the center of mass to the origin then the second added transform's space will be centered on the mass, which is good for optimizing rotation and affine parameters. QUESTION: Therefore I am inclined to conclude that its best practices to map the fixed and moving image to the virtual domain such that the center of mass ( or some other "center feature" ) are at the origin. This would then the scale, rotation and other affine parameters around the center, with out having to used the "Center" parameter the transforms currently have. Is this what others are doing in their registration? For general best practices should this be recommend? @Brian I know I have seen this multi-domain description many time, but I think I may have just gotten it... Is the well describe in the literature some place? I think this may be an important part add the software guide. Thanks, Brad On Jul 29, 2014, at 5:38 PM, Matt McCormick <[email protected]<mailto:[email protected]>> wrote: Hi Brad, Your assessment that the fixed center is important is correct. In most cases, a transform with a Center is the first transform in a Composite transform. There not any issues here. However, if the transform with a Center is further along the chain, then a CenteredTransformInitializer that was applied before the registration is started will not generate the desired result. In this case, we would have to respond to an Event in the registration process and estimated the center on the imaged after the other transforms have been applied. It is more work, but it is a more unusual case. HTH, Matt On Mon, Jul 28, 2014 at 10:11 PM, Bradley Lowekamp <[email protected]<mailto:[email protected]>> wrote: Helloo Nick! I am glad you got back to me. I suspect that I have spent more time this past week looking at the ITK affine transforms than anyone else[1] this week. With that here is my current understanding... 1) The CenterTransformInitializers estimates an initial "Center" and an initial "Translation" parameter from either the geometry or the first moments. From my problematic experiences, getting the initial transform which is a "good" guess is been critical for the optimizer to head to the correct solution. 2) The Center parameter of a transform can either be "Fixed" or an optimizable parameter. The transforms with the "Centered" monkier are the the ones with the Center parameter optimizable. This issue seems to be what you were referring to in to message. In many ways the point I am trying to make is independent of #2. I have observed that it's important to have the "Center" of an affine transform ( or sub parameterization thereof ) at the center of the object you are trying to register. That is the coordinate space of the optimizable parameters' point 0 is near the center. This enables rotation to easily rotate around the center, as well as scaling to maintain the same relative center when "zooming". This issue is independent of wether that center point can be optimized. For example, consider changing the origin of an image for alignment by say 500, and compensating with just an initial translation and not setting the "Center" parameter. If we are trying to optimize rotation and translation, then the optimization path would be very difficult to traverse with these inter-dependent parameters to force it to rotate around the center of the object. This scenario may be more common in microscopy then medical imaging, due to microscopy frequently having multiple subjects across large images. I could write up a IPython Notebook to illustrate the case. So that is my understanding of why using a fixed center is important. I thought this may have been shared knowledge, but perhaps is not or is incorrect... Now I am trying to understand how this interacts with all the coordinate frames involved with the ITKv4 registration framework, and the composite transforms. Specifically the composite transform apply the transforms in "reverse order"[2]. As I understand that that means that the newest transform get applied first. So that transform parameters are being optimized in the images space not in the virtual domains we are working towards. Therefore the center of our transforms is still important as we composite transforms? I hope I explained that clearly, a white board may really be needed.... Thanks, Brad [1] https://github.com/SimpleITK/SimpleITK/compare/master...9bc9bc8440e64d94 099d3519eb23bcc16c7cf015 [2] http://www.itk.org/Doxygen/html/classitk_1_1CompositeTransform.html On Jul 28, 2014, at 8:57 PM, Nicholas Tustison <[email protected]<mailto:[email protected]>> wrote: Hi Brad, I apologize for the delayed response. I¹m still catching up on my workload after moving to CA. We might want to talk to Luis as it was my understanding that the ³center² component of the linear transforms might be a carry-over from all the ³Centered² transforms, e.g., http://www.itk.org/Doxygen/html/classitk_1_1CenteredAffineTransform.htm l which, if I remember correctly, Luis said were somewhat sub optimally conceived but this was such a long time ago that I might be completely misremembering. However, fixing the center for all transforms within a composite transform is certainly not necessary within the specified framework. Rather, the matrix and offset are updated with the ³Center² being updated implicitly. Nick On Jul 24, 2014, at 7:27 AM, Bradley Lowekamp <[email protected]<mailto:[email protected]>> wrote: Hello, There are a lot of transforms involved in the new registration framework. I am trying to figure out what the implications for the fix parameters of the transform "Center" ( ie center of rotation/scaling ) are when combined with composite transforms and the fixed/moving/registration coordinate frames... Based on how the transforms are composed it seems necessary to set the fix "Center" for subsequent transformations. Additionally, I am unsure how one would "improve" ( poorly defined? ) the center for a subsequent transform ie Affine after a similarity... Are there some documented guidance or figures to help with this issue? Do we have a comprehensive diagram of these transforms? Thanks, Brad _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Kitware offers ITK Training Courses, for more information visit: http://kitware.com/products/protraining.php Please keep messages on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/insight-developers _______________________________________________ Community mailing list [email protected]<mailto:[email protected]> http://public.kitware.com/mailman/listinfo/community _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Kitware offers ITK Training Courses, for more information visit: http://kitware.com/products/protraining.php Please keep messages on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/insight-developers ________________________________ Notice: This UI Health Care e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete it. Thank you. ________________________________
MultiStageImageRegistration1.cxx
Description: MultiStageImageRegistration1.cxx
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Kitware offers ITK Training Courses, for more information visit: http://kitware.com/products/protraining.php Please keep messages on-topic and check the ITK FAQ at: http://www.itk.org/Wiki/ITK_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/insight-developers
