<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:arial, helvetica, sans-serif;font-size:10pt"><div>Hello,</div><div style="font-family:arial, helvetica, sans-serif;font-size:10pt"><div style="font-family:times new roman, new york, times, serif;font-size:12pt"><div style="font-family:arial, helvetica, sans-serif;font-size:10pt;"><div><br>I am using MutualInformationHistogramImageToImageMetric + RegularStepGradientDescentOptimizer combination, and have a few doubts on it.<br><br>I understand that the value of the metric should be maximum at a location of optimal registration. But I also expected that the value of the gradient magnitude to be minimum at this location, which is not so (The gradient here is computed as per GetDerivative() function line 186 in itkHistogramImageToImageMetric.txx). I expect this behavior, as once the metric value is maximized (I ensured that this is not a local maxima) the
value of the gradient should be least, and there should also be a change in direction. I do see the conical shape in the metric values, but the optimizer always overshoots it and ends at a location where metric value is lower.
I have observed it in all my test cases.<br><br>For example in the log that I have provided below, around iteration 45, the metric value has peaked (starting from 0.1749 in iteration 31, peak of 0.1808 in iteration 45, and then the value going down afterwards), but the magnitude of the gradient (as defined in line 205, itkRegularStepGradientDescentBaseOptimizer.cxx), I get at this iteration is 5.909, which is not minimum. I also see a pattern of the value of the gradient magnitude to be gradually gradually decreasing.<br>I do not understand this behavior. Can you please throw some light?<br><br>*** I see that the values of m_DerivativeStepLengthScales in function GetDerivative() of itkHistogramImageToImageMetric.txx file, all the values are set to 1.0 ,using <br>m_DerivativeStepLengthScales.Fill(1.0);<br><br>This means that at each step, when the gradient is computed (i.e the parameters are perturbed by
m_DerivativeStepLength), all the parameters are assumed to have uniform scaling. How can this be assumed here?<br>Shouldn't these be scaled appropriately, like the optimizer scale
values used in
RegularStepGradientDescentBaseOptimizer::AdvanceOneStep() function?<br><br>Also the value of m_DerivativeStepLength is always set to 1.0 . Shouldn't this a user defined parameter, or based on the optimizer ?<br><br>--- LOG ----<br><br>Below is the log I am printing to analyze the value of the metric and how the optimizer behaves<br><br>MaximumStepLength : 0.1<br>MinimumStepLength : 0.0001<br>.....<br><br>Format:<br>Gradient_Magnitude Tolerance_value Current_step_Length<br>Iter_Number Metric_Value [Affine Parameters] Angle_in_Degrees Change_in_Metric_value Change_in_angle<br> <br>.......<br>....<br><br>GradMag = 7.494 Tol =0.0001 StepLen = 0.1<br>31 0.1749 [0.5827, -0.06824, 0.06624, 0.6027, -856.8, -931.7] Angle:6.473 Metr:0.0005967 ADiff:-0.02686<br>GradMag = 8.051 Tol =0.0001 StepLen = 0.1<br>32 0.1757 [0.5827, -0.06827,
0.06578, 0.6035, -856.7, -931.7] Angle:6.448 Metr:0.0007982 ADiff:-0.02441<br>GradMag = 7.914 Tol =0.0001 StepLen = 0.1<br>33 0.1766 [0.5826, -0.06834, 0.06529, 0.6043, -856.6, -931.7] Angle:6.423 Metr:0.0009113 ADiff:-0.02468<br>GradMag = 7.315 Tol =0.0001 StepLen = 0.1<br>34 0.1773 [0.5826, -0.06841, 0.06477, 0.6052, -856.5, -931.7] Angle:6.398 Metr:0.0006155 ADiff:-0.02567<br>GradMag = 7.99 Tol =0.0001 StepLen = 0.1<br>35 0.1778 [0.5827, -0.06849, 0.06428, 0.606, -856.4, -931.6] Angle:6.374 Metr:0.0004911 ADiff:-0.02425<br>GradMag = 7.949 Tol =0.0001 StepLen = 0.1<br>36 0.1783 [0.5827, -0.06857, 0.06378, 0.6068, -856.4, -931.6] Angle:6.349 Metr:0.000521 ADiff:-0.02433<br>GradMag = 7.78 Tol =0.0001 StepLen = 0.1<br>37 0.1787 [0.5827, -0.06868,
0.06327, 0.6076, -856.3, -931.5] Angle:6.326 Metr:0.0004448 ADiff:-0.02362<br>GradMag = 7.599 Tol =0.0001 StepLen = 0.1<br>38 0.179 [0.5828, -0.06881, 0.06276, 0.6085, -856.2, -931.5] Angle:6.302 Metr:0.0003036 ADiff:-0.02324<br>GradMag = 7.277 Tol =0.0001 StepLen = 0.1<br>39 0.1795 [0.5829, -0.06896, 0.0622, 0.6093, -856.1, -931.5] Angle:6.278 Metr:0.0004738 ADiff:-0.02439<br>GradMag = 8.089 Tol =0.0001 StepLen = 0.1<br>40 0.1797 [0.5829, -0.0691, 0.06171, 0.6101, -856, -931.4] Angle:6.257 Metr:0.0002499 ADiff:-0.02088<br>GradMag = 8.046 Tol =0.0001 StepLen = 0.1<br>41 0.18 [0.583, -0.06926, 0.06122, 0.6108, -855.9, -931.4] Angle:6.237 Metr:0.0002045 ADiff:-0.01961<br>GradMag = 8.101 Tol =0.0001 StepLen = 0.1<br>42 0.1802 [0.5831, -0.06943,
0.06075, 0.6116, -855.8, -931.4] Angle:6.219 Metr:0.0002648 ADiff:-0.01865<br>GradMag = 7.21 Tol =0.0001 StepLen = 0.1<br>43 0.1803 [0.5831, -0.06964, 0.06021, 0.6124, -855.7, -931.3] Angle:6.199 Metr:6.091e-005 ADiff:-0.02007<br>GradMag = 8.228 Tol =0.0001 StepLen = 0.1<br>44 0.1805 [0.5832, -0.06986, 0.05974, 0.6131, -855.6, -931.3] Angle:6.183 Metr:0.0002261 ADiff:-0.01615<br>GradMag = 7.485 Tol =0.0001 StepLen = 0.1<br>45 0.1808 [0.5833, -0.07012, 0.05922, 0.6139, -855.5, -931.2] Angle:6.166 Metr:0.0002553 ADiff:-0.01641<br>GradMag = 5.909 Tol =0.0001 StepLen = 0.1<br>46 0.1806 [0.5834, -0.07048, 0.05857, 0.6148, -855.4, -931.2] Angle:6.147 Metr:-0.0002007 ADiff:-0.01916<br>GradMag = 6.526 Tol =0.0001 StepLen = 0.1<br>47 0.1803 [0.5835,
-0.07083, 0.058, 0.6156, -855.3, -931.1] Angle:6.132 Metr:-0.000293 ADiff:-0.01502<br>GradMag = 7.555 Tol =0.0001 StepLen = 0.1<br>48 0.1803 [0.5836, -0.07113, 0.05752, 0.6163, -855.3, -931.1] Angle:6.12 Metr:3.284e-006 ADiff:-0.01187<br>GradMag = 6.259 Tol =0.0001 StepLen = 0.1<br>49 0.1803 [0.5837, -0.0715, 0.05695, 0.617, -855.2, -931.1] Angle:6.106 Metr:-3.978e-006 ADiff:-0.01394<br>GradMag = 6.12 Tol =0.0001 StepLen = 0.1<br>50 0.1803 [0.5837, -0.07192, 0.05638, 0.6177, -855.1, -931] Angle:6.095 Metr:6.792e-005 ADiff:-0.01128<br>GradMag = 5.317 Tol =0.0001 StepLen = 0.1<br>51 0.1803 [0.5839, -0.07242, 0.05577, 0.6185, -855, -931] Angle:6.086 Metr:-2.758e-005 ADiff:-0.009191<br>GradMag = 5.665 Tol =0.0001 StepLen = 0.1<br>52 0.1802 [0.584,
-0.07289, 0.05525, 0.6191, -854.9, -930.9] Angle:6.079 Metr:-0.0001544 ADiff:-0.006347<br>GradMag = 5.083 Tol =0.0001 StepLen = 0.1<br>53 0.1802 [0.5842, -0.07338, 0.05469, 0.6198, -854.8, -930.9] Angle:6.072 Metr:3.066e-006 ADiff:-0.007603<br>GradMag = 4.135 Tol =0.0001 StepLen = 0.1<br>54 0.1802 [0.5844, -0.07398, 0.05404, 0.6206, -854.7, -930.8] Angle:6.064 Metr:2.267e-005 ADiff:-0.007948<br>GradMag = 3.486 Tol =0.0001 StepLen = 0.1<br>55 0.1797 [0.5848, -0.07466, 0.05324, 0.6215, -854.7, -930.8] Angle:6.052 Metr:-0.0005252 ADiff:-0.01135<br>GradMag = 4.202 Tol =0.0001 StepLen = 0.1<br>56 0.1792 [0.585, -0.07522, 0.05258, 0.6222, -854.6, -930.7] Angle:6.043 Metr:-0.0004813 ADiff:-0.009771<br>GradMag = 3.866 Tol =0.0001 StepLen = 0.1<br>57
0.1789 [0.5853, -0.07577, 0.05185, 0.6229, -854.5, -930.7] Angle:6.029 Metr:-0.0002625 ADiff:-0.01337<br>GradMag = 3.972 Tol =0.0001 StepLen = 0.1<br>58 0.1786 [0.5856, -0.07628, 0.05115, 0.6235, -854.4, -930.6] Angle:6.017 Metr:-0.0003571 ADiff:-0.01262<br>GradMag = 4.536 Tol =0.0001 StepLen = 0.1<br>59 0.1781 [0.5858, -0.07669, 0.05057, 0.6239, -854.3, -930.5] Angle:6.005 Metr:-0.0004517 ADiff:-0.0113<br>GradMag = 3.345 Tol =0.0001 StepLen = 0.1<br>60 0.1777 [0.5861, -0.07723, 0.04978, 0.6244, -854.3, -930.4] Angle:5.99 Metr:-0.000372 ADiff:-0.0155<br></div><div style="font-family:arial, helvetica, sans-serif;font-size:10pt;"> -....<br><br><br><br>
<br>
Thanks for any clues,<br>
<br>
Sharath Venkatesha<br><br><br><div style="font-family:times new roman, new york, times, serif;font-size:12pt;"><font size="2" face="Tahoma"><hr size="1"><b><span style="font-weight:bold;">From:</span></b> Luis Ibanez <luis.ibanez@kitware.com><br><b><span style="font-weight:bold;">To:</span></b> Sharath Venkatesha <sharath20284@yahoo.com><br><b><span style="font-weight:bold;">Cc:</span></b> Insight users <insight-users@itk.org><br><b><span style="font-weight:bold;">Sent:</span></b> Thursday, July 2, 2009 4:28:59 PM<br><b><span style="font-weight:bold;">Subject:</span></b> Re: [Insight-users] Registration - Mutual Information + Affine - Fine tuning of parameters for optimization<br></font><br>
<br>Hi Sharath,<br><br><br>0) It is great that you have verified the Initialization<br><br><br>1) Translation initialization is "usually" enough...<br> but only if the rotation and scaling misalignments<br> are small....<br>
<br> A typical registration will not be able to correct <br> for more than a 30 degrees rotation, or a scaling<br> beyond a factor of 1.5.<br><br> So, please provide all the initialization that you can.<br>
<br> You may want to consider the use of the <br> <br> LandmarkBasedTransformInitializer<br><br><br>2) If you results start diverging from the expected answer<br> in the very first iterations, then you should suspect:<br>
<br> a) Incorrect balance of the rotation vs translation<br> values in the parameter scaling array<br><br> or<br><br> b) Your step lenghts are too large in the optimizer<br>
<br> or<br><br> c) You may have set the Optimizer to Maximize, when<br> the Metric needs to be Minimized (or the other way<br> around).<br><br> <br>3) You are correct that if the Registration is walking in the<br>
right direction, the Joint histograms should be getting <br> more concentrated in the diagonal...<br><br> BUT ONLY IF<br><br> you are working with two images of the same modality.<br><br> Otherwise, if the images are of different modality, the<br>
histogram will never be concentrated in the diagonal.<br><br><br>4) Unfortunately we don't have much material written on<br> how to analyze the histograms.<br><br> I will suggest that you save them as images using the<br>
Histogram to Image filter, and then load them as a <br> series of 2D+T images into VV<br><br><span><span> <a target="_blank" href="http://www.creatis.insa-lyon.fr/rio/vv">http://www.creatis.insa-lyon.fr/rio/vv</a></span></span><br><br> so that you can play them as an animation.<br>
<br><br>5) The typical use of the vtkRegistrationMonitor will be:<br><br> #include "vtkRegistrationMonitor.h"<br> #include "vtkKWImage.h"<br> #include "vtkContourFilter.h"<br> #include "vtkImageData.h"<br>
<br> vtkRegistrationMonitor monitor;<br><br> vtkSmartPointer< vtkKWImage > fixedKWImage = vtkSmartPointer< vtkKWImage >::New();<br> vtkSmartPointer< vtkKWImage > movingKWImage = vtkSmartPointer< vtkKWImage >::New();<br>
<br> fixedKWImage->SetITKImageBase( const_cast<FixedImageType *>( fixedImage ) );<br> movingKWImage->SetITKImageBase( const_cast<MovingImageType *>( movingImage ) );<br><br> vtkSmartPointer< vtkContourFilter > fixedContour = vtkSmartPointer< vtkContourFilter >::New();<br>
vtkSmartPointer< vtkContourFilter > movingContour = vtkSmartPointer< vtkContourFilter >::New();<br><br> fixedContour->SetInput( fixedKWImage->GetVTKImage() );<br> movingContour->SetInput( movingKWImage->GetVTKImage() );<br>
<br> fixedContour->SetValue( 0, 200.0 ); // level for iso-contour (DEPENDS on your data)<br> movingContour->SetValue( 0, 200.0 ); // level for iso-contour (DEPENDS on your data)<br><br> monitor.SetFixedSurface( fixedContour->GetOutput() );<br>
monitor.SetMovingSurface( movingContour->GetOutput() );<br><br> monitor.SetNumberOfIterationPerUpdate( 1 ); (RECONSIDER CHANGING...)<br><br> std::string screenshotDirectory = argv[7];<br> itksys::SystemTools::MakeDirectory( screenshotDirectory.c_str() );<br>
<br> std::cout << screenshotDirectory << std::endl; <br><br> monitor.SetScreenshotOutputDirectory( screenshotDirectory.c_str() );<br> monitor.SetScreenshotBaseName( "registrationScreenshot" );<br>
<br> monitor.Observe( optimizer, rigidTransform );<br><br><br>You will find the vtkKWImage classes also in <br><br> InsightApplications/Auxiliary/vtk<br><br><br>---<br><br> Regards,<br><br><br> Luis<br><br><br>----------------------------------------------------------------------<br>
<div class="gmail_quote">On Thu, Jul 2, 2009 at 3:01 PM, Sharath Venkatesha <span dir="ltr"><<a rel="nofollow" ymailto="mailto:sharath20284@yahoo.com" target="_blank" href="mailto:sharath20284@yahoo.com">sharath20284@yahoo.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex;">
<br>
Hi,<br>
<br>
Thanks for the all the help and information provided in the earlier mails.<br>
<br>
I have ensured that I am using correct initialization ( translation only) and correct values of optimizer scales.<br>
<br>
*** Is it sufficient if I provide initialization parameters for translation (approx values) only? It is difficult for me estimate the scale and rotation parameters.<br>
<br>
I am using MutualInformationHistogramImageToImageMetric +<br>
MultiResolutionImageRegistrationMethod + AffineTransform +<br>
RegularStepGradientDescentOptimizer/GradientDescentOptimizer<br>
<br>
I am having problem with tuning of parameters (MaxStepSize and MinStepSize for RegularStepGradientDescentOptimizer, and LearningRate for GradientDescentOptimizer) . My results start diverging off the correct route in the first few iterations itself.<br>
<br>
I have looked into<br>
(1) Plotting of joint histograms (section 8.5.3 of ITK manual)<br>
(2) vtkRegistrationMonitor<br>
<br>
<br>
I am having trouble interpreting the results of the joint histograms.I understand that with correct parameters for registration, I should get a histogram which has highest density only along the diagonal. I have the outputs of the JointHistograms after very iteration for very level , and finding hard to follow the changes.<br>
<br>
*** Is there is a defined procedure for understanding the results?<br>
<br>
Can you please point me to where I can get more information on (1) and (2) ?<br>
<br>
Thanks,<br>
Sharath Venkatesha<br>
<br>
<br>
------------------------------<br>
Luis wrote:<br>
<br>
<br>
Hi Sharath,<br>
<br>
0) Whenever you get the exception saying that too many points<br>
mapped outside of the moving image, it means that the<br>
current Transform is such that when mapping the moving image<br>
into the Fixed image coordinate system the overlap between<br>
the two image is so small that it is unlikely that the<br>
registration will recover in further iterations.<br>
<br>
This is typically due to:<br>
<br>
<br>
A) Poor initilization of the Transform<br>
<br>
B) Poor selection of Scaling parameters<br>
(the array that normalizes the dynamic<br>
range of the different Transform parameters,<br>
e.g. radians versus millimeters)<br>
<br>
C) Optimizers that are set to perform jumps<br>
that are too large, and bring the Transform<br>
out of the range of the image.<br>
<br>
1) You want to check this potential suspects in order.<br>
<br>
That is.<br>
<br>
First, verify that the initial transform<br>
is reasonably placing the Moving image on top of the<br>
Fixed image. You can do this by using the Resample<br>
image filter, passing the moving image as input,<br>
using the initial transform as Transform, and using<br>
the Fixed image as reference. Then compare the<br>
fixed image to the resampled moving image.<br>
<br>
If the initial image looks ok,<br>
then you want to check the values of the ParameterScaling.<br>
It should be such that when you look at the Transform<br>
parameters at ever iteration (using an Observer), the<br>
values should change from iteration to iteration, according<br>
to the expected dynamic range.<br>
<br>
For example, transform parameters that correspond to rotations<br>
should change by increments smaller than 0.01 (since they are<br>
measured in Radians). While transform parameters that correspond<br>
to translations should change at increments of 1 ~ 10<br>
<br>
<br>
Finally, you should identify the parameter of the optimzer<br>
that is responsible for selecting the size of jumps that<br>
are performed in the parameteric space. (e.g. as you have<br>
done for the learning rate in the GradientDescent optimizer).<br>
<br>
You want to reduce the size of that jump, until you get the<br>
Transform to have small increments at every iteration.<br>
<br>
<br>
2) These parameters must be set up for every "family" of<br>
registration problems. That is, the parameters that may be<br>
good for registering a T1 to T2 MRI brain images, may not be<br>
appropriate for registering a confocal microscopy image to<br>
another.<br>
<br>
However, once you fine tune the paramters for a pair of<br>
T1-T2 images, it is likely that the same set of parameters<br>
will work for another pair of the same type.<br>
<br>
There is a need for a "smart layer" above the registration<br>
framework, that could take away from the user the burden<br>
of finding proper parameter settings....<br>
<br>
any ideas are welcome :-)<br>
<br>
<br>
3) Visual monitoring of the registraiton process<br>
will help to make the fine-tunning process less<br>
frustrating.<br>
<br>
You may want to give it a try at the VTK helper<br>
classes:<br>
<br>
InsightApplications/Auxiliary/vtk/<br>
vtkRegistrationMonitor.h<br>
vtkRegistrationMonitor.cxx<br>
<br>
they will display renderings from iso-surfaces<br>
at every iteration of the registration process.<br>
<br>
This is usually very informative...<br>
<br>
<br>
<br>
Regards,<br>
<br>
<br>
Luis<br>
<br>
<br>
--------------<br>
sharath v wrote:<br>
> Hi,<br>
><br>
> Thanks for the help. Changing the learning parameter worked...<br>
><br>
><br>
For Viola MI + Affine and with learning rate of 0.01 and 100<br>
iterations, I get good results on BrainProtonDensitySliceR10X13Y17<br>
image. Whereas on the BrainProtonDensitySliceR10X13Y17S12 image, it<br>
requires atleast 200 iterations to give correct results.<br>
><br>
><br>
I want to use an optimizer which has a stopping value (i.e not fixed<br>
number of iterations) like Amoeba/Evolutionary/GradientDescentStep<br>
><br>
> I tried using the Amoeba optimizer with the following<br>
><br>
> OptimizerType::ParametersType simplexDelta( transform->GetNumberOfParameters() );<br>
> simplexDelta.Fill( 5.0 );<br>
> optimizer->AutomaticInitialSimplexOff();<br>
> optimizer->SetInitialSimplexDelta( simplexDelta );<br>
> optimizer->SetParametersConvergenceTolerance( 0.01 ); // quarter pixel<br>
> optimizer->SetFunctionConvergenceTolerance(0.001); // 0.1%<br>
> optimizer->SetMaximumNumberOfIterations( 200 );<br>
><br>
><br>
but I get an exception that sampled point mapped to outside of the<br>
moving image, after 6-7 iterations. Similar issue happens for<br>
OnePlusOne optimizer with<br>
><br>
> typedef itk::Statistics::NormalVariateGenerator GeneratorType;<br>
> GeneratorType::Pointer generator = GeneratorType::New();<br>
> generator->Initialize(12345);<br>
> optimizer->SetNormalVariateGenerator( generator );<br>
> optimizer->Initialize( 10 );<br>
> optimizer->SetEpsilon( 1.0 );<br>
> optimizer->SetMaximumIteration( 4000 );<br>
><br>
> Can you please let me know what values need to be used?<br>
><br>
> And is there a way to make the registration process partially independent of these parameters?<br>
><br>
> Thanks,<br>
> Sharath<br>
<br>
<br>
<br>
<br>
<br>
_____________________________________<br>
Powered by <a rel="nofollow" target="_blank" href="http://www.kitware.com">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at<br><span><span>
<a target="_blank" href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a></span></span><br>
<br><span><span>
Please keep messages on-topic and check the ITK FAQ at: <a target="_blank" href="http://www.itk.org/Wiki/ITK_FAQ">http://www.itk.org/Wiki/ITK_FAQ</a></span></span><br>
<br>
Follow this link to subscribe/unsubscribe:<br><span><span>
<a target="_blank" href="http://www.itk.org/mailman/listinfo/insight-users">http://www.itk.org/mailman/listinfo/insight-users</a></span></span><br>
</blockquote></div><br>
</div></div></div><br>
</div></div><div style="position:fixed"></div></div><br>
</body></html>