[Insight-users] Registration optimizer drawbacks

[Luis Ibanez]

Hi Bjorn,

I completly agree with your analysis of the optimizers.

As you pointed out, the drawback of the GradientDescent
is that the steps depend on the values of the gradient.
This could be an advantage if you where facing a problem
in which the derivatives were smooth and monotonic.
In such a case the GradientDescentOptimizer method will
perform much faster than the RegularStepGradientDescent.

For Image registration it is quite hard to anticipate
how the derivatives of the Image metric will behave.
It is then much safer to use the RegularStepGradientDescent
which will advance at a stable rate.

To summarize:

     - GradientDescent is fast and unstable
     - RegularStepGradientDescent is slow and stable

As ITK is a toolkit, itkOptimizers are not expected to be
used only for image regisration. It is then interesting
to offer a variety of typical optimizers. The main reason
why the GradientDescentOptimizer was implemented using
learning rates is that it was the implementation described
in the original Viola & Wells paper and it was desirable
to provide an equivalent in ITK.

The flexibility of the registration framework in ITK
allows you to change just this component of their
registration method and use the RegularStepOptimizer
instead. (you could also implement an optimizer on your
own and add it to the toolkit).

Note that if the derivatives where monotonic you could
even go faster by using the itkConjugateGradientOptimizer.


Luis


=============================================

Bjorn Hanch Sollie wrote:
> Hi all!
> 
> I have been using the GradientDescent and RegularStepGradientDescent
> optimizers in registration, and I'm currently trying to document the
> positives and the negatives of one versus the other.
> 
> The learning rate of the GradientDescent optimizer obviously has the
> drawback that the length of the step is proportional to the derivative
> of the metric, and it can be pretty hard to get an idea of how this
> derivative will change as the registration progresses.  This means
> that what is a good learning rate for the initial iterations of the
> registration, might become inappropriate later on.
> 
> This is indeed a serious drawback, but I fail to see why it wouldn't
> always be be better to use the RegularStepGradientDescent optimizer
> instead.  I presume both optimizers are in the tookit for good reason.
> So, to put things in perspective, I'd like to know is what the
> drawbacks of the RegularStepGradientDescent optimizer are, because to
> me it really looks like it will always be a better choice.  I'll
> appreciate it if anyone can provide me with any clues.
> 
> Thanks in advance!
> 
> -Bjorn
>