How to avoid:
Make better parameter estimates
Don't "linearize" nonlinear equations
Don't fit nonlinear equations with a linear least squares procedure
Graphically fit parameters before fitting
Eliminate redundant parameters
How to identify problem: Graphical inspection
Parameter values tend to be orders of magnitude away from the XY data range
What's A Local Minimum?
Curve fitting algorithms attempt to minimize the Sum or Squares. The Sum of Squares
space is in N+1 dimensions, where N is the number of parameters in the equation.
A global minimum is the location in the N+1 dimensional space where the Sum of
Squares is minimized, which is our goal.
A local minimum is a location
in the N+1 dimensional space where the the minimum is local to a specific region.
If a curve fitting algorithm has determined this to be the minimum, it’s
a false signal that it has converged.
This emphasizes the point of starting with good initial parameter estimates. The
quality of the estimates will have a strong influence on the curve fit success.
Here's a very simple example:
In this graph, notice that the global minimum is at A=0, while a local minimum
exists at approximately A=3.8, which is what we want to avoid. If you gave the
starting parameter estimate as A=8, the algorithm would continue moving down toward
A=3.8 and stop since there was no change in the Sum of Squares.
By comparison, if you started at A=-8, then the algorithm would find the global
minimum at A=0. This is why having good parameter estimates are very important.
If you don't provide a good starting point for the curve fit, then you may end
up in a local minimum. This is particularly important for very complex models,
where estimation is very useful. A Real World Example
Okay - I admit that the above was a ridiculously simple example. The natural question
to ask is "How to I know that I've encountered a local minimum?"
The answer? The only way to tell if a local minimum has been found is to
draw the curve fit line over the original data set. Often, you will notice that
the parameter values are orders of magnitude from where they should be. There
are no algorithms available to prevent this problem.
Here's an example curve fit from a customer, who was having trouble fitting a
simple model to a data set:
By looking at the graph, notice the following:
1) The curve fit is clearly not fitting the data at all. It should be curved,
not a straight line.
2) Notice the data ranges for the X and Y data. There's a difference of 4 orders
of magnitude from the parameters. Parameters shouldn't be that large when
the XY data is that small, which is another indicator of a local minimum.
It turned out that a linear least squares procedure was being used to fit this
equation. This can also cause you to get trapped in a local minimum. When we used
a nonlinear least squares procedure, we got a much better result:
That's a much better fit. Notice that the parameter values are reasonable and
the curve fit does a better job of describing the data.
If you end up in a local minimum, then there are a couple of things you can do:
1) Try to make better parameter estimates. Usually a lot of trial and error is
required, especially for exponential terms and other complex functions.
2) If possible, modify the step size used (usually smaller step sizes help)
3) Check for any redundant parameters that may be
affecting the model "Linearizing"
A Non-Linear Equation
Sometime people like to cheat and rearrange a non-linear equation so they can
use a linear least squares procedure for the curve fit. Why? It's much easier
from a mathematical perspective.
How do you do this? Well, you simply rearrange the parameters until they are linear.
Here's an example:
That looks fine, doesn't it? Well, there's a problem - you are trying to solve
an implicit function! This also invalidates one of the basic assumptions of curve
fitting, that all of the parameters are statistically independent. In other words,
you will very likely find a local minimum, or get a very bad curve fit.
The rule of thumb is: when fitting a non-linear function, use a nonlinear curve
fitting algorithm.
