9.3 Evaluating Fitting Routines

Just how good are fitting routines at finding the correct parameter values anyway?

9.0 Fitting Overview - - - - 9.1 Parameter Fitting

9.2 Fitting experimental curves with models

All fitting routines look for the best minimum error in the parameter space. There are usually several minima in the mean square error in the parameter space. These represent possible "fits" to the data which vary in quality. The routines search for these minima by measuring the slopes of error function in the vicinity of the last parameter set and trying to guess the way to move in parameter space to find the location of a zero (or near zero) slope. The choice of a move to a new location depends on the search algorithm. The slope determination is repeated at the new location. The process cycles until the mean square error is less than a predetermined value or no better minimum is found. On can test the power of the search method for the particular type of problem one is interested in by asking it to fit data generated with known parameters.

I have illustrated this process below with a triple exponential decay which has time constants which are close to each other thus offering a variety of parameter sets which have minima near each other, a difficult problem for man or algorithm to solve. On important point is to examine how the starting point affects the quality of the "fit" produced.Therefore I have used a couple of different intitial parameter sets with which to start the "Fit to Data" process. When initial parameter values are those used to generate the curve, there are no changes although you might see new red checkmarks on some parameters (indicating that they have been touched but not changed enough to show up as a change in the least singnificant digit) and perhaps a tiny (but insignificant) decrease in the error sum.

Simplex

When the default parameters (1's for all) are used as the starting condition and the "Simplex" method was chosen, the results are shown below.

The mean square error is relatively large and this is obviously a rather poor fit; the amplitude of A is 30% low and that of C 40% high. While the actual values of h, j, and k are quite different from the generating values, the ratios of their values ( 1, 1.9, and 8) are at least in the vicinity of the proper values (1:3: 9).

When values closer to the generating parameter values are selected (h=8, j=4) for the initial guesses, the Simplex method finds a much better fit.

Here the mean square error is 2-3 orders of magnitude lower and the fitted parameter values are in much closer agreement with the generating values.

Praxis

When the default parameters (1's for all) are used as the starting condition and the Praxis method was chosen, an excellent fit was obtained; all of the amplitudes are within 2 parts in 1000 of the correct values and the ratios of the rate constants is quite accurate.

Will this method yield an even better fit if it is started with a parameter set (h=8, j=4) closer to the known? The answer shown below is NO.

I found that Praxis yeilded its answer with a single change to the display on the screen and that in four different fit attenpts under the same initial conditions, the errors ranged from 3 * 10^-9 to 7*10^-10 .

Back to:

9.0 Fitting Overview

9.1 Parameter Fitting

9.2 Fitting experimental curves with models