Statistics Glossary → Nonlinear Regression
Nonlinear Regressionanalysis
A regression that models the relation between an outcome and a predictor with a function whose parameters enter nonlinearly, often one derived from theory (exponential, logistic, Michaelis-Menten and so on), and estimates those parameters by least squares.
Concept
The form of some relations is known from theory: a hot drink approaches room temperature exponentially, a population grows quickly and then levels off at its carrying capacity, an enzyme's rate approaches an upper limit as substrate increases. The parameters of these functions have a direct meaning (an upper limit, a half-life, a growth rate), but because they enter the equation nonlinearly they cannot be estimated with ordinary linear regression. Nonlinear regression fits such models to the data directly.
The method looks for the parameters that minimise the sum of squared differences between the observed values and the model's predictions: Σ (yᵢ − f(xᵢ; θ))². Unlike linear regression, this minimum has no closed-form formula; the solution is found by iterative algorithms that start from initial values and improve the parameters step by step. In the Michaelis-Menten model, for example, rate = Vmax · x / (Km + x): Vmax is the highest attainable rate and Km the substrate concentration at which the rate reaches half of it.
Poor starting values can stop the algorithm from converging or trap it in a wrong local solution, so starting values should be chosen sensibly from the data or theory. Standard errors and confidence intervals are approximate and can mislead with small samples or strong curvature; the bootstrap offers a check in that case. Checking whether the residuals are randomly scattered shows whether the chosen functional form is appropriate, and different forms can be compared with criteria such as AIC.
When to use it
- When the form of the relation is known from theory or earlier work and the parameters themselves (an upper limit, a rate constant, a half-life) are to be interpreted: growth curves, enzyme kinetics, dose-response, decay processes.
- If the relation is curved but its form is unknown, a generalised additive model (GAM) or polynomial regression is a more flexible option.
- If the model can be made linear by a transformation (e.g. taking logs of both sides of a power law), linear regression can also be used; but since this changes the error structure, the results may differ from fitting the curve directly.
An everyday example
A café wants to know how long it takes for a served coffee to cool to a drinkable temperature. The temperature in the cup is measured every two minutes: it starts at 85 °C, falls quickly at first and then ever more slowly, approaching the room temperature of 22 °C. When the exponential decay model, temperature = c + a · e^(−b·t), is fitted to these data, c gives the room temperature, a the initial temperature difference and b the cooling rate. With these parameters you can work out how many minutes it takes for the coffee to reach 60 °C.
Parameters in MerQur
Function form (Exponential decay, Exponential growth, Logistic, Gompertz, Michaelis-Menten, Weibull CDF, Power, Custom formula)
Concept: The functional form is the assumption about which mathematical shape the relation follows. Each built-in form describes a particular process: exponential decay a decline towards a value, logistic and Gompertz S-shaped growth that stops at an upper limit, Michaelis-Menten a rate that saturates, and the power function scale-proportional relations. The form should be chosen from the theory of the process rather than from the look of the data.
Everyday example: Since a coffee's cooling is a decay towards a temperature (the room's), the exponential decay form is the natural choice. If the question were how the café's daily customer numbers grew in the months after opening, a logistic form that levels off at a saturation level would fit better.
What it does: Selects a ready-made model: exp_decay a e^(-bx) + c, exponential a e^(bx), logistic K/(1 + e^(-r(x - x0))), Gompertz a e^(-b e^(-cx)), Michaelis-Menten Vmax x/(Km + x), Weibull CDF 1 - e^(-(x/a)^b), power a x^b; "Custom formula" uses a free expression. Starting values are derived from the data.
Effect on the result: Parameters, SE and fit.
On the validation data: Puromycin Michaelis-Menten: Vmax 212.684 (SE 6.947), Km 0.06412 (0.00828). DNase1 logistic K 2.34518; trees power a 0.08661, b 2.23639. uspop exponential b 0.016089.
Model formula and Start values
Concept: A custom formula lets you write any model beyond the built-in forms. Because nonlinear estimation is iterative, the algorithm needs a starting point; the closer the starting values are to the true solution, the more easily and reliably it converges.
Everyday example: In the coffee model, starting values are easy to guess from the data: 22 for c (room temperature), 85 − 22 = 63 for a (the initial difference), and a small value such as 0.1 for b. With these sensible starts the algorithm reaches the solution in a few steps, while starting at c = 0 could send it in the wrong direction.
What it does: Free model expression (names other than parameters are data columns) and parameter starting values ('a=10, b=0.1'). A good starting value is essential for difficult models (Bates and Watts, 1988). The solver uses the complex-step Jacobian, the switch from LM to TRF and the Bates-Watts relative-offset convergence criterion; if it does not converge, the table is headed "NOT CONVERGED" and the SE, CI and p are left blank.
Effect on the result: Solution and convergence.
On the validation data: Over the NIST 27 datasets x 2 starting points, MerQur's lowest-LRE median is 9.2. The first starting points of MGH17 and BoxBOD converge to the certified values (BoxBOD b1 213.809, RSS 1168.009). For Puromycin, "Vmax=200, Km=0.05" does not change the result.
Confidence Level (%90, %95, %99, custom)
Concept: Sets how much assurance the parameters' confidence intervals give of covering the true values. In nonlinear models these intervals are approximate, and they widen as the level rises.
Everyday example: If the cooling rate is b = 0.08, you can say with 95% confidence that the true rate lies between, say, 0.07 and 0.09; asking for 99% widens the interval.
What it does: Sets the level of the Wald-t confidence interval (t(n - p)).
Effect on the result: Intervals only.
On the validation data: Puromycin Vmax lower bound: 90% 200.092; 95% 197.205; 99% 190.666.
Parameter bounds
Concept: Parameter bounds restrict the range in which estimates are sought, to exclude theoretically meaningless values (e.g. a negative rate or a negative upper limit). If the solution lies inside the bounds, the result is the same as the unbounded estimate; if it sits on a bound, that can be a sign that the model or data need review.
Everyday example: A negative cooling rate b would mean the coffee warms up by itself, which is physically meaningless. Setting a lower bound of 0 for b stops the algorithm drifting to such a solution even with a few noisy measurements.
What it does: Gives lower:upper bounds in the form 'a=0:100, b=0:' (an empty side is unbounded); if bounds are given, the trust-region reflective (TRF) algorithm is applied.
Effect on the result: If the bound is not active, the result is the same as unbounded; if active, the parameter stays at the bound.
On the validation data: "Vmax=0:500, Km=0:" (not active) is the same as unbounded. "Vmax=0:200" (active): Vmax = 200, Km 0.0528, RSS 1593.87; the SE is computed from the full (J'J)^-1.
Weight (WLS)
Concept: Weighted least squares gives each observation's squared error a weight, so that some observations influence the fit more than others. If measurements differ in precision, the more precise ones receive larger weights (usually the inverse of their variance).
Everyday example: If the thermometer is less precise at high temperatures, the measurements from the first minutes get smaller weights and those near room temperature larger ones. The model then sticks more closely to the reliable measurements.
What it does: Minimizes the sum of w r^2 with a positive weight column (weighted least squares).
Effect on the result: Parameters and SE.
On the validation data: trees power, weight Height: a 0.08947, b 2.2259 (unweighted 0.08661 and 2.23639); codling Weibull, weight tot: a 19.046.
Extra output: ANOVA + correlations + iterations
Concept: The ANOVA table compares the variability the model explains with what it leaves unexplained. Parameter correlations show how strongly the estimates of two parameters depend on each other; a very high correlation signals that the data struggle to tell those parameters apart. The iteration history shows how the algorithm approached the solution.
Everyday example: If measurements cover only the first 6 minutes, before the coffee gets anywhere near room temperature, c and a may be strongly correlated: the data cannot separate room temperature from the initial difference well. Extending the measurements to 40 minutes fixes this.
What it does: Gives the ANOVA table based on the uncorrected total sum of squares (regression = uncorrected total sum of squares - residual sum of squares), the parameter correlations, the Bates-Watts convergence criterion and the iteration history.
Effect on the result: An additional table only.
On the validation data: Puromycin regression SS 270213.55, residual SS 1195.449, uncorrected total 271409, corrected total 30858.917; Vmax-Km correlation 0.7651.
Extra output: AIC/BIC + form comparison
Concept: AIC and BIC are information criteria that balance fit against the number of parameters; smaller values indicate a better model. Fitting different functional forms to the same data and comparing them with these criteria shows which shape describes the data better.
Everyday example: If both exponential decay and a power function are fitted to the coffee data and the exponential model has a clearly smaller AIC, the idea that cooling is an exponential process is supported by the data as well.
What it does: Gives the log-likelihood, AIC and BIC (k = p + 1) and the AIC table of all ready-made forms.
Effect on the result: An additional table only.
On the validation data: Puromycin log-likelihood -44.6355, AIC 95.271, BIC 96.726.
Extra output: Bootstrap SE (B = 999)
Concept: The bootstrap generates many artificial datasets by resampling the model's residuals, refits the model to each, and derives standard errors and confidence intervals from the parameters' distribution across these fits. It serves to check whether the approximate formula-based intervals are trustworthy in a small sample.
Everyday example: With only 12 temperature readings, the formula-based interval for the cooling rate may be too narrow. If the bootstrap interval is similar, the result is reliable; if it is much wider, more measurements are needed.
What it does: Performs 999 resamples with centered residuals (seed 20240601); gives the SE and the percentile confidence interval.
Effect on the result: An additional table only.
On the validation data: Puromycin Vmax bootstrap SE 6.264 (asymptotic 6.947); the bootstrap SE carries a seed-dependent Monte Carlo margin.
Extra plots: Confidence/prediction band and Residual plots
Concept: The confidence band shows where the mean curve may lie at a given x; the prediction band shows where a single new observation may fall. The prediction band is always wider because it also includes the random scatter of individual measurements. Residual plots show whether the model leaves a systematic deviation.
Everyday example: At minute 10 the confidence band for the mean coffee temperature might be 49-52 °C, but the prediction band for the next cup is 46-55 °C, because each cup's starting temperature and fill level differ a little. If the residuals are all positive in the first minutes, the model is not describing that part well.
What it does: Draws the confidence and prediction bands at the x quantiles by the delta method; draws residual plots.
Effect on the result: The bands are computed by the delta method; the residual plots are graphics only.
On the validation data: The numerical output does not change.
Example on real data
Data: Treloar 1974 Puromycin (treated), R datasets
Variables: X conc (substrate concentration), Y rate (reaction velocity), function form Michaelis-Menten
- Question: In the puromycin-treated enzyme, does the reaction velocity (rate) follow a Michaelis-Menten relationship with substrate concentration (conc), and what are Vmax and Km?
- Model: rate = Vmax·conc / (Km + conc), nonlinear least squares; N = 12, 2 parameters.
- Result: Vmax = 212.68 (SE = 6.95), 95% CI [197.21, 228.16], t = 30.61, p < .001; Km = 0.0641 (SE = 0.0083), 95% CI [0.0457, 0.0826], t = 7.74, p < .001.
- Model fit: R² = .961, RMSE = 9.98; the model explains the data very well.
- Interpretation: The maximum velocity the enzyme can reach is about 213 units; the substrate concentration at which the velocity reaches half of this is about 0.064.
- APA: Michaelis-Menten model: Vmax = 212.68, SE = 6.95, 95% CI [197.21, 228.16]; Km = 0.064, SE = 0.008, 95% CI [0.046, 0.083]; R² = .96.
Related concepts
Analysis of variance (ANOVA) tableBootstrap (resampling)Nonlinear least squares and starting valuesWeighted least squares (WLS)Prediction intervalInformation criteria (AIC, AICc, BIC)Regression coefficient (B) and parameter estimatesOrdinary least squares (OLS)
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.