Statistics Glossary → Response Surface Regression (RSREG)
Response Surface Regression (RSREG)analysis
A method that models how an outcome in a designed experiment changes with two or more numeric factors using a second-order surface, and searches for the factor levels that make the outcome highest or lowest.
Concept
In a process, several settings can be changed at once: oven temperature and baking time, amounts of fertiliser and water, the doses of two components of a drug. The goal is often to find the 'best' combination. Trying the factors one at a time misses how they affect each other. Response surface methodology uses data from a planned experiment to estimate the shape of the outcome as a surface over the factors and searches for the top or bottom of that surface.
The model is a full second-order one: a linear term and a squared term for each factor, plus a cross-product term for each pair of factors. For two factors: y = β₀ + β₁x₁ + β₂x₂ + β₁₁x₁² + β₂₂x₂² + β₁₂x₁x₂ + ε. The squared terms capture curvature, and the cross term captures how one factor's effect changes with the level of the other. Canonical analysis finds the stationary point where the surface's slope is zero and, from the signs of the eigenvalues, says whether it is a maximum (all negative), a minimum (all positive) or a saddle point (mixed signs). If the stationary point lies outside the experimental region, ridge analysis shows in which direction the best response lies as one moves away from the design centre.
The second-order model is a reliable approximation only within the range of factor levels tried; predictions outside this range can be misleading. If there are replicated runs at the centre point, the lack-of-fit test checks whether the second-order model is adequate. If the design is not balanced (e.g. central composite or Box-Behnken), the coefficients are estimated in a correlated way and interpretation becomes harder.
When to use it
- To model how an outcome (yield, quality, cost) changes with the levels of several continuous factors and to find the best combination, with data collected from a response surface design such as a central composite or Box-Behnken design.
- If it is not yet known which factors matter, two-level factorial or fractional factorial screening experiments come first; response surface methodology is used after the important factors are identified.
- If the relationship is poorly approximated by a second-order surface (significant lack of fit), higher-order models or flexible methods such as LOESS can be considered.
An everyday example
A bakery wants to find the oven temperature and baking time that give its bread the highest tasting score. Thirteen planned runs are made with temperature between 200 and 240 °C and time between 25 and 40 minutes, and each loaf is scored by a tasting panel. The response surface model might show that the score peaks in both factors and that the highest score is expected at, say, 222 °C for 32 minutes. If both eigenvalues in the canonical analysis are negative, this point is truly a maximum.
Parameters in MerQur
Factors
Concept: Factors are the numeric settings the researcher deliberately varies in the experiment. The model automatically builds the linear and squared terms for each factor and the cross-product term for each pair of factors.
Everyday example: In the bakery example the factors are temperature and baking time. The model contains temperature, time, temperature², time² and temperature x time; the last term answers 'is a shorter time enough at a higher temperature?'.
What it does: Selects the numeric factors varied in the experiment; linear, square and cross-product terms enter the model for a second-order surface.
Effect on the result: Determines the model size and all outputs. Only the full second-order model is supported.
On the validation data: smell (Box-Behnken, n = 15, three factors): R2 0.881989, model F 4.152122 (p 0.065691); MBT (n = 12, two factors): R2 0.800257, model F 4.807726 (p 0.041029).
Covariates (optional)
Concept: Covariates are numeric variables that are not part of the surface but affect the outcome. They enter the model before the surface terms to remove their influence; the best-point estimates are given at the covariates' mean values.
Everyday example: If the humidity of the bakery changed across test days and humidity affects the crust, humidity is added as a covariate. The effects of temperature and time are then estimated cleared of the humidity differences.
What it does: Enters numeric covariates that are not part of the surface before the surface terms. Canonical and ridge predictions are given at the covariate means.
Effect on the result: Changes the error sum of squares and the factor tests; the factor coefficients are adjusted for the covariate.
On the validation data: Experiment (Day block): without covariates total error SS 9405.129724, model F 0.530816; adding Grade gives total error SS 0.10082, R2 0.999993.
Target column
Concept: The target is the numeric outcome measured in the experiment that is to be optimised (made highest or lowest).
Everyday example: The target is the 0-100 score the tasting panel gives each loaf.
What it does: Selects the numeric response variable.
Effect on the result: Determines all outputs.
On the validation data: ChemReact: Yield, response mean 80.585714.
Block (optional)
Concept: A block is a categorical variable representing groups of runs made under different conditions (day, batch, machine). When the level difference between blocks is removed from the model, the factor effects are estimated more precisely.
Everyday example: If half the 13 runs were made on Monday and half on Tuesday with a different sack of flour, day is added as a block. The Tuesday flour lowering all scores by 5 points is then not mistaken for an effect of temperature or time.
What it does: Enters a categorical block variable as dummies before the surface terms.
Effect on the result: Separates the block variance from the error; canonical and ridge predictions are given at the block mean.
On the validation data: ChemReact with blocks: R2 0.998082, model F 607.169689, lack-of-fit F 0.530712 (p 0.685088); without blocks: R2 0.282593, model F 0.630254.
Coding (SAS (midrange / half-range), No coding (original units))
Concept: Coding transforms each factor so that the middle of its experimental range is 0 and the edges are -1 and +1. In coded units the coefficients do not depend on the factors' units and can be compared; coefficients in original units translate directly into practical settings.
Everyday example: If temperature was tried between 200 and 240 °C, 220 °C corresponds to coded 0 and 240 °C to coded +1. The question 'does temperature matter more than time?' can then be answered without comparing degrees with minutes.
What it does: In midrange / half-range coding, code = (x - M) / S, M = (max + min) / 2, S = (max - min) / 2; the factors fall in -1..1 (default).
Effect on the result: Runs the canonical and ridge analyses on the coded scale; the ANOVA is not affected by coding.
On the validation data: ChemReact: Time M = 85, S = 7.07; Temp M = 175, S = 7.07.
Decomposition order (SAS RSREG (linear, quadratic, cross), R rsm (FO, TWI, PQ))
Concept: The sequential (Type I) sums of squares divide the explained variation according to the order in which the term groups (linear, quadratic, cross-product) enter the model. If the design is orthogonal the order does not matter; if not, the group entered first may get a larger share.
Everyday example: In a balanced central composite design, the orders 'linear, quadratic, cross' and 'linear, cross, quadratic' give the same table. If some runs failed and were dropped, the design is no longer balanced and the two orders can show different shares.
What it does: Sets the term order of the sequential (Type I) sums of squares: linear, square, cross-product or linear, cross-product, square.
Effect on the result: In an unbalanced design the Square and Cross-product SS can differ between the two orders; in orthogonal designs they are the same.
On the validation data: smell: linear 7143.25, square 11445.233333, cross-product 293.5; all four designs are orthogonal, so both orders give the same values.
Ridge analysis (Maximum response, Minimum response, Both, None)
Concept: Ridge analysis finds, on circles of increasing size around the design centre, the factor combination that makes the predicted response highest (or lowest). If the stationary point is a saddle or lies outside the experimental region, it shows in which direction to look for the best result.
Everyday example: If the canonical analysis found a saddle point, the 'best setting' is not a single point. Ridge analysis shows in which direction (e.g. higher temperature with shorter time) the score rises fastest away from the centre, and guides where to run the next experiment.
What it does: Finds the factor levels that maximise or minimise the prediction at increasing radii from the coded centre (Draper 1963).
Effect on the result: Determines only the ridge output (location, prediction, SE); does not change the model.
On the validation data: MBT: the prediction, SE and factor levels are given for 11 radii.
Maximum radius (default 1.0) and Radius step (default 0.1)
Concept: Sets, in coded units, how far out and at what intervals the ridge analysis goes. Radius 1 is the edge of the experimental region; predictions beyond it extend into untested conditions.
Everyday example: With the default settings, the best settings are listed in 11 steps from the centre to the edge. If the radius is raised to 1.5, remember that suggestions reaching beyond 240 °C are predictions not checked by experiment.
What it does: Sets the radius range of the ridge analysis in coded units (1 = edge of the experimental region).
Effect on the result: Changes only the number of rows and the range of the ridge table.
On the validation data: With the defaults, 11 radii (MBT).
Save to dataset ([Parameters] check box)
Concept: Adds to the data set, as new columns, the response predicted by the model for each run and its difference from the observed value (residual).
Everyday example: Each loaf's row gets 'expected score' and 'deviation from expected' columns. A run with a very large deviation might be, for instance, a bake during which the oven door was left open, and should be re-examined.
What it does: Adds the "<target>_rsm_pred" and "<target>_rsm_resid" columns.
Effect on the result: Does not change the computation.
On the validation data: ChemReact: the Yield_rsm_pred and Yield_rsm_resid columns are added.
Canonical analysis (automatic output)
Concept: Canonical analysis gives the stationary point where the surface's slope is zero and, through the eigenvalues, the curvature at that point. If all eigenvalues are negative the point is a peak (maximum), if all are positive a trough (minimum), and if the signs are mixed a saddle point. An eigenvalue near zero indicates a long ridge along which the response barely changes.
Everyday example: If the stationary point is at 222 °C and 32 minutes and both eigenvalues are negative, this is a true peak. If one eigenvalue is very close to zero, different settings, such as a slightly higher temperature with a slightly shorter time, can be chosen with almost no loss in score, which may also save energy.
What it does: Gives the stationary point (exact inverse), eigenvalues, eigenvectors and the surface type (minimum, maximum, saddle). MerQur writes a warning for small eigenvalues.
Effect on the result: Does not change the model; reports the location and type of the optimum.
On the validation data: smell: prediction at the stationary point -52.024631, surface type minimum; MBT: 83.74194, saddle.
→ Stationary point and canonical analysis (response surface)
Example on real data
Data: ChemReact, Myers and Montgomery, R rsm
Variables: factors Time and Temp (temperature), target Yield, block Block
- Question: Which levels of time (Time) and temperature (Temp) maximise the yield (Yield) of a chemical reaction? (ChemReact, two blocks, n = 14)
- Model: Block + full second-order surface; R² = .998, adjusted R² = .996, Root MSE = 0.163, model F = 607.17, p < .001.
- Decomposition: Block SS = 69.53 (F = 2611.10), linear F = 180.73, square F = 334.05 (p < .001), cross-product F = 2.35 (p = .169).
- Lack of fit: F(3, 4) = 0.53, p = .685; the second-order model is adequate.
- Canonical analysis: Eigenvalues -1.846 and -2.637 (both negative): the stationary point is a maximum; Time = 86.86, Temp = 176.67, predicted yield 82.14.
- Additional finding: The block effect is large (B2 coefficient -4.46); if the block is removed from the model, R² falls to .28.
- APA: Second-order response surface, R² = .998, F(6, 7) = 607.17, p < .001; lack of fit F(3, 4) = 0.53, p = .685; maximum at Time = 86.9, Temp = 176.7.
Related concepts
Ridge analysis (response surface)Blocking (block variable)Coded factors (response surface)Stationary point and canonical analysis (response surface)Types of sums of squares (Type I, II, III)CovariateEigenvalue and eigenvectorDummy (indicator) variable coding
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.