Statistics Glossary → Cost parameter C and the ε-insensitive band
Cost parameter C and the ε-insensitive bandconcept
The C parameter of a support vector machine, which balances the width of the margin against the penalty for training errors, and ε, which sets the unpenalized error band in support vector regression.
A soft-margin SVM tries to separate two classes with as wide a margin as possible while allowing some observations to fall inside the margin or on the wrong side. C sets how heavily each such violation is penalized. A large C tolerates few training errors; the margin narrows and the model fits the training data tightly (risk of overfitting). A small C allows a wider margin and a smoother decision boundary, acting like stronger regularization.
In support vector regression (SVR), predictions that deviate from the true value by less than ε are not penalized at all; only points outside this 'tube' become support vectors and contribute to the loss. A larger ε gives fewer support vectors and a flatter function, while a smaller ε gives a more detailed fit. Because C and ε (and γ for the RBF kernel) interact, they are usually chosen together by cross-validation; scaling the variables is essential.
An everyday example
A rope is to be stretched across a schoolyard to separate the players of two teams. If no player may end up on the wrong side, the rope follows a narrow zigzag path because of a few scattered players (large C). If a few players are allowed to stay on the other side, the rope can be pulled straight, leaving a wide gap between the two teams (small C).
In MerQur
The 'C (cost)' (misclassification penalty; large C means a narrow margin) and 'ε (epsilon)' (width of the SVR epsilon-insensitive band; not used in classification) parameters of the Support Vector Machine.
Analyses where this concept appears
Related concepts
SVM kernel function (rbf, linear, polynomial) and γRegularization (penalty) and the penalty parameter λ/α