MerQuR+Statistics Glossary

Statistics Glossary → Cost parameter C and the ε-insensitive band

Cost parameter C and the ε-insensitive bandconcept

The C parameter of a support vector machine, which balances the width of the margin against the penalty for training errors, and ε, which sets the unpenalized error band in support vector regression.

A soft-margin SVM tries to separate two classes with as wide a margin as possible while allowing some observations to fall inside the margin or on the wrong side. C sets how heavily each such violation is penalized. A large C tolerates few training errors; the margin narrows and the model fits the training data tightly (risk of overfitting). A small C allows a wider margin and a smoother decision boundary, acting like stronger regularization.

In support vector regression (SVR), predictions that deviate from the true value by less than ε are not penalized at all; only points outside this 'tube' become support vectors and contribute to the loss. A larger ε gives fewer support vectors and a flatter function, while a smaller ε gives a more detailed fit. Because C and ε (and γ for the RBF kernel) interact, they are usually chosen together by cross-validation; scaling the variables is essential.

An everyday example

A rope is to be stretched across a schoolyard to separate the players of two teams. If no player may end up on the wrong side, the rope follows a narrow zigzag path because of a few scattered players (large C). If a few players are allowed to stay on the other side, the rope can be pulled straight, leaving a wide gap between the two teams (small C).

In MerQur

The 'C (cost)' (misclassification penalty; large C means a narrow margin) and 'ε (epsilon)' (width of the SVR epsilon-insensitive band; not used in classification) parameters of the Support Vector Machine.

Analyses where this concept appears

Support Vector Machine (SVM)

Related concepts

SVM kernel function (rbf, linear, polynomial) and γRegularization (penalty) and the penalty parameter λ/α