Statistics Glossary → SVM kernel function (rbf, linear, polynomial) and γ
SVM kernel function (rbf, linear, polynomial) and γconcept
The function in a support vector machine that defines the similarity between observations and so determines the shape of the decision boundary (straight, curved, flexible); in kernels such as rbf, γ sets how local this similarity is.
A support vector machine looks for the boundary that separates the classes with the widest margin. If the classes cannot be separated by a straight line, the kernel function implicitly maps the data into a higher-dimensional space and seeks a linear separation there, which corresponds to a curved boundary in the original space. The linear kernel uses the inner product of two observations; the polynomial kernel has the form (γ·u'v + coef0)^degree; and the radial basis function (rbf) kernel, exp(-γ·|u - v|²), makes similarity fall off with distance.
In the rbf kernel, a larger γ confines each observation's influence to a narrower region and the boundary becomes very wiggly, drifting into overfitting; a smaller γ smooths the boundary and brings it closer to the linear kernel. γ should be tuned together with the penalty parameter C (usually by grid search and cross-validation). Because kernels rest on distances and inner products, the variables must be standardized. When the number of predictors is very large relative to the number of observations, a linear kernel is often enough and easier to interpret.
An everyday example
The positions of two plant species are mapped in a garden: one grows in a round patch in the middle, the other around it. No straight line can separate them, but an SVM with an rbf kernel can draw a circular boundary around the inner patch. If γ is set too large, the boundary forms little islands around each individual plant and gets new plants wrong.
In MerQur
It appears as the 'Kernel' (rbf, linear, poly, sigmoid), 'gamma' (0 = 'scale') and 'Degree (poly) and coef0' options in the Support Vector Machine.