Support vector machine

202307201829
Status:
Tags: Machine Learning Supervised learning Statistics Kernel methods

Tldr

The objective of the SVM is to find a hyperplane that separates data points of one class from those of another. The best hyperplane is the one with the largest margin between the two classes.

Margin is the maximal width of the slab parallel to the hyperplane that has no interior datapoints.

Only linearly separable problems can be solved. Practically, the algorithm maximizes the soft margin, allowing a small number of misclassifications.

• Formulated for binary classification problems
  • Multiclass problems are reduced to a series of binary ones
• Good at classification and regression
• Kernels make SVMs flexible & able to handle nonlinear problems
• Features can be transformed using a kernel function.

Type of SVM	Mercer Kernel	Description
Gaussian, Radial Basis Function	$K(x1,x2)=exp(-\frac{\Vert x1-x2\Vert }{2\sigma 2})$	One class learning. $\sigma$ is width of kernel.
Linear	$K(x_1,x_2)=x_1^T{x}_2$	Two class learning
Polynomial	$K(x_1,x_2)=(x_1^T{x}_2+1{)}^Q$	$Q$ is the order of the polynomial
Sigmoid	$K(x_1,x_2)=tanh({\beta }_0{x}_1^T{x}_2+{\beta }_1)$	Only mercer kernel for certain ${\beta }_0$ and ${\beta }_1$ values

Training an SVM → quadratic optimization problem

• Fit hyperplane minimizing soft margin between classes
• Number of transformed features is determined by number of support vectors

---

---

References