Catalogue of Artificial Intelligence Techniques
Support vector machines.
Author(s): Alfred Mazimbe
Support Vector Machines, from now on abbreviated SVM, are a supervised, non-parametric learning technique for classifying a set of data points using a hyperplane. The orientation and position of this plane are determined by a normal vector w and bias b. Although, in practice, the complexity of the boundaries may necessitate use of non-linear surfaces, it turns out that if non-linear problems can be transformed into higher-dimensional space using the so-called kernel functions; the classes in the transformed space are often linearly separable.
Every linear classifier has a maximum margin; defined as the maximum horizontal distance that the boundary could be increased by before hitting a data point. Hence, support vectors are precisely those data points that the classifier pushes up against. Furthermore, according to a theorem from Learning Theory, the linear decision function that maximises the margin of the training set minimises the generalisation error. Consequently, an SVM maximises its margin subject to the constraint that training examples are classified correctly. Its goal is to produce a model which predicts the classes of new data instances based solely on their attributes.
- Richard O. Duda,
Peter E. Hart, Richard O. Duda, David G. Stork, Pattern Classification.