The purpose of this book is to provide up-to-date progress both in Multiple Criteria Programming (MCP) and Support Vector Machines (SVMs) that have become powerful tools in the field of data mining. Most of the content in this book are directly from the research and application activities that our research group has conducted over the last ten years.

Although the data mining community is familiar with Vapnik’s SVM [206] in classification, using optimization techniques to deal with data separation and data analysis goes back more than fifty years. In the 1960s, O.L. Mangasarian formulated the principle of large margin classifiers and tackled it using linear programming. He and his colleagues have reformed his approaches in SVMs [141]. In the 1970s, A. Charnes and W.W. Cooper initiated Data Envelopment Analysis, where linear or quadratic programming is used to evaluate the efficiency of decision-making units in a given training dataset. Started from the 1980s, F. Glover proposed a number of linear programming models to solve the discriminant problem with a small-size of dataset [75]. Since 1998, the author and co-authors of this book have not only proposed and extended such a series of optimization-based classification models via Multiple Criteria Programming (MCP), but also improved a number of SVM related classification methods. These methods are different from statistics, decision tree induction, and neural networks in terms of the techniques of separating data.

When MCP is used for classification, there are two common criteria. The first one is the overlapping degree (e.g., norms of all overlapping) with respect to the separating hyperplane. The lower this degree, the better the classification. The second is the distance from a point to the separating hyperplane. The larger the sum of these distances, the better the classification. Accordingly, in linear cases, the objective of classification is either minimizing the sum of all overlapping or maximizing the sum of the distances. MCP can also be viewed as extensions of SVM. Under the framework of mathematical programming, both MCP and SVM share the same advantage of using a hyperplane for separating the data. With certain interpretation, MCP measures all possible distances from the training samples to separating hyperplane, while SVM only considers a fixed distance from the support vectors. This allows MCP approaches to become an alternative for data separation.