Support vector machine (SVM) is an effective supervised learning method for classification, regression, and outlier detection, particularly in high-dimensional spaces, utilizing a subset of training points for efficiency and memory savings. It aims to classify datasets by identifying a maximum marginal hyperplane (MMH) through two steps: (i) generating optimal hyperplanes to separate classes and (ii) choosing the hyperplane that correctly separates classes.
1. Classification of SVM
Scikit-Learn offers three classes for performing multi-class classification, as described below.
1.1. Support Vector Classifier (SVC)
SVC is a C-support vector classification, implemented using the libsvm library. This class implements multiclass support based on the one-vs-one scheme. SVC model relies on a training data subset, as the cost function ignores points outside the margin during model building. The following example shows how to use this classification approach.
import numpy as np
from sklearn.svm import SVC
data = np.array([[-1, -1], [-2, -1], [1, 1], [2, 1]])
target = np.array([1, 1, 2, 2])
classifier = SVC(kernel = 'linear', gamma = 'scale', shrinking = False)
classifier.fit(data, target)
print(classifier.coef_)
print(classifier.predict([[-0.5,-0.8]]))
print(classifier.n_support_)
print(classifier.support_vectors_)
print(classifier.support_)
print(classifier.intercept_)
print(classifier.fit_status_)Output
[[0.5 0.5]]
[1]
[1 1]
[[-1. -1.]
[ 1. 1.]]
[0 2]
[-0.]
0The example below also illustrates how to apply this classification based on the breast_cancer dataset.
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn import svm
from sklearn import metrics
dataset = datasets.load_breast_cancer()
print("Name of the features:\n", dataset.feature_names, "\n")
print("Label type of cancer:", dataset.target_names, "\n")
print("Shape of the dataset:", dataset.data.shape, "\n")
print("Cancer data features (top 5 records):\n", dataset.data[0:5], "\n")
print("Cancer labels (top 5 records):", dataset.target[0:5], "\n")
data_train, data_test, target_train, target_test = train_test_split(dataset.data, dataset.target, test_size = 0.3, random_state = 109)
classifier = svm.SVC(kernel = 'linear')
classifier.fit(data_train, target_train)
target_prediction = classifier.predict(data_test)
print("Accuracy of the model:", metrics.accuracy_score(target_test, target_prediction))
print("Precision of the model:", metrics.precision_score(target_test, target_prediction))
print("Recall of the model:", metrics.recall_score(target_test, target_prediction))Output
Name of the features:
['mean radius' 'mean texture' 'mean perimeter' 'mean area'
'mean smoothness' 'mean compactness' 'mean concavity'
'mean concave points' 'mean symmetry' 'mean fractal dimension'
'radius error' 'texture error' 'perimeter error' 'area error'
'smoothness error' 'compactness error' 'concavity error'
'concave points error' 'symmetry error' 'fractal dimension error'
'worst radius' 'worst texture' 'worst perimeter' 'worst area'
'worst smoothness' 'worst compactness' 'worst concavity'
'worst concave points' 'worst symmetry' 'worst fractal dimension']
Label type of cancer: ['malignant' 'benign']
Shape of the dataset: (569, 30)
Cancer data features (top 5 records):
[[1.799e+01 1.038e+01 1.228e+02 1.001e+03 1.184e-01 2.776e-01 3.001e-01
1.471e-01 2.419e-01 7.871e-02 1.095e+00 9.053e-01 8.589e+00 1.534e+02
6.399e-03 4.904e-02 5.373e-02 1.587e-02 3.003e-02 6.193e-03 2.538e+01
1.733e+01 1.846e+02 2.019e+03 1.622e-01 6.656e-01 7.119e-01 2.654e-01
4.601e-01 1.189e-01]
[2.057e+01 1.777e+01 1.329e+02 1.326e+03 8.474e-02 7.864e-02 8.690e-02
7.017e-02 1.812e-01 5.667e-02 5.435e-01 7.339e-01 3.398e+00 7.408e+01
5.225e-03 1.308e-02 1.860e-02 1.340e-02 1.389e-02 3.532e-03 2.499e+01
2.341e+01 1.588e+02 1.956e+03 1.238e-01 1.866e-01 2.416e-01 1.860e-01
2.750e-01 8.902e-02]
[1.969e+01 2.125e+01 1.300e+02 1.203e+03 1.096e-01 1.599e-01 1.974e-01
1.279e-01 2.069e-01 5.999e-02 7.456e-01 7.869e-01 4.585e+00 9.403e+01
6.150e-03 4.006e-02 3.832e-02 2.058e-02 2.250e-02 4.571e-03 2.357e+01
2.553e+01 1.525e+02 1.709e+03 1.444e-01 4.245e-01 4.504e-01 2.430e-01
3.613e-01 8.758e-02]
[1.142e+01 2.038e+01 7.758e+01 3.861e+02 1.425e-01 2.839e-01 2.414e-01
1.052e-01 2.597e-01 9.744e-02 4.956e-01 1.156e+00 3.445e+00 2.723e+01
9.110e-03 7.458e-02 5.661e-02 1.867e-02 5.963e-02 9.208e-03 1.491e+01
2.650e+01 9.887e+01 5.677e+02 2.098e-01 8.663e-01 6.869e-01 2.575e-01
6.638e-01 1.730e-01]
[2.029e+01 1.434e+01 1.351e+02 1.297e+03 1.003e-01 1.328e-01 1.980e-01
1.043e-01 1.809e-01 5.883e-02 7.572e-01 7.813e-01 5.438e+00 9.444e+01
1.149e-02 2.461e-02 5.688e-02 1.885e-02 1.756e-02 5.115e-03 2.254e+01
1.667e+01 1.522e+02 1.575e+03 1.374e-01 2.050e-01 4.000e-01 1.625e-01
2.364e-01 7.678e-02]]
Cancer labels (top 5 records): [0 0 0 0 0]
Accuracy of the model: 0.9649122807017544
Precision of the model: 0.9811320754716981
Recall of the model: 0.96296296296296291.2. Nu Support Vector Classification (NuSVC)
This class performs similarly to SVC but supports a slightly different set of parameters. The nu parameter, which differentiates this class from SVC, contains an upper bound on the fraction of training errors and a lower bound on the fraction of support vectors. The example below represents how to apply this classification.
import numpy as np
from sklearn.svm import NuSVC
data = np.array([[-1, -1], [-2, -1], [1, 1], [2, 1]])
target = np.array([1, 1, 2, 2])
classifier = NuSVC(kernel = 'linear', gamma = 'scale', shrinking = False)
classifier.fit(data, target)
print(classifier.coef_)
print(classifier.predict([[-0.5,-0.8]]))
print(classifier.n_support_)
print(classifier.support_vectors_)
print(classifier.support_)
print(classifier.intercept_)
print(classifier.fit_status_)Output
[[0.4 0.4]]
[1]
[1 1]
[[-1. -1.]
[ 1. 1.]]
[0 2]
[-0.]
01.3. Linear Support Vector Classification (LinearSVC)
This class also performs similarly to SVC, but is implemented using liblinear library. It offers more flexible options for penalties and loss functions, as well as better scalability for large samples. The following example indicates how to use this class.
from sklearn.datasets import make_classification
from sklearn.svm import LinearSVC
data, target = make_classification(n_features = 5, random_state = 0)
classifier = LinearSVC(dual = False, random_state = 0, penalty = 'l1', tol = 1e-5)
classifier.fit(data, target)
print(classifier.predict([[0, 0, 0, 0, 0]]))
print(classifier.coef_)
print(classifier.intercept_)Output
[1]
[[0.91673018 0.21709621 0.04026653 0. 0. ]]
[0.27494206]2. Regression With SVM
As mentioned above, SVM is utilized for both classification and regression. Scikit-Learn offers SVR, NuSVR, and LinearSVR implementations for regression.
2.1. Support Vector Regression (SVR)
Scikit-Learn’s SVC can be extended to regression, known as Support Vector Regression (SVR). The SVR model relies on a subset of training data, as its cost function disregards points near the model’s predictions during construction. This is an epsilon-support vector regression based on the libsvm library, which has two parameters: C and epsilon. The following example illustrates how to use this model and displays the results obtained.
from sklearn import svm
data = [[2, 2], [1, 1]]
target = [2, 1]
regressor = svm.SVR(kernel = 'linear', gamma = 'auto')
regressor.fit(data, target)
print(regressor.coef_)
print(regressor.predict([[1,1]]))Output
[[0.4 0.4]]
[1.1]2.2. Nu Support Vector Regression (NuSVR)
NuSVR uses nu to control the number of support vectors, replacing epsilon in SVR, unlike NuSVC which replaces the C parameter. The example below uses this regression method.
import numpy as np
from sklearn.svm import NuSVR
num_samples, num_features = 10, 5
np.random.seed(0)
data = np.random.randn(num_samples, num_features)
target = np.random.randn(num_samples)
regressor = NuSVR(kernel = 'linear', gamma = 'auto', C = 1.0, nu = 0.1)
regressor.fit(data, target)
print(regressor.coef_)Output
[[-0.12568358 -0.10508326 0.1170252 -0.11592755 0.14527754]]2.3. Linear Support Vector Regression (LinearSVR)
LinearSVR, like SVR with a linear kernel, differs as it uses the liblinear library instead of the libsvm library, offering greater flexibility in choosing penalties and loss functions, and better scalability for large samples. The following example shows how to use this method.
from sklearn.datasets import make_regression
from sklearn.svm import LinearSVR
data, target = make_regression(n_features = 5, random_state = 0)
classifier = LinearSVR(dual = False, random_state = 0, loss = 'squared_epsilon_insensitive', tol = 1e-5)
classifier.fit(data, target)
print(classifier.predict([[0, 0, 0, 0, 0]]))
print(classifier.coef_)
print(classifier.intercept_)Output
[-0.09281428]
[97.43345055 45.45394833 11.78184625 85.35650891 42.07996087]
[-0.09281428]References
- Hackeling, G. (2017). Mastering Machine Learning with scikit-learn, 2nd Edition. Packt Publishing Ltd.
- Géron, A. (2019). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition. O’Reilly Media, Inc.
- Tutorials Point. Scikit Learn Tutorial. Retrieved November 20, 2025, from https://www.tutorialspoint.com/.
- DataCamp. Support Vector Machines with Scikit-learn Tutorial. Retrieved December 25, 2025, from https://www.datacamp.com/.
Leave a Reply