AysamByte

Stochastic Gradient Descent (SGD) is an effective optimization algorithm for estimating coefficients/parameters of functions that minimize a cost function. It is utilized in discriminative learning for linear classifiers like SVM and Logistic regression, making it suitable for large datasets by updating coefficients for each training instance.

SGD Classifier

The SGD classifier implements a simple SGD learning, considering various loss functions and penalties for the classification process. The following example uses this classifier and displaying several features, such as the weight vector.

import numpy as np
from sklearn import linear_model

data = np.array([[1, 2], [-3, 2], [-1, 2], [-1, -1]])
target = np.array([2, 2, 1, 1])

classifier = linear_model.SGDClassifier(max_iter = 1000, tol = 1e-3, penalty = "elasticnet")
classifier.fit(data, target)

print(classifier.predict([[2., 2.]]))
print(classifier.coef_)
print(classifier.intercept_)
print(classifier.decision_function([[2., 2.]]))

Output

[1]
[[-5.77589042  6.78156872]]
[-8.46908595]
[-6.45772936]

SGD Regressor

The SGD regressor implements a simple SGD learning routine for fitting linear regression with various loss functions and penalties. The following example applies this classifier and displaying several features of the model.

import numpy as np
from sklearn import linear_model

num_samples, num_features = 10, 5

rnd = np.random.RandomState(0)
data = rnd.randn(num_samples, num_features)
target = rnd.randn(num_samples)

regressor = linear_model.SGDRegressor(max_iter = 1000, penalty = "elasticnet", loss = 'huber', tol = 1e-3, average = True)
regressor.fit(data, target)

print(regressor.coef_)
print(regressor.intercept_)
print(regressor.t_)

Output

[-2.66643137e-03  3.50083938e-03  5.86714922e-05 -4.82618772e-03
  5.90207257e-05]
[-0.00252676]
61.0

References

1. Hackeling, G. (2017). Mastering Machine Learning with scikit-learn, 2nd Edition. Packt Publishing Ltd.
2. Géron, A. (2019). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition. O’Reilly Media, Inc.
3. Tutorials Point. Scikit Learn Tutorial. Retrieved November 20, 2025, from https://www.tutorialspoint.com/.
4. DataCamp. Support Vector Machines with Scikit-learn Tutorial. Retrieved December 25, 2025, from https://www.datacamp.com/.