SVM

Multi-class SVM

Theory

SVMs are inherently binary classifiers. For multi-class problems, we use strategies to combine multiple binary SVMs. One-vs-Rest and One-vs-One are the most common approaches.

Visualization

Mathematical Formulation

Strategies:
• One-vs-Rest (OvR):
  Train K binary classifiers
  Predict class with highest confidence

• One-vs-One (OvO):
  Train K(K-1)/2 classifiers
  Use voting for final class

Trade-offs:
OvR: Faster, K classifiers
OvO: More accurate, K(K-1)/2 classifiers

Code Example

from sklearn import svm
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

# Load multi-class dataset
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.3, random_state=42
)

# One-vs-Rest
clf_ovr = svm.SVC(kernel='rbf', decision_function_shape='ovr')
clf_ovr.fit(X_train, y_train)

# One-vs-One
clf_ovo = svm.SVC(kernel='rbf', decision_function_shape='ovo')
clf_ovo.fit(X_train, y_train)

# Evaluate
y_pred_ovr = clf_ovr.predict(X_test)
y_pred_ovo = clf_ovo.predict(X_test)

print("One-vs-Rest Performance:")
print(classification_report(y_test, y_pred_ovr))

print("\nOne-vs-One Performance:")
print(classification_report(y_test, y_pred_ovo))

Kernel SVM

Content-based RecSys