Traditional programming encodes explicit rules. Machine learning inverts this: given examples of inputs and desired outputs, an algorithm discovers the rules itself. The discovered rules are encoded in a model, which can then predict outputs for new inputs.

Core Vocabulary

  • Feature — an input variable used to make a prediction (e.g., house size, number of rooms)
  • Label — the output we want to predict (e.g., house price)
  • Training set — labelled examples used to fit the model
  • Validation set — held-out examples used to tune hyperparameters
  • Test set — unseen examples used for final evaluation only
  • Overfitting — model memorises the training data and generalises poorly

Supervised vs Unsupervised Learning

  • Supervised learning — train on labelled data to predict a target. Subtypes: classification (discrete labels) and regression (continuous values).
  • Unsupervised learning — find structure in unlabelled data. Examples: clustering (K-means), dimensionality reduction (PCA).
  • Semi-supervised learning — a small labelled set combined with a large unlabelled set.
  • Reinforcement learning — an agent learns by taking actions and receiving rewards.

Evaluation Metrics

metrics.pypython
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score

# Classification
accuracy  = accuracy_score(y_true, y_pred)   # correct / total
precision = precision_score(y_true, y_pred)  # TP / (TP + FP)
recall    = recall_score(y_true, y_pred)     # TP / (TP + FN)
f1        = f1_score(y_true, y_pred)         # harmonic mean of precision & recall

# Regression
from sklearn.metrics import mean_squared_error, r2_score
mse = mean_squared_error(y_true, y_pred)
r2  = r2_score(y_true, y_pred)  # 1.0 = perfect, 0 = baseline mean