Data Preprocessing in Machine Learning: Cleaning, Scaling & Encoding (Beginner's Guide)

So far in this series we've trained models, measured them properly, and learned to avoid overfitting. But every one of those lessons quietly assumed something that's almost never true in real life: that the data was already clean and ready to use.

It isn't. Real data arrives messy — full of blanks, wildly different number ranges, and text categories a model can't even read. Data preprocessing is the work of turning that mess into something an algorithm can actually learn from. There's a saying in machine learning that captures why this matters more than any fancy algorithm:

"Garbage in, garbage out." A brilliant model fed bad data will make bad predictions. Most of a data scientist's time goes here — not on the model.

This lesson covers the four preprocessing steps you'll use in nearly every project.

Step 1: Clean the data

Before anything clever, fix the obvious problems in the raw data:

• Duplicate rows — the same record entered twice skews the model. Drop them.
• Obvious errors — an age of 200, a negative price, a typo in a category ("Mumbai" vs "mumbai").
• Wrong data types — a number stored as text ("42" instead of 42), a date stored as a plain string.
• Irrelevant columns — an ID or a name that carries no predictive signal.

import pandas as pd

df = pd.read_csv("data.csv")

df = df.drop_duplicates()          # remove repeated rows
df = df.drop(columns=["user_id"])  # drop a useless column
df["price"] = df["price"].abs()    # fix negative prices

Step 2: Handle missing values

Real datasets have holes — a blank age, an unanswered survey field. Models generally can't handle these blanks (usually shown as NaN), so you must deal with them. You have two broad choices:

1. Remove the rows or columns with missing data. Fine if only a tiny fraction is missing — wasteful if you'd throw away lots of good data alongside the gap.
2. Impute (fill in) the missing values. Usually the better choice. A common approach: fill missing numbers with the column's median, and missing categories with the most frequent value (the "mode").

# fill missing ages with the median age
df["age"] = df["age"].fillna(df["age"].median())

# fill missing city with the most common city
df["city"] = df["city"].fillna(df["city"].mode()[0])

Why median, not mean? The median isn't dragged around by extreme outliers the way the average is — a single billionaire in an "income" column would wildly inflate the mean. (We covered this in the statistics lesson.)

Step 3: Scale the numbers

Look at these two features: age (range 0–100) and salary (range 0–200,000). To some algorithms — anything distance-based like KNN or SVM — salary looks thousands of times "bigger" than age, so it drowns age out entirely, purely because of its units. That's not fair, and it's not what we want.

Feature scaling puts every number on a comparable range so each feature gets a fair say. Two common methods:

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
# fit on TRAIN only, then apply to both (see warning below)
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled  = scaler.transform(X_test)

Standardization is the safe default for most algorithms. Tree-based models (Decision Tree, Random Forest) don't need scaling at all — but it never hurts them.

Step 4: Encode categorical data

Models do maths, and maths needs numbers — but a column like city = "Delhi", "Mumbai", "Chennai" is text. Encoding converts those categories into numbers. The two main techniques:

• One-Hot Encoding — creates a new 0/1 column for each category. "Delhi" becomes [1, 0, 0], "Mumbai" becomes [0, 1, 0]. Use this for categories with no natural order (cities, colours, brands). It's the safe default.
• Label Encoding — assigns each category a number: Delhi=0, Mumbai=1, Chennai=2. Only use this when the categories have a real order (e.g. "Low", "Medium", "High"), otherwise the model wrongly assumes Chennai (2) is "greater than" Delhi (0).

# one-hot encode the 'city' column (no natural order)
df = pd.get_dummies(df, columns=["city"])

# now: city_Delhi, city_Mumbai, city_Chennai as 0/1 columns

One critical rule: fit on training data only

This trips up almost every beginner, so it's worth stating loudly. When you scale or encode, you learn the settings from the training data only (the mean, the min/max, the list of categories) and then apply those same settings to the test data.

If you scale using the whole dataset before splitting, information from the test set "leaks" into training — called data leakage — and your test score becomes dishonestly high. That's why the code above calls fit_transform on X_train but only transform on X_test. In real projects, a scikit-learn Pipeline handles this ordering for you automatically.

Honestly, this is the step where I've seen the most "mysteriously amazing" results turn out to be a leakage bug. [Optional: swap in a specific time this bit you on a real project, so it reads as your own experience.]

In my own projects, the "mysteriously amazing" results almost always turned out to be a data-leakage bug from scaling before the split — not a genuinely good model.

Key takeaways

• Garbage in, garbage out — clean data matters more than a fancy model, and preprocessing is where most of the real work happens.
• Clean (dupes, errors, types) → handle missing values (remove or impute) → scale numbers (so units don't dominate) → encode text (one-hot for unordered, label for ordered).
• Use median to fill missing numbers (robust to outliers) and the mode for categories.
• Always fit scalers/encoders on training data only to avoid data leakage.

Continue the series: ← Lesson 12: Overfitting & Underfitting  ·  Next: Lesson 14 — Your First ML Project →