Guide 🗃️ Where to find free datasets & clean them — beginner's field guide

Every "learn data" tutorial hands you a spotless toy dataset. Real life hands you 40,000 rows with dates in three formats, names in ALLCAPS and a column called "Unnamed: 7". This guide fixes both halves: where to get free data, and how to make it usable.

🌍 The free dataset goldmines (bookmark all six)

• Kaggle Datasets — the biggest playground; every dataset comes with public notebooks so you can see how others analysed it. Steal like an analyst.
  
• data.gov.in — Indian government open data: agriculture, transport, health, economy. Underused, unglamorous, gold for India-specific projects.
  
• Hugging Face Datasets — the AI community's library; text, images, audio, ready for ML.
  
• Google Dataset Search — searches across thousands of public repositories. When you know WHAT you want but not WHO publishes it.
  
• Our World in Data — beautifully cleaned global stats (great when you need trustworthy economics/health fast).
  
• awesome-public-datasets (GitHub) — the giant curated list of lists.
  

🎯 Pick a learning dataset, not a trophy

1. Under ~50MB — your laptop and your patience will thank you
   
2. Published/updated recently — stale data teaches stale lessons
   
3. A little dirty — that's the point. Spotless CSVs teach nothing.
   

🧹 The 5-step cleaning workflow (works in Sheets AND Python)

1. Duplicate the raw file first. Raw data is sacred — you will butcher something on try #1, everyone does.
   
2. Fix structure: kill blank rows, split merged cells, one column = one kind of thing.
   
3. Fix types: dates as dates, numbers as numbers. Watch for "₹1,200" stored as text and Excel eating leading zeros (phone numbers, PIN codes!).
   
4. Handle gaps: blank ≠ zero. Delete rows only when the missing value makes them useless; otherwise fill with median/average or a loud "Unknown".
   
5. Duplicates & outliers: exact-duplicate rows go. Outliers get investigated, not auto-deleted — that "₹9,99,999 phone bill" might be the real story.
   

🐍 The pandas version (save this)

Code:

import pandas as pd

df = pd.read_csv("messy.csv")
df = df.drop_duplicates()
df["date"] = pd.to_datetime(df["date"], dayfirst=True, errors="coerce")
df["price"] = pd.to_numeric(
    df["price"].astype(str).str.replace(r"[^\d.]", "", regex=True),
    errors="coerce")
df["price"] = df["price"].fillna(df["price"].median())
df["city"] = df["city"].str.strip().str.title()
df.to_csv("clean.csv", index=False)

Ten lines that cover 80% of everyday mess.

🇮🇳 Desi data traps nobody warns you about

• Lakhs & crores — "2.5L" in a salary column. Decide: convert to absolute numbers, always.
  
• DD/MM/YYYY vs MM/DD/YYYY — 07/08/2026 has entered the chat.
  
• Encoding ghosts — names in regional scripts turning into राम means it was read with the wrong encoding (try UTF-8 → latin1).
  
• State names — "TN", "Tamil Nadu", "tamilnadu", three different rows. Pick one canonical spelling and map everything to it.
  

🏆 The portfolio move
Clean a dataset → write up what you found ("3 surprising things in India's railway data") → publish the cleaned CSV + your notes back on Kaggle. That single artifact beats a certificate on any data resume, and we love seeing these in Data science basics.

More tools for the job: The 2026 AI tools map, and yes — AI can write cleaning code for you: see 25 ChatGPT prompts.

Your turn: What's the messiest dataset you've ever met? War stories below. 👇