Why do I get an 'Unnamed: 0' column after merging?

A file was saved with its pandas index as the first column. Pass index=False when writing, or index_col=0 when reading that file, so the old index isn't treated as data.

How do I merge CSV files that use different delimiters?

Read each one with the right sep= value (for example ';' for many European exports) before concatenating. pandas normalises them, and to_csv writes one consistent delimiter.

How do I add the file name as a column?

Assign it before concatenating: df["source_file"] = f.name inside the loop, as in the first example.

How do I merge CSV files with different columns in Python?

pd.concat aligns by column name and fills missing columns with NaN. Use join="inner" in concat if you only want the columns all files share.

How do I avoid repeating the header when combining CSVs?

With pandas it's handled for you, since each file's header becomes column names. With the csv module, write the header only for the first file, as in the example.

Is glob order guaranteed?

No. glob returns files in an arbitrary, filesystem-dependent order. Wrap it in sorted() for predictable output.

Can I merge CSVs without loading them into memory?

Yes, the csv module streams row by row. With pandas you can read in chunks with chunksize= and append each chunk to the output file.

How to merge CSV files in Python

By the Analistable team · Updated 2026-10-10 · 5 min read

To stack many CSVs with the same columns, read each one with pandas and concatenate: pd.concat([pd.read_csv(f) for f in glob.glob("data/*.csv")], ignore_index=True), then to_csv("merged.csv", index=False). To combine two CSVs side by side on a shared column, use pd.merge(a, b, on="id", how="left"). Without pandas, the standard csv module can stack files with constant memory.

Part of our guide: How to merge data in pandas, R and SQL

Stacking vs joining

"Merge" means two different jobs. Stacking (appending) puts the rows of several files with the same columns underneath each other, such as twelve monthly exports. Joining puts columns from two files side by side, matched on a key such as an order ID. Pick the right one first; the code is different.

Stack a folder of CSVs with pandas

from pathlib import Path
import pandas as pd

files = sorted(Path("exports").glob("*.csv"))
frames = []
for f in files:
    df = pd.read_csv(f, dtype={"order_id": str})
    df["source_file"] = f.name          # keep track of where each row came from
    frames.append(df)

merged = pd.concat(frames, ignore_index=True)
merged.to_csv("all_orders.csv", index=False)
print(len(files), "files,", len(merged), "rows")

ignore_index=True gives the result a fresh 0..n index instead of repeating each file's index. concat aligns columns by name, so a file with an extra column produces NaN for the others rather than misaligning data. Reading IDs as str keeps leading zeros. Sorting the file list makes the output order repeatable.

Example run on three monthly exports
File	Rows	Columns
orders_2026-07.csv	1840	9
orders_2026-08.csv	2105	9
orders_2026-09.csv	1977	10
all_orders.csv	5922	11 (10 + source_file)

The September file added a column (say, discount_code), so the merged file has 10 data columns and earlier months show blanks there. Check that 1,840 + 2,105 + 1,977 = 5,922 to confirm nothing was lost.

Stack CSVs with only the standard library

For very large files, or where pandas isn't installed, the csv module streams rows one at a time and never holds a whole file in memory. It writes the header once and skips it in later files.

import csv
from pathlib import Path

files = sorted(Path("exports").glob("*.csv"))
with open("all_orders.csv", "w", newline="", encoding="utf-8") as out:
    writer = None
    for f in files:
        with open(f, newline="", encoding="utf-8-sig") as src:
            reader = csv.DictReader(src)
            if writer is None:
                writer = csv.DictWriter(out, fieldnames=reader.fieldnames)
                writer.writeheader()
            elif reader.fieldnames != writer.fieldnames:
                raise ValueError(f"{f.name} has different columns")
            writer.writerows(reader)

utf-8-sig strips the byte-order mark Excel often adds, which otherwise ends up glued to the first column name. The column check stops the script rather than writing misaligned rows. For multi-gigabyte inputs see merging large CSV files.

Join two CSVs on a key

import pandas as pd

orders = pd.read_csv("orders.csv", dtype={"customer_id": str})
customers = pd.read_csv("customers.csv", dtype={"customer_id": str})

joined = orders.merge(
    customers[["customer_id", "name", "country"]],
    on="customer_id", how="left", validate="many_to_one", indicator=True,
)
print(joined["_merge"].value_counts())
joined.drop(columns="_merge").to_csv("orders_with_customers.csv", index=False)

how="left" keeps every order even if the customer is missing. validate="many_to_one" raises an error if a customer_id repeats in customers.csv, which would otherwise duplicate orders (see why pandas merge creates duplicate rows). If the key column has different names, use left_on= and right_on=. The full set of options is in merging DataFrames in pandas.

Common problems

Different delimiters: European exports often use semicolons. Pass sep=";" to read_csv.
Encoding errors: try encoding="utf-8-sig" or "cp1252" for older Windows exports.
Header rows repeated in the data: if a file was itself made by pasting CSVs together, filter rows where a column equals its own header name.
Columns in a different order: pandas aligns by name, so order doesn't matter; the csv module version above does need identical headers.
Dates: parse them with parse_dates=["order_date"] so sorting and filtering work after the merge.

If you don't want to write code, the free merge CSV tool stacks files in your browser, and Analistable can load several CSVs and answer questions across them without uploading them anywhere. More options are in the CSV merging guide and the pandas, R and SQL hub.

Which approach to use

Use pandas when the combined data fits comfortably in memory and you want to clean, filter or join it in the same script; it handles columns in different orders and gives you useful errors. Use the csv module when the files are huge, when you just need one big file at the end, or when you can't install packages on the machine. If you then need to analyse the result, DuckDB can query the folder of CSVs directly with read_csv_auto('exports/*.csv') and skip the merged file altogether.

Whichever you choose, record three numbers on each run: how many files were read, the rows in each, and the rows written. If the sum of inputs doesn't equal the output for a stack, or a left join's output doesn't equal the left file's rows, stop and investigate before anyone uses the file.

Stack and join in one script

A common job does both: stack twelve monthly order files, then join the result to a customer list. Do the stack first, check its row count, then run the join with validate="many_to_one". Doing it in this order means one join instead of twelve, and a single place to check for unmatched customers.

Frequently asked questions

Why do I get an 'Unnamed: 0' column after merging?: A file was saved with its pandas index as the first column. Pass index=False when writing, or index_col=0 when reading that file, so the old index isn't treated as data.
How do I merge CSV files that use different delimiters?: Read each one with the right sep= value (for example ';' for many European exports) before concatenating. pandas normalises them, and to_csv writes one consistent delimiter.
How do I add the file name as a column?: Assign it before concatenating: df["source_file"] = f.name inside the loop, as in the first example.
How do I merge CSV files with different columns in Python?: pd.concat aligns by column name and fills missing columns with NaN. Use join="inner" in concat if you only want the columns all files share.
How do I avoid repeating the header when combining CSVs?: With pandas it's handled for you, since each file's header becomes column names. With the csv module, write the header only for the first file, as in the example.
Is glob order guaranteed?: No. glob returns files in an arbitrary, filesystem-dependent order. Wrap it in sorted() for predictable output.
Can I merge CSVs without loading them into memory?: Yes, the csv module streams row by row. With pandas you can read in chunks with chunksize= and append each chunk to the output file.