🧰 File Operations · 7 min read

CSV Delimiter in pandas

Pass sep to read_csv: sep=";" for semicolons, sep="\t" for tabs. The default is a comma, and delimiter= is an alias for the same argument. If you leave the default on a file that uses something else, pandas does not complain — it returns one column whose name is the entire header line. That silent wrong shape is the reason this question keeps coming up: nothing in the output says "I could not find a comma", so the mistake surfaces later as aKeyError on a column that should exist.

The default is a comma, and it fails quietly

pandas documents sep as a string with a default of ','. There is no delimiter detection unless you ask for it. So a file exported by software that writes semicolons — the common case in locales where the comma is the decimal mark — is not rejected. It is parsed as a single column of text.

Measured on pandas 2.3.3, on a three-row file whose header is id;name;city:

The first row is the failure mode worth memorising. A shape of (3, 1) with a long column name is what a wrong separator looks like — not an exception. The general version of this symptom, in Excel as well as pandas, is covered in why a CSV opens in one column.

sep and delimiter are the same argument

The documentation calls delimiteran "alias for sep", so the two are interchangeable: read_csv(path, delimiter=";") and read_csv(path, sep=";") produce the same frame. Use one style in a codebase and keep it. sep is the shorter name and the one the rest of the documentation uses.

The cases where you must pass something: semicolon-delimited European exports, tab-delimited files that were saved with a .csv name, pipe-delimited output from older tools, and files whose first column is an ID containing no commas so that the mistake is easy to miss.

Let pandas guess: sep=None

From the read_csv documentation: "sep=Nonedetects the separator from the first valid row of the file with Python's builtin sniffer tool, csv.Sniffer; it is supported only by the Python parsing engine, which will be used automatically."

Two details matter in practice. It sniffs the first valid row, so a file with a preamble of comment lines or a title row can lead to a wrong guess. And it runs on the Python engine, which is slower on large files than the C engine. For a one-off look at an unfamiliar export, sep=None, engine="python" is the right call; for a nightly job reading the same file shape every day, hard-code the separator.

Multi-character separators are regex, and the C engine refuses them

The same documentation paragraph continues: separators longer than 1 character and different from \s+"will be interpreted as regular expressions and will force the use of the Python parsing engine. Note that regex delimiters are prone to ignoring quoted data."

If you force the C engine anyway, you get an error instead of a quiet engine switch. Measured on pandas 2.3.3 with sep=r";+", engine="c":

ValueError: the 'c' engine does not support regex separators
(separators > 1 char and different from '\s+' are interpreted as regex)

The same regex separator with engine="python" returned the expected shape (3, 3). That is the whole trade: regex separators buy flexibility and cost speed, and they can misread quoted fields. If a quoted value legitimately contains the separator pattern, the regex path will split it. Run the file through the delimiter converter once and read a clean file afterwards.

Tabs, and the delim_whitespace deprecation

A tab-separated file loaded with the comma default shows the same failure. Measured: a two-line TSV read with defaults returned shape (2, 1) with a single column named id followed by a tab character and name; with sep="\t" it returned shape (2, 2).

Older code often uses delim_whitespace=True. On pandas 2.3.3 it still works but warns: "The 'delim_whitespace' keyword in pd.read_csv is deprecated and will be removed in a future version. Use sep='\s+'instead." The replacement is not identical in spirit — \s+ matches any run of whitespace, tabs included, which is usually what people wanted from the keyword in the first place.

When the separator is not the problem

Two other faults produce frames that look like a separator mistake.

A UTF-8 byte order mark attaches itself to the first column name, so df['id'] raises a KeyError while df.columns[0] prints something that looks right. Fix it with encoding="utf-8-sig". The encoding guide covers how to confirm which encoding you actually have.

Missing or unbalanced quotes make a parser merge rows or shift fields right. That usually surfaces as a field-count error rather than a one-column frame, which the field-count guide and the quoting rules deal with. And if you want the plain-language version of reading a file at all, opening a CSV in Python starts from zero.

Fixing it in the file instead

Passing sep is the right fix for a one-off analysis. It is the wrong fix when anything else will open the file: a spreadsheet, an upload form, a client. In that case the file should be a real comma CSV, and changing the delimiter explains what has to stay intact when you rewrite it — quoting, embedded line breaks, and the decision not to use find-and-replace on a file whose values contain commas.

A related question is what the extension claims. A tab-separated file called data.csv will be read as CSV by every tool that trusts names; CSV vs TXT explains why the extension matters even though pandas ignores it.

Frequently Asked Questions

Sources

• pandas documentation, pandas.read_csv: sep is a string with a default of ','; sep=None detects the separator from the first valid row with csv.Sniffer and is supported only by the Python engine, which is used automatically; separators longer than one character and different from \s+ are interpreted as regular expressions and force the Python engine; regex delimiters are prone to ignoring quoted data; delimiter is an alias for sep. Retrieved 2026-10-31.
• Local measurement, 2026-10-31, pandas 2.3.3 on Python 3.13.5: semicolon file id;name;city with three data rows gave shape (3, 1) and one column named id;name;city with the default separator, shape (3, 3) with sep=";", the same with delimiter=";", and shape (3, 3) with sep=None, engine="python". sep=r";+" raised ValueError: the 'c' engine does not support regex separators (separators > 1 char and different from '\s+' are interpreted as regex) on the C engine and returned shape (3, 3) on the Python engine. A tab-separated file gave shape (2, 1) with a column named id\tname by default and (2, 2) with sep="\t". delim_whitespace=True emitted the deprecation FutureWarning quoted above.