The 2 Best ways to read CSV files into a List of Dictionaries with Python

Why CSV files?

CSV files is one of the most widely used formats for sharing tabular data, data in row and column format. Here we will learn how to read CSV files with Python.

Why CSV files?
  • CVS file format existed before the first personal computer existed.
  • The IBM Fortran compiler supported the format back in 1972.
  • CSV files are easy to create and share.
  • The CSV file is human readable, unlike XLS or XLSX formats.
  • Almost any editor can read CSV files.
  • The CSV file format is easy to parse.
  • CSV files have a simple schema, which has made them popular.
  • You can manipulate a CSV is easy, due to the simple text format.

How are CSV files formatted?

CSV file format.
  • CSV files are in plain text.
  • The first line is the column names.
  • Each name is separated by a comma.
  • Then each of the following rows is a data record, where each entry is comma separated.

Read CSV files into a List of Dictionaries

Why read CSV file content into a List of Dictionaries?

List of Dictionaries
  • Each row of data contains the same items from the columns.
  • It is easy to access a specific row of data with dictionaries.

Alternatively, you could have the data in a List of Lists. The disadvantage would be you would need to keep track of the column names and what entries in the list it has. This is the problem Dictionaries solve.

Read CSV files with Python

First, we will look at the classical way to read CSV files into Python and understand why we want to read it into a list of Dictionaries.

Then we will demonstrate the two most convenient ways to read CSV files into a list of dictionaries in Python.

See this more general introduction to CSV files.

Learn about it on YouTube

The classical way to read CSV files in Python

To make the demonstration we need a CSV file.

Save the following content in NameRecords.csv

First name,Last name,Age
Connar,Ward,15
Rose,Peterson,18
Paul,Cox,12
Hanna,Hicks,10

Then we will read the content with the default CSV reader in Python.

import csv
with open('NameRecords.csv') as csvfile:
    csv_reader = csv.reader(csvfile)
    rows = list(csv_reader)

Then if you want to get the content it will be in a list of lists represented in rows.

for row in rows:
    print(row)

Resulting in the following output.

['First name', 'Last name', 'Age']
['Connar', 'Ward', '15']
['Rose', 'Peterson', '18']
['Paul', 'Cox', '12']
['Hanna', 'Hicks', '10']

Which illustrates two problems.

  1. The column names are given in the first item of the list.
  2. The row is a list of the items, and you need to keep track of what they represent.

Hence, if we could read the content directly into a list of dictionaries, it would make our life as a programmer easier.

What do I mean? See the two methods below.

Method 1: Using DictReader

This is possibly the classical way to do it and uses the standard Python library CSV.

First, you need a CSV file to work with. Save the following content in NameRecords.csv

First name,Last name,Age
Connar,Ward,15
Rose,Peterson,18
Paul,Cox,12
Hanna,Hicks,10

Then the following will read the content into a list of dictionaries.

import csv
with open("files/NameRecords.csv", "r") as f:
    csv_reader = csv.DictReader(f)
    name_records = list(csv_reader)

Access to the content can be achieved as follows.

print(name_records[0])
print(name_records[0]['First name'])
  • Advantage of approach: No additional library needs installation.
  • The disadvantage of the approach: You cannot access remote files.

Method 2: Using pandas read_csv()

This approach can read remote data.

import pandas as pd
url = "https://raw.githubusercontent.com/LearnPythonWithRune/LearnPython/main/files/NameRecords.csv"
name_records = pd.read_csv(url)
name_records = name_records.to_dict('records')

Again the data is structured in a list of records in dictionaries.

  • Advantage of approach: Can read remote files directly (like from GitHub)
  • The disadvantage of the approach is: Need to install the pandas library.

For an example see this project with pandas read_csv.

What next?

I am happy you asked.

If this is something you like and you want to get started with Python, then this is part of an 8 hours FREE video course with full explanations, projects on each level, and guided solutions.

The course is structured with the following resources to improve your learning experience.

  • 17 video lessons teaching you everything you need to know to get started with Python.
  • 34 Jupyter Notebooks with lesson code and projects.
  • 2 FREE eBooks to support your Python learning.

See the full FREE course page here.

Learn Python

Learn Python A BEGINNERS GUIDE TO PYTHON

  • 70 pages to get you started on your journey to master Python.
  • How to install your setup with Anaconda.
  • Written description and introduction to all concepts.
  • Jupyter Notebooks prepared for 17 projects.

Python 101: A CRASH COURSE

  1. How to get started with this 8 hours Python 101: A CRASH COURSE.
  2. Best practices for learning Python.
  3. How to download the material to follow along and create projects.
  4. A chapter for each lesson with a descriptioncode snippets for easy reference, and links to a lesson video.

Expert Data Science Blueprint

Expert Data Science Blueprint

  • Master the Data Science Workflow for actionable data insights.
  • How to download the material to follow along and create projects.
  • A chapter to each lesson with a Description, Learning Objective, and link to the lesson video.

Machine Learning

Machine Learning – The Simple Path to Mastery

  • How to get started with Machine Learning.
  • How to download the material to follow along and make the projects.
  • One chapter for each lesson with a Description, Learning Objectives, and link to the lesson video.

Leave a Comment