Holla, Welcome back to another exciting tutorial on “How to load CSV file into Pandas Data frame”. In this Python tutorial, you’ll learn the pandas read_csv method. The method read and load the CSV data into Pandas Dataframe.
You’ll also learn various optional and mandatory parameters of the pandas read_csv method syntax. In the end, you will see the live coding demo for better understanding. Let’s begin our tutorial with an introduction to the CSV file, followed by an introduction to Python Pandas and Pandas Dataframe.
Table of Contents
Pandas Read_CSV Python.
Holla, Welcome back to another exciting Python tutorial on “How to load CSV file into Pandas Data frame”. In this Python tutorial, you’ll learn the pandas read_csv method. The method read and load the CSV data into Pandas Dataframe.
You’ll also learn various optional and mandatory parameters of the pandas read_csv method syntax. In the end, you will see the live coding demo for better understanding. Let’s begin our tutorial with an introduction to the CSV file, followed by introduction to Python Pandas and Pandas Dataframe.
What is a CSV file?
1077501,1296599,5000 1077430,1314167,2500 1077175,1313524,2400
What are Delimiters in CSV file?
What is Pandas in Python?
Pandas is an open-source library written for the Python programming language. Pandas is a robust, prominent, and comprehensive data analysis library. It provides various methods such as read, writes, and dataset update methods.
It’s used for machine learning in the form of data-frames. Pandas allow various data manipulation operations such as group by, join, merge, etc.
The Data frame is an object that is useful in representing data in the form of rows and columns. Pandas data frame is generally created from .csv (comma-separated) files, Excel spreadsheets, tuples, lists.
what is a pandas Dataframe?
Pandas Read_CSV Syntax.
# Python read_csv pandas syntax with
# minimum set of parametrs.
pd.read_csv(filepath,
sep=',',
dtype=None,
header=None,
skiprows=None,
index_col=None,
skip_blank_lines=True,
na_filter=True)Now, let’s understand the importance of these parameters.
- filepath: The filepath parameter specifies the file location. A local file could be passed as://localhost/path/to/table.csv.
- sep: The sep parameter specifies the delimiter which is used in the file.
- dtype: The dtype parameter specifies the column datatype (i.e. integer or float).
- header: The header parameter specifies the column header row. A list of values can be used while reading a CSV file.
- skiprows: The skiprows parameter use to skip initial rows, for example, skiprows=05 means data would be read from 06th row.
- index_col: The index_col parameter use to specify the column as the row labels of the data frame.
- skip_blank_lines: The parameter is used to skip blank lines while reading data from the dataset using read_csv pandas.
- na_filter: The parameter is used to drop NaN value from the dataset.
- low_memory: Internally process the file in chunks, resulting in lower memory use while parsing, but possibly mixed type inference.
- encoding: Encoding to use for UTF when reading/writing (ex. ‘utf-8’).
Python read_CSV Pandas Example.
Now, let’s dirty our hands with actual code. I am using Jupyter Notebook for read_csv pandas demo. We would be using a loan.csv file, you can download a CSV file from the internet or you can use your own CSV file.
Please note, if your CSV file is in the same directory, then you are not required to specify the full path. If your file location is different then you need to specify the complete location of the file.
Example #1 The aim of this python tutorial is to show how to load data from CSV file into pandas dataframe by using read_csv pandas.
import numpy as np # import numpy as np
import pandas as pd # import pandas as pd
# Reading and load loan file into df.
df_loan = pd.read_csv("loan.csv", sep=",",
encoding = "ISO-8859-1",
index_col=None,
low_memory=False,
dtype={'id':np.int32}, nrows=16, skiprows=0)
df_loan.head(3)
The pandas read_csv method code snippet, read data from loan.csv file and load into df_loan dataframe.
| id | member_id | loan_amnt | funded_amnt | funded_amnt_inv | term | int_rate |
|---|---|---|---|---|---|---|
| 0 1077501 | 1296599 | 5000 | 5000 | 4975 | 36 months | 10.65% |
| 1 1077430 | 1314167 | 2500 | 2500 | 2500 | 60 months | 15.27% |
| 2 1077175 | 1313524 | 2400 | 2400 | 2400 | 36 months | 15.96% |
Example #2 The second example is to show how to load data from CSV file into pandas dataframe by using read_csv pandas. But this time different encoding option is used.
If there is a data conversion error, while reading the CSV file then you must try different encoding options. In the below example, encoding='utf-8' is used despite encoding = "ISO-8859-5".
import numpy as np # import numpy as np
import pandas as pd # import pandas as pd
# Reading and load loan file into df.
df_loan = pd.read_csv("loan.csv", sep=",",
encoding='utf-8',
index_col='id')
# Display first 3 rows from the pandas dataframe.
df_loan.head(3)Note: In the above example, the ‘id’ column is set as the index of dataframe.
Example #3 The third example is to show how to load data from CSV file into pandas dataframe by using pandas read_csv. The index column is set to none i.e. python will set index value to default sequence.
import numpy as np # import numpy as np
import pandas as pd # import pandas as pd
# Reading and load loan file into df.
df_loan = pd.read_csv("loan.csv", sep=",",
encoding = "ISO-8859-1",
index_col=None,
low_memory=False)
df_loan.head(3)
Example #4 The fourth example is to show how to load data from CSV file into pandas dataframe by using pandas read_csv and convert column type while reading the CSV file.
import numpy as np # import numpy as np
import pandas as pd # import pandas as pd
# Reading and load loan file into df.
df_loan = pd.read_csv("loan.csv", sep=",",
encoding = "ISO-8859-1",
index_col=None,
low_memory=False,usecols=[0,1,2,3,4,5],
dtype={'id':np.int32})
df_loan.head(3) Example #5 The fifth example is to show how to load data from CSV file into pandas dataframe by using pandas read_csv and print first and last two rows.
import numpy as np # import numpy as np
import pandas as pd # import pandas as pd
# Reading and load loan file into df.
df_loan = pd.read_csv("loan.csv", sep=",",
encoding = "ISO-8859-1",
index_col=None,
low_memory=False)
df_loan.head(2)
# Print last two rows. df_loan.tail(2)
You can use the Python help command to get details about the syntax and possible parameters.
# Get help from python regarding read_csv syntax. help (pd.read_csv)
# Python read_csv pandas all parameters list.
read_csv(filepath_or_buffer,
sep=',',
delimiter=None,
header='infer',
names=None,
index_col=None,
usecols=None,
squeeze=False,
prefix=None,
mangle_dupe_cols=True,
dtype=None,
engine=None,
converters=None,
true_values=None,
false_values=None,
skipinitialspace=False,
skiprows=None,
skipfooter=0,
nrows=None,
na_values=None,
keep_default_na=True,
na_filter=True,
verbose=False,
skip_blank_lines=True,
parse_dates=False,
infer_datetime_format=False,
keep_date_col=False,
date_parser=None,
dayfirst=False,
iterator=False,
chunksize=None,
compression='infer',
thousands=None,
decimal=b'.',
lineterminator=None,
quotechar='"',
quoting=0,
doublequote=True,
escapechar=None,
comment=None,
encoding=None,
dialect=None,
tupleize_cols=None,
error_bad_lines=True,
warn_bad_lines=True,
delim_whitespace=False,
low_memory=True,
memory_map=False,
float_precision=None)
