How to Replace NA or NaN Values in Pandas DataFrame with fillna()
John on June 10, 2021
To replace NA or NaN values in a Pandas DataFrame, use the Pandas fillna() function. This function can be applied in a variety of ways depending on whether you need all NaN values replacing in the table or only in specific areas.
DataFrame.fillna() Syntax
Here is the full syntax of the Pandas fillna() function and what each argument does:
DataFrame.fillna(self, value=None, method=None, axis=None, inplace=False, limit=None, downcast=None, **kwargs)
| Name | Description | Type/Default Value | Required / Optional |
|---|---|---|---|
| value | Value to use for replacing NaN/NA. | scalar, dict, Series, or DataFrame | Required |
| method | Method to use for filling holes in reindexed Series pad / ffill. | {‘backfill’, ‘bfill’, ‘pad’, ‘ffill’, None} Default Value: None | Optional |
| axis | Axis along which to fill missing values. | {0 or ‘index’} | Optional |
| inplace | If True modify the original DataFrame | bool Default Value: False | Optional |
| limit | If method is specified, this is the maximum number of consecutive NaN values to forward/backward fill. | int Default Value: None | Optional |
| downcast | A dict of item->dtype of what to downcast if possible, or the string ‘infer’. | dict Default Value: None | Optional |
Replace all NaN Values with 0 Using DataFrame.fillna()
To replace all NaN and NA values in a DataFrame, pass the value as the first argument of fillna() and nothing else.
import pandas as pd
import numpy as np
data = {
'name': ['John', 'Amy', 'Alice'],
'age': [np.nan, 25, 23],
'weight': [160, np.nan, 140],
'height': [172, 150, np.nan]
}
df = pd.DataFrame(data)
print('Original DataFrame:\n----------')
print(df)
df = df.fillna(0)
print('\n\nNew DataFrame with NaN replaced:\n---------')
print(df)
Original DataFrame:
----------
name age weight height
0 John NaN 160.0 172.0
1 Amy 25.0 NaN 150.0
2 Alice 23.0 140.0 NaN
New DataFrame with NaN replaced:
---------
name age weight height
0 John 0.0 160.0 172.0
1 Amy 25.0 0.0 150.0
2 Alice 23.0 140.0 0.0
Note – fillna() does not modify the original DataFrame so either overwrite the original DataFrame variable or store the result in a new one. The only exception to this is when inplace=True.
Replace NaN with Column Specific Values
To set a NaN replacement value for different columns supply a value= argument to fillna() and set its value to a dictionary corresponding to the column names in your DataFrame.
import pandas as pd
import numpy as np
data = {
'name': ['John', 'Amy', 'Alice'],
'age': [np.nan, 25, 23],
'weight': [160, np.nan, 140],
'height': [172, 150, np.nan]
}
df = pd.DataFrame(data)
print('Original DataFrame:\n----------')
print(df)
df = df.fillna(value={'name': 0, 'age': 1, 'weight': 2, 'height': 3})
print('\n\nNew DataFrame with NaN replaced:\n---------')
print(df)
Original DataFrame:
----------
name age weight height
0 John NaN 160.0 172.0
1 Amy 25.0 NaN 150.0
2 Alice 23.0 140.0 NaN
New DataFrame with NaN replaced:
---------
name age weight height
0 John 1.0 160.0 172.0
1 Amy 25.0 2.0 150.0
2 Alice 23.0 140.0 3.0
Modify the Original DataFrame with fillna()
To modify the original DataFrame with fillna() apply the inplace=True argument.
import pandas as pd
import numpy as np
data = {
'name': ['John', 'Amy', 'Alice'],
'age': [np.nan, 25, 23],
'weight': [160, np.nan, 140],
'height': [172, 150, np.nan]
}
df = pd.DataFrame(data)
print('Original DataFrame:\n----------')
print(df)
df.fillna(value=0, inplace=True)
print('\n\nNew DataFrame with NaN replaced:\n---------')
print(df)
Original DataFrame:
----------
name age weight height
0 John NaN 160.0 172.0
1 Amy 25.0 NaN 150.0
2 Alice 23.0 140.0 NaN
New DataFrame with NaN replaced:
---------
name age weight height
0 John 0.0 160.0 172.0
1 Amy 25.0 0.0 150.0
2 Alice 23.0 140.0 0.0
