python - Imputation of missing values for categories in pandas

Question

The question is how to fill NaNs with most frequent levels for category column in pandas dataframe?

In R randomForest package there is na.roughfix option : A completed data matrix or data frame. For numeric variables, NAs are replaced with column medians. For factor variables, NAs are replaced with the most frequent levels (breaking ties at random). If object contains no NAs, it is returned unaltered.

in Pandas for numeric variables I can fill NaN values with :

df = df.fillna(df.median())

Answer 1

You can use df = df.fillna(df['Label'].value_counts().index[0]) to fill NaNs with the most frequent value from one column.

If you want to fill every column with its own most frequent value you can use

df = df.apply(lambda x:x.fillna(x.value_counts().index[0]))

UPDATE 2018-25-10 ?

Starting from 0.13.1 pandas includes mode method for Series and Dataframes. You can use it to fill missing values for each column (using its own most frequent value) like this

df = df.fillna(df.mode().iloc[0])