python - Convert percent string to float in pandas read_csv

Question

Is there a way to convert values like '34%' directly to int or float when using read_csv in pandas? I would like that it is directly read as 0.34.

Using this in read_csv did not work:

read_csv(..., dtype={'col':np.float})

After loading the csv as 'df' this also did not work with the error "invalid literal for float(): 34%"

df['col'] = df['col'].astype(float)

I ended up using this which works but is long winded:

df['col'] = df['col'].apply(lambda x: np.nan if x in ['-'] else x[:-1]).astype(float)/100

Answer 1

You can define a custom function to convert your percents to floats

In [149]:
# dummy data
temp1 = """index col 
113 34%
122 50%
123 32%
301 12%"""
# custom function taken from https://stackoverflow.com/questions/12432663/what-is-a-clean-way-to-convert-a-string-percent-to-a-float
def p2f(x):
    return float(x.strip('%'))/100
# pass to convertes param as a dict
df = pd.read_csv(io.StringIO(temp1), sep='s+',index_col=[0], converters={'col':p2f})
df
Out[149]:
        col
index      
113    0.34
122    0.50
123    0.32
301    0.12
In [150]:
# check that dtypes really are floats
df.dtypes
Out[150]:
col    float64
dtype: object

My percent to float code is courtesy of ashwini's answer: What is a clean way to convert a string percent to a float?