Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
821 views
in Technique[技术] by (71.8m points)

python - How to efficiently handle European decimal separators using the pandas read_csv function?

I'm using read_csv to read CSV files into Pandas data frames. My CSV files contain large numbers of decimals/floats. The numbers are encoded using the European decimal notation:

1.234.456,78

This means that the '.' is used as the thousand separator and the ',' is the decimal mark.

Pandas 0.8. provides a read_csv argument called 'thousands' to set the thousand separator. Is there an additional argument to provide the decimal mark as well? If no, what is the most efficient way to parse a European style decimal number?

Currently I'm using string replace which I consider to be a significant performance penalty. The coding I'm using is this:

# Convert to float data type and change decimal point from ',' to '.'
f = lambda x: string.replace(x, u',', u'.')
df['MyColumn'] = df['MyColumn'].map(f)

Any help is appreciated.

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

For European style numbers, use the thousands and decimal parameters in pandas.read_csv.

For example:

pandas.read_csv('data.csv', thousands='.', decimal=',')

From the docs:

thousands :

str, optional Thousands separator.

decimal :

str, default ‘.’ Character to recognize as decimal point (e.g. use ‘,’ for European data).


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...