Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
396 views
in Technique[技术] by (71.8m points)

python - How could I detect subtypes in pandas object columns?

I have the next DataFrame:

df = pd.DataFrame({'a': [100, 3,4], 'b': [20.1, 2.3,45.3], 'c': [datetime.time(23,52), 30,1.00]})

and I would like to detect subtypes in columns without explicit programming a loop, if possible.

I am looking for the next output:

column a = [int]
column b = [float]
column c = [datetime.time, int, float]
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

You should appreciate that with Pandas you can have 2 broad types of series:

  1. Optimised structures: Usually numeric data, this includes np.datetime64 and bool.
  2. object dtype: Used for series with mixed types or types which cannot be held natively in a NumPy array. The series is structured as a sequence of pointers to arbitrary Python objects and is generally inefficient.

The reason for this preamble is you should only ever need to apply element-wise logic to the second type. Data in the first category is homogeneous by nature.

So you should separate your logic accordingly.

Regular dtypes

Use pd.DataFrame.dtypes:

print(df.dtypes)

a      int64
b    float64
c     object
dtype: object

object dtype

Isolate these series via pd.DataFrame.select_dtypes and then use a dictionary comprehension:

obj_types = {col: set(map(type, df[col])) for col in df.select_dtypes(include=[object])}

print(obj_types)

{'c': {int, datetime.time, float}}

You will need to do a little more work to get the exact format you require, but the above should be your plan of attack.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...