Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
279 views
in Technique[技术] by (71.8m points)

python - Pandas combining sparse columns in dataframe

I am using Python, Pandas for data analysis. I have sparsely distributed data in different columns like following

| id | col1a | col1b | col2a | col2b | col3a | col3b |
|----|-------|-------|-------|-------|-------|-------|
|  1 |   11  |   12  |  NaN  |  NaN  |  NaN  |  NaN  |
|  2 |  NaN  |  NaN  |   21  |   86  |  NaN  |  NaN  |
|  3 |   22  |   87  |  NaN  |  NaN  |  NaN  |  NaN  |
|  4 |  NaN  |  NaN  |   NaN |  NaN  |  545  |   32  |

I want to combine this sparsely distributed data in different columns to tightly packed column like following.

| id | group |  cola |  colb |
|----|-------|-------|-------|
| 1  |  g1   |   11  |   12  |
| 2  |  g2   |   21  |   86  |
| 3  |  g1   |   22  |   87  |
| 4  |  g3   |  545  |   32  |

What I have tried is doing following, but not able to do it properly

df['cola']=np.nan
df['colb']=np.nan
df['cola'].fillna(df.col1a,inplace=True)
df['colb'].fillna(df.col1b,inplace=True)
df['cola'].fillna(df.col2a,inplace=True)
df['colb'].fillna(df.col2b,inplace=True)
df['cola'].fillna(df.col3a,inplace=True)
df['colb'].fillna(df.col3b,inplace=True)

But I think there must be more concise and efficient way way of doing this. How to do this in better way?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

You can use df.stack() assuming 'id' is your index else set 'id' as index. Then use pd.pivot_table.

df = df.stack().reset_index(name='val',level=1)
df['group'] = 'g'+ df['level_1'].str.extract('col(d+)')
df['level_1'] = df['level_1'].str.replace('col(d+)','')
df.pivot_table(index=['id','group'],columns='level_1',values='val')

level_1    cola  colb
id group
1  g1      11.0  12.0
2  g2      21.0  86.0
3  g1      22.0  87.0
4  g3     545.0  32.0

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...