Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
641 views
in Technique[技术] by (71.8m points)

python 3.x - Pandas - Conditional drop duplicates

I have a Pandas 0.19.2 dataframe for Python 3.6x as below. I want to drop_duplicates() with the same Id based on a conditional logic.

import pandas as pd
import numpy as np
np.random.seed(1)
df = pd.DataFrame({'Id':[1,2,3,4,3,2,6,7,1,8],
              'Name':['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'K'],
              'Size':np.random.rand(10),
              'Age':[19, 25, 22, 31, 43, 23, 44, 20, 51, 31]})

What would be the most efficient (if possible vectorised) way to achieve this based on the logic I describe below?

1) Before dropping duplicates, sum the Size of duplicate Id entries.

2) Drop duplicates for same Id records, keeping the one that has a larger Age.

The desired output would be:

   Age  Id Name      Size
1   25   2    B  0.812662
3   31   4    D  0.302333
4   43   3    E  0.146870
6   44   6    G  0.186260
7   20   7    H  0.345561
8   51   1    I  0.813790
9   31   8    K  0.538817
See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

Use GroupBy.transform for aggregated values with same size as original DataFrame with sort_values and drop_duplicates for remove dupes:

df['Size'] = df.groupby('Id')['Size'].transform('sum')
df = df.sort_values('Age').drop_duplicates('Id', keep='last').sort_index()
print (df)
   Id Name      Size  Age
1   2    B  0.812663   25
3   4    D  0.302333   31
4   3    E  0.146870   43
6   6    G  0.186260   44
7   7    H  0.345561   20
8   1    I  0.813789   51
9   8    K  0.538817   31

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...