Efficient groupby() coding for pandas in python

Question

I have written the following code, and I get what I need. However, I am wondering whether there's an easier/efficient way of achieving the same.

job_hist.groupby('employee_id').count()[job_hist.groupby('employee_id').count()['start_date'] > 1]

See that I have repeated job_hist.groupby('employee_id').count() inside [ ] after count().

Thank you.

question from:https://stackoverflow.com/questions/65849229/efficient-groupby-coding-for-pandas-in-python

Answer 1

If you do want all the columns (e.g. because some of them may contain NaN), then:

cnt = job_hist.groupby('employee_id').count()
out = cnt.loc[cnt['start_date'] > 1]

But a more customary goal is simply to count how many rows there are for each employee_id:

cnt = job_hist.groupby('employee_id').size()
out = cnt.loc[cnt > 1]

Or, in one go:

out = job_hist.groupby('employee_id').size().to_frame('cnt').query('cnt > 1')