indexing - Pandas merging rows with the same value and same index

Question

I have a DataFrame with an index called SubjectID and a column Visit. Subjects have multiple Visits and either an integer value or an N/A for Value1 and Value2. I want to collapse the rows that have the same SubjectID and the same Visit number.

Here is my data frame:

SubjectID    Visit    Value1    Value2    
B1           1         1.57      N/A
B1           1         N/A       1.75
B1           2         N/A       1.56

I want to it to look like this:

Subject ID    Visit     Value1    Value2
B1            1          1.57      1.75
B1            2          N/A       1.56

I was trying to use groupby() to solve this problem but I'm not sure how to make it take into account both the index and the values in the Visit column.

Answer 1

You can use groupby.first or groupby.last to get the first/last non-null value for each column within the group. For the example data, the output would be the same for either method:

df = df.groupby(['SubjectID', 'Visit']).first().reset_index()

The resulting output:

  SubjectID  Visit  Value1  Value2
0        B1      1    1.57    1.75
1        B1      2     NaN    1.56