Scrolling an element which loads further on scrolling using Selenium in Python

Question

I am trying to create an Instagram scraping bot that collects a list of Followers and Following using Python + Selenium.

However, that list keeps on loading when the user scrolls until the list is exhausted. I am attaching a screenshot for reference (some content hidden due to privacy reasons):

Now, I believe I have two ways to achieve this:

1. Keep reading the usernames, and then keep on scrolling.
2. Keep scrolling till the end, and then read all usernames together from the source code.

I've been trying to figure this out using the second method. However, I am not able to figure out how to know when there is no more content to scroll. How can I achieve this (provided that I don't know anything about the length of this element)?

Reason for not using Method 1: When scrolling, the DOM keeps getting refreshed, so it is hard to keep track of which usernames have been read.

question from:https://stackoverflow.com/questions/65905354/scrolling-an-element-which-loads-further-on-scrolling-using-selenium-in-python

Answer 1

One way to do this is is to keep track of the amount of child elements the div has that contains the li elements for the followers. If it doesn't increase after a scroll event, you've reached the end of the list.