javascript - I can't get Instagram comments because of the pagination

Question

My friend and I are trying to use Instagram Scraping. When we try to get Instagram post's comments we didn't manage to get results. Because of we couldn't get the request URL from Network tab/XHR and Fetch. To more clear here our codes;

$.ajax({
url:"https://www.instagram.com/p/CKbQLm3qXhN/?__a=1",
type:'get',
success:function(response){
var list = [];
var list2 = [];
var lengthh = response.graphql.shortcode_media.edge_media_to_parent_comment.edges.length;
var comments = response.graphql.shortcode_media.edge_media_to_parent_comment.count;
console.log(comments);
console.log(lengthh);
console.log(response.url);
for (var i=0;i<lengthh;i++)
{
  list.push(response.graphql.shortcode_media.edge_media_to_parent_comment.edges[i].node.text);
  list2.push(response.graphql.shortcode_media.edge_media_to_parent_comment.edges[i].node.owner.username);
}
console.log(list);
console.log(list2);
}});

We can get just a little bit comments with that codes. But we want all of the comments.

When we use the code that I shared below this code automatically runs the Show more comments button every time it runs. So Instagram shows us more comments, actually 12 more commments. Each new 12 comments are uploaded to a separate page.

document.querySelector("#react-root > section > main > div > div.ltEKP > article > div.eo2As > div.EtaWk > ul > li > div > button > span").click();

The comments are uploaded in pages to another URL. We also need to reach those URLs and get the comments. How can we get the URL which I shared below.

Click to see image

question from:https://stackoverflow.com/questions/65894379/i-cant-get-instagram-comments-because-of-the-pagination

Answer 1

Waitting for answers