web scraping - How to send a post requests in python

Question

Hi Experts I want to scrape data from this link https://www.orpha.net/consor/cgi-bin/ClinicalLabs_Diagnostictest.php?lng=EN&search=ClinicalLabs_Diagnostictest_TechPurpose

I want to use filter by country wise but the url is not changed when I filter it and in the network tab I see a post requests is sent to this url with data and params How can I send a post requests with correct data and params. Please Guide me And also here is my code which I try to send a post request

import requests
url = 'https://www.orpha.net/consor/cgi-bin/ClinicalLabs_Diagnostictest.php?lng=EN&search=ClinicalLabs_Diagnostictest_TechPurpose'
data = {'checkFilterLgnBox_AU':'AU'}
r = requests.post(url, data=data)

question from:https://stackoverflow.com/questions/65902018/how-to-send-a-post-requests-in-python

Answer 1

The address shouldn't return a modified address on your POST. Is there some kind of documentation on the API?

https://www.orpha.net/consor/cgi-bin/ClinicalLabs_Diagnostictest.php?lng=EN&search=ClinicalLabs_Diagnostictest_TechPurpose&checkFilterLgnBox_AU=AU

This works but this just changes the website activating the checkbox. So it's actually just some state handling for this page.