scraping a fake <a> with python

Question

I'm a beginner in Python, and I'm using BeautifullSoup to scrape data from an html page.

So far, everything fine. But some links are weird, and perhaps the purpose is not to be scraped. This page : https://francechansons.net/alain-souchon-liste-de-chansons/ has a list of links, with href being themselves links, instead of url. My current code is :

from urllib.request import urlopen
from bs4 import BeautifulSoup as bs

html_page = urlopen('https://francechansons.net/alain-souchon-liste-de-chansons/')
soup = bs(html_page, 'lxml')

entry_content_div=soup.find("div", class_="entry-content") 
ul = entry_content_div.find("ul")
li = ul.find('li')
children = li.findChildren("a")
for child in children:
    print(child)

I get

 <a href="alain_souchon-18_ans_que_j_t_ai_a_l_oeil">18 ans que j’t’ai à l’?il</a>

instead of :

<a href="https://francechansons.net/alain_souchon-18_ans_que_j_t_ai_a_l_oeil/">18 ans que j’t’ai à l’?il</a>'

Hope someone understands this convoluted message

question from:https://stackoverflow.com/questions/65937565/scraping-a-fake-a-with-python

Answer 1

Just concatenate the href with a base url like this:

baseUrl = 'https://francechansons.net/'
    for child in children:
        print(baseUrl+child['href'])

But check if there is already an http in the href:

if 'http' in child['href']:
    print(child['href'])
else:
    print(baseUrl+child['href'])

Example

from urllib.request import urlopen
from bs4 import BeautifulSoup as bs

html_page = urlopen('https://francechansons.net/alain-souchon-liste-de-chansons/')
soup = bs(html_page, 'lxml')

entry_content_div=soup.find("div", class_="entry-content") 
ul = entry_content_div.find("ul")
li = ul.find('li')
children = li.findChildren("a")
baseUrl = 'https://francechansons.net/'
for child in children:
    if 'http' in child['href']:
        print(child['href'])
    else:
        print(baseUrl+child['href'])

Output

https://francechansons.net/alain_souchon-18_ans_que_j_t_ai_a_l_oeil