Scrapy Xpath returns empty result

Question

I tried to scrape some data from this url:https://www.cargurus.com/Cars/inventorylisting/viewDetailsFilterViewInventoryListing.action?zip=92612
I first debugged and tested in Scrapy shell. Since I just wanted the model name of the car, I copied the Xpath from Chrome. Here is how I did this:

1. I right clicked the name of model and clicked inspect
2. I found the text of model name and copied the Xpath

After I got the Xpath, I typed the command below:

response.xpath('//*[@id="cargurus-listing-search"]/div[1]/div/div[2]/div[2]/div[4]/div[1]/div/a/div[3]/div/div[1]/h4/text()').extract()

But the shell returned an empty list.

question from:https://stackoverflow.com/questions/65929291/scrapy-xpath-returns-empty-result

Answer 1

You could use the following to extract all the car names

response.xpath(//div[@class="titleWrap"]/h4/text()).extract()