I'm gonna try to explain this as clearly as I can, but it's very confusing to me so bear with me.(我将尽我所能清楚地解释这一点,但这让我很困惑,所以请耐心等待。)
For this project, I'm using Node.js with the modules Axios and Cheerio.(对于这个项目,我将Node.js与Axios和Cheerio模块一起使用。)
I am trying to fetch HTML data from a webshop (similar to Amazon/eBay), and store the product information in a dictionary.(我正在尝试从网上商店(类似于Amazon / eBay)获取HTML数据,并将产品信息存储在字典中。) I managed to store most things (title, price, image), but the product description is on a different page.(我设法存储了大多数东西(标题,价格,图像),但是产品说明在另一个页面上。) To do a request to this page, I'm using the URL I got from the first request, so they are nested.(要对此页面进行请求,我使用的是从第一个请求中获得的URL,因此它们是嵌套的。)
This first part is done with the following request:(第一部分通过以下请求完成:)
let request = axios.get(url)
.then(res => {
// This gets the HTML for every product
getProducts(res.data);
console.log("Got products in HTML");
})
.then(res => {
// This parses the product HTML into a dictionary of product items
parseProducts(productsHTML);
console.log("Generated dictionary with all the products");
})
.then(res => {
// This loops through the products to fetch and add the description
updateProducts(products);
})
.catch(e => {
console.log(e);
})
I'll also provide the way I'm creating product objects, as it might clarify the function where I think the problem occurs.(我还将提供创建产品对象的方式,因为它可以阐明我认为出现问题的功能。)
function parseProducts(html) {
for (item in productsHTML) {
// Store the data from the first request
const $ = cheerio.load(productsHTML[item]);
let product = {};
let mpUrl = $("a").attr("href");
product["title"] = $("a").attr("title");
product["mpUrl"] = mpUrl;
product["imgUrl"] = $("img").attr("src");
let priceText = $("span.subtext").text().split("xa0")[1].replace(",", ".");
product["price"] = parseFloat(priceText);
products.push(product);
}
}
The problem resides in the updateProducts function.(问题出在updateProducts函数中。) If I console.log the dictionary afterwards, the description is not added.(如果以后我用console.log字典,则不添加描述。) I think this is because the console will log before the description gets added.(我认为这是因为控制台将在添加描述之前记录日志。) This is the update function:(这是更新功能:)
function updateProducts(prodDict) {
for (i in prodDict) {
let request2 = axios.get(prodDict[i]["mpUrl"])
.then(res => {
const $ = cheerio.load(res.data);
description = $("div.description p").text();
prodDict[i]["descr"] = description;
// If I console.log the product here, the description is included
})
}
// If I console.log the product here, the description is NOT included
}
I don't know what to try anymore, I guess it can be solved with something like async/await or putting timeouts on the code.(我不知道要尝试什么,我想可以用async / await之类的方法或在代码上设置超时来解决。) Can someone please help me with updating the products properly, and adding the product descriptions?(有人可以帮助我正确更新产品并添加产品说明吗?) Thank you SO much in advance.(非常感谢您。)
ask by Tom de Visser translate from so
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…