Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
534 views
in Technique[技术] by (71.8m points)

javascript - 如何使用嵌套的HTTP请求更新字典?(How can I update my dictionary with nested HTTP request?)

I'm gonna try to explain this as clearly as I can, but it's very confusing to me so bear with me.(我将尽我所能清楚地解释这一点,但这让我很困惑,所以请耐心等待。)

For this project, I'm using Node.js with the modules Axios and Cheerio.(对于这个项目,我将Node.js与Axios和Cheerio模块一起使用。) I am trying to fetch HTML data from a webshop (similar to Amazon/eBay), and store the product information in a dictionary.(我正在尝试从网上商店(类似于Amazon / eBay)获取HTML数据,并将产品信息存储在字典中。) I managed to store most things (title, price, image), but the product description is on a different page.(我设法存储了大多数东西(标题,价格,图像),但是产品说明在另一个页面上。) To do a request to this page, I'm using the URL I got from the first request, so they are nested.(要对此页面进行请求,我使用的是从第一个请求中获得的URL,因此它们是嵌套的。) This first part is done with the following request:(第一部分通过以下请求完成:) let request = axios.get(url) .then(res => { // This gets the HTML for every product getProducts(res.data); console.log("Got products in HTML"); }) .then(res => { // This parses the product HTML into a dictionary of product items parseProducts(productsHTML); console.log("Generated dictionary with all the products"); }) .then(res => { // This loops through the products to fetch and add the description updateProducts(products); }) .catch(e => { console.log(e); }) I'll also provide the way I'm creating product objects, as it might clarify the function where I think the problem occurs.(我还将提供创建产品对象的方式,因为它可以阐明我认为出现问题的功能。) function parseProducts(html) { for (item in productsHTML) { // Store the data from the first request const $ = cheerio.load(productsHTML[item]); let product = {}; let mpUrl = $("a").attr("href"); product["title"] = $("a").attr("title"); product["mpUrl"] = mpUrl; product["imgUrl"] = $("img").attr("src"); let priceText = $("span.subtext").text().split("xa0")[1].replace(",", "."); product["price"] = parseFloat(priceText); products.push(product); } } The problem resides in the updateProducts function.(问题出在updateProducts函数中。) If I console.log the dictionary afterwards, the description is not added.(如果以后我用console.log字典,则不添加描述。) I think this is because the console will log before the description gets added.(我认为这是因为控制台将在添加描述之前记录日志。) This is the update function:(这是更新功能:) function updateProducts(prodDict) { for (i in prodDict) { let request2 = axios.get(prodDict[i]["mpUrl"]) .then(res => { const $ = cheerio.load(res.data); description = $("div.description p").text(); prodDict[i]["descr"] = description; // If I console.log the product here, the description is included }) } // If I console.log the product here, the description is NOT included } I don't know what to try anymore, I guess it can be solved with something like async/await or putting timeouts on the code.(我不知道要尝试什么,我想可以用async / await之类的方法或在代码上设置超时来解决。) Can someone please help me with updating the products properly, and adding the product descriptions?(有人可以帮助我正确更新产品并添加产品说明吗?) Thank you SO much in advance.(非常感谢您。)   ask by Tom de Visser translate from so

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)

To refactor this with async/await one would do:(要使用async / await重构它,可以这样做:)

async function fetchAndUpdateProducts() => { try { const response = await axios.get(url); getProducts(response.data); console.log("Got products in HTML"); parseProducts(productsHTML); console.log("Generated dictionary with all the products"); await updateProducts(products); } catch(e) { console.log(e); } } fetchAndUpdateProducts().then(() => console.log('Done')); and(和) async function updateProducts(prodDict) { for (i in prodDict) { const response = await axios.get(prodDict[i]["mpUrl"]); const $ = cheerio.load(response.data); description = $("div.description p").text(); prodDict[i]["descr"] = description; } } This will not proceed to conclude the call to fetchAndUpdateProducts unless the promise returned by updateProducts has been resolved.(除非已解决updateProducts返回的承诺,否则不会继续结束对fetchAndUpdateProducts的调用。)

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...