I have started small scraping project and stop myself on the start. While I can do the parsing part, I have problems to load the page. I've reduced the code to as little as possible and still not working for requested URL
// Required Libraries
const cheerio = require("cheerio");
const Nightmare = require("nightmare");
const scraping = (url) => {
url = "https://www.portaldrazeb.cz/drazba/144ex02170-20-62-rb3p1";
// All scraping will be done here
let nightmare = new Nightmare({ show: true });
const nightmarePromise = new Promise((resolve, reject) => {
nightmare
.goto(url)
.wait("main")
.evaluate(() => {})
.end()
.then((response) => {
var json = {
title: "",
picture: "",
price: "",
starting_price: "",
description: "",
address: "",
map: "",
customer: { name: "", email: "", phone: "" },
};
resolve(json);
});
});
return nightmarePromise.then((jsonObj) => {
nightmare = null;
return jsonObj;
});
};
module.exports = {
scraping: scraping,
};
While the code works reliably for whatever URL it is not working for the URL requested. The output is correct only in 30% calls, in the rest the call starts Nightmare browser, the page starts loading, then several resets occurs in a second and the browser gets closed and following error is displayed
(node:18468) UnhandledPromiseRejectionWarning: Error: navigation error
at unserializeError (C:UsersTomasDropbox (Personal)ServerScrapReality
ode_modules
ightmarelibipc.js:162:13)
at EventEmitter. (C:UsersTomasDropbox (Personal)ServerScrapReality
ode_modules
ightmarelibipc.js:89:13)
at Object.onceWrapper (events.js:422:26)
at EventEmitter.emit (events.js:315:20)
at ChildProcess. (C:UsersTomasDropbox (Personal)ServerScrapReality
ode_modules
ightmarelibipc.js:49:10)
at ChildProcess.emit (events.js:315:20)
at emit (internal/child_process.js:876:12)
at processTicksAndRejections (internal/process/task_queues.js:85:21)
(node:18468) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict
(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 3)
question from:
https://stackoverflow.com/questions/65835846/js-nightmare-not-loading-specific-url 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…