Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
244 views
in Technique[技术] by (71.8m points)

javascript - JS Nightmare not loading specific URL?

I have started small scraping project and stop myself on the start. While I can do the parsing part, I have problems to load the page. I've reduced the code to as little as possible and still not working for requested URL

// Required Libraries
const cheerio = require("cheerio");
const Nightmare = require("nightmare");

const scraping = (url) => {
  url = "https://www.portaldrazeb.cz/drazba/144ex02170-20-62-rb3p1";
  // All scraping will be done here
  let nightmare = new Nightmare({ show: true });
  const nightmarePromise = new Promise((resolve, reject) => {
    nightmare
      .goto(url)
      .wait("main")
      .evaluate(() => {})
      .end()
      .then((response) => {
        var json = {
          title: "",
          picture: "",
          price: "",
          starting_price: "",
          description: "",
          address: "",
          map: "",
          customer: { name: "", email: "", phone: "" },
        };
        resolve(json);
      });
  });

  return nightmarePromise.then((jsonObj) => {
    nightmare = null;
    return jsonObj;
  });
};

module.exports = {
  scraping: scraping,
};

While the code works reliably for whatever URL it is not working for the URL requested. The output is correct only in 30% calls, in the rest the call starts Nightmare browser, the page starts loading, then several resets occurs in a second and the browser gets closed and following error is displayed

(node:18468) UnhandledPromiseRejectionWarning: Error: navigation error at unserializeError (C:UsersTomasDropbox (Personal)ServerScrapReality ode_modules ightmarelibipc.js:162:13) at EventEmitter. (C:UsersTomasDropbox (Personal)ServerScrapReality ode_modules ightmarelibipc.js:89:13) at Object.onceWrapper (events.js:422:26) at EventEmitter.emit (events.js:315:20) at ChildProcess. (C:UsersTomasDropbox (Personal)ServerScrapReality ode_modules ightmarelibipc.js:49:10) at ChildProcess.emit (events.js:315:20) at emit (internal/child_process.js:876:12) at processTicksAndRejections (internal/process/task_queues.js:85:21) (node:18468) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 3)

question from:https://stackoverflow.com/questions/65835846/js-nightmare-not-loading-specific-url

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
Waitting for answers

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...