javascript - react native fetch not getting the same content as post man

Question

Im having a little problem with my request on getting an html from https://readnovelfull.com/beauty-and-the-beast-wolf-hubby-xoxo/chapter-1-i-would-not-be-responsible.html as example.

I can get all the html on the other url eg novel detalj, latest upgated etc.

but not when im getting the detali for the chapters.

I tested those url on postman and also on https://codebeautify.org/source-code-viewer as well and there is no problem on getting the content of the chapter of which it exist under the div #chr-content

So I am a bit lost now, what am I doing wrong?

Here is my fetch calls which is working on other novel sites.

  static async getHtml(
    url: string
  ): Promise<HTMLDivElement> {
    console.log(`Sending html request to ${url}`);
    var container = parse('<div>test</div>') as any;
    try {
      let headers = new Headers({
        Accept: '*/*',
        'User-Agent':
          'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36'
      });

      var data = await fetch(url, {
        method: 'GET',
        headers: headers,
      });
      if (!data.ok) {
        const message = `An error has occured:${data.status}`;
        console.log(message);
      } else {
        var html = await data.text();
        console.log('Data is ok. proceed to parse it');
        container = parse('<div>' + html + '</div>') as any;
      }
    } catch (e) {
      console.log(e);
    }
    return container as HTMLDivElement;
  }

I should mention that am not getting any error what so ever, its just that the html I am getting is not the same as postman and other site is getting.

Update

Ok so i did some research on the site and this is what i come up with.

the site need X-CSRF-TOKEN and i was able to extract those and find those values

const csrf = 'x09Q6KGqJOJJx2iHwNQUa_mYfG4neV9EOOMsUBKTItKfNjSc0thQzwf2HvCR7SQCqfIpC2ogPj18jG4dQPgVtQ==';
const id = 774791;

which i need to send a request to https://readnovelfull.com/ajax/increase-chapter-views with the values above. and this will send back true/false

now i tried to inc the csrf on my fetch call after but its still the same old same no data.

any idee if i am doing something wrong still?

question from:https://stackoverflow.com/questions/65875970/react-native-fetch-not-getting-the-same-content-as-post-man

Answer 1

Looks like you have an issue with CORS. To make sure just try to send request through cors proxy. One of the ways you can quickly do that is add prefix URL:

https://cors-anywhere.herokuapp.com/https://readnovelfull.com/beauty-and-the-beast-wolf-hubby-xoxo/chapter-1-i-would-not-be-responsible.html`

NOTE: Using this CORS proxy on production is not recommended, because it's not secure

If after that you'll receive data, that means that you faced with CORS, and you need to figure out how to solve it in your specific case.

Reproducable example:

const parse = (str) => str;

const getHtml = async (url) => {
    console.log(`Sending html request to ${url}`);
    var container = parse('<div>No content =(</div>')
    try {
      let headers = new Headers({
        Accept: '*/*',
        'User-Agent':
          'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36'
      });
      var data = await fetch(url, {
        method: 'GET',
        headers: headers,
      });
      if (!data.ok) {
        const message = `An error has occured:${data.status}`;
        console.log(message);
      } else {
        var html = await data.text();
        console.log('Data is ok. proceed to parse it');
        container = parse('<div>' + html + '</div>');
      }
    } catch (e) {
      console.log(e);
    }
    return container;
  }

getHtml('https://cors-anywhere.herokuapp.com/https://readnovelfull.com/beauty-and-the-beast-wolf-hubby-xoxo/chapter-1-i-would-not-be-responsible.html').then(htmlContent => document.querySelector('div').innerHTML = htmlContent);

<div>loading...</div>