javascript - How to make Puppeteer work with a ReactJS application on the client-side

Question

I am fairly new to React and I am developing an app which will take actual screenshots of a web page and the app can draw and add doodles on top of the screenshot taken. I initially used html2canvas and domToImage to take client-side screenshots but it doesn't render the image exactly as it is shown in the web page.

Reddit user /pamblam0 suggested I look into Google's Puppeteer. How it works is that it launches a headless chromium browser which goes to my react app on localhost then gets a screenshot of that whole page easily. My problem however, is that puppeteer doesn't play nice inside a react app. It gives me a ws error which, as explained on a google search can be fixed by simply installing ws (which doesn't work by the way).

What happens now my puppeteer script works out my react app. From what I understand it doesn't work with client side app (I might be wrong). What I want to happen is that whenever I click the button from my react app, puppeteer should execute and return a base64 string which will then be passed to a component in my react app.

Here is what I've done so far.

puppeteerApp.js

const puppeteer = require('puppeteer');

const takeScreenshot = async () => {
    puppeteer.launch().then(async browser => {
        const page = await browser.newPage();
        const options = {
            path: 'saved_images/webshot.png',
            encoding: 'base64'
        }
        await page.goto('http://localhost:3000/', { waitUntil: 'networkidle2' });
        const elem = await page.$('iframe').then(async (iframe) => {
            return await iframe.screenshot(options)
        });

        await browser.close()
    });
}

takeScreenshot();

Code from react app. App.js

import React, { Component } from 'react';
import ScreenshotsContainer from './containers/ScreenshotsContainer/ScreenshotsContainer'
import ImageContainer from './containers/ImageContainer/ImageContainer';
import html2canvas from 'html2canvas';
import domtoimage from 'dom-to-image';
import Button from './components/UI/Button/Button'
import classes from './App.module.css';
import { CSSTransition } from 'react-transition-group'
import { ToastContainer, toast } from 'react-toastify';
import 'react-toastify/dist/ReactToastify.css';


class App extends Component {

  constructor(props) {
    super(props);
    this.state = {
      imgURIArray: [],
      img: null,
      showImageContainer: false,
      showScreenshotContainer: false,
      selectedImageURI: null,
      showSaveAnimation: false,
      showNotify: false
    }
  }


  storeImageToArrayHandler = (imgURI) => {
    if (imgURI !== "") {
      this.setState({ imgURIArray: [...this.state.imgURIArray, imgURI] }, () => {
        this.setState({ showImageContainer: !this.state.showImageContainer })
      })
    }
  }

  getScreenshotHandler = () => {
   //use puppeteer here!!!
  }



  getSelectedImageFromContainerHandler(selectedImageURI) {
    this.setState({
      selectedImageURI: selectedImageURI,
      showImageContainer: !this.state.showImageContainer
    })

  }

  showImageContainerHandler(showImageContainer) {
    this.setState({ showImageContainer: showImageContainer })
  }

  showScreenshotContainerHandler = () => {
    this.setState({ showScreenshotContainer: !this.state.showScreenshotContainer })
  }
  notify = (submitSuccessful, msg) => {
    let message = msg ? msg : ""
    submitSuccessful ?
      toast.success(message, {
        autoClose: 3000,
        position: toast.POSITION.TOP_CENTER
      })
      :
      toast.error(message, {
        position: toast.POSITION.TOP_CENTER
      });

  }
  render() {
    let buttonOps = (
      <CSSTransition
        in={!this.state.showScreenshotContainer}
        appear={true}
        timeout={300}
        classNames="fade"
      >
        <div className={classes.optionButtons}>
          <Button icon={"fas fa-camera"} type={"button-success"} gridClass={""} buttonName={""} style={{ width: "100%", height: "70px" }} onClick={() => this.getScreenshotHandler} />
          <Button icon={"fas fa-images"} type={"button-primary "} gridClass={""} buttonName={""} style={{ width: "100%", height: "70px" }} onClick={() => this.showScreenshotContainerHandler} />
        </div>
      </CSSTransition>
    )

    return (
      <div>
        {
          this.state.showImageContainer ?
            <div>
              <ImageContainer
                img={this.state.img}
                showImageContainer={showImageContainer => this.showImageContainerHandler(showImageContainer)}
                storeImageToArrayHandler={imgURI => this.storeImageToArrayHandler(imgURI)}
                notify={(submitSuccessful, msg) => this.notify(submitSuccessful, msg)}
              />
            </div>
            : null
        }
        <CSSTransition
          in={this.state.showScreenshotContainer}
          appear={true}
          timeout={300}
          classNames="slide"
          unmountOnExit
          onExited={() => {
            this.setState({ showScreenshotContainer: false })
          }}
        >
          <ScreenshotsContainer
            imgURIArray={this.state.imgURIArray}
            getSelectedImageFromContainerHandler={imgURI => this.getSelectedImageFromContainerHandler(imgURI)}
            showScreenshotContainerHandler={() => this.showScreenshotContainerHandler}
            notify={(submitSuccessful, msg) => this.notify(submitSuccessful, msg)}
          />

        </CSSTransition>
        {this.state.showImageContainer ? null : buttonOps}
        {/* <button onClick={this.notify}>Notify !</button> */}
        <ToastContainer />

      </div >
    );
  }
}

export default App;

Any help would be appreciated. Thanks!

Answer 1

Your React.js application runs on the client-side (in the browser). Puppeteer cannot run inside that environment as you cannot start a full browser inside the browser.

What you need is a server which does that for you. You could ether offer a HTTP endpoint (option 1) or expose your puppeteer Websocket (option 2):

Option 1: Provide a HTTP endpoint

For this option, you setup a server which handles the incoming request and runs the task (making a screenshot) for you:

server.js

const puppeteer = require('puppeteer');
const express = require('express');

const app = express();

app.get('/screenshot', async (req, res) => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto(req.query.url); // URL is given by the "user" (your client-side application)
    const screenshotBuffer = await page.screenshot();

    // Respond with the image
    res.writeHead(200, {
        'Content-Type': 'image/png',
        'Content-Length': screenshotBuffer.length
    });
    res.end(screenshotBuffer);

    await browser.close();
})

app.listen(4000);

Start the application with node server.js and you can now pass the URL to your server and get a screenshot back from your server: http://localhost:4000/screenshot?url=https://example.com/

The response from the server could then be used as as the source of an image element in your application.

Option 2: Exposing the puppeteer Websocket to the client

You could also control the browser (which runs on the server) from the client-side by by exposing the Websocket.

For that you need to expose the Websocket of your server like this:

server.js

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const browserWSEndpoint = browser.wsEndpoint();
    browser.disconnect(); // Disconnect from the browser, but don't close it
    console.log(browserWSEndpoint); // Communicate the Websocket URL to the client-side
    // example output: ws://127.0.0.1:55620/devtools/browser/e62ec4c8-1f05-42a1-86ce-7b8dd3403f91
})();

Now you can control the browser (running on the server) form the client-side with a puppeteer bundle for the client. In this scenario you could now connect to the browser via puppeteer.connect and make a screenshot that way.

I would strongly recommend using option 1, as in option 2 you are fully exposing your running browser to the client. Even with option 1, you still need to handle user input validation, timeouts, navigation errors, etc.