Cloud Run Outbound API Calls being throttled?

Question

I have an instance that on some requests needs to make multiple calls to an external API, sometimes up to 2,000+ calls.

When running my application locally, each call to the external api returns at sub 200ms every time without fail, and the entire process for all 2,000 calls takes appx 15 seconds.

I noticed running on cloud run, my API calls are split into three categories: appx 1/4 take ~200ms same as local. about 1/2 take exactly 12007ms and some take exactly 63000ms, the whole process is taking 20+ minutes for these same API calls.

I have tried batching, using async/eachLimit set to 10, 20, 50.. but the same thing occurs.

The endpoint, data and calls are identical on my local to what's on cloud run. Cloud run is running through a NAT with static IP as well (which may be intercepting/throttling?).

Has anyone encountered this before? Adding my docker image to a VM in the same VPC/NAT Gateway has the same effect as my local (all calls < 200ms and complete is < 15 seconds).

Has anyone encountered this in run and how to get around?

Snippet running (Note I have played around with the limit (50 in below snippet):

const productsWithAvailabilities = await mapLimit(products, 50, async product => {
      console.time(product.ProductID)
      const availabilityOther = await productClient.execute(
        'GetAvailabilityA',
        {
          productid: product.ProductID,
          viewid: 'WEB',
          connectid: this.pricingToken,
        },
        false,
      )

      let AvailabilityOther

      try {
        AvailabilityOther = availabilityOther?.GetAvailabilityAResult?.diffgram?.Warehouses?.ProductAvailability?.map(
          a => ({
            ...a,
            QtyAvail: parseFloat(a.QtyAvail),
            QtyOnHand: parseFloat(a.QtyOnHand),
            QtyOnOrder: parseFloat(a.QtyOnOrder),
            QtyInTransit: parseFloat(a.QtyInTransit),
            Available: parseFloat(a.QtyAvail),
          }),
        )
      } catch (e) {
        console.log({ product, e })
      }

      console.timeEnd(product.ProductID)
      return {
        ...product,
        AvailabilityOther,
        Availability: AvailabilityOther?.find(a => a.LocationID === LOCATIONS.MEL),
        QtyAvailableOther: AvailabilityOther?.filter(a => a.LocationID !== LOCATIONS.MEL)
          .map(a => a.QtyAvail)
          .reduce((result, current) => result + current, 0),
      }
    })

    console.timeEnd('availabilities')
    return productsWithAvailabilities as MoProProduct[]
  }

ProductClient.execute is making a SOAP post request using the node 'soap' library.

VPC Connector throughput is 200 - 1000 (default setting) and I have a single external IP / router connected to the NAT.

question from:https://stackoverflow.com/questions/65931674/cloud-run-outbound-api-calls-being-throttled

Answer 1

Waitting for answers