How to Add Automatic Retry Support to Fetch in Node.js

For this tutorial, we're going to use CheatCode's full-stack JavaScript framework, Joystick. Joystick brings together a front-end UI framework with a Node.js back-end for building apps.

To begin, we'll want to install Joystick via NPM. Make sure you're using Node.js 16+ before installing to ensure compatibility (give this tutorial a read first if you need to learn how to install Node.js or run multiple versions on your computer):

Terminal

npm i -g @joystick.js/cli

This will install Joystick globally on your computer. Once installed, next, let's create a fresh project:

Terminal

joystick create app

After a few seconds, you will see a message logged out to cd into your new project and run joystick start. Before you run that, we need to install one more dependency, node-fetch:

Terminal

cd app && npm i node-fetch

This will give us access to a Node.js friendly implementation of the Fetch API. After this is installed, you can go ahead and start up your app.

Terminal

joystick start

After this, your app should be running and we're ready to get started.

Writing a wrapper function for Fetch

To get started, we're going to write our wrapper function first as well as one other function to help us create a delay between retry attempts. Because we'd consider code like this "miscellaneous" or part of our app's "standard library," we're going to create a file inside of the /lib (short for "library") folder at the root of the project we created above.

Because we'll be writing code that's only meant for a Node.js environment, we're going to create another folder within /lib called /node which will signal to Joystick that our file should only be built for a Node-available environment.

/lib/node/retryFetch.js

import fetch from 'node-fetch';

const retryFetch = (url = '', options = {}) => {
  const { retry = null, retryDelay = 0, retries = 5, ...requestOptions } = options;
  return fetch(url, requestOptions);
};

export default retryFetch;

Above, we kick off our file by importing the fetch dependency we installed earlier via the node-fetch package. Here, fetch is the actual Fetch function that we'll call to perform our request. Just below this, we've defined a function retryFetch which takes two arguments:

1. url which is the URL we're going to "fetch."
2. options which is the options object that will be handed off to fetch().

Just inside of our retryFetch function's body, we're doing something special. Here, we're using JavaScript destructuring to "pull apart" the passed in options object. We want to do this because we're going to "piggyback" on this object to include our retry-related configuration (Fetch doesn't support this and so we don't want to pass it to Fetch accidentally).

To prevent that, here we "pluck off" three properties from the options object that we're anticipating:

1. retry a boolean true or false value letting us know if we should retry a request should it fail.
2. retryDelay an integer representing the number of seconds to wait before retrying a request.
3. retries an integer representing the number of retry attempts we should make before stopping.

After these, we've written ...requestOptions to say "scoop up the rest of the object into a variable called requestOptions that will be available below this line." We've accented rest here as the ... is known as the "rest/spread" operator in JavaScript. In this context, ... literally says "get the rest of the object."

To round off our foundational code, we return a call to fetch() passing in the url string as the first argument and the options object passed to our retryFetch function as the second argument.

This gives us the basics, but at the moment our retryFetch function is a useless wrapper around fetch(). Let's extend this code to include the "retry" functionality:

/lib/node/retryFetch.js

import fetch from 'node-fetch';

let attempts = 0;

const retryFetch = async (url = '', options = {}) => {
  const { retry = null, retryDelay = 0, retries = 5, ...requestOptions } = options;

  attempts += 1;

  return fetch(url, requestOptions).then((response) => response).catch((error) => {
    if (retry && attempts <= retries) {
      console.warn({
        message: `Request failed, retrying in ${retryDelay} seconds...`,
        error: error?.message,
      });

      return retryFetch(url, options, retry, retryDelay);
    } else {
      throw new Error(error);
    }
  });
};

export default retryFetch;

This is the majority of the code for this function. Focusing back on the body of our retryFetch function we've added some more code. First, just below our destructuring of options, we've added a line attempts += 1 which increments the attempts variable initialized above our retryFetch function. The idea here is that we want to keep track of each call to retryFetch so that we can "bail out" if we've reached the maximum retries allowed (if specified).

Worth noting, in the destructuring of options, you'll notice that we "plucked off" retries as retries = 5. What we're saying here is "pluck off the retries property from the options object, and if it's not defined, give it a default value of 5." This means that even if we don't pass a specific number of retries, by default we'll try 5 times and then stop (this avoids our code running infinitely and wasting resources on a request that can't be resolved).

Next, notice that we've extended our call to fetch(), here adding on the .then() and .catch() callbacks for a JavaScript Promise (we expect fetch() to return a JavaScript Promise).

Because our goal is to only handle a failed request, for the .then() callback, we just take the passed response and immediately return it (while technically unnecessary—we could just omit .then()—this adds clarity to our code for maintenance sake).

For the .catch()—what we really care about—we check to see if retry is true and that our attempts variable's current value is less than or equal to the specified number of retries (either what we've passed or the default of 5).

If both of those things are truthy, first, we want to give ourselves a heads up that the request failed by calling to console.warn() passing an object with two things: a message letting us know that the request failed and that we'll try in the allotted retryDelay and the error message we received from the request.

Most importantly, at the bottom, we make a recursive call to retryFetch() passing the exact same arguments it was initially called with.

This is the "trick" of this function. Even though we're inside of the retryFetch function, we can still call it from within itself—trippy. Notice that we've prefixed a return on the front, too. Because we're calling return in front of our original fetch() call, the return in front of our recursive retryFetch call will "bubble up" back to the return fetch() and ultimately, be the return value of our initial retryFetch() call.

In the event that we haven't enabled retry functionality or we've run out of attempts, we take the error that occurred and throw it (this allows it to bubble to the .catch() of the call to retryFetch() properly).

Before we can say "done," there's a slight gotcha. As this code stands, notice that we're not making use of the retryDelay we anticipate being passed. To make use of this, we're going to write another function above our retryFetch definition that will give us the ability to "pause" our code for an arbitrary number of seconds before continuing.

/lib/node/retryFetch.js

import fetch from 'node-fetch';

let attempts = 0;

const wait = (time = 0) => {
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve();
    }, time * 1000);
  });
};

const retryFetch = async (url = '', options = {}) => {
  const { retry = null, retryDelay = 0, retries = 5, ...requestOptions } = options;

  attempts += 1;

  return fetch(url, requestOptions).then((response) => response).catch(async (error) => {
    if (retry && attempts <= retries) {
      console.warn({
        message: `Request failed, retrying in ${retryDelay} seconds...`,
        error: error?.message,
      });

      await wait(retryDelay);

      return retryFetch(url, options, retry, retryDelay);
    } else {
      throw new Error(error);
    }
  });
};

export default retryFetch;

This is now the complete code. Above retryFetch, we've added another function wait which takes in a time as an integer in seconds and returns a JavaScript Promise. If we look close, inside of the returned Promise is a call to setTimeout() taking the passed time and multiplying it by 1000 (to get the seconds in the milliseconds that JavaScript expects). Inside of the setTimeout()'s callback function, we call to the resolve() function of the returned Promise.

Like the code suggests, when JavaScript calls the wait() function, if we tell it using the await keyword, JavaScript will "wait" for the Promise to resolve. Here, that Promise will resolve after the specified time has elapsed. Cool, eh? With this, we get an asynchronous pause in our code without bottlenecking Node.js.

Putting this to use is quite simple. Just above our recursive call to retryFetch(), we call to await wait(retryDelay). Notice, too, that we've appended the async keyword to the function we're passing to .catch() so that the await here doesn't trigger a runtime error in JavaScript (await is known as a "reserved keyword" in JavaScript and won't work unless the parent context where it's used is flagged as async).

That's it! Let's write some test code to take this for a spin.

Calling the wrapper function

To test out our code, let's jump over to the /index.server.js file at the root of the project that was created for us earlier when we ran joystick create.

/index.server.js

import node from "@joystick.js/node";
import api from "./api";
import retryFetch from './lib/node/retryFetch';

node.app({
  api,
  routes: {
    "/": (req, res) => {
      res.render("ui/pages/index/index.js", {
        layout: "ui/layouts/app/index.js",
      });
    },
    "*": (req, res) => {
      res.render("ui/pages/error/index.js", {
        layout: "ui/layouts/app/index.js",
        props: {
          statusCode: 404,
        },
      });
    },
  },
}).then(async () => {
  retryFetch('https://thisdoesnotexistatallsowillfail.com', {
    retry: true,
    retryDelay: 5,
    retries: 3,
    method: 'GET', // NOTE: Unnecessary, just showcasing passing regular Fetch options.
  }).then(async (response) => {
    // NOTE: If all is well, handle the response.
    console.log(response);
  }).catch((error) => {
    // NOTE: If the alotted number of retry attempts fails, catch the final error.
    console.warn(error);
  });
});

The part we want to focus on here is the .then() we've tacked on the end of node.app() near the bottom of the file. Inside, we can see that we're calling the imported retryFetch() function, passing the url we want to call as a string and an options object that will be passed to fetch(). Remember that on the options object, we've told our code to expect three additional options: retry, retryDelay, and retries.

Here, we've specified the behavior for our function along with a standard fetch() option method. On the end of our call to retryFetch(), we add on a .then() to handle a successful use case, and a .catch() to handle the error that's returned if we run out of retry attempts before we get a successful response.

If we open up the terminal where we started our app, we should see an error being printed to the terminal (the passed URL does not exist and will immediately fail). With the above settings, we should see 3 errors printed 5 seconds apart and then a final error letting us know that the request ultimately failed.

Wrapping up

In this tutorial, we learned how to write a wrapper function around the Node.js fetch() implementation that allowed us to specify retry logic. We learned how to wrap the fetch() function while feeding it arguments from the wrapper as well as how to recursively call the wrapper function in the event that our request failed. Finally, we learned how to create a function for delaying our code by an arbitrary number of seconds to pause in between request attempts.