Skip to content
HeyNode

Stream to an HTTP Response with Node.js

In this tutorial we will use streams to efficiently send a large file as an HTTP response in a Node.js application. Reading a large file into memory before sending it back as a response is an inefficient use of time and resources. We can create a readstream from a file and pipe it directly to an HTTP response body, significantly reducing the memory footprint and time required to serve the file.

By the end of this tutorial, you should be able to:

  • Understand streaming to a response body
  • Implement fs.createReadStream and pipe the content into an HTTP response

Goal

Send a stream of data back to the client as a response to a request in a Node.js application.

Prerequisites

Watch: Stream to an HTTP Response

Clone the starter repo which contains a basic Express server and the file we will send as a response. Enter the directory and install the dependencies.

Terminal window
git clone https://github.com/OsioLabs/http-stream-file.git
cd http-stream-file
npm i

Inside the index.js file you’ll see the basic Express server which looks like this:

const express = require("express");


const app = express();


const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`Server listening on port ${PORT}`));

Before we can send a file as a stream, require the fs module at the top of the file:

const express = require("express");
const fs = require("fs");


const app = express();


const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`Server listening on port ${PORT}`));

The fs module contains the createReadStream method which we will need to create a read stream from a file on disk.

Now create a route and GET handler:

app.get("/download", (req, res, next) => {
  // download file
});

Once we’ve created the handler, we can create a stream from a file to prepare to pipe it to the response:

app.get("/download", (req, res, next) => {
  const fileStream = fs.createReadStream(`${__dirname}/data/planets.csv`);
});

To create the stream, we pass the path of the file to the createReadStream method of fs. This returns a file stream that represents the data from the file. If you checked out the repository at the beginning of the tutorial, you’ll have a directory in the root of the project called data that contains the file planets.csv.

After you’ve created the read stream, you can pipe the stream directly into the response to send the file:

app.get("/download", (req, res, next) => {
  const fileStream = fs.createReadStream(`${__dirname}/data/planets.csv`);
  fileStream.pipe(res);
});

If you run the server by executing node ./index.js and load localhost:3000/download in your browser, you should see the CSV data load on the page. Press CTRL+C to terminate the Node.js process.

This simple approach just sends the file as a stream, but we’re not quite done! The file wasn’t actually downloaded; it was only sent to the browser, and we aren’t handling any errors yet.

First, let’s make sure we’re handling the case where the file doesn’t actually exist. We can wait until the open event is emitted from the file stream, to make sure we are opening a file that exists. The open event is specific to working with files and createReadStream, not streams in general:

app.get("/download", (req, res, next) => {
  const fileStream = fs.createReadStream(`${__dirname}/data/planets.csv`);


  fileStream.on("open", () => {
    fileStream.pipe(res);
  });
});

Waiting to start piping until we receive the open event ensures that the file can be read.

If the file doesn’t exist, or the process doesn’t have permission to read the file, an error event will be emitted by the file stream. In that case, we want to handle the error by passing it to an Express error handler using next:

app.get("/download", (req, res, next) => {
  const fileStream = fs.createReadStream(`${__dirname}/data/planets.csv`);


  fileStream.on("open", () => {
    fileStream.pipe(res);
  });


  fileStream.on("error", err => {
    // error opening the file
    next(err);
  });
});

We are manually handling the errors from the file stream instead of using the pipeline method because pipeline destroys streams passed to it when an error occurs. That would destroy the res stream, preventing us from sending back a response.

In most other cases, pipeline is recommended over pipe, but for this particular case handling the errors manually is more useful.

Currently, the browser is displaying the file inline, meaning it is loading the file directly in the browser as if it was a web page or part of a web page. That’s because we haven’t set any headers on the response to tell it about the content it is receiving. When directly streaming to a response, Express doesn’t know much about the content, so it won’t set these headers for you.

To get the browser to download the file, we need to set a Content-Type header on the response that specifies what type of file we are sending back. We also need to set a Content-Disposition header to attachment, with an optional file name, to tell browsers we are sending an attached file as the payload.

Express provides the res.attachment method to help us set Content-Type, Content-Disposition, and file name:

app.get("/download", (req, res, next) => {
  const fileStream = fs.createReadStream(`${__dirname}/data/planets.csv`);


  fileStream.on("open", () => {
    // set relevant file headers
    res.attachment("myFile.csv");
    fileStream.pipe(res);
  });


  fileStream.on("error", err => {
    // error opening the file
    next(err);
  });
});

Express uses the file name passed to res.attachment to infer the file type, as well as set the name of the file that’s downloaded. We passed a filename with the .csv extension, so it will set the Content-Type header on the response as text/csv.

Starting the server again and opening localhost:3000/download will start downloading the file, and save it with the specified file name myFile.csv.

Recap

In this tutorial we used streams to send a file as a response to an HTTP request. We created a read stream from a file on disk, and piped it to the response. We attached listeners to the open and error events on the file stream to make sure we handle any errors that are encountered when creating a stream from the file. To make a browser download the file, we used the Express res.attachment method to add Content-Type and Content-Disposition headers to the response.

Further your understanding

  • HTTP client libraries often implement a streaming interface as well. How might you stream the response from an API client request originating from your server to a response?
  • Express has built in methods to help you send files which use the send package to create a file stream. You can read the documentation and source code to learn more about streaming files.

Additional resources