Use Streams to Extract, Transform, and Load CSV Data

Using the fs module and streams, we can take a local file and perform ETL steps on it to transform the file into JSON and write it to disk. We can use a helpful node module from npm, csvtojson, to convert the csv data for us. With these tools, we can:

  • Create a readstream from a local csv file
  • Pipe the csv data to csvtojson to transform it to JSON
  • Alter the data using a Transform stream
  • Write the transformed JSON data to a file using a WriteStream
  • Handle error events

By the end of this tutorial, you should be able to:

  • Explain how to use the fs module to create read/write streams
  • Implement Transform streams to modify data as it passes through a stream
  • Understand how to handle error events with streams
  • Convert CSV data to JSON using streams
Log in or Sign up to view the complete tutorial.

Sign in with your Osio Labs account
to gain instant access to our entire library.

Data Brokering with Node.js