Upload and stream multipart content with Nodejs

Getting started with streaming your uploaded content is pretty straight forward in Node js. In this example below, we will be taking a look at how you can achieve this using request.js a pretty nifty HTTP request utility that should be a must have in your utility belt. That’s if you’re going to be using Node js for the long haul.

Why streaming is important

For those of you who might be new to the concept of streams. The problem with loading a large file into memory is that you can actually run out of memory and cause your application to crash. if it’s a production application that could be a nightmare you would want to avoid that at all costs.

The beauty of streams is they help you work around this memory problem by acting on chunks of data hence allowing you to minimize your memory footprint. You can imagine a stream like a tap in your kitchen sink. Once a stream is opened, data flows in chunks from its origin to the process consuming it. So, in theory, you can stream unlimited amounts of data.

In order to illustrate the streaming in action, we create our client code which will perform uploading of our content.

On the server side, we will use busboy npm package to actually do the stream of the uploaded content. So imagine our Node Js server is a public facing service that you need to upload multi-part content to.  One thing you want to keep in mind is when using Busboy it only parses application/x-www-form-urlencoded and multipart/form-data requests.

You will need to run the client as a separate node app while the server runs on a separate node app. See full source code here