What Is Node.js?

Node.js, or node if you are more familiar with it, is a non-blocking and asynchronous runtime for JavaScript. This means that we, as front end developers, no longer have to learn a second language in order to create web servers and the back end logic behind our websites. No longer will we need to learn languages like Java or Python or even Ruby and the Rails framework associated with it. We can now build our own back ends with the language we already know and love.

Installation of Tools

Before we get into the nitty gritty and start writing code we will first need to install a few things. If this is not your first time with Node or JavaScript then please feel free to skip this section as I will be covering how to install Node and make sure that it is working on your machine.

We are going to head over to the official Node website over here. The website will look like this:

First things first, depending on when you are reading this post you might see a different version there for the LTS that is fine. As long as you choose the LTS version you will be ok. The steps for installing node are pretty similar on all operating systems. A file will be downloaded and you will click through the install wizard. To make sure that everything was installed correctly you can type:

node -v

npm -v

Those of you knew to Node will notice the second command. No where in the install wizard or download page does it tell you anything about npm. Npm stands for Node Package Manager. You will rely on this very heavily throughout your JavaScript programming career. It doesn't just install packages for Node development. You will also use npm to install other packages such as React, a front end framework that I will cover in another tutorial. For now just run those two commands and make sure that you don't receive any errors. If you everything is good you will receive something that looks like the following:

Now that you have all the tools installed let's get started with writing some code.

Simple Web Server

Here's an example of how you can build a basic web server with node:

require('http')

.createServer((req, res) => res.end('hello world!'))

.listen(8080);

The code above will create a server and have it return a response to the browser that displays a message.

And that is all it takes to create a basic web server. three lines of code, which can technically fit on one line but for readability it's best to break it up.

Our First API

The rest of this post is going to focus on a production e-commerce app that can be modified to fit any starting place for just about any company that may need one. The app will showcase PC parts for sale for those that want to build their own computers. We will be building this API to be used by mobile apps, CLIs and as a library. We are going to create a JSON file that will store the information of the products we are selling. Here's what it will look like:

products.json

[

{

"id": "jrP28HZBVI1",

"description": "ASUS Motherboard",

"imgThumb": "https://images.unsplash.com/photo-1522920192563-6df902920a8a?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=400&q=60",

"img": "https://images.unsplash.com/photo-1522920192563-6df902920a8a?ixlib=rb-1.2.1&ixid=eyJhcHBfaWQiOjEyMDd9&auto=format&fit=crop&w=400&q=60",

"userId": "Z1uhdL3Lmw",

"userName": "Thomas Jensen",

"userLink": "https://unsplash.com/@thomasjsn",

"tags": [

"motherboard",

"ASUS",

"computer",

"parts",

]

},

]

Each product in the listing should have enough info for out client to use. The listing is already in a format that a client can readily use, so the API can start simple. We can make this API available at the endpoint http://localhost:8080/products.

Now we can start fleshing this out. We are going to:

• Create an express server

• Listen for a GET request on our /products route

• Create a request handler to read from our products.json file and send the response

We will also set up some more advanced features:

• Create a client library

• Write a test verifying that it works as expected

• View our products via a separate web app

Express Server

We will start simple:

server.js

const fs = require('fs').promises;

const path = require('path');

const express = require('express');

const port = process.env.PORT || 8080;

const app = express();

app.get('/products', listProducts);

app.listen(port, () => console.log(`Server listening on port ${port}`));

async function listProducts(req, res) {

const productsFile = path.join(__dirname, '../products.json')

try {

const data = await fs.readFile(productsFile);

res.json(JSON.parse(data));

} catch (err) {

res.status(500).json({ error: err.message });

}

}

The above code sets up a very basic web server using Express. Express is a very popular library that allows us to create servers fairly easy and quickly. We will require the fs module which allows us to access the file system and we also use the path module to get the path to our products.json file so that we can use the data inside to display the data to the website in the browser. If you are unfamiliar with the try/catch block the reason that we use this is so if there is an error we can catch it and produce a somewhat decent and understandable error message. The try block will, as the name says try to run the code within that portion of the block. If the code fails then the catch block will display the error message as to what failed so we can fix it.

Our API should now be up. The API will only return JSON and won't care how the data is necessarily being used or rendered. In other words we have many ways that we can use it. The API can be hosted on a CDN, a CLI utility such as curl, a mobile app, or an HTML file. As long as the client used understands HTTP, everything should work fine.

We can run the API to see if it works by opening a terminal, if you are using VS Code you can go up to the top of the screen and select view then terminal and it will open a terminal right there within the editor already at the spot you will need. Then you will run:

node server.js

Basic Front End

We will start with a very simple HTML page that will fetch the API endpoint and use console.log() to display it.

index.html

<!DOCTYPE html>

<html lang="en">

<head>

<meta charset="utf-8">

<title>Awesome Parts</title>

</head>

<body>

<script>

const url = "http://localhost:8080/products";

fetch(url).then(res => res.json()).then(products => console.log(products));

</script>

</body>

</html>

If you are using VS Code you can install a really handy extension called Live Server that will let you right click on the html file and select "Open with live server" it will automatically open a browser and launch your file.

You should now be able to press F12 or right click and select inspect if you are in Google Chrome, and then if you aren't on it by default select the console tab. You should now see something like this:

Since our HTML page is being served with a different origin than what our API is using, our browser is blocking the request. For security reasons, browsers will not load data from other origins/domains unless the other server returns the correct CORS(Cross-Origin Resource Sharing) headers. If the HTML file was server directly from our server we wouldn't have this issue, but as we want to be able to have any web client access our API we need to add a single line to our route handler like so:

res.setHeader('Access-Control-Allow-Origin', '*');

res.json(JSON.parse(data));

If you got something like this to show up in the console within the browser pat yourself on the back as you just got CORS working. If you find yourself getting an error when trying to get your data from an API in the future make sure that you have that one line of code and if not add it then try again.

Since this post is about Node and setting up APIs not directly dealing with the front-end we are going to move on but if you are familiar with writing front-end code and want to add a front-end to consume your API by all means feel free.

Creating A Library

We now have a basic API setup and running. We are now ready to modularize our app so that we can keep things neat and clean for when we start adding more functionality. A single-file production API would be extremely difficult to maintain.

The module system that Node uses is one of the best things about using Node. We are able to leverage this to make sure that our app's functionality is divided into individual modules with clear boundaries.

So far we have been using the require() method to pull in core modules that are built directly in to Node such as the http and third party modules like express. We are going to see how we can use the require() method to import local files. We will be pulling our route handlers out into their own file. We are going to name the file app.js and we will call the file inside of our server.js file like so:

server.js

const express = require('express');

const api = require('./app');

const port = process.env.PORT || 8080;

const app = express();

app.get('/products', api.listProdcuts);

app.listen(port () => console.log(`Server listening on port ${port}`));

Here is our app.js:

const Products = require('./products') module.exports = { listProducts } async function listProducts (req, res) { res.setHeader('Access-Control-Allow-Origin', '*') try { res.json(await Products.list()) } catch (err) { res.status(500).json({ error: err.message }) } }

We use require() pull in another module, ./products. This will be our data model for our products. We will keep the data model separate from the API module because the data model doesn't need to know about HTTP request and response objects. The API module is responsible for converting HTTP requests into HTTP responses. This is achieved by the module leveraging other modules. Another way of thinking about this is that this module is the connection point to the outside world and our internal methods. If you come from a MVC background such as Django or Ruby on Rails this would be like the Controller.

Let's build the products module

products.js

const fs = require('fs').promises;

const path = require('path');

const productsFile = path.join(__dirname, '../products.json');

module.exports = {

list

}

async function list() {

const data = await fs.readFile(productsFile);

return JSON.parse(data);

}

As of right now, our products module only has a single method, list(). Also take note that this module is general-purpose, we want to keep it this way so that it can be used by our API, a CLI tool or even tests. A similar thought process is to think about the server module being on the outside. This module is responsible for creating a web server object, setting up middleware (we will touch on this shortly), and connecting routes to route handler functions. Basically our server module connects external URL endpoints to internal route handler functions.

Our API module is a collection of route handlers. Each route handler is responsible for accepting a request object and returning a response. The route handler does this primarily by using model methods. Our goal is to keep these route handler functions high-level and readable.

Last but certainly not least we have our model module. Our model is on the inside, and should be agnostic about how it's being used. The route handler functions can only be used in a web server because they expect response objects and are aware of things like content-type headers. In other words the model methods are only concerned with getting and storing data.

Query Parameters

Our API returns the same data each time. This means that there's no reason for our API to exist. The products.json file, which is a static file that could be served from a server or CDN, could technically achieve what our API does. We need to have our API respond with different data depending on the client's needs. The first thing a client will need is a way to request only the data that it wants.

The goal is to accept two query parameters that are only necessary if the client wants to fetch only a specific amount of data and not all products at once. These parameters are going to be limit and query. By allowing the client the ability to specify how many results they receive at a time and how many results to skip, we can allow the client to scan the catalog at its own pace.

An example could be if the client wants the first 50 items, it can make a request to /products?limit=50, and if the client wants the next 50 items, it can make a request to /products?limit=50&offset=50.

Express can parse query parameters for us. This means that when a client hits a route like /products?limit=50&offset=50 we should be able to access an object like:

{

limit: 50,

offset: 50

}

Let's update our request handler to use the built in feature of express.

app.js

async function listProducts(req, res) {

res.setHeader('Access-Control-Allow-Origin', '*');

const { offset = 0, limit = 50 } req.query;

try {

res.json(await Products.list({

offset: Number(offset),

limit: Number(limit)

}))

} catch(err) {

res.status(500).json({ error: err.message });

}

}

The above changes show us creating two variables offset and limit from the req.query object that express provides. We also make sure that they have default values if one or both are not set. We also coerce the limit and offset variables into numbers as by default query parameter values are strings.

Let's make a minor change to our file:

products.js

async function list (opts = {}) {

const { offset = 0, limit = 25} = opts;

const data = await fs.readFile(productsFile);

return JSON.parse(data).slice(offset, offset + limit);

}

We should now be in a decent spot to see if our changes are working. Go ahead and run your server and navigate to http://locahost:8080/products?limit=1&offset=1

In another post I will show you how we can view our APIs with a really awesome tool called Postman. For now using the browser will work. But keep in mind with most API development, the tool of choice is to use Postman or something similar.

Filtering our Products

At this point we should now have a good understanding of how to add functionality to our app. We are going to use the same approach to add a filter by tag. We can accept a specified tag filter via a query parameter. All we need to modify is the app.js file so that we can change our route handler to look for a tag query parameter.

app.js

async function listProducts(req, res) {

res.setHeader('Access-Control-Allow-Origin', '*');

const { offset = 0, limit = 50, tag } = req.query

try {

res.json(await Products.list({

offset: Number(offset),

limit: Number(limit),

tag

}))

} catch(err) {

res.status(500).json({ error: err.message })

}

}

We are making sure to pull the tag property from the req.query object and pass it as an option to Products.list(). Because all query parameters come in as strings, and we expect tag to be a string, we don't need to coerce it to a type as the default type is string.

Our handler is updated but right now, our Products.list() method has not been changed to accept the tag option. If we were to test this endpoint, filtering wouldn't work yet. Let's update our products.js file

products.js

async function list (opts = {}) {

const { offset = 0, limit = 50, tag } = opts;

const data = await fs.readFile(prodcutsFile);

return JSON.parse(data)

.filter((p, i) => !tag || p.tags.indexOf(tag) >= 0)

.slice(offset, offset + limit);

}

We have restricted our response to a single tag by adding in the filter() method.

Fetching a Single Item

The app is coming along pretty well. We are able to page through all of our products and to filter results by tag. The next thing we need is the ability to get a single product.

We're going to create a new route, handler and model to add this feature. Let's add the new route to our server.js file

server.js

const express = require('express');

const api = require('./app');

const port = process.env.PORT || 8080;

const app = express();

app.get('/products', api.listProducts);

app.get('/products/:id', api.getProduct);

const server = app.listen(port, () => console.log(`Server listening on port ${port}`);

)

The route we are using is a pretty standard route. We want express to listen to any route that starts with /products/ and is two levels deep. This route will match the following urls, /products/1, /products/xyz, or /products/some-super-long-string. Express will make anything in the place of :id available to our route handler on the req.params.id property.

Let's now add the getProduct() method.

app.js

module.exports = {

getProduct,

listProducts

}

async function getProducts(req, res, next) {

res.setHeader('Access-Control-Allow-Origin', '*');

const { id } = req.params;

try {

const product = await Products.get(id);

if (!product) return next()

res.json(product)

} catch(err) {

res.status(500).json({ error: err.message });

}

}

Here we are seeing something new. The next() method in our handler function is used in the case that our item isn't found. The other new thing is the req.params to get the desired product ID from the request. I will touch more on next() later.

Let's update Products.js

products.js

async function get(id) {

const products = JSON.parse(await fs.readFile(productsFile));

for (let i = 0; i < products.length; i++) {

if (products[i].__id === id) return products[i]

}

return null;

}

We are just iterating through the items until we find the one with the matching ID. If we don't find one we return null. The section will go into more detail about middleware.

Next() up Middleware

First we have some refactoring we need to do. If you open the app.js file you will see there is some repetitiveness that can be taken care of. Both of methods set the Access-Control-Allow-Origin header for CORS as well as use a try/catch to send JSON. We can use some middleware to put this logic in a single place.

We have used the request handlers that immediately correspond to HTTP methods such as GET. There are others that we will touch on later such as PUT, POST and DELETE. I want to talk about middleware here though. We can setup a call that, regardless of what the HTTP method or URL expected, will still function. Lets update our server.js file.

server.js

app.use(middleware.cors);

app.get('./products', api.listProducts);

app.get('./products/:id', api.getProducts);

We will create a new module middleware.js to store our functions.

middleware.js

const cors (req, res, next) => {

const origin = req.headers.origin;

res.setHeader('Access-Control-Allow-Origin', origin || '*');

res.setHeader(

'Access-Control-Allow-Methods',

'POST, GET, PUT, DELETE, OPTIONS, XMODIFY'

)

res.setHeader('Access-Control-Allow-Credentials', true);

res.setHeader('Access-Control-Max-Age', '86400');

res.setHeader(

'Access-Control-Allow-Headers',

'X-Requested-With, X-HTTP-Method-Override, Content-Type, Accept'

)

next();

}

This file is storing a bunch of CORS headers. By default browsers are very picky about what request types are allowed. We can set these headers to tell the browser to be more permissive when requests are made from pages hosted on other domains/origins. As you may have noticed we are also calling next().

Next() is a function that is provided to all request handlers; this includes both route handlers and middleware functions. Next() allows our functions to pass control of the request and response objects to other handlers in a series.

The order in which these handlers is defined is by the order in which they are called. If you look back at your file you will see app.use(middleware.cors) before we call app.get() for each of our route handlers. This means that our middleware function will run before either of those route handlers have a chance to. And if we didn't use next() the route handlers wouldn't be called.

Let's update app.js

app.js

const Products = require('./products');

module.exports = {

getProduct,

listProducts

}

async function getProduct(req, res, next) {

const { id } = req.params;

try {

const product = await Products.get(id);

if (!product) return next();

res.json(product)

} catch(err) {

res.status(500).json({ error: err.message });

}

}

async function listProducts(req, res) {

const { offset = 0, limit = 50, tag } = req.query;

try {

res.json(await Products.list({

offset: Number(offset),

limit: Number(limit),

tag

}))

} catch(err) {

res.status(500).json({ error: err.message });

}

}

CORS still works? Crazy right? Our middleware.js module now handles that for us instead of the app.js file.

Error Handling

We have officially centralized our CORS header. We can now do something similar with our error handling. We are using the try/catch block in both of our route handler functions. Let's move that out into a function in our middleware.

middleware.js

function handleError (err, req, res, next) {

console.error(err);

if (res.headersSent) return next(err);

res.status(500).json({ error: 'Internal Error' });

}

The first argument we pass to this new function is err. This function will receive the error object that is passed to next() along with the other standard arguments, req and res. Console.error() will log the errors for us and not in the response for the client to see. Error logs often contain sensitive information, and it can be a security risk to expose them to API clients.

Express comes with it's own 404 Not Found error handling but we want our error to be handled with JSON. Let's go ahead and build that function. I will use ES6 functions just to show you the difference. This syntax is the newer way to create functions but you can do both.

middleware.js

const notFound = (req, res) => {

res.status(404).json({ error: 'Not Found' });

}

The function above is a normal middleware function so we are not looking for an error object. The trick is that we only want this to run after all routes ahve been checked. If no route handlers match the request's URL, this function should run. This function will also run if any route handler matches, but calls next(), with no arguments.

Let's add this functionality to our server.js.

server.js

app.use(middleware.cors);

app.get('/products', api.listProducts);

app.get('/products/:id', api.getProduct);

app.use(middleware.handleError);

app.use(middleware.notFound);

The logic for how express will handle the requests is pretty straight forward. It will basically go from top to bottom. First, all requests will run through middleware.cors(). The request URL will then be matched against our two route handlers. If a route handler does not match the request URL, middleware.notFound() is ran, if there is the corresponding route handler will run. While that route handler is running, if there's an error and next(err) is called, middleware.handleError() will run.

HTTP POST, PUT, AND DELETE

I said that we would be implementing these other HTTP methods soon. As promised let's go ahead and implement them. GET requests are one of the most simplest and common request, but they limit the info that a client can send to an API. We aren't going to go into too much detail in this post. So for now we are only going to concentrate on receiving and parsing data from the client.

When a GET request is sent the only information transferred is the URL path, host and request headers. GET requests are not supposed to have any permanent effect on the server. To provide richer functionality to the client we need to support other methods like POST.

A POST can contain a request body. The request body can be any type of data such as a JSON document, form field, or a movie file. An HTTP Post is the primary way for a client to create documents on a server.

Our admin users will be able to use the client to create new products. For now, all users will have this power, as I won't be covering authentication and authorization for admin users in this post.

Let's update our server.js file.

server.js

async function createProduct (req, res, next) {

console.log('request body:', req.body);

res.json(req.body);

}

We are interested in storing a new product for use later. Before we can store a product, we should be able to access it. Since we're expecting the client to send the product as a JSON document for the request body, we should be able to see it via console.log(), then send it back using res.json().

In order to get the product data back we need to actually parse the data. Express doesn't parse request bodies for us. We are going to use the express recommended middleware.

Let's add that to our server.js

server.js

const express = require('express');

const bodyParser = require('body-parser');

const api = require('./app.js');

const middleware = require('./middleware');

const port = process.env.PORT || 8080;

const app = express();

app.use(middleware.cors); app.use(bodyParser.json()); app.get('/products', api.listProducts); app.post('/products', api.createProduct);

app.put('/products/:id', api.editProduct);

app.delete('/products/:id', api.deleteProduct);

As you can see we also add the put and delete methods at the bottom. The put method in case you were wondering is another way to update an existing piece of data.

Let's update the app.js file with the methods for put and delete.

app.js

async function editProduct(req, res, next) {

res.json(req.body);

}

async function deleteProduct(req, res, next) {

res.json({ success: true });

}

Summary

I hope that you enjoyed learning about Node and building out a simple API that can be easily converted to an e-commerce API for just about anything. I had a lot of fun writing this post and if more people are interested I will definitely see about creating more content for it. Thanks for reading.