How I built a URL Shortener with Cloudflare Workers and KV - A Tutorial

This article was originally published on my blog.

Introduction

In this tutorial, you will learn how to develop apps on Cloudflare workers and Cloudflare KV by building a URL shortening service. You can try a demo of the url shortener you will build here! Before we start, we have to complete some prerequisites. We will install wrangler, Cloudflare's command line tool and then we'll login wrangler to our workers account. You can skip these steps if you have previously done them.

Prerequisites

1. Installing wrangler
2. Login into wrangler

Installing and logging into wrangler

Install wrangler globally by running npm install -g @cloudflare/wrangler

When wrangler is installed, run wrangler login from your terminal. You will be asked to choose whether to open a browser window and login from there or to enter an API key manually. If you decide to open a browser, a browser window will be opened and all you have to do is to login and authorize the app. If the first method doesn't work or you prefer to enter the API key manually, you will have to create an API key and then run wrangler config and paste your API token. You can find your API token at https://dash.cloudflare.com/profile/api-token or create a new one with the "Edit Cloudflare Workers" template. After completing these steps you should be ready to move on with the tutorial.

Creating our workers app

We will create our app from the cloudflare workers default template. Open a terminal at the directory where you want the app to be created and run wrangler generate short-linker https://github.com/cloudflare/worker-template. This will clone the starter repository for you. You can replace short-linker with any name you like.

The main files in the short-linker folder that was created will be similar to this

--short-linker
   |--index.js
   |--wrangler.toml

As you may have already guessed, wrangler.toml contains the configuration data for our worker, while index.js is the entry point of the worker.

One more step we need to complete before we start development is to add your Cloudflare account_id to the wrangler configuration file of your project. To find your account_id, run wrangler whoami from a terminal. Copy your account ID and paste it in the space provided in wrangler.toml Your wrangler.toml should look like this now // https://developers.cloudflare.com/workers/cli-wrangler/authentication

name = "short-linker"
type = "javascript"
account_id = "veryawesomeaccountid"
workers_dev = true
route = ""
zone_id = ""

Starting the workers development server

Open a terminal from the short-linker folder and run wrangler dev. This will create a preview deployment that will be accessible from localhost:8787. You should see 'Hello Worker!' when you visit localhost:8787. If that doesn't work, then you should make sure you have logged wrangler into your cloudflare account, you have added your account-id to wrangler.toml, and the computer you are running the development server on has internet access.

Most of our work now will happen in index.js since we are done with the configuration. Open index.js. the content should look as below

 addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})
/**
 * Respond with hello worker text
 * @param {Request} request
 */
async function handleRequest(request) {
  return new Response('Hello worker!', {
    headers: { 'content-type': 'text/plain' },
  })
}

At the top, we have an event listener listening for fetch vents. In that event listener, we pass the event object to a handler that returns a response based on the data contained in the object passed to it. Our request handler here is the asynchronous function handleRequest. The handleRequest function here is simple, we simply return a Response object with a body and some headers. Replace Hello worker with Hi worker, save, In handleRequest, we respond with a Response object. You can replace Hello worker! with anything else you like. When you save, the preview server will automatically detect the changes and redeploy the changed code. Reload localhost:8787; you should see Hi worker being returned.

The request object that is passed to the handleRequest function contains all the data in the request body like the url that the request was made to, the method that the request was sent with, the body of the request if it was a POST or similar request and other data.

Some of the most useful properties and methods of the request variable passed to the handler are

• request.method: it contains the method of the request e.g. "POST", "GET", "UPDATE"
• request.url contains the url where the request was made to.
• request.text() returns the request body as plain text
• request.json() returns the body as a json object You can find a full list of the properties of the request object here.

To grasp the relevance of the properties of the request object, we have to explain something about workers. A plain worker, like the one that we just created returns the same response for all requests regardless of the url or method. A GET request to localhost:8787/some/sub/folder, a POST request to localhost:8787/mock-api, and a GET request to localhost:8787/ will all have the same response - Hi worker in our case. You can try that and see for yourself. It is with this request object that we customize our response based on the method, the url, or any other parameter of the request.

Test these by running these curl commands from your command line

• curl -X GET http://localhost:8787/some/sub/folder
• curl -X POST http://localhost:8787
• curl -X GET http://localhost:8787

Modify index.js to look as follows and save.

 addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})
/**
 * Respond with hello worker text
 * @param {Request} request
 */
async function handleRequest(request) {
  if(request.method == "POST") {
    return new Response('Hello POST worker!', {
      headers: { 'content-type': 'text/plain' },
    })
  }
  return new Response('Hi worker', {
    headers: { 'content-type': 'text/plain' },
  })
}

What this does is that it checks if we made a POST request, and if we made a post request, it will return Hello POST worker. Run curl -X POST http://localhost:8787 from your command line. You should see Hello POST worker as your response, but curl -X GET http://localhost:8787 still returns Hi worker.

Now you should have a good understanding of the purpose of the request object and we are ready to start building our URL shortener

How our URL shortener will work

Here is little description of the dynamics of our worker. When you make a POST request to localhost:8787 or any of its subfolders, with the body of the request containing the url you want to redirect to, the worker will generate a random string, say aj3ad. We will tie or create a map from this random string to the url that was contained in the request body. On every visit to my-worker-subdomain.worker.app/aj3ad, we will extract the random string at the end, find what url that string is tied to and finally redirect the user to that URL. Cool right?

In order to do this, we need some form of persistent storage to store the random strings and the URL the string is tied to. This is when we need to use cloudflare KV. Cloudflare KV(key-value storage) is a persistent storage just like Redis. The storage is similar to a big JSON file where you can assign values to keys, and also read values from keys whenever you want to.

Creating our namespace

In cloudflare KV we have namespaces. A namespace is what you will call a table in a regular database, but the table has only two columns - the key and the value. We can write to a namespace with NAMESPACE.put("key", "value") and get a value with NAMESPACE.get("key"). To create a namespace with the name URL_SPACE, run wrangler kv:namespace create "URL_SPACE" from a terminal. You can replace URL_SPACE with any other name you like. When it runs, you will be prompted to add an extra line to your configuration file(wrangler.toml). What this does is that it binds the namespace we created to a variable in our worker. Binding means to assign a variable that we will access our namespace from. This means that the name of your namespace can be JACK but it will be accessed from the worker as JILL. The line you will be prompted to add looks like this.

kv_namespaces = [
         { binding = "URL_SPACE", id = "namespaceid" }
]

However, since we are working on a preview deploy, we will have to add a preview namespace too so you will modify the above line to look like below in wrangler.toml.

kv_namespaces = [
         { binding = "URL_SPACE", id = "namespaceid", preview_id = "namespaceid" }
]

Wrangler.toml should look like this now.

name = "short-linker"
type = "javascript"
account_id = "veryawesomeaccountid"
workers_dev = true
route = ""
zone_id = ""
kv_namespaces = [ { binding = "TEST_SPACE", id = "namespaceid", preview_id = "namespaceid" }]

You will receive a warning about using the same namespace for a production and preview deploy but you can ignore it for now. However, if you are working on a real app, you should be careful to create a new namespace for your preview deploys to prevent breaking things during development. You will have to restart your development server manually. You can end the old server process by pressing Cmd + C or Ctrl + C, and then star the server again with wrangler dev.

Creating an endpoint to generate short URLs

The first thing we will do is to create a handler for post requests. When a post request is made to our worker, the handler will create a key-value entry in our KV storage. The key that it will be stored with will be the a randomly generated string that will be the end part of the short url. The url that we want to redirect to will be contained in the request body and will be read with request.text().

Add the following lines to index.js

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})
/**
 * Respond with hello worker text
 * @param {Request} request
 */
async function handleRequest(request) {

    if( request.method === "POST"){ 
        const neededURL = await  request.text()
        const rand = Math.random().toString(30).substr(2, 5);
        await URL_SPACE.put(rand, neededURL)
        return new Response(rand)
    }

  return new Response('Hello worker!', {
    headers: { 'content-type': 'text/plain' },
  })
}

What this does is that it checks if our request is a POST request and if it is so, it gets the url that we want to redirect to from the request body by calling request.text. It then generates a pseudo-random string with Math.random().toString(30).substr(2, 5) and then store the pair in our KV with TEST_SPACE.put(rand, neededURl).

When you save, your code will be redeployed. You can make a post request to locahost:8787 by running curl -d "https://www.google.com" -H "Content-Type: text/plain" -X POST http://localhost:8787/ from your terminal. You should receive an about six characters long response which is the random string. If this works, then kudos to you. You are ready to proceed.

Automatically adding request protocols

Another thing you may want to do is to automatically redirect users to a https version of a site if they don't add a protocol to their request. We do this with a regex search that appends https:// if the redirect url does not contain either http:// or https://. To add this modify requestHandler to look like this.

async function handleRequest(request) {

    if( request.method === "POST"){ 
        const neededURL = await  request.text()
        let cleanUrl
        if(!neededURL.match(/http:\/\//g) && !neededURL.match(/http:\/\//g)){
            cleanUrl = "https://" + neededURL
        }
        else {
            cleanUrl = neededURL
        }
        const rand = Math.random().toString(30).substr(2, 5);
        await TEST_SPACE.put(rand, neededURL)
        await TEST_SPACE.put(rand, cleanUrl)
        return new Response(rand)
    }

  return new Response('Hello worker!', {
    headers: { 'content-type': 'text/plain' },
  })
}

Creating the short-url redirect

The next step is to create the redirect functionality. We want someone that visits localhost:8787/random-string to be redirected to the page that the random-string was generated for. We will use regex to strip off the https://.../ or http://.../ part of the url and search and redirect to the URL tied to the random string in our namespace.

async function handleRequest(request) {

    if( request.method === "POST"){ 
        ...
    }

    if( request.method === "GET" ){
        let shortCode = request.url.replace(/https:\/\/.+?\//g, "")
        shortCode = shortCode.replace(/http:\/\/.+?\//g, "")

        if(shortCode !== "") {
            const redirectTo = await TEST_SPACE.get(shortCode)
            return Response.redirect(redirectTo, 301)
        }
        else {
            return new Response( 'Welcome to shortlinker, make a post request to add a shortlink', {
                headers: { 'content-type': 'text/plain' },
            })
        }
    }

  return new Response('Hello worker!', {
    headers: { 'content-type': 'text/plain' },
  })
}

Our new code addition intercepts if the request method is GET and we use the request.url.replace(/https:\/\/.+?\//g, "") to remove the initial part of our url and assign the random part left to the variable shortCode. If the part left is not an empty string, then it wasn't the homepage that we visited and then our worker goes on to redirect our user to the URL shortCode is tied to in our namespace.

You can test thus by running curl http://localhost:8787/short-url replacing short-url with one of the random strings that was generated after making a POST request. You now have a fully functional url-shortening worker and everything should work as expected. You can make a POST request to your worker and visit the url that is returned.

Further improvements

In the following code block, we have the complete code for the worker together with a frontend for the homepage that you can use to make the requests without the terminal. I also made the API to return the full shortlink url.

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

const htmlBody = `
<html>
    <head>
        <style>            
            #url {
                font-size: 2em;
                width: 100%;
                max-width: 500px;
            }
            #submit {
                font-size: 2em;
            }
        </style>        
        <script>
            const submitURL = () => {
                document.getElementById("status").innerHTML="creating short url"
                //await call url for new shortlink and return
                fetch('/', {method: "POST", body: document.getElementById("url").value})
                    .then(data => data.text())
                    .then(data => {

                        console.log('ready')
                        document.getElementById("status").innerHTML="Your short URL: " + data
                    } )
            }
        </script>
    </head>
    <body>
        <h1 id="title">URL Shortener</h1>
        <input type="text" id="url" placeholder="enter url here" />
        <button id="submit" onclick="submitURL()">Submit</button>
        <div id="status"></div>
    </body>
</html>
`

/**
 * Respond with hello worker text
 * @param {Request} request
 */
async function handleRequest(request) {
    if( request.method === "POST"){ 
        const neededURL = await  request.text()
        let cleanURL
        //add necessary protocols if needed to url
        if(!neededURL.match(/http:\/\//g) && !neededURL.match(/https:\/\//g)){
            cleanURL = "https://" + neededURL
        }
        else {
            cleanURL = neededURL
        }

        const rand = Math.random().toString(30).substr(2, 5);
        await URL_SPACE.put(rand, cleanURL)
        let fullURL = "localhost:8787/" + rand;
        return new Response(fullURL)
    }

    if( request.method === "GET" ){
        const shortCode = request.url.replace(/https:\/\/.+?\//g, "")
        if(shortCode !== "") {
            const redirectTo = await URL_SPACE.get(shortCode)
            return Response.redirect(redirectTo, 301)
        }
        else {
            return new Response( htmlBody, {
                headers: { 'content-type': 'text/html' },
            })
        }
    }

}

View a working demo here!