TL;DR

This is the second post in a series focussing on the use of Cloudflare Workers as APIs. In this post some DevOps challenges are explored and an approach to blue / green deployments is achieved by extending the Serverless framework to dynamically update routes and their associated workers to activate and rotate different versions of the API in and out of service. The post culminates with some video demos of blue / green deployments and CI/CD in action.

The challenge

A common DevOps requirement is to have the ability to deploy new versions and quickly rollback to the previous version without any downtime. Blue / green deployments is a pattern often applied to realise such a requirement. So how would we go about blue / green deployments if we're using Cloudflare Workers as our API?

Architecture

To deploy Cloudflare Workers you simply define a route matching pattern and point it at a Worker. Suppose you're modelling a RESTful API with Cloudflare Workers and we want to implement some account management features, we could create a Worker that acts as our "account" resource and then map a route accordingly:

Now all HTTP requests that arrive at the API domain and match that route will be passed on to our account Worker, for example:

GET https://api.peasey.co.uk/account/123
GET https://api.peasey.co.uk/account/123/something
POST https://api.peasey.co.uk/account/123/something

Simple as that. Obviously your DNS has to resolve to Cloudflare and you have to process the HTTP method and path in your Worker and work out what to do etc, but that's a post for another time.

Concept

I've borrowed the concept of deployments slots from Azure.

We could have as many slots as we want and call them what we want, but to follow the norm we'll stick to having a blue and a green slot.

We use a naming convention for the Worker that dynamically embeds the slot into the Worker name at the point of deployment. For now we'll just add the slot in the naming convention but in later posts you'll see we use the same mechanism to embed other information such as the environment.

When a slot is deployed a slot route will be dynamically created at the point of deployment to allow testing of the slot, the original route from the service definition will only be created and mapped to the appropriate Worker when a slot is made active. At any time we can determine the active slot by inspecting the name of the Worker mapped to the original route.

This approach allows the developer experience to remain unaffected by the deployment architecture as additional routes and the naming convention are applied unobtrusively during the deployment.

The diagram below shows the deployed version of the previous example with the blue slot currently active:

Automation

There are a number of options available when it comes to automating the deployment of Workers.

The first is the wrangler cli tool which requires you to define your routes/Workers in a TOML file and then publish them via a cli command.

Underpinning that is an API that enables you to have finer control over how you manage your routes/Workers.

Then there's a plugin for the Serverless framework that allows us to use Cloudflare Workers as a provider. This is an intriguing option because the Serverless framework is very extensible and I can use it to consistently provision resources in the architecture from several providers.

Approach

I decided to take the Serverless framework / Cloudflare API approach due to the versatility of the plugin architecture and that I can use it to deploy resources in AWS too:

We won't be deploying resources in AWS just yet, but it's something I know I'll need to do in a future post.

Serverless plugin hooks

The plan is to create a plugin and hook into the Serverless hooks, specifically the before:deploy hook, to pre-process the service definition file with our slot information before the deploy command is executed. During the pre-processing stage we can query the Cloudflare APIs to understand which routes are deployed and which Workers they're mapped to in order to determine the slot.

Take this simplified service definition file for our example:

...
functions:
  account:
    name: account
    script: src/integration/cloudflare/workers/account
    webpack: true
    events:
      - http:
          url: api.peasey.co.uk/account/*
...

When pre-processed it might become a blue slot before it's deployed:

...
functions:
  account:
    name: blue-account
    script: src/integration/cloudflare/workers/account
    webpack: true
    events:
      - http:
          url: blue-api.peasey.co.uk/account/*
...

Note the blue prefix to both the function name and the url.

We also need to think about the inverse operation, i.e. to remove the slot, for that we'll hook into the before:remove hook and pre-process the service definition file such that the plugin removes the appropriate routes/Worker.

There are also further things to consider around activating and rotating slots.

Serverless plugin commands

Hooking into the deploy/remove commands allows us to augment the service definition file with our slot information, but deploying is not the final step in activating our service. Deploying gives us the opportunity to test the slot, then when we're ready to release it to end users we can activate it, or rotate it into service (in coordination with the cutover of other resources perhaps).

We'll create the following commands as part of our plugin:

Activate slot

This command enables a specific slot to be activated, regardless of which slot is currently active, i.e. activating the currently active slot is a no-op. So for example to activate the blue slot we can run the following:

> serverless activate-slot --slot blue

Rotate

This command enables the next sequential slot to be activated, in this case we only have blue and green slots so rotating is activating the other slot. This command is useful when you don't know (or care) which slot is currently active:

> serverless rotate

We'll likely integrate this into an automated CI/CD process, so to simplify that we'll ensure that the rotate command acts like the activate slot command when there are no active slots.

Current slot

It might also be useful to have a command that returns the current active slot:

> serverless current-slot
> green

Other considerations

The plugin commands should be able to act on a single function defined in the service definition file, or all of them. All of them are assumed unless you specify the function with the --function (-f) flag.

Proof of concept

I've published a minimal proof of concept repo on GitHub with a local Serverless framework plugin that demonstrates this concept for a basic Worker. You can follow along by creating a Cloudflare account, cloning the above repo, configuring it and then following the steps below.

Let's run-through a typical release process using the plugin to rotate between blue and green slots. I've created some npm scripts in the repo to simplify the commands as we're using dotenv to load configuration.

We start with an empty Cloudflare environment:

After finalising the development for the initial release of our account Worker the first thing we do is deploy:

> npm run deploy

The plugin notices that nothing exists in the Cloudflare environment so the Worker gets deployed to the default slot (in this case the blue slot):

The plugin has created a blue slot route so we can test the Worker:

> curl https://blue-api.peasey.co.uk/account
> "Processing account resource from blue slot..."

Once the Worker is tested we decide to release it to our end users by activating the blue slot:

> npm run activate-slot:blue

The plugin notices that the original route from the service definition doesn't exist, which means no slots are currently active, therefore it creates the original route that our end users will be using and maps it to the blue Worker:

All good, our end users are happily using the blue slot of our API:

> curl https://api.peasey.co.uk/account
> "Processing account resource from blue slot..."

Some time passes and we've finished the development of a new feature for our Worker and we're ready to deploy again:

> npm run deploy

This time the plugin notices that a blue route is deployed and active, so it deploys the Worker to the green slot:

Again, the plugin has created a green slot route so we can test the Worker:

> curl https://green-api.peasey.co.uk/account
> "Processing account resource from green slot..."

Once the Worker is tested we decide to release it to our end users by rotating it into service:

> npm run rotate

The plugin notices that the original route is mapped to the blue Worker and so maps it to the green Worker instead:

Now our end users are happily using the green slot of our API:

> curl https://api.peasey.co.uk/account
> "Processing account resource from green slot..."

And so on and so forth the cycle continues as we incrementally improve our API without any disruption to our end users.

Demo

Watch a demo of blue / green deployments in action:

CI/CD

As I mentioned previously you probably want to integrate this into a CI/CD process. An example of how you might do that in GitHub is when a PR is merged into your release branch, after all tests/checks have passed, the deploy command is run and deploys to the inactive slot. Once that's tested and you are ready to release it to your end users, you create a new release and have that trigger some automation (like a GitHub Action) to run the rotate command.

You can watch a demo of CI/CD using GitHub Actions from my post on Enhancing the development experience for Cloudflare Workers:

Coming next

In the next post I'll be looking at how you can create a faster local development experience.

Make sure you check out the other posts in this series:

1. Delivering APIs at the edge with Cloudflare Workers
2. Blue / Green deployments for Cloudflare Workers
3. Enhancing the development experience for Cloudflare Workers
4. A middleware architecture for Cloudflare Workers
5. API observability for Cloudflare Workers