Newest project hot off the keyboard: A/B(/C/D/etc…) split testing with Astro. Going into this project you will first need to know two major things:

  1. What is Astro
  2. What is A/B Split testing

What is Astro?

Astro is my favorite web framework for it’s simplicty and server-side nature. Astro is the framework I’ve build this website in but also sites like tegan.training and foss.cooking - which you can read about here. The two main things I like about are:

CSS scoping means that on one page, /index.html for example, I can write some custom css for a div, then on another page, let’s say /foo.html, the div won’t be styled. You can, of course, override this behavior but it’s nice to not need 1000 different css classes, just to get some unique styling on a specifc page, or part of page.

The other great thing being the backend tooling. Astro provides helper functions for doing simple things like getting cookies, but also for more advanced things such as creating custom (pseudo-)html classes. Also, since Astro tries to keep all javascript away from the front end, we can easily pass backend variables into the html to be rendered then sent to the frontend. This will be useful in a second!

What is A/B testing?

aka: A/B testing, Split testing, A/B split testing

A/B testing is where we serve different versions of a website(/advertisement/whatnot) to the users, then track how many sales were gotten with which version of the media. Let me explain with a little diagram:

user_1 	-> website_a -> doesn't buy

user_2 	-> website_b -> buys

user_3 	-> website_a -> buys

user_4 	-> website_b -> doesn't buy

user_5 	-> website_a -> doesn't buy

user_6 	-> website_b -> buys

Here we can see that website_a lead to one purchase, but website_b lead to two purchases. This must mean that website_b had better copy or styling and thus made us more sales! Neat!

Implementation

Note: Incase I change stuff on the repo later I made a tag writeup so you can go back and look at the code as I’m describing it here.

Note: If you haven’t worked with .astro files before, you can think of them just like normal .html files, except that there is a fenced area at the top (denoted by ---). Inside the fence is typescript code that is run either on every page load (how this project is configured), or just once when generating the static site.

Link to the repository.

Here is the main flow of the website:

  1. User requests any page on the site

  2. As the request comes into the server, middleware (middleware.ts) intercepts it and checks the user’s cookies

    1. If the user has the cookie with the name “split” then don’t do anything, we have already tagged this user

    2. If there is no cookie detected, then we choose a split at random and set a cookie to reflect that choice

    3. We then set the Astro.locals.split object to be the split we have detected/chosen for this browser, as seen here. This is to allow us to make descisions while rendering the page, based on the split chosen (we will see this in just a second)

    4. Finally - pass the request onto the next stop - in this case to be rendered

  3. Inside the code for any given page (like index.astro) we can switch on the Astro.locals.split object. Then depending on what split this user is requesting we can serve modified versions of the webpage to them!

  4. When the user is checking out - as seen in the in the check for POST requests here - we, similar fashion to how we modifed index.astro based on the split, read and save split identifier when processing the transaction! (If this doesn’t make sense, here’s a footnote to a more in-depth explanation1)

From here the obvious next steps would be to connect this to a database to store identifiers so we could later count our results with some SQL or whatever your DB users as a query language.

Perhaps for the next project I could make a proxy that automatically does split testing for you, and just routes between two different backend webservers. That is, if I ever sit down and learn how to write a reverse proxy…

Footnotes

  1. When a user requests the /checkout url, their browser will submit a GET request to that endpoint. On line 9 we see that we have a check for POST reqeusts, so none of that code will get run, and the user will recieve the html that is written below the fence (---). This html includes a form. As you can see on line 39, the form’s method is POST. This means that when the form is submitted it will request this same endpoint (/checkout) but this time making a POST request instead of a GET request. Now all the code that is inside that if statment’s body will be run, which does some “validation” (not really), then prints out the user’s email and split identifier to the server’s console! ↩