Scaling an Open edX platform can become unwieldy. Let’s take a look at how Terraform can help you maintain control of everything inside your AWS account

Note: get the source code for this article at https://github.com/lpm0073/cookiecutter-openedx-devops. Follow the instructions in the README.

Open edX is a beast! How do you tame it?

In all fairness, that question is prone to coming up for any successful, modern web platform that goes through a growth spurt. In this article we’ll explore how I manage not just one, but several very large Open edX installations. Here are what I consider to be the key success factors:

  • Infrastructure as code. I use Terraform, but there are other good alternatives. Terraforms gives me the ability to version control my backend infrastructure service configurations so that I can safely fallback when I make a mistakes, and it gives me complete automation of the entire life cycle of each service which saves me lots of time.
  • Dedicated VPC. I use a dedicated VPC for each Open edX installation, which helps to optimize the network for each installation as well as to keep systems from bleeding into each other, and, it also helps with tear-downs.
  • Managed Services. All of my Open edX platforms run on AWS, and I’m biased towards using their managed services such as RDS for MySQL, DocumentDB for MongoDB, EKS for Kubernetes, and Elasticache for Redis. This dramatically reduces the number of failure points for which you are directly responsible.
  • Kubernetes. Paradoxically, adding Kubernetes simplifies most aspects of system management
  • Simple security policies. We’ll talk more below about firewall settings, user accounts, admin accounts, and exposing your backend services to the outside world.

Earlier this year I open-sourced my personal Terraform and Github Actions scripts in the form of a Cookiecutter template repository named Cookiecutter Openedx Devops. You can use this Cookiecutter to create your own Open edX devops repository, perfectly configured with your custom domain name and AWS account information. Cookiecutter Open edX Devops is a highly opinionated set of tools for creating and maintaining an AWS backend for Open edX that satisfies all five of these principals.

The Terraform modules of Cookiecutter Open edX Devops

Cookiecutter Open edX Devops leverages Terraform and Github Actions to provide 1-click backend solutions incorporating the current best practices for each service with regard to feature set, configuration, maintainability and security. This is mostly achieved by restricting the Terraform modules that it leverages to those supported by Hashicorp directly and of Terraform AWS modules, which is a community of AWS service users spanning dozens of large organizations and thousands of individual contributors. For each backend service it:

  • creates and configures the service
  • stores admin account credentials in Kubernetes Secrets
  • creates security groups, IAM policies and anything else that necessary for the service to work correctly with the Open edX applications
  • creates Route53 DNS subdomain records
  • reconfigures the Open edX applications to use the new remote service

Fully integrated backend

Individual services

  • Kubernetes. Uses AWS Elastic Kubernetes Service to implement a Kubernetes cluster onto which all applications and scheduled jobs are deployed as pods. Tutor natively deploys Open edX applications as individual containers for LMS, CMS, Workers, Forum, etcetera. All backend service admin account credentials are automatically stored in Kubernetes Secrets. The Kubernetes configuration itself is intentionally as simple as possible. Simple is good.
  • MySQL. uses AWS RDS for all MySQL data, accessible inside the VPC as mysql.yourdomain.edu:3306. Instance size settings are located in the environment configuration file, and other common configuration settings are located here. Passwords are stored in Kubernetes Secrets accessible from the EKS cluster.
  • MongoDB. uses AWS DocumentDB for all MongoDB data, accessible inside the VPC as mongodb.master.yourdomain.edu:27017 and mongodb.reader.yourdomain.edu. Instance size settings are located in the environment configuration file, and other common configuration settings are located here. Passwords are stored in Kubernetes Secrets accessible from the EKS cluster.
  • Redis. uses AWS ElastiCache for all Django application caches, accessible inside the VPC as cache.yourdomain.edu. Instance size settings are located in the environment configuration file. This is necessary in order to make the Open edX application layer completely ephemeral. Most importantly, user’s login session tokens are persisted in Redis and so these need to be accessible to all app containers from a single Redis cache. Common configuration settings are located here. Passwords are stored in Kubernetes Secrets accessible from the EKS cluster.
  • Container Registry. uses this automated Github Actions workflow to build your tutor Open edX container and then register it in Amazon Elastic Contai