Learn how to safely upgrade your Open edX installation to the most recent version. This detailed how-to walks you thru backing up, preparing your environment, and how to trouble-shoot common upgrade-related issues.

Summary

This methodology is only recommended for minor updates to the same named version. If on the other hand you want to upgrade from named version X to named version Y then you should follow these instructions:

  1. Backup your production instance: Open edX Complete Backup Solution
  2. Build a new Open edX instance: Open edX Step-By-Step Production Installation Guide
  3. Restore your backup onto your new Open edX instance: Restore Open edX From a Backup

Otherwise, if you just need a good script then follow these instructions and god speed.

cd ~
git clone https://github.com/lpm0073/edx.scripts.git
sudo chmod 755 edx.scripts/*
sudo nohup edx.scripts/edx.platform-upgrade.sh > upgrade.out &

This script does the following:

  1. Stops all running Open edX services
  2. creates a timestamped backup of /edx/app/edxapp/edx-platform/
  3. creates timestamped backups of each of the four json configuration files located in /edx/app/edxapp/
  4. creates a timestamped backup of /edx/app/edxapp/edxapp_env/, the Python virtual environment associated with pip
  5. creates a timestamped backup of /edx/app/edxapp/venvs/, the Ruby virtual environment
  6. creates a timestamped backup of /edx/app/edxapp/nodeenvs/, the Node virtual environment
  7. runs the built-in Open edX upgrade script

and assumes the following…

  • Your Open edX instance is built following the instructions published here: openedx.atlassian.net/Native Open edX Ubuntu 16.04 64 bit Installation
  • Your Open edX platform runs on a single AWS EC2 virtual server
  • You have SSH access to your EC2 instance and sudo capability
  • You are at least an active novice at Linux commands and Bash scripting
  • You at least have a general idea of what Ansible is

If you’d like more context on what this script does as well as some tips on how to prepare for a successful upgrade then read on.

Background

Open edX itself is an awe-inspiring amalgamation of many of the open source community’s greatest achievements. Of the many subsystems that form Open edX’s platform architecture, most of these have a certain “correct” way to be upgraded, but fortunately the devops team at edX have developed a fine set of Ansible playbooks that take care of 100% of these details for us. The Ansible playbooks came pre-shipped with your Native build and they are called automatically by Open edX’s built in upgrade utility. And so, you already have everything that you need.

Lets take a look at how updates are handled across the various subsystems that makeup the platform. The following diagram, which is created from memory and is therefore certain to be incomplete, organizes the platform into categories based on how they are maintained.

Each of the square boxes in the diagram above represents a large open source code project; each consisting of millions of lines of code. It behooves you to study the upgrade process at this high level in order to understand how each of these boxes is upgraded and whether or not you’d ever need to delve into installation/configuration details of any of these. Almost always the answer is a definitive, NO.

Tier Comments
1. Application Software Everything comes from Github repositories that are stored somewhere within https://github.com/edx. The code is basically downloaded from a repository and executed “as is”. That is, there is no compilation nor binaries that need to be created. There is no “make” process. The application code base is mostly Python/Django. If you ever break or damage this code, you can just download it again from Github and you’ll be back up and running in a matter of minutes.
2. Support Platforms These are pretty mature open source projects. All of these are stateless. That is, there is no user data associated with any of these code packages. Furthermore, all of these packages are installed and configured by the Ansible playbooks. So, if anything were to go wrong with any of these supporting platforms then you should be able to simply delete its code from your Ubuntu server and then re-run the Ansible playbook which will detect the missing software and subsequently launch the appropriate installation playbook.
3. System Software These are low-level service sub-systems that are mostly updated by apt-get, the Ubuntu package manager. Again, Ansible manages this for you. Beware that most of these sub-systems are stateful, and, they contain data that has been persisted by either the instructors or the learners. For this and other reasons it is imperative that you create a complete backup of your server before attempting to upgrade the Open edX platform. I prefer AWS server snapshots / Amazon Machine Images as either of these gives you an exact run-time copy of the entire server.

Preparing to Upgrade: Some Best Practices

It is probable that the upgrade will not succeed on your first attempt. With that in mind, here are the preparatory steps that I would suggest that you at least consider before pressing forward with an upgrade to your Open edX platform.

1. Create an AWS Machine Image (AMI)

This is unquestionably your absolute best means of restoring your current production environment in the events that things don’t go as planned. It only takes a few minutes to create a snapshot, and more importantly, it only takes a few minutes to recreate your current production server.

2. Review the steps to launch your AMI

It is quick and simple to stage a replacement server using an AWS AMI, unless you’ve never done it before. If this is new to you then you should definitely do a practice run before venturing into a production software upgrade.

3. Backup the MySQL databases and MongoDB’s

If you already have an up to date AWS AMI then it is unlikely that you’d also need backups of your databases. However, in order to maximize the restore options that are available to you — again, in the event that things don’t go as planned — then it’d be a great idea to have full backups of all of your databases before your venture ahead. Here is a link to a robust backup script: github.com/lpm0073/edx.scripts/edx.backup.sh

4. Check Available Disk Space

The Upgrade script that I reference in this post creates full backups of around 5Gb of files in your /edx file system. Ideally you should have multitudes more than this much space available on your Ubuntu Linux volume. You can check the available space on your volume with this command

df -H
5. Review your steps to merge customizations and configuration into the upgraded code base

Aside from your MySQL and MongoDB data, several other data sources are unique to your installation including for example

  • The passwords.yml file that is hopefully still stored in your home folder following the Native build process that you followed to initially created your Open edX instance. The Open edX update script references this file during the upgrade process.
  • UI Customizations. Hopefully you’ve implemented these as a custom theme. If not then you should seriously consider biting the bullet and migrating your UI customizations to a custom theme beforehand. Otherwise, you’re definitely going to be in for a rough ride.
  • Configuration Files. The entire platform is configured via four json files located in /edx/app/edxapp. The upgrade scripts backs these up for you, but it’d be prudent to automate this yourself.
  • Nginx virtual server settings. Any customizations you’ve made to the Nginx virtual servers will get overwritten by the upgrade process, so make sure to back these up yourself prior to attempting an upgrade!
6. Confirm You Have Enough Time To Complete The Upgrade.

The upgrade takes a long time. Ideally, you’ll perform the upgrade on a low-volume day. If the upgrade goes perfectly then you’ll need around four hours. But, it probably won’t go perfectly.

7. Perform the upgrade.

Warning: Make a complete snapshot of your instance before attempting to upgrade the software.

There are two possible commands that you can use to upgrade the Open edX software.

  1. running /edx/bin/update edx-platform master <– this is the preferred method
  2. re-running the “native build” installation script

You should first attempt to upgrade using the built-in update method. I occasionally have problems with version incompatibilities with Pip and other libraries. If, after 15 minutes or so, I’m unable to trouble-shoot these problems then I’ll switch to the alternative method of re-installing the software which is effectively the following:

wget https://raw.githubusercontent.com/edx/configuration/master/util/install/native.sh
sudo chmod 755 native.sh
# HEADS UP! -- you need to edit this script, and set the following variable
#                     OPENEDX_RELEASE="master"
#
sudo rm -r /edx/app/       #NOTE: this will delete all modifications and configuration of your instance.
sudo ./native.sh

I hope you found this helpful. Please help me improve this article by leaving a comment below. Thank you!