Install Open edX Notes & Annotations

Summary

Open edX Notes & Annotations is one of the coolest, most value-added upgrades that you can add to your Open edX Native build. Hopefully some day it will ship with the platform’s core functionality, but until then this post will help you get Notes & Annotations up and running on your Ginkgo installation (or later) in just a couple of hours. Notes itself is a Django app which largely follows the conventions of norms of the rest of the Open edX ecosystem. Technically speaking, it’s beautifully designed, works great, and really showcases the edX design team’s ability to thread the needle when it comes to enhancing the user experience while keeping the platform manageable.

Setup Procedure

We’ll use Ansible for most of the installation, leaving us with just a few chores beforehand and some cleanup activities afterwards. Here’s our installation and setup procedure:

1. Setup Oauth between LMS and the Notes API

First, we’ll use the Django admin console of the LMS to setup seamless Oauth authentication between the LMS and the Notes API. You’ll need to login using a super user account. You can review the “Managing Open EdX Tips and Tricks” page for instructions on how to convert your account. For the Python/Django uninitiated, Django apps like LMS and CMS come with a “back end” admin console where additional configuration parameters are available beyond what you’ll find in the four JSON files in /edx/app/edxapp/. Refer to this screen shot for the URL path and guidelines for creating your Oauth client.

2. Set Configuration Parameters

Take note that what follows is an unorthodox approach compared to most other configuration activities you’ll perform on your Open edX installation, but so far I haven’t come up with a better alternative.

A. Set Ansible Parameters

First, we’ll edit the Ansible Notes API defaults parameters file that is part of the edx “Configuration” github repository: /edx/app/edx_ansible/edx_ansible/playbooks/roles/edx_notes_api/defaults/main.yml. Following are the exact parameters that we’ll modify.

• EDX_NOTES_API_MYSQL_DB_PASS – Choose a new strong password that the Notes user will pass to MySQL when connecting to the Notes database.
• EDX_NOTES_API_MYSQL_HOST – This is “localhost” by default on Native installations unless you have migrated your MySQL environment elsewhere.
• EDX_NOTES_API_ELASTICSEARCH_URL – set this to “localhost:9200”
• EDX_NOTES_API_DATASTORE_NAME – This is the name of the MySQL database that Ansible will create for you. Use a simple, sensible name like “notes”
• EDX_NOTES_API_SECRET_KEY – Choose a new strong password of at least 16 characters.
• EDX_NOTES_API_CLIENT_ID – This is the “Client ID” value from the Oauth setup screen from the previous step
• EDX_NOTES_API_CLIENT_SECRET – This is the “Client Secret” value from the Oauth setup screen from the previous step
• EDX_NOTES_API_ALLOWED_HOSTS – add a new item to this list containing the fully qualified domain name of your LMS. See my example below (Row 50)

Following is an example showing how your Oauth configuration from step 1 should map to your Ansible parameter values in this step.

B. Set Application Parameters

Next we’ll edit the LMS environment configuration to enable Notes in the front end: /edx/app/edxapp/lms.env.json. There are three sets of changes within the Open edX platform json configuration files:

1. EDXNOTES_INTERNAL_API & EDXNOTES_PUBLIC_API (row 154): The Notes application relies on a REST api that is authenticated by the LMS via Oauth. The API provides the annotation highlight meta data along with descriptive and tag data that the learner might have provided. These two parameter values by default are set to 127.0.0.1 (your local host). You’ll need to explicitly set both of these parameter to your fully qualified domain name. http[s]://[your-domain.com]:18120/api/v1
2. JWT_ISSUER (rows 234 & 239): JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. These tokens are used by the Oauth authentication processes. Change each of these parameters from 127.0.0.1 to your fully qualified domain name, as you did in the previous step above.
3. ENABLE_EDXNOTES (row 190): Set this to true.

NOTE: Running Ansible sometimes results in the configuration files in /edx/app/edxapp being overwritten.

3. Create Users with Ansible

March 19, 2019 – Note regarding Ginkgo native installations and newer. I got a run-time error when I ran this step on a Ginkgo.2 native installation. As a workaround I created the MySQL user manually using mysql from the command line. The Ansible script in this step reads the parameters in your main.yml file and then creates a user in MySql with full access to the edX Notes database.

Ok, we’re ready to put Ansible work, which is a two-step process for this installation. In this step we’ll use Ansible to create a new Linux user, “edx_notes_api”, plus create a MySQL user and then finally add permissions so that our new user can perform basic CRUD operations on the database. This is quick. This should take one or two minutes.

source /edx/app/edx_ansible/venvs/edx_ansible/bin/activate
cd /edx/app/edx_ansible/edx_ansible/playbooks
sudo ansible-playbook -i 'localhost,' -c local ./run_role.yml -e 'role=edxlocal' -e@roles/edx_notes_api/defaults/main.yml

4. Install Notes Software with Ansible

Now we’re ready to install the Notes Django app. This is also quick. This should take a minute or two to complete.

source /edx/app/edx_ansible/venvs/edx_ansible/bin/activate
cd /edx/app/edx_ansible/edx_ansible/playbooks/edx-east
sudo ansible-playbook -i 'localhost,' -c local ./notes.yml -e@/edx/app/edx_ansible/server-vars.yml

5. Run Database Migrations

For anyone new to Django, there’s a process that is generally referred to as “Database Migrations” whereby Django introspects it’s own code to deduce what database tables, fields and relationships are necessary to persist the data described in the various Python objects referred to in the code. This is one of the most compelling features of the Django framework but it can also make software installations seem buggy or at least problematic the first time you slug through the process.

Following are the commands to run database migrations for the edX Notes API, noting that DB_MIGRATION_USER refers to the MySQL user, which probably should be either “root” or “admin”, and DB_MIGRATION_PASS refers to the password for this user. Note also that the “admin” user is created as part of the Native build. You’ll find the password for this user in  /home/ubuntu/my-passwords.yml, stored as the parameter value COMMON_MYSQL_ADMIN_PASS on or around row 12 of this file.

export EDXNOTES_CONFIG_ROOT=/edx/etc/
export DB_MIGRATION_USER=root
export DB_MIGRATION_PASS=replacethisstringwithyoursuperduperaswesomepassword
/edx/bin/python.edx_notes_api /edx/bin/manage.edx_notes_api migrate --settings="notesserver.settings.yaml_config"

The first time your run this you should see a few screens worth of output. You can run migrations any time and as often as you want. It only modifies your database structures if/when its really needed.

6. Compile Assets & Restart edX Applications

The Open edX developer team created a defined process to sweep up and organize all static assets in the LMS and CMS, generically referred to as, “Compiling Assets”. Any time you add or modify either of these two applications you’ll need to run this process again. Otherwise you’re prone to getting some really quirky behavior in the UI. In the worst of cases the app will fail to start, leaving you with a stoic photo image of a sinking ship – yikes.

Following are the operating system commands to manually compile assets for both the LMS and CMS:

sudo -H -u edxapp bash
source /edx/app/edxapp/edxapp_env
cd /edx/app/edxapp/edx-platform
paver update_assets cms --settings=aws
paver update_assets lms --settings=aws

Note: this process takes up to 15 minutes to complete during which time the LMS and CMS will be unavailable to end user.

After this process finishes you should restart the two applications using these commands, and noting that the “:” symbols are intentional and should be included the commands:

/edx/bin/supervisorctl restart edxapp:
/edx/bin/supervisorctl restart edxapp_worker:

7. Enable Notes In One Or More Of Your Courses

For Notes to appear as an option to your online learners you first have to explicitly enable this feature on a course-by-course basis from the “Advanced Options” selection of the “Configuration Menu” in Studio for each course. You’ll find a true/false parameter option “Enable Notes” around halfway down this screen.

8. Verify The Installation

After logging in to the LMS as a student user and entering any course which you’ve enabled Notes you should see a new window tab, “Notes” where annotations will appear. Highlighting text anywhere in this course should result in a popup floating menu with Annotation options.

Trouble Shooting

• Log Files. You’ll see a 500 error in your browser if your configuration is not perfect. You’ll need to review the edX Notes logs to get more diagnostic information on the true nature of your problem. These are located in two different locations: accidentally I assume: /edx/var/log/edx-notes-api and /edx/var/log/edx_notes_api.
• Configuration File. The configuration file for edX Notes is located in /edx/etc/edx_notes_api.yml. You can edit this file using a text editor like vi in case you need to make changes. You have to restart the LMS for changes to take effect.
• Firewall. You should pay close attention to the ports used by the Notes API. You might need to open port 18120 on your firewall.
• Nginx. If you’re using SSL on your site then you probably need to make adjustments to the Nginx virtual server configuration file /edx/app/nginx/sites-available/edx_notes_api. See screen shot below for an example of a site that uses a Letsencrypt SSL certificate.

I hope you found this helpful. Please help me improve this article by leaving a comment below. Thank you!