Nginx is commonly used as a web server in Django sites. Nginx is a web server that can also be used as a reverse proxy, load balancer, mail proxy and HTTP cache. Nginx accelerates content and application delivery, improves security, facilitates availability and scalability for the busiest web sites on the Internet. It is a small, lightweight and very fast alternative to Apache web server.
Memcached is a performance optimization technology. The basic idea behind Memcached is quite simple. It takes a lot of processor overhead to instantiate Python objects, but once they’ve been instantiated its pretty effortless to serialize and retain these instance-by-instance in RAM memory in a simple key-value storage format. Memcached plays an extremely important role in Open edX’s impressive backend performance.
Technically speaking, it is a general-purpose distributed memory-caching system. It is often used to speed up dynamic database-driven websites by caching data and objects in RAM to reduce the number of times an external data source must be read. Memcached is free and open-source software, licensed under the Revised BSD license.
Gunicorn is a connector between Nginx and Django. In Django projects Nginx is specially configured so that for any URL that Nginx cannot serve directly from the Ubuntu file system (for example, a static file such as an image or CSS or JS file) it assumes that the URL should be served by Django via Gunicorn. Technically speaking, the Gunicorn “Green Unicorn” is a Python Web Server Gateway Interface HTTP server. It is a pre-fork worker model, ported from Ruby’s Unicorn project. The Gunicorn server is broadly compatible with a number of web frameworks, simply implemented, light on server resources and fairly fast.
This article explains more about how Nginx / Gunicorn / Django work together so serve up web pages, “How to use Django with uWSGI”
Celery/RabbitMQ are backend subsystems that are commonly found in Django projects. These two technologies work together to enable most application processing on behalf of end users to be performed on background threads. If for example, a learner clicks a “submit” button on a homework assignment problem, the homework submission is queued to RabbitMQ and then eventually sent to Celery to be processed. Once the processing has completed, Celery can broadcast the processing results such that the learner’s browser will magically update itself at the appropriate moment, displaying the grading results.
The design objective of including Celery/RabbitMQ in the application stack is to provide a more immediate and performant user experience for the learner. The learner clicks the “submit” button and the browser responds immediately, enabling the learner to continue to interact with other controls on the browser page as opposed to the page completely freezing momentarily while the backend calculates the grading result.
Celery – Distributed Task Queue is a simple, flexible, and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system. It’s a task queue with focus on real-time processing, while also supporting task scheduling. Celery has a large and diverse community of users and contributors, you should come join us on IRC or our mailing-list.
MySQL has a 20-year history in web backends, beginning with its use as the persistence layer in LAMP stacks. MySQL is an open-source relational database management system. In Open edX software MySQL is used to store user data such as usernames, passwords, and and learner grading results. Course data on the other hand, with only a couple of exceptions, is stored in MongoDB.
Open edX stores user data in MySQL, such as usernames and passwords, course enrollment records, and learner submissions to assignment problems and exams. I often use MySQL Workbench to browse table data contents as a way to better understand Python/Django code on which I’m working.
Open edX stores most course content in MongoDB, a cross-platform document-oriented database program. Classified as a NoSQL database program, MongoDB uses JSON-like documents with optional schemas. MongoDB is ideal for storing course content specifically because JSON is an ideal, semi-structured data language.
MongoDB is one of the cornerstone technologies that powers major Internet sites like Facebook, Youtube and Twitter. It’s appeal to platform architects stems from both its flexibility and its stability. It is flexible because the data language can encode as much information as is possible with a relational database; albeit without the rigidity of a relational database. Meanwhile, because the back-end data storage is simply text representations of JSON objects, it can not only be persisted directly to a Linux file system without the need to convert the contents to a binary format (as is the case with relational databases) but also this opens the possibility to use replication to “shard” the data, making the back-end platform incredibly resilient.
Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Elasticsearch is developed in Java.