Setup SMTP Email on Open edX

With Open edX Ginkgo, integrating an SMTP email account like Google’s Gmail is as simple as setting a few parameter values in the four JSON configuration files for the LMS and Course Management Studio, each of which are located in /edx/app/edxapp/. Get SMTP email working on your Open edX instance in less than an hour with this step-by-step how-to guide.

Summary

The native build of Open edX relies on a simple email generator utility that comes with Ubuntu 16.04. While this technically works, the messages that it generates for new user activations, password reset instructions, and welcome messages are generally universally flagged as spam by common email management client applications like MS Outlook, Mac Mail and so on. Fortunately however, the architects of Open edX have made it easy for you to integrate an SMTP email account to your instance. Take note that the configuration procedure has been simplified over time, but much of the online documentation that I find has not kept pace with these changes, and this can make it appear as though getting SMTP email working is more complicated that it really is.

We’ll integrate a Google Gmail account in this tutorial, but you can use just about any other SMTP mail provider and these instructions will still be valid. Google’s Gmail email service is designed to be accessed by third party applications. The email management software that you use on your Desktop and mobile devices for example are third-party apps that provide the means to quickly connect and synchronize your local email message storage system to Gmail’s email services.

SMTP is a standardized protocol that Gmail and many other email providers use to allow third party applications to integrate and synchronize with the central email servers of the service provider. Apple and Android both provide really good out-of-the-box mobile apps for email that let you connect to a Gmail account; usually requiring no more than a user name and password. Open edX uses the same approach, but without the nice GUI interface.

Setup Procedure

1. Configure Your Gmail Account

There is one small but important change that you’ll need to make to your default Gmail account settings in order for Gmail to allow Open edX to connect. After logging in to your Gmail account from a web browser, navigate to https://myaccount.google.com/lesssecureapps to change the setting for “Allow less secure apps” to “On”.

2. Edit /edx/app/edxapp/lms.auth.json

lms.auth.json contains authorization credentials for the various subsystems that support the Learning Management System (LMS) application; email username and password being two such credentials. Each time the LMS attempts to send an email message it refers to the username and password from this file to establish a connection and send the message using the Gmail SMTP email service.

Pro Tip: Keep a backup copy of the original versions of each of the four JSON files in /edx/app/edxapp. JSON syntax is as rigid as it is unforgiving, and any syntax errors in these four JSON files will prevent your Open edX platform from restarting.

3. Edit /edx/app/edxapp/cms.auth.json

cms.auth.json contains authorization credentials for the various subsystems that support the Course Management Studio (CMS) application. The structure of this file is similar, but not identical to that of lms.auth.json from the previous step. You’ll also add your Gmail username and password to this file.

4. Edit /edx/app/edxapp/lms.env.json

lms.env.json contains application parameters and option values for the LMS. You’ll do two things in this file. First, you’ll add your Google Gmail SMTP settings, and second, you’ll set your gmail email address in various locations in the file.

Note that you’ll find other parameters in this file with an “EMAIL_” prefix. If I don’t mention them in this article then they’re not required for setting up SMTP email.

EMAIL_BACKENDThe default value might look similar, so check this carefully to ensure that you’re instructing Django to use SMTP rather than the default Ubuntu email generator utility.
EMAIL_HOSTThis is the fully qualified domain name for the email service that you’re using (Google Gmail in the case of this tutorial). If you’re using a different mail provider then you should ensure that you provide a Fully Qualified Domain Name (FQDM) here.
EMAIL_PORTThe port assignment is specific to the service provider that you’re using, and it’s arbitrary, so don’t try to guess.
EMAIL_USE_TLSThis value is specific to your provider. This is a true/false flag indicating whether email connections will be established using secure HTTPS protocol with TLS (a new improved alternative to SSL).

Note: If you’re using Ginkgo then you can disregard this step. Next you will potentially update several email addresses, depending on the version you are running. Until Ginkgo, the Open edX platform recognized nearly a dozen different email addresses for roles ranging from new user activation to bug reports. If you’re using a version other than Ginkgo then you’ll need to do a search-replace on “@example.com” to correctly set these values.

5. Edit /edx/app/edxapp/cms.env.json

This file is virtually identical in both layout and substance to lms.env.json. You should duplicate all edits that you made in the previous step.

6. Restart Open edX

You’ll need to restart the Open edX platform in order for these changes to take effect. Keep in mind that restarting the platform takes around 10 minutes and that it will be unavailable to learners and instructors during this time. Also keep in mind that any syntax errors you might have made while editing the JSON files will prevent the Open edX platform from starting.


# For Ginkgo and newer
/edx/bin/supervisorctl restart lms
/edx/bin/supervisorctl restart cms
/edx/bin/supervisorctl restart edxapp_worker:

For older versions of Open edX (prior to Ginkgo) you should use this alternative syntax. Note that The “:” character is part of each of these two commands.


/edx/bin/supervisorctl restart edxapp:
/edx/bin/supervisorctl restart edxapp_worker:

7. Test Your Work

You can easily verify your results by requesting a password reset email using the “Forgot Password?” from your Open edX login screen. This simple utility will attempt to use your new SMTP email settings to generate the password reset email using Google Gmail. If it works then you’ll not only instantly receive the email but you’ll also be able to see a copy of the message in the “Sent Mail” for your Gmail account.

8. Troubleshooting

There are a limited set of things that can go wrong, but also a limited set of ways to check your results and so troubleshooting is pretty straightforward. Your three best diagnostic tools are a.) the log file in /edx/var/log/lms/edx.log, b.) the Javascript console in Google Chrome and c.) the “Sent Mail” folder of your Google Gmail account.

If the password reset screen freezes or reports an error then Check the Javascript console for errors. If you find an HTTP 500 error then this means that Open edX was unsuccessful in its attempt to connect to Gmail. You might be able to find more information in the Javascript object tree explorer.

In newer versions of Open edX (Ginkgo and later) the UI will generate a positive response message regardless of whether an email was really generated and sent by your platform. You’ll find any related errors messages in /edx/var/log/lms/edx.log. Errors will take the form:

Oct  7 21:28:53 ip-172-31-12-255 [service_variant=lms][edx_ace.channel.django_email][env:sandbox] ERROR [ip-172-31-12-255  30626] [django_email.py:94] - (530, '5.5.1 Authentication Required. Learn more at\n5.5.1  https://support.google.com/mail/?p=WantAuthError o14-v6sm9007712wrp.57 - gsmtp', u'your-email-address@gmail.com')
Traceback (most recent call last):
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_ace/channel/django_email.py", line 91, in deliver
html_message=rendered_template,
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/mail/__init__.py", line 62, in send_mail
return mail.send()
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/mail/message.py", line 348, in send
return self.get_connection(fail_silently).send_messages([self])
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/mail/backends/smtp.py", line 111, in send_messages
sent = self._send(message)
File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/mail/backends/smtp.py", line 127, in _send
self.connection.sendmail(from_email, recipients, message.as_bytes(linesep='\r\n'))
File "/usr/lib/python2.7/smtplib.py", line 736, in sendmail
raise SMTPSenderRefused(code, resp, from_addr)
SMTPSenderRefused: (530, '5.5.1 Authentication Required. Learn more at\n5.5.1  https://support.google.com/mail/?p=WantAuthError o14-v6sm9007712wrp.57 - gsmtp', u'your-email-address@gmail.com')

I hope you found this helpful. Please help me improve this article by leaving a comment below. Thank you!

By |2018-10-07T15:40:33+00:00June 28th, 2018|Categories: Open edX|13 Comments

About the Author:

Lawrence is a full stack developer specializing in the Open edX platform, Django, Angular, Ionic, Wordpress and Amazon Web Services. He lives in Puerto Escondido, Oaxaca, Mexico.

13 Comments

  1. Spurgeon October 15, 2018 at 7:12 am - Reply

    Hi Lawrence,

    Thanks a ton!!

    This helped me very much.

    One more issue that needs to fixed is replacing “localhost” in one of the configuration files / django admin i guess…

    The change password link that arrives in the mail has ‘localhost’ as the base url.

    https://localhost/password_reset_confirm/e-50g-9d06d31234c92def02439/?utm_content=c4f86b80-7483-4bea-aafb-9a1d868bebd5&utm_medium=email&utm_source=student&utm_campaign=passwordreset

    Any idea which one needs to be changed, where?

    Thanks once again!!

  2. SungNam Park October 7, 2018 at 11:34 am - Reply

    Hello, thank you for the reply.

    I think this is the message that i have for finding password. I think sending e-mail has problem.

    5650 UserAPIInternalError: An unexpected error occurred when calling ‘request _password_change’ with arguments ‘(u’sw2db@naver.com’, False)’ and keyword argum ents ‘{}’: SMTPAuthenticationError(535, ‘5.7.8 Username and Password not accepte d. Learn more at\n5.7.8 https://support.google.com/mail/?p=BadCredentials q76-v 6sm24354673pfa.18 – gsmtp’)

    • SungNam Park October 7, 2018 at 11:42 am - Reply

      For the link i actually changed the password; what I have written in the code. But still does not work.

      I have tried 465 port but I don’t think this is it because

      SSL: 465
      TLS/STARTTLS: 587

    • admin October 7, 2018 at 2:51 pm - Reply

      ok, the error messages provides all of the information that you need to trouble shoot your problem. based on the error “Username and Password not accepted” you can deduce that the port, hostname and TLS settings are correct (these must be correct in order for authentication to be attempted). if you follow the link that is provided in the message then you’ll see “Allow less secure apps: If you don’t use 2-Step Verification, you might need to allow less secure apps to access your account.” as the 3rd option. this very likely is your problem.

  3. SungNam Park October 7, 2018 at 8:33 am - Reply

    Hello, I am using AWS EC2 instance to run the production line open edx. When I turn off and on the AWS instance, activation e-mail system does not work. I checked everything that you mentioned. What kind of other problems do i have to consider?

    my website is ilkmooc.kr

    Thank you

    • admin October 7, 2018 at 10:18 am - Reply

      Hi SungNam and good luck with your next steps. you should look at the log file in /edx/var/log/lms/edx.log for any errors related to SMTP or email. any error messages you find will guide you to your next step in trouble shooting. having said that however, it sounds like you might be having troubles with a subsystem named Celery (also known as RabbitMQ). the activation email is generated by a Celery task, so if the email is not working then sometimes this is the source of the problem. you can read more about trouble shooting Celery here: https://blog.lawrencemcdaniel.com/open-edx-trouble-shooting-celery-rabbitmq/.
      Btw, i really like the home page on your site! 🙂

  4. Alan August 11, 2018 at 10:35 pm - Reply

    Hello,

    I use devstack for Hawthorn version.
    I follow this tutorial carefully, but not succeeded at setting smtp.

    Could you please help me?

    • admin August 12, 2018 at 6:21 am - Reply

      Hi alan, there weren’t any differences in Ginkgo devstack/fullstack that would affect SMTP setup. Hawthorn is still too new for me to be able to comment. I’d suggest you post a question on Slack just in case you might have run across a bug: https://openedx.slack.com

  5. Phaneendra July 31, 2018 at 12:03 am - Reply

    Hello Lawrence,

    I followed your blog regarding the smtp setup . I got an error saying like There has been 500 error like that.

    • admin July 31, 2018 at 5:22 am - Reply

      reboot the server, note the time, then check the log for the LMS, located here: /edx/var/log/lms/edx.log. you should see some errors generated during the boot process that will provide insights on your problem lies. keep in mind that the time stamps in the log are GMT-0.

  6. MarlonOrozco June 29, 2018 at 9:56 am - Reply

    Excellent

Leave A Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.