With Open edX Ginkgo, integrating an SMTP email account like Google’s Gmail is as simple as setting a few parameter values in the four JSON configuration files for the LMS and Course Management Studio, each of which are located in /edx/app/edxapp/. Get SMTP email working on your Open edX instance in less than an hour with this step-by-step how-to guide.
Summary
The native build of Open edX relies on a simple email generator utility that comes with Ubuntu 16.04. While this technically works, the messages that it generates for new user activations, password reset instructions, and welcome messages are generally universally flagged as spam by common email management client applications like MS Outlook, Mac Mail and so on. Fortunately however, the architects of Open edX have made it easy for you to integrate an SMTP email account to your instance. Take note that the configuration procedure has been simplified over time, but much of the online documentation that I find has not kept pace with these changes, and this can make it appear as though getting SMTP email working is more complicated that it really is.
We’ll integrate a Google Gmail account in this tutorial, but you can use just about any other SMTP mail provider and these instructions will still be valid. Google’s Gmail email service is designed to be accessed by third party applications. The email management software that you use on your Desktop and mobile devices for example are third-party apps that provide the means to quickly connect and synchronize your local email message storage system to Gmail’s email services.
SMTP is a standardized protocol that Gmail and many other email providers use to allow third party applications to integrate and synchronize with the central email servers of the service provider. Apple and Android both provide really good out-of-the-box mobile apps for email that let you connect to a Gmail account; usually requiring no more than a user name and password. Open edX uses the same approach, but without the nice GUI interface.
Setup Procedure
1. Configure Your Gmail Account
There is one small but important change that you’ll need to make to your default Gmail account settings in order for Gmail to allow Open edX to connect. After logging in to your Gmail account from a web browser, navigate to https://myaccount.google.com/lesssecureapps to change the setting for “Allow less secure apps” to “On”.
2. Edit /edx/app/edxapp/lms.auth.json
lms.auth.json contains authorization credentials for the various subsystems that support the Learning Management System (LMS) application; email username and password being two such credentials. Each time the LMS attempts to send an email message it refers to the username and password from this file to establish a connection and send the message using the Gmail SMTP email service.
Pro Tip: Keep a backup copy of the original versions of each of the four JSON files in /edx/app/edxapp. JSON syntax is as rigid as it is unforgiving, and any syntax errors in these four JSON files will prevent your Open edX platform from restarting.
3. Edit /edx/app/edxapp/cms.auth.json
cms.auth.json contains authorization credentials for the various subsystems that support the Course Management Studio (CMS) application. The structure of this file is similar, but not identical to that of lms.auth.json from the previous step. You’ll also add your Gmail username and password to this file.
4. Edit /edx/app/edxapp/lms.env.json
lms.env.json contains application parameters and option values for the LMS. You’ll do two things in this file. First, you’ll add your Google Gmail SMTP settings, and second, you’ll set your gmail email address in various locations in the file.
Note that you’ll find other parameters in this file with an “EMAIL_” prefix. If I don’t mention them in this article then they’re not required for setting up SMTP email.
| EMAIL_BACKEND | The default value might look similar, so check this carefully to ensure that you’re instructing Django to use SMTP rather than the default Ubuntu email generator utility. |
| EMAIL_HOST | This is the fully qualified domain name for the email service that you’re using (Google Gmail in the case of this tutorial). If you’re using a different mail provider then you should ensure that you provide a Fully Qualified Domain Name (FQDM) here. |
| EMAIL_PORT | The port assignment is specific to the service provider that you’re using, and it’s arbitrary, so don’t try to guess. |
| EMAIL_USE_TLS | This value is specific to your provider. This is a true/false flag indicating whether email connections will be established using secure HTTPS protocol with TLS (a new improved alternative to SSL). |
Note: If you’re using Ginkgo then you can disregard this step. Next you will potentially update several email addresses, depending on the version you are running. Until Ginkgo, the Open edX platform recognized nearly a dozen different email addresses for roles ranging from new user activation to bug reports. If you’re using a version other than Ginkgo then you’ll need to do a search-replace on “@example.com” to correctly set these values.
5. Edit /edx/app/edxapp/cms.env.json
This file is virtually identical in both layout and substance to lms.env.json. You should duplicate all edits that you made in the previous step.
6. Restart Open edX
You’ll need to restart the Open edX platform in order for these changes to take effect. Keep in mind that restarting the platform takes around 10 minutes and that it will be unavailable to learners and instructors during this time. Also keep in mind that any syntax errors you might have made while editing the JSON files will prevent the Open edX platform from starting.
# For Ginkgo and newer
/edx/bin/supervisorctl restart lms
/edx/bin/supervisorctl restart cms
/edx/bin/supervisorctl restart edxapp_worker:
For older versions of Open edX (prior to Ginkgo) you should use this alternative syntax. Note that The “:” character is part of each of these two commands.
/edx/bin/supervisorctl restart edxapp:
/edx/bin/supervisorctl restart edxapp_worker:
7. Test Your Work
You can easily verify your results by requesting a password reset email using the “Forgot Password?” from your Open edX login screen. This simple utility will attempt to use your new SMTP email settings to generate the password reset email using Google Gmail. If it works then you’ll not only instantly receive the email but you’ll also be able to see a copy of the message in the “Sent Mail” for your Gmail account.
8. Troubleshooting
There are a limited set of things that can go wrong, but also a limited set of ways to check your results and so troubleshooting is pretty straightforward. Your three best diagnostic tools are a.) the log file in /edx/var/log/lms/edx.log, b.) the Javascript console in Google Chrome and c.) the “Sent Mail” folder of your Google Gmail account.
If the password reset screen freezes or reports an error then Check the Javascript console for errors. If you find an HTTP 500 error then this means that Open edX was unsuccessful in its attempt to connect to Gmail. You might be able to find more information in the Javascript object tree explorer.
In newer versions of Open edX (Ginkgo and later) the UI will generate a positive response message regardless of whether an email was really generated and sent by your platform. You’ll find any related errors messages in /edx/var/log/lms/edx.log. Errors will take the form:
Oct 7 21:28:53 ip-172-31-12-255 [service_variant=lms][edx_ace.channel.django_email][env:sandbox] ERROR [ip-172-31-12-255 30626] [django_email.py:94] - (530, '5.5.1 Authentication Required. Learn more at\n5.5.1 https://support.google.com/mail/?p=WantAuthError o14-v6sm9007712wrp.57 - gsmtp', u'your-email-address@gmail.com') Traceback (most recent call last): File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/edx_ace/channel/django_email.py", line 91, in deliver html_message=rendered_template, File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/mail/__init__.py", line 62, in send_mail return mail.send() File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/mail/message.py", line 348, in send return self.get_connection(fail_silently).send_messages([self]) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/mail/backends/smtp.py", line 111, in send_messages sent = self._send(message) File "/edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/django/core/mail/backends/smtp.py", line 127, in _send self.connection.sendmail(from_email, recipients, message.as_bytes(linesep='\r\n')) File "/usr/lib/python2.7/smtplib.py", line 736, in sendmail raise SMTPSenderRefused(code, resp, from_addr) SMTPSenderRefused: (530, '5.5.1 Authentication Required. Learn more at\n5.5.1 https://support.google.com/mail/?p=WantAuthError o14-v6sm9007712wrp.57 - gsmtp', u'your-email-address@gmail.com')
I hope you found this helpful. Please help me improve this article by leaving a comment below. Thank you!
error: [Errno 101] Network is unreachable
if this error
SMTPAuthenticationError(535, ‘5.7.8 Username and Password not accepted.
enable unsecure
https://support.google.com/mail/?p=BadCredentials
Hi,
How do we send email to admin in parallel when anyone register in a course?
all SMTP email services that i’ve used allow multiple users to connect a mail box, thus, several people can monitor the send mail from your open edx instance, giving you a way to not only monitor course registrations but also stay abreast of any platform issues that might surface.
Anyway to disable SMTP authentication since the mail server I am using to route mail does not need authentication. Thanks
Hello Lawrence,
I successfully set up the smtp and requested the email reset, received the email in spam, clicked on the link, and it could not connect.
Any ideas?
that behavior might be specific to your email account or mail client.
Hi Lawrence, is there a way to configure multiple SMTP server, for multisite configuration? Let say if open edx server configured for multiple domain: domainA.com and domainB.com.
If user registers from domainA.com, then an email for activation will be sent from registration@domainA.com, via smtp.domainA.com, while another user who registers from domainB.com will get activation email from registration@domainB.com, via smtp.domainB.com.
Thank you
hi chris. no, not out of the box. however, if you’re a django guy then it might be feasible to create a small module to override the default registration behavior. at least, tracking down the python code for the registration pipeline process is somewhat straightforward.
Hi, So is there no need to do changes in cms/envs/common.py, lms/envs/aws.py ?
Also, can you suggest me what will be Email_Host, if I am using mandrill App as email service.
no need to change anything in common.py. googling “mandrill app email host” i found this page: https://mandrill.zendesk.com/hc/en-us/articles/205582217-How-to-Send-Mandrill-Email-with-Popular-CMS-Providers which says the host is smtp.mandrillapp.com.
Hi Lawrence,
Thanks a ton!!
This helped me very much.
One more issue that needs to fixed is replacing “localhost” in one of the configuration files / django admin i guess…
The change password link that arrives in the mail has ‘localhost’ as the base url.
https://localhost/password_reset_confirm/e-50g-9d06d31234c92def02439/?utm_content=c4f86b80-7483-4bea-aafb-9a1d868bebd5&utm_medium=email&utm_source=student&utm_campaign=passwordreset
Any idea which one needs to be changed, where?
Thanks once again!!
Got it!
Thanks for your solution at https://blog.lawrencemcdaniel.com/open-edx-trouble-shooting-guide/#2
Hello, thank you for the reply.
I think this is the message that i have for finding password. I think sending e-mail has problem.
5650 UserAPIInternalError: An unexpected error occurred when calling ‘request _password_change’ with arguments ‘(u’sw2db@naver.com’, False)’ and keyword argum ents ‘{}’: SMTPAuthenticationError(535, ‘5.7.8 Username and Password not accepte d. Learn more at\n5.7.8 https://support.google.com/mail/?p=BadCredentials q76-v 6sm24354673pfa.18 – gsmtp’)
For the link i actually changed the password; what I have written in the code. But still does not work.
I have tried 465 port but I don’t think this is it because
SSL: 465
TLS/STARTTLS: 587
ok, the error messages provides all of the information that you need to trouble shoot your problem. based on the error “Username and Password not accepted” you can deduce that the port, hostname and TLS settings are correct (these must be correct in order for authentication to be attempted). if you follow the link that is provided in the message then you’ll see “Allow less secure apps: If you don’t use 2-Step Verification, you might need to allow less secure apps to access your account.” as the 3rd option. this very likely is your problem.
It was it!!! Thank you Very much!!! I solved it
Hello, I am using AWS EC2 instance to run the production line open edx. When I turn off and on the AWS instance, activation e-mail system does not work. I checked everything that you mentioned. What kind of other problems do i have to consider?
my website is ilkmooc.kr
Thank you
Hi SungNam and good luck with your next steps. you should look at the log file in /edx/var/log/lms/edx.log for any errors related to SMTP or email. any error messages you find will guide you to your next step in trouble shooting. having said that however, it sounds like you might be having troubles with a subsystem named Celery (also known as RabbitMQ). the activation email is generated by a Celery task, so if the email is not working then sometimes this is the source of the problem. you can read more about trouble shooting Celery here: https://blog.lawrencemcdaniel.com/open-edx-trouble-shooting-celery-rabbitmq/.
Btw, i really like the home page on your site! 🙂
Hello,
I use devstack for Hawthorn version.
I follow this tutorial carefully, but not succeeded at setting smtp.
Could you please help me?
Hi alan, there weren’t any differences in Ginkgo devstack/fullstack that would affect SMTP setup. Hawthorn is still too new for me to be able to comment. I’d suggest you post a question on Slack just in case you might have run across a bug: https://openedx.slack.com
Hi Alan, I am also same problem.
Hello Lawrence,
I followed your blog regarding the smtp setup . I got an error saying like There has been 500 error like that.
reboot the server, note the time, then check the log for the LMS, located here: /edx/var/log/lms/edx.log. you should see some errors generated during the boot process that will provide insights on your problem lies. keep in mind that the time stamps in the log are GMT-0.
Excellent