Mailman Suite Installation

This is a step-by-step installation guide for Mailman Suite, also sometimes referred as Mailman 3 Suite or just Mailman 3. There are more than one ways to have Mailman 3 running on your machine.

Mailman 3 in Docker

Abhilash Raj maintains container images for Mailman 3 which you can use directly without having to go through all the steps to download dependencies and configuring Mailman. You can find the detailed documentation on how to use them. If you have any issues using the container images, please report them directly on the Github repo for docker-mailman.

Distribution Packages

Distribution specific packages for Mailman 3 are not yet available. There are efforts going on to package for Debian based systems. If you would like to help package Mailman 3 for your favorite Linux distro, please get in touch at mailman-developers@python.org.

Dependencies

  • Python 2.7 and Python3.5+ both. Mailman Core (v3.1) runs exclusively on Python3.4+, but the support for Python 3.4 might be deprecated in future versions. The web frontend (in Django) runs exclusively on Python 2.7.

  • MTA: Mailman 3 in theory would support any MTA or mail server which can send emails to Mailman over LMTP. Officially, there are configurations for Postfix, Exim4, qmail and sendmail. Mailman Core has a fairly elaborate documentation on setting up your MTA. Look below at `setting up mailman core`_ to find out the location of configuration file mailman.cfg which is mentioned in the documentation above.

    The Web Front-end is based on a Python web framework called Django. For email verification and sending out error logs, Django also must be configured to send out emails.

  • A sass compiler. Syntactically Awesome Stylesheets or Sass is an extension of CSS, which is used to style webpages, that adds more power to the old CSS syntax. It allows using variables, nested rules, inline imports etc. which CSS originally doesn’t support. Hyperkitty uses this to generate CSS styles.

    You can either use the Ruby implementation or C/C++ implementation. Please look at the installation guide for sass to see how you can get one.

    For apt based systems:

    $ sudo apt install ruby-sass
    

    For yum based systems try:

    $ sudo yum install rubygem-sass
    

    or:

    $ sudo dnf install rubygem-sass
    

    After installing this, you’d have to configure Django to use these compilers. A basic configuration would look like:

    # To be added to Django's settings.py
    
    COMPRESS_PRECOMPILERS = (
      ('text/x-scss', 'sass -t compressed {infile} {outfile}'),
      ('text/x-sass', 'sass -t compressed {infile} {outfile}'),
    )
    

    You can replace sass above with whatever is the name of the binary is. For the Ruby implementation it is generally sass, for C/C++ it is sassc.

  • A full text search engine like Whoosh or Xapian. Whoosh is the easiest to setup. First, install Whoosh:

    $ sudo pip2 install whoosh
    

    See also

    What is pip?

    Then configure Django to use it:

    # To be added to Django's settings.py
    
    HAYSTACK_CONNECTIONS = {
          'default': {
          'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine',
          'PATH': os.path.join(BASE_DIR, "fulltext_index"),
          },
    }
    

    For setup instruction on other search backends, look at the documentation for django-haystack.

Installing Mailman Core

Mailman Core is responsible for sending and receiving emails. It exposes a REST API that different clients can use to interact with over an HTTP protocol. The API itself is an administrative API and it is recommended that you don’t expose it to outside of your host or trusted network. To install Core run:

$ sudo pip3 install mailman

This _should_ install the latest release of Mailman Core, which is 3.1.0 as of writing of this document. After this, create a configuration file at /etc/mailman.cfg for Mailman Core.

After this, running mailman info will give you an output which looks something like below:

$ mailman info
GNU Mailman 3.1.0 (Between The Wheels)
Python 3.5.3 (default, Jan 19 2017, 14:11:04)
[GCC 6.3.0 20170118]
config file: /etc/mailman.cfg
db url: sqlite:////var/lib/mailman/data/mailman.db
devmode: DISABLED
REST root url: http://localhost:8001/3.1/
REST credentials: restadmin:restpass

Installing Web UI

Postorius and Hyperkitty are Mailman’s official Web UI and Archiver. Both of them are Django based apps and can be integrated into an existing Django installation/website.

Installing Postorius & Hyperkitty

Postorius and Hyperkitty are Django “apps” and can be directly installed using the commands below:

$ sudo pip2 install postorius
$ sudo pip2 install hyperkitty

Setting up Django Project

See also

What is Django?

If you have absolutely no idea about Django, just clone/download this mailman-suite repo . It should have a documented settings.py and a pre-configured urls.py which you can use to run new Django installation. If you find any setting that is not documented, please look at Django’s settings reference for all available settings.

Exact commands would look something like this:

# Download and install the latest release of django.
$ sudo pip2 install django

# Clone the repo locally.
$ git clone https://gitlab.com/mailman/mailman-suite.git
$ cd mailman-suite/mailman-suite_project/

# Create the tables in the database and load fixtures.
$ python2 manage.py migrate

# Copy all the static files to one single location.
$ python2 manage.py collectstatic

# Run the Django's "development" server at localhost:8000
$ python2 manage.py runserver

Setting up a WSGI server

See also

What is WSGI?

These instructions are to setup your Django website behind a webserver. We are using a uwsgi as the wsgi server to communicate between the webserver and Django. To install uwsgi, run:

$ sudo pip3 install uwsgi

Note

The configuration below doesn’t serve static files, so if you are just “trying-it-out” and want static files to be served, you need to add some additional configuration and steps. See serving static files with uwsgi.

Then you can configure it using the following configuration file:

[uwsgi]
# Port on which uwsgi will be listening.
http-socket = 0.0.0.0:8000

# Move to the directory wher the django files are.
chdir = /path-to-django-project-directory/

# Use the wsgi file provided with the django project.
wsgi-file = wsgi.py

# Setup default number of processes and threads per process.
master = true
process = 2
threads = 2

# Drop privielges and don't run as root.
uid = 1000
gid = 1000

# Setup the django_q related worker processes.
attach-daemon = ./manage.py qcluster

# Setup the request log.
req-logger = file:/path-to-logs/logs/uwsgi.log

# Log cron seperately.
logger = cron file:/path-to-logs/logs/uwsgi-cron.log
log-route = cron uwsgi-cron

# Log qcluster commands seperately.
logger = qcluster file:/path-to-logs/logs/uwsgi-qcluster.log
log-route = qcluster uwsgi-daemons

# Last log and it logs the rest of the stuff.
logger = file:/path-to-logs/logs/uwsgi-error.log

Note that in the above configuration, there is a command called python manage.py qcluster which run the django-q processes. You can remove this from here if you want to manage this yourself via some other init process.

Have a look at uwsgi documentation to learn more about different configuration options. One minor optimization that can be done is to replace:

http-socket = 0.0.0.0:8000

with a more performant option:

uwsgi-socket = 0.0.0.0:8000

However, this requires support for uwsgi protocol in the webserver. Nginx and Apache, the two most popular web-servers have the support and the Nginx configuration below uses uwsgi-socket.

Nginx Configuration

You can reverse proxy the requests to uwsgi server using Nginx. Uwsgi has a special protocol called uwsgi protocol that is available in Nginx as a plugin. Add the following configuration to your sites-availbale in Nginx

Note

This configuration relies on uwsgi listening on a uwsgi-socket instead of a http-socket. Please make sure before using this configuration.

Also, if you must use the http-socket option for some reason, replace the line below with uwsgi_pass 0.0.0.0:8000; with proxy_pass http://0.0.0.0:8000;.

server {

   listen 443 ssl default_server;
   listen [::]:443 ssl default_server;

   server_name MY_SERVER_NAME;
   location /static/ {
        alias /path-to-djang-staticdir/static/;
   }
   ssl_certificate /path-to-ssl-certs/cert.pem;
   ssl_certificate_key /path-to-ssl-certs/privkey.pem;

   location / {
           include uwsgi_params;
           uwsgi_pass 0.0.0.0:8000;

   }

}

Fill in the appropriate paths above in the configuration before using it.