Pre-Installation Guide

What is the current state of Mailman 3?

Mailman 3 is a complete re-write of Mailman 2.1 and has been split into several modular components which collectively we call Mailman Suite. We also often just call it Mailman 3. Let us look very briefly into these components:

  • Mailman Core or just Core is the main engine that is responsible for handling emails. It does all the task of managing users, subscriptions, mailing lists, email addresses, talking to the MTA etc. However, the things that it doesn’t manage include user authorization & user authentication.

    However, users can manage their settings using email based commands which includes subscribing, unsubscribing, changing a few basic settings for their accounts etc.

    Most functionalities in Core can be accessed using an administrative REST API which provides full control over the Core. Core expects the clients that consume this API would take up the responsibility of authenticating and authorizing users and provide interfaces for them to manage their settings and subscriptions.

  • Mailman Client is just sometimes called as Client and is the official Python bindings to the Core’s REST API.

  • Postorius is the official web front-end over the Core’s REST API and is built on Django web framework. In Django’s terminology, Postorius is an “app” which can be be plugged into any Django installation or “project”. Postorius has been built to be deployed alongside other Django “apps” which makes it a little bit difficult to deploy since you need to have a “project” of your own if you just need it to work!

    See also

    What is Django?

    If you don’t have any familiarity with Django, we do provide an example Django “project” with pre-populated settings which should run Postorius out of the box. However, because of the several different ways in which you can configure Django, it might not suite everybody’s needs, in which case you might have to dive into Django’s (and other helper library’s) settings related documentation to figure out the best settings for you.

  • Hyperkitty is the official archiver and, similar to Postorius, is a Django “app”. It is also built along the same principles of re-usability and can be deployed alongside Postorius with the provided Django “project”.

How Can I upgrade from Mailman 2.1.x?

Mailman 2.1 series is the current most popular series of Mailman. The short version is that as of now, upgrading from Mailman 2.1 to Mailman 3.1 is fairly straightforward.

Now the long version. Because of the changes in Database Schema, migrating from Mailman 2.1 to Mailman 3.1 can have issues. There is a mailman import21 command that works well to import list settings and membership. The issues involve held moderator requests and pending digests and they can be handled with some scripting.

Archives can be imported into Hyperkitty easily, but URLs to attachments are going to break because the URL paths are different in Hyperkitty. Although, you might be able to retain your HTML archives and/or archived attachments from Mailman 2.1, at least if they are public, so the old URLs still work.

What do I need to know before deploying Mailman 3?

The installation guide presumes some knowledge of general Python based web applications and their ecosystem. This requirement however is only required if you follow the installation guide provided here. When the distro packages for Mailman 3 are out, you probably won’t need any of this information. If you are new to Python and have no idea about what pip, django and wsgi means, this section is for you!

Here are the three most basic terminologies that you will often encounter when trying to deploy Mailman 3:

  • pip is the Python’s official package manager and can be used to download and install libraries and packages from PyPI a.k.a Python CheeseShop. On most Linux based distributions, you can install it from the distro’s package manager. Here are the instructions for apt & yum (or dnf) based systems:

    $ sudo apt install python3-pip
    

    and:

    $ sudo yum install python3-pip
    

    If you have any other Linux distro, please check its documentation on how to install. After that, you can use it to install python packages using the command like below:

    $ sudo pip3 install <packagename>
    

    The above command is equivalent to:

    $ sudo python3 -m pip install <packagename>
    
  • Django is the Python web framework that Postorius & Hyperkitty are based on. It allows you to run several different web “apps” under a single “project”. People often write their web applications as reusable Django “apps” which can then be plugged into any running Django “project”. A Django “project” is also sometimes called a Django “installation”.

    A typical Django project has a structure of something like this:

    toplevel_project:
    \
    |---- urls.py
    |---- settings.py
    |---- manage.py
    |---- wsgi.py
    

    It is important to understand the above structure even when using Django, because the configuration of a Django project requires you to edit these files.

    Here is a brief overview of these files:

    • urls.py: This is how Django routes requests. If you want to install a new Django “app”, you have to add its base_url to this file.

    • settings.py : This is Django’s configuration file. It is the home for all the different configuration options that are required.

    • manage.py : This is a generic helper script used to perform administrative tasks from the command line on a Django project. You should never edit this file after creating a project.

    • wsgi.py : This is the WSGI or Web Server Gateway Interface application for the Django project. You will need this file later to interface Django with an actual webserver like Nginx or Apache. Usually, you don’t need to change anything in this file.

  • wsgi or the Web Server Gateway Interface is the protocol by which Python web applications talk to web servers. Django comes with a built-in web server, which you can invoke by using the following command:

    $ python manage.py runserver
    

    This will start a development server on http://localhost:8000/. You can also specify a different host:port to bind to. See python manage.py runserver --help for more instructions.

    To deploy any Python based web application, you need an intermediate WSGI server which mediates the interaction between Python and a web server. There are several of them out there but we recommend using uwsgi. Uwsgi has several advantages over others including the ability to configure it entirely using environment variables, which helps in container based deployments. Also, Nginx and Apache web servers have plugins built-in for Uwsgi which makes it an even more compelling candidate.

    However, you are not tied to using Uwsgi and are free to choose any other WSGI server.

  • When running Django using the built-in development web server, Django serves its static files which makes it easy for the developers. However, in production environment, it is advised to serve the static files separately.

    To collect all the static files for all the Django projects in one single place:

    $ python manage.py collectstatic
    

    This will collect all the static files in the location mentioned in Django’s settings as STATIC_ROOT, which is usually under static directory in Django’s project’s root path.

    You need to serve this directly with your web server using a proxy or alias rule, depending on your webserver.

    Here is the relevant portion of the configuration for Ningx:

    server {
        ...
    
        location /static/ {
             alias /path/to/django/STATIC_ROOT;
        }
    }
    
You can do this in Apache using a configuration that looks something like

this:

<VirtualHost *:443>
   ...

   Alias /static /opt/mailman/web/static
   Alias /favicon.ico /opt/mailman/web/static/hyperkitty/img/favicon.ico
   ProxyPassMatch ^/static/ !
   ...
</VirtualHost>