This post is part of the Managing Email series.

Migrating Imap, Gmail and Exchange, mail accounts from GUI clients to Astroid


Initially, I had planned this post to start with a brief history of the decline of email clients for Linux. That quickly got out of hand, and was therefore spun out into a post of its own (TBD). To keep things brief. Thanks to the incredible ineptitude of the Thunderbird steering committee, I ended up requiring a new mail client. Having despaired of the GUI based bloat heavy approaches of most clients, I decided to go the old fashioned route and build one up in a modular manner.

The Accounts

In no order of preference, for a variety of reasons, I have 23 distinct email accounts. However, these are actually broken down into a few basic types and associated mail fetching software.

Generic IMAP
Yahoo, and the rest are managed with isync
A school account of mine uses Office 365, and will be handled with davmail in conjunction with isync
There are a few of these, two personal, and two institutional, all of which are handled with lieer


For general IMAP accounts (anything which isn’t backed by gmail or outlook) an mbsync approach works best. Exhange accounts need davmail and are described in a separate sub-section. Before we get to the script and astroid, it is a good idea to run each of these as we set them up, especially as the first run can take quite a long time (around an hour for 9205 messages).

1mbsync -V blah

General IMAP

For a standard IMAP account (anything which isn’t Exchange or Gmail) the only thing we need to keep track of is a good way to obfuscate passwords. We will use pass for this 1.

1pass git init $GPG_KEY # Optional, use a secure private git repo
2pass init $GPG_KEY
3pass add mail/rgoswami.iitk

With this, we can configure a general IMAP account (an IITK webmail in this instance, but it could be anything).

 1IMAPAccount rgoswami.iitk
 3User rgoswami
 4PassCmd "pass show mail/rgoswami.iitk"
 5AuthMechs LOGIN
 8IMAPStore rgoswami.iitk-remote
 9Account rgoswami.iitk
11MaildirStore rgoswami.iitk-local
12SubFolders Verbatim
13Path /run/media/Storage/.mail/rgoswami_iitk/
14INBOX /run/media/Storage/.mail/rgoswami_iitk/Inbox
15Trash trash:///
17Channel rgoswami.iitk
18Master :rgoswami.iitk-remote:
19Slave :rgoswami.iitk-local:
20Patterns *
21Create Both
22SyncState *

Exchange Accounts

For Exchange accounts we need to setup davmail. Thankfully the process is actually excessively simple for a single account. For each account, a .properties file needs to be generated. The password section is kept blank in this case, since we will authenticate with the O365Interactive protocol.

1# The default setup
2davmail ~/

Use the following properties in the file to prevent timeout errors:


Further information is available at the official docs.

Figure 1: davmail setup

Figure 1: davmail setup

Figure 2: Interactive login workflow

Figure 2: Interactive login workflow

Now we can configure the .mbsyncrc in a manner analogous to the standard IMAP setup, but with the port and host where davmail is running.

 1IMAPAccount rog32
 2Host localhost
 4Pass dummy
 5Port 1143
 6SSLType None
 7AuthMechs LOGIN
 9IMAPStore rog32-remote
10Account rog32
11MaildirStore rog32-local
12SubFolders Verbatim
13Path /run/media/Storage/.mail/rog32/
14Inbox /run/media/Storage/.mail/rog32/Inbox
16Channel rog32
17Master :rog32-remote:
18Slave :rog32-local:
19Patterns *
20Create Both
21SyncState *

The SSLType and AuthMechs are important parameters. Note that the Pass is truly not important since we an OAuth token is stored in the properties file.

Figure 3: First run of a large inbox with davmail and isync

Figure 3: First run of a large inbox with davmail and isync


The general setup gmi init works well for personal accounts. However, there was an additional step for the IEEE account.


Some issues with tags led to the following changes in the .gmaileer.json file

1{"replace_slash_with_dot": false, "account": "", "timeout": 600, "drop_non_existing_label": false, "ignore_empty_history": false, "ignore_tags": ["TODO","new"], "ignore_remote_labels": ["CATEGORY_SOCIAL", "CATEGORY_FORUMS", "CATEGORY_PROMOTIONS", "CATEGORY_PERSONAL", "CATEGORY_UPDATES"], "remove_local_messages": true, "file_extension": ""}

Essentially just the addition of two new ignore_tags.


At this point, we have all mail synced into local directories, but we have no access to view or interact with them. We will start by setting up notmuch to search and index our mail. This is pretty basic for now.


There are a lot more options which may be configured, but this is enough to get started.

Sending Email

At this stage we need a way to actually send email. A simple configuration can be setup in /.config/msmtp/config is given below (where we use pass again):

 2port 587
 3tls on
 4auth on
 5logfile ~/.config/msmtp/.msmtp.log
 6tls_trust_file /etc/ssl/certs/ca-certificates.crt
 8account r95g10
12tls_starttls on
13tls on
14auth on
15port 587
16passwordeval pass show mail/
18account rog32
19host localhost
22tls_starttls off
23tls off
24auth plain
25port 1025
26password dummy

Note that the exchange account again uses a dummy password, and the real password will be prompted for on the first run. The permissions of the file above should be 600. It is prudent to test at-least the exchange section to login.

1echo "Hello world" | msmtp --account=rog32


The bulk of this setup is by the numbers, with the exception of the poll script. However, minimally configure astroid with the location of our notmuch configuration.

1astroid --new-config
2vim ~/.config/astroid/config

The relevant sections are:

1"notmuch_config": "\/home\/whoever\/.notmuch-config",

The accounts section is fairly self explanatory, but we will need to use the following sendmail line as well:

1"sendmail": "msmtp --read-envelope-from -i -t",

Nix Python Poll Script

Natively only bash appears to be supported. However, with nix, we can use a reproducible python script with the sh library to call system functions instead.

 1from pathlib import Path
 2import sh
 4# For generic IMAP maildirs
 6ISYNC_LABELS = ["rgoswami.iitk", "rog32"]
 8for isync in ISYNC_LABELS:
 9    sh.mbsync("-V",isync,_bg=True)
12# Gmaileer
13GMAIL_IDENTIFIERS = ["gmail", "univ", "ieee"]
15path = Path(r"/run/media/haozeke/Storage/.mail/")
17for dirs in path.iterdir():
18    if dirs.is_dir():
19        for gmi in GMAIL_IDENTIFIERS:
20            if gmi in
21                print(f"Syncing {}")
22                sh.gmi("sync", _cwd=dirs, _fg=True)

This needs to be coupled with the standard nix shebang in the file:

1#!/usr/bin/env nix-shell
2#!nix-shell -i python3 -p "python38.withPackages(ps: [ ps.numpy ])"

For working with HTML emails, we need to highlight the notice about potentially sketchy HTML using (defaults) j or k and then hit enter or o.

Figure 4: Unhelpful default

Figure 4: Unhelpful default

Figure 5: After viewing the sketchy bit

Figure 5: After viewing the sketchy bit

Note that since most email is actually sent with text/html it might make more sense to configure the thread_view in the configuration file.

 1"thread_view": {
 2    "open_html_part_external": "false",
 3    "preferred_type": "html",
 4    "preferred_html_only": "false",
 5    "allow_remote_when_encrypted": "false",
 6    "open_external_link": "xdg-open",
 7    "default_save_directory": "~",
 8    "indent_messages": "false",
 9    "gravatar": {
10        "enable": "true"
11    },
12    "mark_unread_delay": "0.5",
13    "expand_flagged": "true"


There are again, two different approaches to deletion.

For gmail accounts it is simple, just adding a trash tag will do the trick, so we can select multiple emails with t and then hit + to add trash and everything works out
Other Accounts
We can delete mail forever (from the server as well), by using a safe-tag and then using notmuch

The generic non-lieer method requires:

1notmuch search --output=files --format=text0 tag:killed | xargs -r0 rm

A pre-new hook will work.


Thankfully, astroid supports GPG encryption as well as markdown support. This makes for simple integration with any popular editor.

Figure 6: Composition with emacs and astroid

Figure 6: Composition with emacs and astroid


This is enough to get started for a while, but it isn’t yet at the stage where I can replace thunderbird unfortunately. However, there are several pain points to be addressed, which will be covered in a future post. Some of these are essentially related to network fluctuations, and the awkward deletion setup.


  1. The GPG key itself can be stored with keybase ↩︎

Series info

Managing Email series

  1. Reclaiming Email with Astroid <-- You are here!