Running multiple instances of Astroid mail

I am trying to set up a better email solution for myself. Currently I am using only webmail clients. They are not great, but I honestly find them more attractive than the other turnkey solutions on Linux. In the last years I have tried Thunderbird, Evolution, Geary, Kmail, Trojitá, Nylas N1, Sylpheed, Claws, and maybe others; all easy to get started with, but none of them was pleasant enough for me to keep using. Unsatisfying search interfaces and stupid formatting of sent emails were the most common issues for me.

Anyway, the point here is not to rant about email clients. I have been considering for a long time to try mutt (or maybe neomutt?) or Astroid. Now the plan is:

  • mbsync to sync IMAP accounts to local maildirs.
  • msmtp to talk to SMTP servers.
  • notmuch to index and search emails.
  • Astroid as the actual interface to all these things.

Clearly there are many things to learn. This post is about how to create separate instances of this whole setup.

One instance per IMAP account

As I use the same computer for both work and personal use, my idea is to run multiple instances of my email application(s), one for personal email and another for work. Who knows, maybe in the future even more instances. Technically speaking, Astroid can handle multiple accounts, but there are reasons not to:

  • I want to be able to forget about work in my free time. And sometimes also forget about personal emails when I am working. It just seems easiest to run one application for each, and then I can choose which ones I keep open.
  • Also, in the past when I have had email clients with multiple accounts, way too often I accidentally sent things from the wrong account. At present I am running two different webmail clients, which is great in this sense. They look different and also have separate address books, so accidentally emailing colleagues from my personal email and vice versa is extremely unlikely. In fact, I really like that they look different, and I've been thinking I should have different color themes for my work and personal Astroid setups, if this ever starts working.
  • Last, for purely technical reasons it seems nice to have separate settings for work and personal email, to simplify the transfer of account settings and/or data selectively from one computer to another.

How I tried to create separate instances of everything

Step 1: Compiling and running one Astroid instance

The core of the procedure was to follow the v0.14 README.md:

I clone the master branch (at that time commit aae4c52091cc7ae28b336c19d2f0ac4a3a4056bf):

git clone https://github.com/astroidmail/astroid.git

Then, as per the instructions for Ubuntu:

# Add `protobuf 3.6` PPA for Ubuntu 18.04
sudo add-apt-repository ppa:maarten-fonville/protobuf

# And install dependencies
sudo apt update
sudo apt-get install cmake git g++ libnotmuch-dev libglibmm-2.4-dev \
libgtkmm-3.0-dev libwebkit2gtk-4.0-dev libgmime-2.6-dev libsass-dev \
libpeas-dev libgirepository1.0-dev libboost-all-dev libgmime-3.0-dev \
libprotobuf-dev libvte-2.91-dev protobuf-compiler ruby-ronn

At this point the compilation ran fine (albeit with a bunch of deprecation warnings etc):

cd astroid
cmake -H. -Bbuild -GNinja # to use the ninja backend
cmake --build build

Getting the tests to pass

Running the tests:

cd build
ctest

At this point, all tests except Test #3: markdown and Test #18: quote_html passed. It turned out to be two easy missing dependencies:

sudo apt-get install cmark # for markdown
sudo apt-get install w3m # for quote_html

Installing

Still just following the README.md.

cmake -H. -Bbuild -GNinja -DCMAKE_INSTALL_PREFIX=/usr/local
sudo cmake --build build --target install

OK! Now it works.

Step 2: Configuring two instances

The problem is that Astroid fundamentally seems built on the assumption of running in one instance only. Concretely, it assumes a single location .config and .cache directories, etc. Specifically, looking in the src/config.cc file, we find the following paths:

Directory name First option Fallback option
.config $XDG_CONFIG_HOME/.config $HOME/.config
.data $XDG_DATA_HOME/.data $HOME/.local/share/.data
.cache $XDG_CACHE_HOME/.cache $HOME/.cache
.runtime $XDG_RUNTIME_HOME/.runtime $HOME/.cache

Most importantly, Astroid's only option for a config file is at .config/astroid/config. Clearly I need two different config files. The obvious solution is to start Astroid something like this:

$XDG_CONFIG_HOME=~/mail/work/.config astroid

This does work, but a potential problem is that other applications and libraries also rely on $XDG_CONFIG_HOME, and I find it difficult to predict how they will behave when they are started from the Astroid process. For example, running the above program, I found directories like ~/mail/work/.config/ibus etc. I don't know how this will affect the system in the long run, and I don't want to learn the hard way.

Hacking Astroid to use other environment variables

So I made a quick and dirty hack. Open Astroid's src/config.cc and replace XDG_ by ASTROID_:

diff --git a/src/config.cc b/src/config.cc
index 0b71440..8011ae9 100644
--- a/src/config.cc
+++ b/src/config.cc
@@ -70,7 +70,7 @@ namespace Astroid {
     if (test) {
       std_paths.config_dir = std_paths.home;
     } else {
-      char * config_home = getenv ("XDG_CONFIG_HOME");
+      char * config_home = getenv ("ASTROID_CONFIG_HOME");
       if (config_home == NULL) {
         std_paths.config_dir = std_paths.home / path(".config/astroid");
       } else {
@@ -85,7 +85,7 @@ namespace Astroid {
     std_paths.plugin_dir = std_paths.config_dir / path ("plugins");

     /* default data */
-    char * data = getenv ("XDG_DATA_HOME");
+    char * data = getenv ("ASTROID_DATA_HOME");
     if (data == NULL) {
       std_paths.data_dir = std_paths.home / path(".local/share/astroid");
     } else {
@@ -93,7 +93,7 @@ namespace Astroid {
     }

     /* default cache */
-    char * cache = getenv ("XDG_CACHE_HOME");
+    char * cache = getenv ("ASTROID_CACHE_HOME");
     if (cache == NULL) {
       std_paths.cache_dir = std_paths.home / path(".cache/astroid");
     } else {
@@ -101,7 +101,7 @@ namespace Astroid {
     }

     /* default runtime */
-    char * runtime = getenv ("XDG_RUNTIME_HOME");
+    char * runtime = getenv ("ASTROID_RUNTIME_HOME");
     if (runtime == NULL) {
       std_paths.runtime_dir = std_paths.cache_dir;
     } else {

Recompile and install this new Astroid.

Configuring separate instances

Create a launch-account script like this:

#!/usr/bin/env bash

ACCOUNT=$1

BASE_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
ACCOUNT_DIR=$BASE_DIR/accounts/$ACCOUNT

ACCOUNT_DIR=$ACCOUNT_DIR \
ASTROID_CONFIG_HOME=$ACCOUNT_DIR/.config \
ASTROID_DATA_HOME=$ACCOUNT_DIR/.data \
ASTROID_CACHE_HOME=$ACCOUNT_DIR/.cache \
ASTROID_RUNTIME_HOME=$ACCOUNT_DIR/.runtime \
NOTMUCH_CONFIG=$ACCOUNT_DIR/.notmuch-config \
astroid

This launch-account script can now be placed in some convenient location, in my case at ~/mail/launch-account, but it does not matter as long as the accounts directory is in the same place. Inside the accounts directory, each account has its own directory where config files etc can be placed. In my case it looks like this:

├── accounts
│   ├── personal
│   │   ├── .config
│   │   │   └── astroid
│   │   │       ├── config
│   │   │       ├── keybindings
│   │   │       └── poll.sh
│   │   ├── .mbsyncrc
│   │   ├── .msmtprc
│   │   ├── .notmuch-config
│   │   └── mail
│   └── work
│       ├── .config
│       │   └── astroid
│       │       ├── config
│       │       ├── keybindings
│       │       └── poll.sh
│       ├── .mbsyncrc
│       ├── .msmtprc
│       ├── .notmuch-config
│       └── mail
└── launch-account

There are a few locations where the different config files must have account-specific paths. The most important examples I can think of now:

  • The directory accounts/{account}/mail contains the account-specific maildir and the notmuch database. Of course .mbsyncrc and .notmuchconfig must be configured accordingly.
  • Astroid must be configured to invoke msmtp with the account-specific config file, something like msmtp --file=/home/rasmus/mail/accounts/work/.msmtprc -i -t. (Or, alternatively, I guess one could use a global .msmtprc and use the auto-from feature.)
  • The Astroid config must also point to the account-specific mail directories, etc, e.g., in the Astroid settings accounts.{id}.save_sent_to and startup.queries.

However, the accounts/{account}/.config/astroid/poll.sh script is actually the same for both my accounts, because it can use the $ACCOUNT_DIR variable set in launch-account. In my case it's currently just these few lines:

#!/usr/bin/env bash
set -e # Exit as soon as one of the commands fail.
mbsync --config $ACCOUNT_DIR/.mbsyncrc -a
notmuch --config=$ACCOUNT_DIR/.notmuch-config new

Now, launching either of the configurations is as simple as this:

~/mail/launch-account work

Step 3: How to run them simultaneously?

Launching either one of these configurations works just fine. But running both simultaneously does not. If I first start one (say work) and then the other (personal), then the second launch just results in another window for the account that launched first. Obviously the instances are sharing something that I don't want them to share.

Installing and running another copy of Astroid with a different prefix makes no difference.

Thus, at the moment I did not really achieve much. I could just as well run Astroid without my little patch, and instead make a little script to move in/out the account-specific .config/astroid, .cache/astroid etc before launching.

To be continued, I suppose...