Bloerg
       

Syncing mails with mbsync (instead of OfflineIMAP)

A lot of people recommend OfflineIMAP for syncing mails between a remote IMAP account and a local maildir mailbox for mutt1 pleasure. However, OfflineIMAP’s quality is far from what I think is necessary for a tool handling sensitive data such as mail. Looking for an alternative, I stumbled upon mbsync which was maintained by Ted Ts’o for a while. Setting it up to synchronize with the Google mail servers is pretty simple once you know the quirks of Google’s IMAP implementation and mbsync’s weird configuration approach.

To get it working with Google I had to compile mbsync from source — the packaged version didn’t work for me. mbsync just wouldn’t download any mails. So, go ahead, download the mbsync source and configure && make && make install it. mbsync groups its .mbsyncrc configuration into account details, stores and channels that connect stores for synchronization. A typical GMail account would look like this:

IMAPAccount gmail
Host imap.gmail.com
User foo.bar@gmail.com
Pass secret
UseIMAPS yes
CertificateFile /etc/ssl/certs/ca-certificates.crt

Saving the password plaintext is not the best idea. Fortunately, the latest mbsync sources come with the PassCmd setting, which is just a command that is run in a subprocess of mbsync. With a tiny Python script such as this:

#!/usr/bin/env python

import argparse
import keyring
import getpass

if __name__ == '__main__':
    SERVICE = 'mbsync'

    parser = argparse.ArgumentParser()
    group = parser.add_mutually_exclusive_group(required=True)
    group.add_argument('--set', '-s', type=str, help='Account to save password')
    group.add_argument('--get', '-g', type=str, help='Account to get password')

    args = parser.parse_args()

    if args.set:
        password = getpass.getpass()
        keyring.set_password(SERVICE, args.set, password)
    else:
        print(keyring.get_password(SERVICE, args.get))

and PassCmd set to:

PassCmd "imap-pass -g foo.bar@gmail.com"

we have a relatively secure way of retrieving the login password through our system’s keyring. Now we have to define a proxy store for the IMAP account and a store for our local maildir

IMAPStore gmail-remote
Account gmail

MaildirStore gmail-local
Path ~/mail/gmail/
Inbox ~/mail/gmail/INBOX

To sync between those stores, you have to set up a channel. Most options are self-explanatory, however you must take care to use the correct label names for the sent, draft, starred etc. mail. The names depend on the language set in the web interface! For me, the verbose flag -V came in handy because this displays the communication with the IMAP server, including listing all the available IMAP directories.

Channel gmail
Master :gmail-remote:
Slave :gmail-local:
Patterns "INBOX" "[Gmail]/Sent Mail" "[Gmail]/Drafts" "[Gmail]/Starred"
Create Both
Expunge Both
SyncState *

Now, running mbsync gmail syncs mail on the Google servers with your local maildir in ~/mail/gmail.

  1. I wanted to write a post about mutt for quite a while now, but I am not ready with my configuration nor do I have much time to document my progress. Hence, I will present different aspects one by one.

Discussion

Alexander Baier
Wed, May 21 2014

Hello Matthias!

Thank you for sharing this setup. I, too, use mbsync albeit with gnus. I have, however, configured multiple Accounts that do not share the same password. Does your script handle such a situation?

Matthias
Wed, May 21 2014

Hi Alexander,

yes it should work, because the first argument that is passed, is the account for which the password is stored. By the way, I generalized that script a little bit to store the password under any kind of “service” or domain.

Alexander Baier
Sat, May 24 2014

This is nice to hear :)

I saved the script you linked to to a file and tried to execute it from the command line, when it printed the following backtrace:

Traceback (most recent call last):
File "/home/delexi/bin/keyring-query.py", line 4, in
import keyring
ImportError: No module named 'keyring'

I don’t know python, but from what I can guess it needs library or module it does not find. I have gnome-keyring and python2-gnomekeyring installed. This is an Arch Linux system. Do you have an idea where the problem is here?

Thanks in advance!

Matthias
Sun, May 25 2014

I use Debian-based systems, but as far as I can tell, you could install this package from AUR which provides the required Python module.

Eduardo Mercovich
Mon, Apr 04 2016

Hello Matthias!

I hope you don’t mind for bumping such and old article, but did you created the maildir anew, or had a previously existent maildir and just changed the package used to sync?

I ask because I have many Gb. of Maildir (even for old, non existent accounts) So I am almost forced to use my actual ~/Maildir…

Thanks a lot. :)

Best…

Matthias
Thu, Apr 07 2016

Hi Eduardo, I did synced new mail dirs using mbsync. But in principle it should also work to copy it using a different method. By the way, sorry for the late reply.

Thomas
Wed, Sep 28 2016

I want to backup my imap server with isync/mbsync to my home server. This should be done automatically once per week. Storing the password with keyring seems not to work without user interaction. Do you have another idea of storing the password so that the backup can be executed automatically by anacron without user asking for the keyring password? Maybe storing the passwords in a gpg encrypted file with a keyfile instead of user-typed password?

Matthias
Thu, Sep 29 2016

Well my original idea is to let the user own the only secret. If you are going to go this way, then you can also store the password in plaintext. Assuming the attacker has access to the machine, he can then just use the keyfile to decrypt your IMAP password.

A better approach would be to store the password in an environment variable and pass that to mbsync.

ottowayne
Thu, Oct 27 2016

Hi there,

However, OfflineIMAP’s quality is far from what I think is necessary for a tool handling sensitive data such as mail.

Would you mind elaborating on why you think this is the case?

Matthias
Thu, Oct 27 2016

Maybe was the case. At that time, OfflineIMAP tended to be pretty unstable crashing at seemingly random places. Also, the code quality was a bit … low. I haven’t checked since then, so maybe all these observations are nil and void.

Post a comment

Name required

E-mail required, not published

Website optional

Comment Markdown accepted