(Good!!!)imapfilter + offlineimap + msmtp + mutt + abook = email

Original article url:

http://blog.grossmeier.net/2009/01/25/imapfilterofflineimapmsmtpmuttabook/

So, I’ve spent a little over a week setting up my new email consumption/creation system. As you can see from the title of this blog post, there are a few parts to it. Why would I do something crazy like edit config files for 4 different apps JUST to read and write email? Well, I wasn’t happy with Thunderbird (yes, I’ll try 3.0 when it hits the repos) and Evolution wasn’t at all what I wanted. I do have gmail so why not just stick with the web interface? Because I am wanting to do more self-hosted solutions for web apps. Also, since I have more than one account, I want different messages to be sorted different and archived differently.

In Thunderbird I had an extension that allowed me to press “y” and the current message would be “archived” to the gmail All Mail folder. This was great, but it only supported one account. If I was reading my work email in Thunderbird (which is also hosted by gmail) and I hit “y” the message would go to my personal gmail account’s All Mail folder, not the work account one. Not good (and a dumb limitation).

So, what email program allows you to have complete control over those types of settings?Mutt. And yes, (Al)pine also. But, I have friends local to me who use mutt so exchanging .muttrc files and such is easier and we can meet in person to share tips.

What I want to do with this blog post, though, is not convince you that Mutt is the best solution for you. I do want to, however, share what I did to set everything up for use with Mutt. In fact, all the rest of the pieces of this setup can work equally well with some like Alpine or even Thunderbird.

(since it is a long post, I didn’t want to spam your reader, click for the rest of it)

imapfilter
imapfilter does just what the name implies: it filters email on IMAP servers. The general idea is you have imapfilter running via a cron job on some always-on computer. It is then run every 3-5 minutes, connects to any number of IMAP servers, and filters the email. And the filtering doesn’t even need to happen only on the INBOX (like gmail) but can be for any mailbox. Eg: if you have an UbuntuBugs folder you can have all new Launchpad emails go there, then when you have read them it could then filter them to your Archive folder (or whatever).

Anyway, here is my imapfilter.config.lua file. It is written in the Lua programming language which is simple to read.

offlineimap:
Again, another straight-forward software name (l love software that is named what it does, kind of like kitchen appliances: blender, toaster, refrigerator).Offlineimap will connect to an IMAP server and download all (or some, you can decide) of the messages to a local directory in Maildir format. It is very robust and quick; syncs take very little time. This app is actually only run once (you can do a gnome-session, run it manually, or use an init script) and it syncs every 10 (or 5, or whatever you want) minutes automatically.

This method is great for a number of reasons. First, you always have an up-to-date backup of all your email on a local drive. Should your mail server die one night you will still be ok. Second, it effectively batches my email for me since I have it sync every 10 minutes. So every 10 minutes is the most often I will get new email. Great for non-interruption.

The syntax for this one is pretty simple too. Here is my .offlineimaprc file. It checks one gmail account and one other imap account. Here is afully commented config file with every option.

msmtp
msmtp is a simpler alternative to sendmail. It sends mail. Mutt is a MUA (Mail User Agent) not a MTA (Mail Transpot Agent) and as such does not send email; you have to set up something else to do it for Mutt. This isn’t that bad, really (ed: it was the hardest part for Greg, don’t let him fool you). Although, I did have a problem with msmtp not being able to connect to my hosting provider’s smtp server via port 25. Getting around that required a quick ssh tunnel script (and a bit of sanity checking from Asheesh, thanks again!).

Here is msmtp config file. And the accompanyingssh tunnel script so sending via port 25 works. The script simply stands in for msmtp and opens the tunnel, then sends the arguments to msmtp.

Mutt
Mutt has the most complex config file, but luckily there are a ton of examples out there and you can fairly easily copy/paste certain sections (like colors). My muttrc has two sub files (which are sourced in muttrc): crypto (for GPG) andmuttmailboxes (for my list of mailboxes). Separated out mainly for readability. I might later separate out my colors section, macros, formating, etc. No practical need to, yet.

Question about GPG and mutt: I only see two options (so I must be blind): either A) sign all mail you send or B) only sign mail that is a reply to a signed email. That leaves out a big use-case of: sign when I want to sign. What am I missing?

Some points of interest with my muttrc: the Folder Hooks section. That allows you to set variables based on which folder you are in (which account you are reading mail in). This is useful for me when I have “y” set to archive a message. In gmail, the message goes to GMAIL/Archive, in my personal domain account it goes to GROSSMEIER/Archive. Also, that is where you set which “persona” you send mail from. It’ll make sense when you see it in my config.

Also, I am using mutt-patched from Debian/Ubuntu. This mainly adds a patch to have a side bar with the list of mailboxes (and associated all/unread/new numbers). Pretty useful. Thus, there are a few settings in my muttrc for that specifically that wouldn’t work with a non-patched Mutt.

Without further ado, my muttrc.

abook
Oh yes, an address book. Mutt has “aliases” that you can set up in a .mutt-alias file.abook helps with the managing of that file. It also can convert your LDIF Thunderbird mailbox into a mutt-alias file. You can see my settings for abook/aliases in my muttrc file at the top, the last four lines of “Basic Setup.”

Thats it. That is all it took to have email just the way I want it. I have all these files version controlled via bzr and pushed to a globally (authorization controlled) accessible repository so I can share them between my desktop and laptop.

All the files:

• imapfilter.config.lua
• offlineimaprc
• msmtprc
• smtpforward.sh
• muttrc
• crypto
• muttmailboxes

Any other tips for how you setup any of these applications? Anything you would change with my setup? Leave a comment.

(potentially) related posts:

1. Searching your email
2. New Laptop: Setting up Email
3. My daily backup "script"
4. Another "script" I use daily

Posted by Greg at 1:25 pm on January 25th, 2009. 16 comments... »
Categories: Uncategorized. Tags: freedom, news, sharing, ubuntu.

16 comments.

1. 

   mutt has had built in smtp for a while now. Check out mutt -v for compile options.

   Posted by Donnie Pennington on January 26th, 2009.

2. 

   @Donnie: Yeah, I guess it does :)

   Oh well, if I erroneously setup msmtp that isn’t a big deal. The hardest part was working around the port block, which would have been true either way.

   Posted by Greg on January 26th, 2009.

3. 

   So does offlineimap mark an email as read once it copies it? If not then you can read an email in mutt and then go to GMail and it shows as unread.

   Posted by etank on January 28th, 2009.

4. 

   @etank:
   offineimap downloads the email in whatever state it is in on the imap server (so in this case “unread”). Then, in my case, mutt will read it and mark the copy on my harddrive as read. Then, when offlineimap syncs with the imap server it marks the copy on the server as read.

   Posted by Greg on January 28th, 2009.

5. 

   Thanks for this, really useful!

   I had a bit of a problem with the offlineimaprc which I copied from you as it didn’t like the inline comments like:
   autorefresh = 10 # minutes until next sync

   It seems fine if the whole line is a comment though, like:
   #### GMAIL Account Settings ####

   Hope that’s of use to anyone setting this up!

   John.

   Posted by John Lawrence on March 24th, 2009.

6. 

   @John: Thanks for the comment. I actually added those comments in only for the version I put online (iow: it was untested). I just checked the version I have running myself and it does not included those inline comments.

   My bad :)

   Posted by Greg on March 24th, 2009.

7. 

   Great post, thank you for sharing.
   I would replace the “$1 $2 $3 $4 $5 $6 $7 $8 $9″ list in smtpforward.sh with “$@”, so that you do not need to know how many arguments will be there (just in case).

   Posted by Redshift on March 26th, 2009.

8. 

   Hey Greg – thanks for the article. I was able to get up and running with mutt, offlineimap, msmtp, and abook across multiple gmail accounts.

   Much appreciated!

   Posted by Adam Bair on August 11th, 2009.

9. 

   Great article. Looks a lot like my setup. One thing you should consider to complete your workflow is msmtp-enqueue so you can “send” emails from mutt even if you’re offline.

   Posted by Jojo on November 18th, 2009.

10. 

    @Jojo: Great idea. I was actually just thinking about how I was going to set up that functionality. Hadn’t got to the google-searching part of the process so you just saved me some time! Thanks!

    Posted by Greg on November 18th, 2009.

11. [...] my email configuration addiction continues. As you may recall, I did an extensive (now out-dated) post about my email setup. I’m still using the basic configuration outlined there (all of the same [...]

    Posted by Consilience — New Laptop: Setting up Email on January 28th, 2010.

12. 

    saved me a bunch of hours configuring imapfilter. Thanks!

    Posted by Prashant Borole on April 10th, 2010.

13. [...] spent quite a bit of yesterday and today fiddling around with some things that my buddy Greg posted about a month ago. See, I have had Mutt doing the generic IMAP thing with GMail and it works fairly well. The [...]

    Posted by Who has the perfect IMAP setup? - Richard A. Johnson's Blog on February 12th, 2011.

14. [...] I mean the suite of tools that bring email sent to me to my eyes. I’ve written before on my setup (note: that is out of date now) and its migration across systems. As you can see, I’m a fan [...]

    Posted by Consilience — Searching your email on January 5th, 2012.

15. 

    Hi, do you know any way to invoke Imapfilter everytime a new Mail arrives in Mutt? Exist there any Hook or something similar to it?

    Posted by Paloran on April 17th, 2012.

16. 

    Hi Paloran,

    I don’t but there might be a way. I just have offlineimap call imapfilter before it syncs. So if you use offlineimap, do it that way. If you don’t use offlineimap I’m at a loss right now. I’m sure if it is possible via a mutt config option there is a guide out there somewhere :)