jab2870
/
Website
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Content

Browse Source
master
Jonathan Hodgson 5 years ago
parent eea39d661d
commit 355b5203ed
4 changed files with 1848 additions and 105 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 1163
      assets/mutt/email-diagram.svg
  2. 105
      content/blog/006-building-a-zsh-prompt.md
  3. 62
      content/blog/006-freelance-business-security.md
  4. 623
      content/blog/007-mutt-setup.md

1163
assets/mutt/email-diagram.svg
View File

File diff suppressed because it is too large Load Diff
Side by Side

After

Width:  |  Height:  |  Size: 46 KiB

105
content/blog/006-building-a-zsh-prompt.md
Unescape View File

@ -1,105 +0,0 @@
---
title: Building a ZSH prompt
tags:
- Linux
- ZSH
description: There are many frameworks out there to help you build a prompt, I thought I'd see how far I could get without one.
date: 2020-03-24
draft: true
---
Until recently, I was using [oh-my-zsh](https://github.com/ohmyzsh/ohmyzsh) with the [Power Level 9k](https://github.com/Powerlevel9k/powerlevel9k) prompt.
In part of my effort to understand how different parts of my system works, and to reduce my reliance on some libraries, I decided to see if I could get what I needed without them. In this blog post, I am going to focus on my prompt configuration.
If you want to follow along, my configuration can be found in [my dotfiles](https://github.com/Jab2870/dotfiles/blob/master/shells/zsh/includes/promptconfig.zsh).
My prompt currently looks something like this:
![My Prompt](/assets/prompt/prompt.png)
## Basics
Zsh simply prints the variables `PROMPT` and `RPROMPT` when drawing the left and right prompts respectively. `PROMPT` is roughly equivalent to the `PS1` variable in bash.
By setting the variable, you can change how the prompt works. E.g.
```
oldPrompt $ PROMPT="new prompt > "
new prompt > ls
Documents Downloads GitRepos
new prompt > █
```
## ZSH Hooks
Zsh has a concept called hooks. These allow you to run a function (or functions) automatically at certain times.
For the purposes of setting my prompt, I want to run a function to set `PROMPT` and `RPROMPT` before the prompt is drawn. This can be done with the `precmd` hook.
```zsh
set_prompts(){
PROMPT="$ "
RPROMPT=""
}
zle -N set_prompts
autoload -U add-zsh-hook
add-zsh-hook precmd set_prompts
```
This defines a function `set_prompts` and it will be run just before the prompt is drawn, every time. We can therefore use it to build the prompt.
## Drawing segments
I like the arrow style that was made popular by [Powerline](https://github.com/powerline/powerline). To achieve this, you will need a font that contains the powewline glyphs. If you don't have one, I highly recommend the [Nerd Fonts](https://www.nerdfonts.com/) project that aggregates a lot of icons as well as the powerline symbols. It even includes a script that allows you to add them to your favourite font.
## Exit Code
In ZSH (and most other shells), the exit code of the previous command is available in the special `$?` variable. If we want the exit code of the last command run, we need to assign this to a variable first.
```bash
# Get the return status of the previous command
local RETVAL=$?
```
## Git
I use Git a lot. It is useful to tell, at a glance, which branch I'm on, which tag I'm on and what the state of repository is. The function is quite simple:
```bash
prompt_git(){
local repoTopLevel="$(command git rev-parse --show-toplevel 2> /dev/null)"
if [ -n "$repoTopLevel" ]; then
local branch="$(git branch --show-current 2> /dev/null)"
local tag="$(git describe --tags --exact-match HEAD 2> /dev/null)"
local color="green"
local ret=""
[ -n "$branch" ] && ret="$branch "
[ -n "$tag" ] && ret+="$tag "
[ -n "$ret" ] || ret="$(git rev-parse --short HEAD 2> /dev/null)"
local repoTopLevel="$(command git rev-parse --show-toplevel 2> /dev/null)"
local untrackedFiles=$(command git ls-files --others --exclude-standard "${repoTopLevel}" 2> /dev/null)
local modified=$(command git diff --name-only 2> /dev/null)
local staged=$(command git diff --staged --name-only 2> /dev/null)
if [ -n "$untrackedFiles" ]; then
ret+=" "
color="orange1"
fi
if [ -n "$modified" ]; then
ret+=" "
color="orange1"
fi
if [ -n "$staged" ]; then
ret+=" "
color="orange1"
fi
fi
ret="$(echo "$ret" | sed -e 's/ *$//')"
echo "$ret"
echo "$color"
}
```

62
content/blog/006-freelance-business-security.md
Unescape View File

@ -0,0 +1,62 @@
---
title: 3 Simple Steps to Improve Your Freelance Business's Security
date: 2020-04-09
description: All too often, security advice is aimed at large companies who can afford to spend hundreds or thousands of pounds on security procedures. In this blog, I will try and lay out some advice for freelancers and small businesses can follow, without breaking the bank.
tags:
- Security Advice
---
All too often, security advice is aimed at large companies who can afford to spend hundreds or thousands of pounds on security procedures. What follows is hopefully advice that anyone can follow.
That being said, if you can afford to hire a security professional, do so. They will be able to target advice at your business specifically.
I work as a security consultant. I am often tasked with trying to break into a website or computer system to identify a client's risks. A huge proportion of my successful attacks would have been unsuccessful if the following procedures were followed.
As a small business, your aim should be minimising risk - you don't want to be the low hanging fruit. The hard truth is, if a nation state or highly skilled hacker targeted you, you are likely to fall. What you want to do is make it hard. Stop the "bored teenager in their bedroom" from being able to get into your systems.
## Use a Password Manager
We all hate passwords. Unfortunately, they are a necessary evil until we come up with something better. My advice to all of you is use a password manager. A password manager is a program that generates random passwords and remembers them for you. This way, you never have to re-use passwords, and you have no excuse for not changing default passwords.
We all hear that re-using passwords is bad, but why? Websites get hacked. This often results in your email address and password being leaked. If that is the case, and you use the same combination of username and password on other sites, an attacker is going to be able to log in using your credentials. Don't take my word for it, put your email address(es) into [this](https://haveibeenpwned.com/) website to see if it has been leaked. Unless it is a new email address, it has probably been in at least one breach. If it hasn't, there is a good chance that it will be in a future leak. My point is, passwords get leaked, and if you re-use them, you are giving hackers the key to your other accounts. Password managers generate random, unique passwords for every site you visit. That means, even if one account is compromised, your other accounts won't be.
A common criticism of password managers is that if your password manager is hacked, all your accounts are compromised. Normally, however, this is not the case. Any reputable password manager will encrypt all your passwords with a single, strong password that only you know. This means that even if the provider is compromised, your passwords will stay safe.
I would suggest looking at [Dashlane](https://www.dashlane.com/) or [LastPass](https://www.lastpass.com/). Both are mature projects that have browser extensions and mobile apps.
You will need to remember one strong password for your password manager. I suggest a passphrase rather than a password. That means 4-6 random words, joined with special characters. They are generally much easier for humans to remember, but harder for hackers to guess. If you're interested, read about [how to create a stong password here](https://fellowship.agency/blog/its-not-difficult-to-create-a-strong-password/).
Once you've signed up, use it. Go through all of your online accounts and generate new passwords, using your password manager.
## Don't Ignore Your Updates
Updates are another necessary evil. Updates often include security fixes, and if you ignore them, your computer or website will be vulnerable to whatever the update fixed. Once hackers learn about security problems, normally this is only a matter of days (at most) after the update, they can scan the internet for vulnerable devices - they don't have to be targeting you.
The [WannaCry](https://en.wikipedia.org/wiki/WannaCry_ransomware_attack) attack in May 2017 crippled many organisations, including the NHS. Microsoft released an [update](https://docs.microsoft.com/en-us/security-updates/SecurityBulletins/2017/ms17-010), fixing the vulnerability that was exploited 2 months earlier. If people had updated their systems in a timely manner, hundreds of millions of pounds would have been saved. It [was reported](https://www.telegraph.co.uk/technology/2018/10/11/wannacry-cyber-attack-cost-nhs-92m-19000-appointments-cancelled/) to have cost the NHS alone £92 million pounds. All because an update wasn't applied.
If you can enable automatic security updates, you should. It is very rare that an automatic security update from an established vendor will break anything. Even if it does, the cost of not updating could be far worse. Most websites management systems and computers will have this option. If it doesn't, it is up to you to make sure that they are done in a timely manner.
## Don't Trust Emails
Emails have changed the way we do business. They allow us to communicate with people quickly and effortlessly. Because of this, we don't treat them with the scrutiny we should.
It is very easy for an attacker to send you an email and make it look like it is coming from someone else. Attackers do this to trick you into disclosing sensitive information in a process called [phishing](https://en.wikipedia.org/wiki/Phishing).
Often, the attacker will have made a website that looks exactly the same as a login screen you use. He or she will direct you there using a link in an email and when you fill in your username and password, they will be sent to the attacker, rather than the website you were trying to log into.
Another common scenario is an email attachment that contains malicious code. This could be as simple as a Word document, but hiding a virus that could give the attacker access to anything on your computer.
My advice, treat emails with caution. Unless you are expecting an email, don't click on links or download attachments. If you receive an email, and the link asks you to sign in, make absolutely sure that you are on the website that you think you are on. If in any doubt, don't fill in the form. Go to the website as you normally would, through your bookmarks or by typing in the website address.
If you receive an email with an attachment, especially one that you weren't expecting, you should be very cautious. Even if you trust the person who sent the email, you can't be sure that their computer or email account hasn't been compromised. If you can, phone them and confirm that they (and not a virus on their computer) sent the email.
## In Summary
The above should prevent a lot of low-skilled attacks. They are some of the first things I look for when attempting to break into a computer or website. These won't make you immune to attack, but they will make you more resilient.
So:
* Use a password manager, that way you can set strong passwords for everything, without re-using them.
* Do your updates, that way known vulnerabilities can be fixed before they are exploited.
* Don't trust emails, this will make it much harder for attackers to trick you into giving up sensitive information.

623
content/blog/007-mutt-setup.md
Unescape View File

@ -0,0 +1,623 @@
---
title: My Email Setup using Mutt and Vim
date: 2020-05-04
description: I describe my email setup. It mainly consists of Mutt and Vim although there are various supporting tools as well
tags:
- Mutt
- Vim
---
My email system is based around my email client of choice, [Mutt](http://www.mutt.org/); or more specifically [NeoMutt](https://neomutt.org/), a fork of Mutt with several of the most common patches applied. I will refer to Mutt throughout this blog although, unless otherwise stated, I am referring to NeoMutt.
In this blog post I will document how Mutt and the many surrounding cogs interlock, to get my emails the way I like them. My goals for setting up my email were as follows:
* **Exchange** - Like many companies, my work emails are handled by MS Exchange. Obviously, it is important that I can read my work email.
* **Offline** - It is fairly common for me to work in remote locations, often resulting in the need to check my emails before I am connected to the internet.
* **Powerful filtering** - I get a lot of emails and keeping them organised can be a lot of work. The more this can be automated, the better.
* **Terminal based** - I spend a lot of time in the terminal. If email can be done in the terminal, I would like to.
* **Vim** - I use Vim extensively for almost all writing tasks, whether it is programming or letters. It would be ideal if emails could be included on that list.
With Mutt, and some supporting tools, I fulfil all of the above requirements.
Below are the tools I use. I will detail how I use all of them in this blog post.
* [Neomutt](https://neomutt.org/) - Email client
* [NeoVim](https://neovim.io/) - Email editor
* [Mbsync](http://isync.sourceforge.net/mbsync.html) - Email downloader
* [Notmuch](https://notmuchmail.org/) - Email indexer (for searching)
* [Msmtp](https://marlam.de/msmtp/) - Email sender
* [Davmail](http://davmail.sourceforge.net/) - Bridge for MS Exchange
* [ImapFilter](https://github.com/lefcha/imapfilter) - Email filterer
* [Abook](http://abook.sourceforge.net/) - Contact manager
If you want to follow along, and understand how everything works, I suggest you get yourself a beer, this will probably take a couple of hours to set up first time.
Install the above tools, on Arch that would be:
```bash
sudo pacman -S neomutt isync notmuch msmtp abook
yay -S davmail imapfilter
```
I will try to describe the options necessary to get a working system, although this isn't a walk through of all of my configuration choices or a guide to using Mutt. To do so would make an already long blog post ridiculously long. However, feel free to peruse [my dotfiles](https://git.jonathanh.co.uk/jab2870/Dotfiles). I try to document options in the configuration file.
## Connecting to Exchange - Davmail
Davmail is only required if you need to interface with a MS Exchange server that does not allow IMAP / SMTP connections. If this isn't a requirement for you, [skip this step](#downloading-emails---mbsync).
To start with, install and create a configuration file in `~/.config/davmail/davmail.properties`.
The first option I set is to ensure that Davmail will only accept connections from localhost.
```ini
# Don't allow remote connections
davmail.allowRemote=false
```
For the purposes of this blog post, I will be setting it up so the various clients can connect to Davmail without an encrypted connection. That would mean, if a malicious actor or process was already on your computer, they could potentially steal credentials. If this isn't acceptable to you, it would be worth consulting [Davmail's documentation](http://davmail.sourceforge.net/sslsetup.html) about setting up an SSL.
```ini
# Don't require client to use ssl when connecting
davmail.ssl.nosecurecaldav=false
davmail.ssl.nosecureimap=false
davmail.ssl.nosecureldap=false
davmail.ssl.nosecuresmtp=false
```
By default, Davmail listens on ports over 1000 so that it doesn't need extra permissions to run. If you want to change the ports it listens on, you can do so with the following options:
```ini
# Ports to listen on
davmail.caldavPort=1080
davmail.imapPort=1143
davmail.ldapPort=1389
davmail.smtpPort=1025
```
Although Davmail doesn't store any credentials for your Exchange server, you do need to give it a URL to connect to (OWA or EWS). I also find it useful to provide a default domain.
```ini
# Exchange details
davmail.mode=Auto
davmail.defaultDomain=<PUT YOUR EXCHANGE DOMAIN HERE>
davmail.url=<PUT YOUR EXCHANGE URL HERE>
```
Finally, we have a couple of other preference options that should be relatively self explanatory.
```ini
# Don't start GUI
davmail.server=true
# Delete messages immediately on IMAP STORE \Deleted flag
davmail.imapAutoExpunge=true
# When a message is sent, put it in the sent folder
davmail.smtpSaveInSent=false
# Send keepalive character during large folder and messages download
davmail.enableKeepAlive=true
```
Start the Davmail server with `davmail ~/.config/davmail/davmail.properties`. You could put this in a start-up script although I quite like running it manually.
For more information on configuring Davmail, see their [official documentation](http://davmail.sourceforge.net/serversetup.html).
## Downloading Emails - Mbsync
Mbsync is the tool I use to download all my emails to a local directory. It manages a two way sync between a local MailDir directory and in IMAP server. In my case, the IMAP server is on localhost and provided by Davmail.
The default configuration file for Mbsync is `~/.mbsyncrc`. I like to keep my home directory clean so I instead keep the file in `$XDG_CONFIG_HOME/isync/mbsyncrc`. To make things easy for myself, I have the following alias:
```bash
alias mbsync="mbsync -c \"$XDG_CONFIG_HOME/isync/mbsyncrc\""
```
To start with, we create an IMAPAccount, which is, as the name suggests, the details for an IMAP account.
```ini
IMAPAccount work
# Address to connect to
Host 127.0.0.1
Port 1143
User USERNAME
PassCmd "pass work/email"
SSLType None
AuthMechs LOGIN
```
In my case, this is provided by Davmail, and as such, the host and port are both non-standard. If you were to use this with a normal IMAP account, you would want to change them. You would also want to use a form of encryption. Mbsync supports both STARTTLS and IMAPS.
The `PassCmd` option will run a command in order to retrieve a password, this means it isn't necessary to store your password in plain text in a configuration file. I am using [Pass](https://www.passwordstore.org/) although there are many other tools that would work. Most password managers, including [Lastpass](https://github.com/lastpass/lastpass-cli), provide a command line interface.
If you are happy with the risk (or lazy), you could replace the `PasCmd` option with `Pass` and just give your password.
Next we create a store for our account. A store in Mbsync is simply one of the locations being synced. In this example, I call it work-remote.
```ini
IMAPStore work-remote
Account work
```
Then, we create another store for our local MailDir copy.
MailDir is a standard format for storing emails that Mutt is able to read and interface with.
```ini
MaildirStore work-local
# Copy folder hierarchy
Subfolders Verbatim
# The trailing "/" is important
Path ~/.mail/work/
Inbox ~/.mail/work/Inbox
```
The `Subfolders` option specifies how a hierarchical folder structure should be represented. I've never had to use anything other than `Verbatim`.
The `Path` and `Inbox` options specify where the MailDir and Inbox folders should be stored.
Lastly, we create a channel. This is what joins the two stores together in a Master-Slave relationship.
```ini
Channel work
Master :work-remote:
Slave :work-local:
# Include everything
Patterns *
# Automatically create missing mailboxes, both locally and on the server
Create Both
# Save the synchronization state files in the relevant directory
SyncState *
```
The `Patterns` option can be used to include / exclude certain folders. I prefer to keep things simple and sync everything.
That should be enough to clone your mailbox. Simply run
```bash
mbsync work
```
This will download your whole mailbox, including attachments. Be prepared for it to take a while.
## Viewing Emails - Mutt
Mutt is an extremely powerful and configurable email manager.
To start with, you will want to create the file `~/.config/mutt/muttrc`.
In there, you will want to tell Mutt about the folder that Mbsync has just created; where it is and where some of the important folders are in there.
```ini
# Folder with emails
set folder = "~/.mail/work"
# Type of mailbox
set mbox_type = Maildir
# Directory to poll for new mail
set spoolfile = +Inbox
# Directory to save sent messages into
set record = +Sent
# Sets the drafts folder
set postponed = +Drafts
# File that headers will be cached
set header_cache = ~/.cache/mutt
```
With just the configuration above, you should be able to open Mutt and read your emails. However, you won't have a very nice time of it. There are some settings that you will probably want.
```ini
# Sort by threads
set sort = threads
# Sort threads by last date recieved - newest first
set sort_aux = reverse-last-date-received
# Show date in year/month/day hour:minute format
set date_format="%y/%m/%d %I:%M%p"
```
This obviously isn't a full list of options, but it should give you an email client that is in some way similar to other clients you will have used.
### HTML Email
Unfortunately, most people don't use plain text emails. This means that we need to tell Mutt how to handle HTML and multipart messages. This starts with a `mailcap` file.
```ini
# Mailcap file is used to tell mutt how to open different types of file
set mailcap_path = "~/.config/mutt/mailcap"
```
This tells Mutt what to do when it comes across different types of mime type. The following should put you in good stead for opening HTML emails:
```mailcap
text/html; $BROWSER %s
text/html; w3m -I %{charset} -T text/html -dump; copiousoutput;
```
The syntax for this file is quite simple. The line is delimited by semi-colons. Field 1 is the mime type, field 2 is the command to run, field 3 and higher are optional flags.
You will see that on the second line, the `copiousoutput` flag is passed which means the command is non-interactive and, as such, Mutt can simply echo the output in its normal pager. Mutt will prefer options with this flag, if trying to view an email using `auto_view`.
Note, I am using the text based web browser w3m, you may have to install it or use an alternative such as lynx. Additionally, you will need to have the `$BROWSER` environment variable set or specify the browser command you want to use in its place.
```ini
# Tells Mutt to automatically view files with these mime types
auto_view text/html
# Order to try and show multipart emails
alternative_order text/plain text/enriched text/html
```
The lines above tell Mutt to try and automatically view HTML emails, although it should prefer plain text.
In my experience, this works for the majority of emails. However, for maybe 5% of emails, this isn't enough. For those emails that are image heavy, or use colour to distinguish different sections, you can open the email in your normal browser. To do that, you will need to push `v` when viewing the email which will take you to the attachment view:
```
I 1 <no description> [multipa/alternativ, 7bit, 4.5K]
I 2 ├─><no description> [text/plain, 7bit, us-ascii, 0.6K]
I 3 └─><no description> [text/html, 7bit, us-ascii, 3.7K]
```
By selecting the text/html option and pushing enter, it will be opened in your browser.
## Sending emails
The last essential piece to any email system is sending emails.
Neomutt has built in SMTP support, so for basic email needs all you need is:
```ini
# Use an external command to get the password
set my_smtp_pass = `pass show work/email`
# Set the smtp url
set smtp_url="smtp://USERNAME:$my_smtp_pass@127.0.0.1:1025"
```
If you want to include your password directly in the configuration file, you can omit the `my_smtp_pass` variable and simply provide your password like so:
```ini
set smtp_url="smtp://USERNAME:PASSWORD@127.0.0.1:1025"
```
This supports both `smtp` and `smtps`, simply by changing the protocol in the URL. If you need to use `startls`, you can do so with the following:
```ini
set ssl_starttls = yes
```
### Composing Emails
One of the biggest advantages (for me) of Mutt is the ability to use Vim for composing my emails. Whether you use Vim or not, you probably have a favourite text editor.
If you do any programming, you are likely to have spent a non-trivial amount of time configuring your editor. As a result, it makes a lot of sense to use that editor for as much as you can.
To use Vim, set the following:
```ini
# Use nvim but don't force text width (looks terible if read on a phone)
set editor = "nvim +':set textwidth=0'"
```
Note that I set my `textwidth` to 0 so Vim doesn't try to hard wrap the lines. I do this because if you hard wrap, the text will look terrible on narrow devices (such as phones).
If you don't like Vim, you can use almost any text editor you like, the only catch is that the editor might need to be set to only return once you close it. This is often done with a `wait` flag, although you will need to consult you editor's documentation. See a couple of examples below:
```ini
# Use Sublime Text to compose email.
# -w stops sublime returning until you close
set editor = "subl -w"
# Use VS Code to compose email.
# -w stops Code returning until you close
set editor = "code -w"
# Emacs
set editor = "emacs"
# Nano
set editor = "nano"
```
### HTML Emails
If you do need to send HTML emails, please spare a thought for your recipient; it is not just terminal email client users that could suffer. According to the [National Eye Institute](https://www.nei.nih.gov/learn-about-eye-health/eye-conditions-and-diseases/color-blindness), one in twelve men are colour blind; so please don't use only colour to distinguish items. Additionally, many people suffer from vision-loss blindness, who are likely to use screen readers or braille displays. Complex layouts or heavy use of images are going to give these people a poor experience.
That being said, HTML emails can be used for aesthetic purposes, whilst still providing a plain text version for those who want it. That is the method I suggest adopting.
Firstly, you need to set up msmtp which is an SMTP client used to send emails. This is done with a configuration file in `~/.config/msmtp/config`. It consists of 1 or more blocks like this:
```ini
# Work
account work
host localhost
port 1025
tls off
tls_starttls off
auth on
user USER
passwordeval "pass show work/email"
from jonathan.hodgson@example.com
```
In the msmtp file, you can replace `passwordeval` with `password` if you wish not to use a password manager.
Next I use a wrapper script called `send-from-mutt` which is used to determine which account to send from and whether or not to convert it to an html multipart email.
```bash
#!/bin/sh
# Put the message, send to stdin, in a variable
message="$(cat -)"
# Look at the first argument,
# Use it to determine the account to use
# If not set, assume work
# All remaining arguments should be recipient addresses which should be passed to msmtp
case "$(echo "$1" | tr '[A-Z]' '[a-z]')" in
"work") account="work"; shift ;;
"home") account="home"; shift ;;
*) account="work"; ;;
esac
cleanHeaders(){
# In the headers, delete any lines starting with markdown
cat - | sed '0,/^$/{/^markdown/Id;}'
}
echo "$message" | sed '/^$/q' | grep -q -i 'markdown: true' \
&& echo "$message" | cleanHeaders | convertToHtmlMultipart | msmtp --file="$config" --account="$account" "$@" \
|| echo "$message" | cleanHeaders | msmtp --file="$config" --account="$account" "$@"
```
What is important is that I can put a fake header in my email:
```
markdown: true
```
Which will cause a conversion from markdown to HTML.
I also use [this script](https://git.jonathanh.co.uk/jab2870/Dotfiles/src/branch/master/bin/.bin/emails/convertToHtmlMultipart) to do the Markdown -> HTML conversion. Credit should go to [Francisco Lopes](https://github.com/oblitum) from whom I took it. If you wish to use this, you will need to have Pandoc installed.
The final thing to do is to make Mutt use the script and add a way to change that fake header. This can be accomplished quite easily by adding the following to the `muttrc` file.
```ini
# Use my msmtp / markdown wrapper script to send emails using the work account
set sendmail = "send-from-mutt work"
# Puts email headers in Vim
set edit_headers=yes
# Adds a header that is used to determine whether my send script should convert the markdown to html
my_hdr Markdown: false
```
Now, when you compose an email, you will get something like this in Vim
```
From: Jonathan Hodgson <jonathan.hodgson@example.com>
To: your-recipient@example.com
Cc:
Bcc:
Subject: Your Subject
Reply-To:
Markdown: false
Your Message Here
```
Now, all you would need to do is change the Markdown header from false to true in order to make an HTML email.
Make sure that you leave an empty line between the headers and your message.
## Contacts - Abook
Contacts are an important part of any email system. I use [Abook](http://abook.sourceforge.net/) which was designed for use with Mutt. Simply install it and run `abook`. You will have a terminal interface in which you can add / remove / view your contacts.
```ini
##############
# Contacts #
##############
# When looking for contacts, use this command
set query_command= "abook --mutt-query '%s'"
# Add current sender to address book
macro index,pager a "<pipe-message>abook --add-email-quiet<return>" "Add this sender to Abook"
# Auto-complete email addresses by pushing tab
bind editor <Tab> complete-query
```
The settings above make it easy to interface with Abook in Mutt. When Mutt prompts you to enter an email address, start typing a name or email address, push tab and you will be able to select the address you want.
If viewing an email, push `a` in order to add the sender to your address book.
Unfortunately, Abook cannot sync with LDAP, which is what Davmail exposes when trying to get Exchange contacts. However, the address book is a very simple plain text format that looks like this:
```ini
# abook addressbook file
[format]
program=abook
version=0.6.1
[0]
name=Bob Bobbington
email=bob.bobbington@example.com
workphone=01234 567890
[1]
name=Ed Eddington
email=ed.eddington@example.com
manager=Bob Bobbington
workphone=01234 456789
```
So, I wrote a little shell script that would make LDAP queries and put them into the format above.
```bash
id=0
for i in {a..z}; do
while read line; do
entry=$(echo "$line" | tr '§' '\n' | egrep '^(cn|mail|title|manager|mobile):' | sed 's/mail/email/' | sed 's/cn/name/' | sed 's/mobile/workphone/' | sed 's/: /=/')
[ $(echo "$entry" | sed '/^$/d' | wc -l) -gt 0 ] && echo -e "[$id]\n$entry\n" && id=$((id + 1))
done <<<"$( ldapsearch -H 'ldap://localhost:1389/' -D 'DOMAIN\USERNAME' -w "$(pass show work/email)" -b "ou=people" "mail=$i*" | awk -v RS="\n\n" -v ORS="\n" '{gsub("\n","§",$0); print $0}' )"
done
```
I won't go into too much detail about how this works because you will almost certainly need to change some of the substitutions. One point to note is that it makes a separate LDAP query for each letter. I.e. everyone starting with an A, then a B. This is because each query seemed to be limited to 100 results. I don't know if this is to do with Davmail or Exchange but seeing as I only need to run this script when I need to update the address book, I thought it would probably be okay to loop through each letter.
## Search
### Normal Search
The built in search in Mutt is quite powerful. For Vim / Less style searching, you can push the `/` key and enter a search term. Pushing enter will take you to the first instance, you can then jump again using the `n` key. To make it more like Less, you might want to add the following mapping which makes `Shift+N` jump backwards.
```ini
# Search back
bind index N search-opposite
```
Note that, by default, `Shift+N` is used for marking a message as unread. If this is something you use, you may wish to map it to something different.
The search terms can be quite complex. For example:
```
'~s "mutt" ~f ("Bob +Bobbington"|"Ed +Eddington")'
```
Will match emails from either Bob Bobbington or Ed Eddington that contain the word mutt in the subject.
For more examples of what you can do, consult the [official documentation](https://neomutt.org/guide/advancedusage.html#3-%C2%A0patterns-searching-limiting-and-tagging).
### Limiting Search
A limiting search is more like a filter in other email clients. Rather than jumping between matches, it will show you a list containing only emails that match the search term. You can use the same powerful searching features that can be used in the normal search. To perform a limit search, use the `l` key. I find these searches particularly useful when searching for an email by date:
```
~d 15/1/2020*2w ~f "boss@example.com"
```
This will show me emails from my boss 2 weeks either side of 15^th^ January
### Faster, Full Body Searches - Notmuch
One limitation of both of the previous types of search is that, by default, they won't search through the body of emails. You can force it but it is slow, particularly if you have a large mailbox.
To perform more complex searches, I use [Notmuch](https://notmuchmail.org/) which indexes emails and provides much faster, full body searches.
The homepage claims that Notmuch is still fast when dealing with the order of "millions of messages". I can't say that I have been able to test that claim, although for all of my mailboxes, it is fast!
Notmuch looks in the environment variable `NOTMUCH_CONFIG` for its configuration file, defaulting to `~/.neomutt-config`. Again, I like my home directory to stay clean so I simply set the environment variable to `~/.config/notmuch/config`.
After setting the environment variable, you can just run `notmuch setup` and it will ask you for various details.
After running the setup command, you can simply run `notmuch new` in order to index your emails. Depending on the size of your mailbox, this could take a while.
Once it is done, you can test it out on the command line. The search options here are even more powerful, for a full list type `neomutt help search-terms`. As an example though, the following will show me all messages that contain the word exam in the body that have been sent / received in the past 2 weeks.
```bash
notmuch search 'body:exam date:-2weeks..now'
```
To make this work nicely in Mutt, add the following to your configuration file:
```ini
######################
# NotMuch Settings #
######################
# All the notmuch settings are documented here: https://neomutt.org/feature/notmuch
# Points to the notmuch directory
set nm_default_url = "notmuch://$HOME/.mail/work"
# Makes notmuch return threads rather than messages
set nm_query_type = "threads"
# Binding for notmuch search
bind index \\ vfolder-from-query
```
This will allow pushing the `\` key in the index view to perform a notmuch search. It will work in a similar way to Mutt's built in limiting search in that you will be presented with a list of emails that match your search.
Another nice thing that Notmuch gives us is the ability to create Virtual mailboxes. These are similar to "Smart" mailboxes in other email clients.
```ini
virtual-mailboxes "Today's Email" "notmuch://?query=date:today"
```
## Filtering - ImapFilter
The last, but still important, part of my setup is filtering. Of course, if you are happy to do it manually, you can do so with Mutt, but who has time for that?
Again, it doesn't respect the `XDG_CONFIG_HOME` variable so I set an alias for it:
```bash
alias imapfilter="imapfilter -c \"$XDG_CONFIG_HOME/imapfilter/config.lua\""
```
This should give you a hint about why I like it. It is configured in LUA, which is a Turing complete programming language.
That means your filtering functions can be as complex as you like.
There are a couple of options that I set at the beginning of the file:
```lua
-- According to the IMAP specification, when trying to write a message
-- to a non-existent mailbox, the server must send a hint to the client,
-- whether it should create the mailbox and try again or not. However
-- some IMAP servers don't follow the specification and don't send the
-- correct response code to the client. By enabling this option the
-- client tries to create the mailbox, despite of the server's response.
-- This variable takes a boolean as a value. Default is “false”.
options.create = true
-- By enabling this option new mailboxes that were automatically created,
-- get auto subscribed
options.subscribe = true
-- How long to wait for servers response.
options.timeout = 120
```
Full details can be found in the man page [imapfilter_config(5)](https://linux.die.net/man/5/imapfilter_config).
Next, you will want to add details of your accounts. Note that ImapFilter has the ability to interface with multiple accounts and (if you wanted it to) move emails between accounts. As with everything else in this post, I will be only be setting it up with one account.
```lua
-- Gets password from pass
status, password = pipe_from('pass show work/email')
-- Setup an imap account called work
work = IMAP {
server = "localhost",
port = 1143,
username = "USERNAME",
password = password
-- ssl = auto
}
```
Again, since I am using Davmail, I will be connecting to localhost and won't be encrypting the connection. If you plan on connecting to an IMAP server that is not on localhost, you will almost certainly want to include the ssl option.
Now to start filtering. With this, you are limited only by your programming ability and your imagination, here is a simple example to get you started:
```lua
-- This function takes a table of email addresses
-- and flags messages from them in the inbox.
function flagSenders(senders)
for _, v in pairs(senders) do
messages = work["Inbox"]:contain_from(v)
messages:mark_flagged()
end
end
flagSenders {
"bob.bobbington@example.com",
"ed.eddington@example.com"
}
```
## Conclusion
I am very happy with this setup. This blog explains how the different pieces interlock, although doesn't explain each and every configuration choice I have made.
If you are interested in my full configuration, you can find them all in [my dotfiles repository](https://git.jonathanh.co.uk/jab2870/Dotfiles). For specific configuration:
* [Mutt Configuration](https://git.jonathanh.co.uk/jab2870/Dotfiles/src/branch/master/mutt/.config/mutt)
* [Davmail Configuration](https://git.jonathanh.co.uk/jab2870/Dotfiles/src/branch/master/davmail/.config/davmail/davmail.properties)
* [Mbsync Configuration](https://git.jonathanh.co.uk/jab2870/Dotfiles/src/branch/master/isync/.config/isync/mbsyncrc)
* [Msmtp Configuration](https://git.jonathanh.co.uk/jab2870/Dotfiles/src/branch/master/msmtp/.config/msmtp/config)
* [ImapFilter Configuration](https://git.jonathanh.co.uk/jab2870/Dotfiles/src/branch/master/imapfilter/.config/imapfilter/config.lua)
* [Abook Configuration](https://git.jonathanh.co.uk/jab2870/Dotfiles/src/branch/master/abook/.config/abook/abookrc)
* [Vim Configuration](https://git.jonathanh.co.uk/jab2870/vim)
* [Email Specific Scripts](https://git.jonathanh.co.uk/jab2870/Dotfiles/src/branch/master/bin/.bin/emails)
Write Preview
Loading…
Cancel
Save