Added a couple of blogs and tidy categories

master
Jonathan Hodgson 2 years ago
parent 72c1b072cc
commit 39791363fa
  1. 2
      content/blog/006-freelance-business-security.md
  2. 2
      content/blog/010-wordpress-username-enumeration.md
  3. 288
      content/blog/015-advent-of-code-day-1.md
  4. 200
      content/blog/016-multipart-emails-in-neomutt.md

@ -3,7 +3,7 @@ title: 3 Simple Steps to Improve Your Freelance Business's Security
date: 2020-04-09
description: All too often, security advice is aimed at large companies who can afford to spend hundreds or thousands of pounds on security procedures. In this blog, I will try and lay out some advice for freelancers and small businesses can follow, without breaking the bank.
tags:
- Security Advice
- Security
---
All too often, security advice is aimed at large companies who can afford to spend hundreds or thousands of pounds on security procedures. What follows is hopefully advice that anyone can follow.

@ -6,7 +6,7 @@ description: |
can protect your own site against them.
date: 2020-09-05
tags:
- Security Advice
- Security
- Websites
---

@ -0,0 +1,288 @@
---
title: "Advent of Code: Day 1"
tags:
- Advent of Code
description: Day one of the Advent of Code
date: 2021-12-01
---
So, it's December all ready. I haven't written anywhere near as many blogs this
year as I would like to. Hopefully the [Advent of
Code](https://adventofcode.com/2021/) this year will give me some things to
write about and make this years list of blogs look better.
This is the first year I am doing the Advent of Code so don't really know what
to expect. However, with day 1 done, I think I'll probably enjoy it.
If you're reading this, it is no longer day 1. I will refrain from publishing
the solutions until at least the day after they go live. However, if previous
years are anything to go by, you should still be able to view the puzzles and
have a go for yourself:
<https://adventofcode.com/2021/day/1>
I am going to use the project puzzles this year to try and learn C. I have used
it a little bit but certainly wouldn't call myself proficient. This is supposed
to be a way for me to get better so if you're looking for well formed C code by
someone who knows what they are doing, you should probably look elsewhere.
My solutions are available here:
<https://git.jonathanh.co.uk/jab2870/Advent-of-Code/src/branch/master/day1>
## Part 1
The first part of today's challenge involved taking a list of numbers and
counting the number of times a number is higher than the previous number.
The example they gave is with this list of numbers:
```
199
200
208
210
200
207
240
269
260
263
```
I put those in a text file called example.txt.
### Bash
I started by implementing it in bash. Why? Because I know bash and wanted to
make sure I had understood the problem before trying to battle with C.
```bash
#!/usr/bin/env bash
file="$1"
increasing=0
last=99999999
while read line; do
if [ "$line" -gt "$last" ]; then
increasing="$(( increasing + 1 ))"
fi
last="$line"
done < <(cat "$file")
echo "$increasing were increasing"
```
There is not a huge amount going on here. If you are at all familiar with bash,
you can probably see what is going on here. I initially set up variables to hold
the file name (passed in as the first argument), a counter for the number of
times we increase from one line to the next and the value of the last line we've
seen that I initially set to a large number.
Then we enter the loop. It is worth noting that the loop had to be of the form:
```bash
while read line; do
# Do stuff
done < <(cat "$file")
```
Rather than:
```bash
cat "$file" | while read line; do
# Do stuff
done
```
The reason for this is that pipes are executed in a sub shell meaning they can't
change variables in the initial shell. In other words, with the second method,
we wouldn't be able to update the increasing and last variables from inside the
loop.
Inside the loop, I check to see if the current value is larger than the previous
value and if it is I increment the `increacing` variable by 1. I then set the
`last` variable so it can be used for the next iteration.
Finally, once all the lines have been done, I print out the number of lines that
were increasing.
### C
So, in a surprise to nobody, my C implementation is not as short as the bash
one. However, it wasn't as bad as I thought it would be.
Step 1 was trying to work out how to read a file in to c one file at a time.
Given that I knew the format of the lines, I was able to use fscanf to get each
of them.
Although the syntax is slightly different, the logic is very similar between the
Bash and C implementations.
```c
#include <stdio.h>
int main( int argc, char *argv[] ){
if( argc == 2 ) {
unsigned int last = ~0;
unsigned int increasing = 0;
unsigned int value;
// Read the values into an array
FILE *fp=fopen(argv[1], "r");
while (!feof (fp)){
fscanf(fp, "%d\n", &value);
if ( value > last )
increasing++;
last = value;
}
fclose(fp);
printf( "%d were increasing\n", increasing );
return 0;
} else {
printf("You need to provide a file\n");
return 1;
}
}
```
## Part 2
The extension for part 2 was to change the program so that rather than comparing
lines with the previous line, the sum of 3 consecutive lines should be compared.
```
199 A
200 A B
208 A B C
210 B C D
200 E C D
207 E F D
240 E F G
269 F G H
260 G H
263 H
```
So, the sum of the numbers marked with A would be compared to the sum of those
marked with B. Then B with C and so on.
I think the intended way of doing this is to first create an array type
structure of sums then use the code from part 1 to work out how many of these
sums are increasing. However, a short cut can be taken. The sum of the next
group of numbers is always going to be the sum of the previous numbers plus the
next digit, minus the first. It follows therefore that we can compare the groups
by comparing the differences of every 3 numbers rather than adjacent numbers.
So, for example
$$ A = 199 + 200 + 208 $$
$$ B = 200 + 208 + 210 $$
We can confirm that group B increases from group A because 210 is greater than
199.
In order to do this, I want to be able to easily reference individual entries by
their index so rather than doing the comparisons on the first pass through the
file, I first create an array. I can then compare an entry with the entry 3
before it.
### Bash
```bash
#!/usr/bin/env bash
file="$1"
increasing=0
readarray -t "arr" < "$file"
length="${#arr[@]}"
for i in $(seq 3 "$length"); do
curr="${arr[$i]}"
prev3="${arr[$((i - 3))]}"
if [ -n "$curr" ] && [ "$curr" -gt "$prev3" ]; then
increasing="$((increasing + 1))"
fi
done
echo "$increasing were increacing"
```
You'll see that bash has a nice built-in for turning a file into an array.
However, any bonus points it it might get for that, it looses by having the
worst syntax ever for getting the length of that array. Who thought that
`${#arr[@]}` would ever be a good idea‽
### C
```c
#include <stdio.h>
int countLines(char filename[]){
FILE *fp;
fp=fopen(filename, "r");
long int lines =0;
if ( fp == NULL ) {
return -1;
}
while (EOF != (fscanf(fp, "%*[^\n]"), fscanf(fp,"%*c")))
++lines;
fclose(fp);
return lines;
}
int main( int argc, char *argv[] ){
if( argc == 2 ) {
// 1 less because file ends with trailing new line
int lines=countLines(argv[1]);
int values[lines];
int line=0;
FILE *fp=fopen(argv[1], "r");
int increasing = 0;
// Read the values into an array
while (!feof (fp)){
fscanf(fp, "%d\n", &values[line]);
if ( (line >= 3) && (values[line] > values[line-3]) ){
increasing++;
}
line++;
}
fclose(fp); // closing file
printf( "Increasing: %d\n", increasing );
return 0;
} else {
printf("You need to provide a file\n");
return 1;
}
}
```
So this is obviously longer than the bash version but mainly because there isn't
a nice way of putting a text file into an array. I have to first iterate through
the file to determine how many lines there are, then allocate that much space for
the array, then re-run through the file to get the values.
Initially, I then had a third loop where I iterated through the newly created
array to determine if there was an increase. However, I then realised I could do
that whilst populating the array.

@ -0,0 +1,200 @@
---
title: "Multipart Emails in Neomutt"
tags:
- Mutt
description: Mutt now supports multipart email. I guess it will be easy to set it up right?
date: 2022-05-27
---
It recently came to my attention that mutt now supports sending multipart
emails. I thought that this would mean that in half an hour or so I would have
html emails working. Turns out, I was wrong. What instead happened was weeks of
trial and error and reading RFCs.
I now have a system I am happy with. I write an email in markdown and mutt
(along with some surrounding scripts) will convert that markdown to html, attach
inline images and create a multipart email.
## A note about HTML emails
If you do need to send HTML emails, please spare a thought for your recipient;
it is not just "weird" terminal email client users that could suffer. According
to the [National Eye Institute](https://www.nei.nih.gov/learn-about-eye-health/eye-conditions-and-diseases/color-blindness),
one in twelve men are colour blind; so please don't use only colour to
distinguish items. Additionally, many people suffer from vision-loss blindness,
who are likely to use screen readers or braille displays. Complex layouts or
heavy use of images are going to give these people a poor experience.
That being said, HTML emails can be used for aesthetic purposes, whilst still
providing a plain text version for those who want it. That is the method I
suggest adopting if you need html emails, and the method this blog post will
describe.
## Multipart / Related emails
To begin with, we need to understand a little bit about how emails are
structured. Below is an example tree structure of a standard email.
```
Multipart Related
├─>Multipart Alternative
│ ├─>Plain Text Email
│ └─>HTML Email
└─>Inline Image Attachment
Non-Inline Attachment
```
Starting at the lowest level, we see a plain text email and an HTML email. These
are both wrapped in a multipart **alternative** wrapper. This tells email
clients receiving the email that they are alternative versions of the same
document. The email client will normally choose which to display based on the
mime type and user preferences.
The multipart alternative wrapper and an image attachment are then wrapped in a
multipart **related** wrapper. This tells the email client that the contents are
related to one another, but not different version of the same document. This is
where inline images are attached.
Finally, there is another attachment that is outside of the multipart related
wrapper. This will show up as another attachment but cannot be displayed inline.
## Neomutt Configuration
The conversion from markdown to html will be handled by an external script. It
will create files and instruct mutt to attach them.
We can start with the following:
```vimrc
macro compose Y "<first-entry>\
<pipe-entry>convert-multipart<enter>\
<enter-command>source /tmp/neomutt-attach-macro<enter>
```
We specify a macro to run when `Y` is pushed. First, we select the first entry.
This is in case we have attached anything manually, the first entry should be
the markdown file.
We then pipe the selected entry (the markdown file) to an external script, in
this case a bash script called `convert-multipart`. Finally we source a file
called `/tmp/neomutt-commands`. This will be populated by the script and will
allow us to group and attach files inside neomutt.
## Converting to HTML
Let's start with a simple pandoc conversion.
```bash
#!/usr/bin/env bash
commandsFile="/tmp/neomutt-commands"
markdownFile="/tmp/neomutt-markdown"
htmlFile="/tmp/neomutt.html"
cat - > "$markdownFile"
echo -n "push " > "$commandsFile"
pandoc -f markdown -t html5 --standalone --template ~/.pandoc/templates/email.html "$markdownFile" > "$htmlFile"
# Attach the html file
echo -n "<attach-file>\"$htmlFile\"<enter>" >> "$commandsFile"
# Set it as inline
echo -n "<toggle-disposition>" >> "$commandsFile"
# Tell neomutt to delete it after sending
echo -n "<toggle-unlink>" >> "$commandsFile"
# Select both the html and markdown files
echo -n "<tag-entry><previous-entry><tag-entry>" >> "$commandsFile"
# Group the selected messages as alternatives
echo -n "<group-alternatives>" >> "$commandsFile"
```
The above bash script will create an html file using pandoc, and create a file
of neomutt commands. This instructs neomutt to attach the html file, set its
disposition, and group the markdown and html files into a "multipart
alternatives" group.
Neomutt's attachment view should look something like this.
```
I 1 <no description> [multipa/alternativ, 7bit, 0K]
- I 2 ├─>/tmp/neomutt-hostname-1000-89755-7 [text/plain, 7bit, us-ascii, 0.3K]
- I 3 └─>/tmp/neomutt.html [text/html, 7bit, us-ascii, 9.5K]
```
## Inline attachments
The next part of the puzzle is inline attachments. These need to be attached and
then grouped within a multipart related group.
To reference the file from within the html email, each inline image needs a
unique cid. I use md5 sums for this. They are not cryptographically secure, but
for the purposes of generating unique strings for images in an email, they are
fine.
```bash
grep -Eo '!\[[^]]*\]\([^)]+' "$markdownFile" | cut -d '(' -f 2 |
grep -Ev '^(cid:|https?://)' | while read file; do
id="cid:$(md5sum "$file" | cut -d ' ' -f 1 )"
sed -i "s#$file#$id#g" "$markdownFile"
done
```
We loop through all the images in the markdown file, and replace the paths for
cids (assuming they are not already cids or remote images).
As the markdown has changed, we need to attach the new one and detach the old.
```bash
if [ "$(grep -Eo '!\[[^]]*\]\([^)]+' "$markdownFile" | grep '^cid:' | wc -l)" -gt 0 ]; then
echo -n "<attach-file>\"$markdownFile\"<enter><first-entry><detach-file>" >> "$commandsFile"
fi
```
To attach the images, we loop through the original file and add to the file
neomutt sources. Neomutt will be instructed to attach, set the disposition, set
the content ID and tag the image.
```bash
grep -Eo '!\[[^]]*\]\([^)]+' "${markdownFile}.orig" | cut -d '(' -f 2 |
grep -Ev '^(cid:|https?://)' | while read file; do
id="$(md5sum "$file" | cut -d ' ' -f 1 )"
echo -n "<attach-file>\"$file\"<enter>" >> "$commandsFile"
echo -n "<toggle-disposition>" >> "$commandsFile"
echo -n "<edit-content-id>^u\"$id\"<enter>" >> "$commandsFile"
echo -n "<tag-entry>" >> "$commandsFile"
done
```
```bash
if [ "$(grep -Eo '!\[[^]]*\]\([^)]+' "$markdownFile" | grep '^cid:' | wc -l)" -gt 0 ]; then
echo -n "<first-entry><tag-entry><group-related>" >> "$commandsFile"
fi
```
Finally, if there were any images attached, we select the first entry (the
multipart alternative we've already created), tag it and mark everything tagged
as multipart related.
```
I 1 <no description> [multipa/related, 7bit, 0K]
I 2 ├─><no description> [multipa/alternativ, 7bit, 0K]
- I 3 │ ├─>/tmp/neomutt-markdown [text/plain, 7bit, us-ascii, 0.3K]
- I 4 │ └─>/tmp/neomutt.html [text/html, 7bit, us-ascii, 9.5K]
I 5 └─>/tmp/2022-05-27T15-02-20Z.png [image/png, base64, 0.5K]
```
At this point, the user is free to attach additional, non inline documents as
normal. This email should be good for both text based and graphical email
clients.
For the full source changes, see [this commit](https://git.jonathanh.co.uk/jab2870/Dotfiles/commit/08af357f4445e40e98c715faab6bb3b075ec8afa).
Loading…
Cancel
Save