jab2870/Website
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Update build system and content

Browse source
This commit is contained in:
Jonathan Hodgson 2020-03-29 17:15:00 +01:00
parent aa21b867b3
commit 6f1abd3fe5
44 changed files with 1406 additions and 92 deletions
Download patch file Download diff file Expand all files Collapse all files

2
content/001-fuzzy-search-hashcat-modes.md
View file

@ -198,7 +198,7 @@ printf "\e[5n"
These are special escape sequences that are for determining if the terminal is functioning correctly. When `\e5[n` is sent to the terminal, it should respond with `\e[0n`. We listen for this and ask bash to redraw the current line. For more on this, check `man console_codes`.
Unfortunately, this relies on a terminal that supports DSR sequences. Most do, but not all. If you experience a problem where you select a hash and the whole line is replaced by the hash you chose, it is very likely that your terminal emulator doesn't support these codes. A possible work around is to push ctrl-l which will clear your terminal screen and will also cause the current line to be re-drawn. If you do have this problem, you might be interested in switching to Zsh. The ZSH solution doesn't rely on these DSR codes.
Unfortunately, this relies on a terminal that supports DSR sequences. Most do, but not all. If you experience a problem where you select a hash and the whole line is replaced by the hash you chose, it is very likely that your terminal emulator doesn't support these codes. A possible work around is to push ctrl-l which will clear your terminal screen and will also cause the current line to be re-drawn. If you do have this problem, you might be interested in [switching to Zsh](/blog/the-switch-to-zsh.html). The ZSH solution doesn't rely on these DSR codes.
### ZSH

64
content/002-the-switch-to-zsh.md Normal file
View file

@ -0,0 +1,64 @@
---
title: The switch to ZSH
tags:
- Pentesting
- Linux
description: ZSH is great shell but in this blog I explain some of the annoyances I've had with it, as well as some of the benefits.
date: 2020-02-29
---
[ZSH](https://www.zsh.org/) is an alternative to shells like sh, bash or fish. I have nothing against any of them but I choose to use ZSH. There are hundreds of articles that cover the benefits, but very few that I have found also cover the downsides. In this blog, I will discuss some of the reasons I use ZSH and some of the annoyances I face. I work as a penetration tester, so some of the annoyances are specific to that. I am not going to pretend that this is a comprehensive list, it is the things that made my decision.
To start with, I'll talk about some of the features that make my life easier, the pros. It should come as no surprise that, as I use ZSH on a daily basis, the pros outway the cons for me; that is not to say however that I think zsh is perfect. It isn't.
## PRO: Bash Familiarity
ZSH is not strictly compatible with all BASH or SH features, (more on that later) but it is a lot more similar than some alternative shells. For me, this is important because I spend a lot of time SSHed into machines that have BASH or SH installed. It is often not possible for me to install ZSH so having a similar working environment is hugely beneficial.
## PRO: Tab Completion
The tab completion on ZSH is hugely superior to that of bash. As well as giving descriptions of arguments, you can use the tab or arrow keys to cycle through each option
![ZSH tab completion](/assets/switch-to-zsh/zsh-tab-complete.gif)
Obviously, the tab completion is better with some tools than others, but zsh is able to fall back to the completion you would get in bash, with the bonus of being able to cycle through options.
## PRO: Syntax Highlighting
We all use syntax highlighting in our code editors, why not also have it in our shell? You can see from the screenshot below that as soon as a valid command is input, you get feedback by it turning green. If an invalid command is input, it is red. It saves a lot of the embarrassing typos where you type `gti` instead of `git`, or `sl` instead of `ls`.
![ZSH syntax highlighting](/assets/switch-to-zsh/zsh-syntax-highlighting.gif)
It should be noted that this isn't a native ZSH feature, but a [plugin](https://github.com/zsh-users/zsh-syntax-highlighting).
## PRO: Autosuggest
Another plugin is the [autosuggestion](https://github.com/zsh-users/zsh-autosuggestions) plugin, it adds the last command from your history that matches what you have already typed. You can then use the right arrow key to accept the suggestion. This makes writing long but common commands much quicker.
![ZSH autosuggestion](/assets/switch-to-zsh/zsh-autosuggest.gif)
## PRO and CON: ZLE
Zsh uses a tool called zle (zsh line editor) as an alternative to readline that most other shells use. Readline is also the tool used by the `read` command that many shell scripts use for getting user input.
ZLE is, subjectively, much more advanced than readline. Personally, my favourite feature is its superior vim emulation, meaning I can edit the command I am currently writing with the same keys I use to edit almost everything else I write (including this blog).
Hoverer, coming to my first annoyance, as it doesn't use readline, your readline configuration isn't read. This makes sense but readline is everywhere. Metasploit, pappy and bash all use readline as their line editor. You can configure them all in one place: `.inputrc`, that is all of them except zsh/zle.
That is not to say that you cannot configure zle to do the same things, you absolutely can, but it means making those configuration changes in multiple places.
## CON: /dev/tcp (or lack thereof)
`/dev/tcp/*` is a non-standard feature that bash added. ZSH, does not implement this. There is a tool called `ztcp` which, like zle, is more powerful than the bash alternative but, in my opinion, not as easy.
That being said, I only really use that feature when I don't have netcat available, that is when I am on a remote machine. When that is the case, I am normally in a bash environment anyway.
## CON: stty raw -echo
When you get a netcat reverse shell, one of the first things you want to do is upgrade it to a fully interactive shell. When using Bash, you would do this by pressing Ctrl+z to background the current process, then typing `stty raw -echo` then typing `fg` and pushing enter twice.
If you try this in ZSH, it won't work. Why? Because ZSH's implementation resets the `-echo` option every time the prompt is drawn. If you're interested in why, it is discussed [here](https://www.zsh.org/mla/workers/2014/msg00870.html). But, what that means, is we need to foreground the shell, before the prompt is drawn. The solution:
```zsh
stty raw -echo; fg <enter><enter>
```

139
content/003-exchange-mutt.md Normal file
View file

@ -0,0 +1,139 @@
---
title: Using Mutt with Exchange
tags:
- Linux
- Vim
- Mutt
- FZF
description: Like many organisations, the company I work at uses an Exchange server for email and calendar. My preferred email client is NeoMutt. I use Davmail to make these two work together.
date: 2020-03-10
---
[Mutt](http://www.mutt.org/) is an extremely powerful, vim like email client. Unfortunately, like many, the company I work for uses Exchange for email and calenders. As a Linux user, I can't (and don't want) to install outlook and I don't want to have to open a web browser, every time I want to check an email. My mission was to find a way to manage my work emails using Mutt.
For the purpose of this blog, I assume that you cannot connect to the Exchange server directly with IMAP or SMTP. If you are unsure, check at `yourdomain.com/owa/#path=/options/popandimap`. If you can use IMAP natively, use it.
## DavMail
The solution to my dilemma was [Davmail](http://davmail.sourceforge.net/). This works as a middle~~man~~ (person) between the Exchange server and any standards compliant email client, in my case Mutt. Davmail will translate IMAP and SMTP requests to the Exchange equivalents, as well as LDAP for contacts. The result: you can use whichever email client you like to manage your cooperate emails.
## Configuring DavMail
The [setup help](http://davmail.sourceforge.net/linuxsetup.html) for DavMail is good, so I am not going to try to reinvent the wheel. I suggest you read the documentation.
If you find it easier to configure the server using a GUI, you can. Run `davmail` and fill in the boxes. Nothing groundbreaking here.
If you would like to run Davmail without a GUI in the future, simply change the `davmail.server` property from `false` to `true` in the configuration file (by default this is in your home directory).
```bash
sed -i '/davmail\.server=/s/false/true/' .davmail.properties
```
Now, you can start the server with
```bash
nohup davmail davmail.properties &
```
## Configuring Mutt
You can now connect to the locally running Davmail server as though it was an IMAP / SMTP server. Unless you have configured it differently, you will need to connect using port 1143 for IMAP and 1025 for SMTP.
Your credentials will be you Active Directory username and password and, with that, you should be able to send and receive emails using the exchange server.
```muttrc
set folder = "imap://127.0.0.1:1143/"
set imap_user = "domain/user.name"
set imap_pass = "password"
set spoolfile = +INBOX
mailboxes = +INBOX
set realname = 'Your Name'
set from = 'your-email@company.com
set use_from = yes
set smtp_url="smtp://$imap_user:$imap_pass@127.0.0.1:1025"
set ssl_force_tls = no
set ssl_starttls = no
```
Note here I have disabled `tls` authentication. This is for the communication between the mail client and the local server, so the information never leaves the computer.
Also, you might prefer not to put your password in a configuration file in clear text. You can run shell commands to populate variables so, in my case, I use the command line password manager [Pass](https://www.passwordstore.org/).
This allows me to change the `imap_pass` line to
```
set imap_pass = `pass show domainCreds | head -n 1`
```
## Adding contacts
One potential solution is to use [this perl script](https://github.com/namato/dotfiles/blob/e7ce282c8883dd1971356a0ea53c75b47105c8fa/scripts/ldap.pl). After setting Mutt's `query_command`, it will query the LDAP server to search for contacts when typing in the recipient fields.
It works but, in my opinion, has 2 shortcomings.
Firstly, it relies on an internet connection, meaning it is not possible to compose an email offline to send it when you get online. Admittedly, it is rare that I will be doing much with emails offline, but I like to have the option.
Secondly, and more importantly for me, the lookup is slow, particularly if you have to connect through a slow VPN.
### Local contacts
Storing contacts locally solves solves both problems, with the cost of using up disk space.
I created a contacts folder and ran the following in bash.
```bash
for i in {a..z}; do
ldapsearch -H 'ldap://localhost:1389/' -D 'DOMAIN\USERNAME' -w 'PASSWORD' -b ou=people "mail=$i*" > "$i.people"
done
```
Change the credentials as needed and you will end up with a file for each letter of the alphabet. I did this because the Exchange server in question wouldn't return more than 100 results at a time. Making a request for each letter meant I didn't have to work out how to deal with paging with Davmail.
To use this in Mutt, I created a script called lookup that takes all the contacts and puts them in the form `email<tab>Name<tab>title`.
```bash
#!/usr/bin/env bash
cat $HOME/Contacts/*.people | awk -v RS="\n\n" -v ORS="\n" '{gsub("\n","§",$0); print $0}' | while read line; do
echo "$line" | tr '§' '\n' | egrep '^(cn|mail|title):' | cut -d':' -f2- | tr '\n' '\t'
done
```
Then in my Mutt configuration, I have the line:
```muttrc
set query_command = "/home/jonathan/Contacts/lookup | grep '%s'"
```
This means I can start typing a name or email address, press `ctrl`+`t`, and you can select a contact.
## Contact lookup in VIM
If I am writing an email to more than one person, I find it much easier to add the recipients in vim using FZF. To make this happen, I pair the [fzf.vim](https://github.com/junegunn/fzf.vim) plugin with the following snippet in my vimrc file:
```vim
function! s:make_email_list(lines)
let l:emails = []
"return type( a:lines )
for email in a:lines
let l:address = substitute(split( email, "\t" )[0], " ","","" )
let l:name = split( email, "\t" )[1]
let l:emails = add( l:emails, l:name . " <" . l:address . ">" )
endfor
return join(l:emails, ', ')
endfunction
inoremap <expr> <c-c> fzf#vim#complete({
\ 'source': '$HOME/Contacts/lookup',
\ 'reducer': function('<sid>make_email_list'),
\ 'options': '--multi',
\ 'down': '30%' })
```
This uses the same lookup script as before, but this time it is passed to FZF. I can fuzzy search for contacts by pressing `ctrl`+`c`.
This is only useful if you have `edit_headers` set in your `.muttrc` file which allows you to set and modify the email headers in your editor.

156
content/004-tools-that-make-my-life-easier.md Normal file
View file

@ -0,0 +1,156 @@
---
title: Terminal tools that make my life easier
tags:
- Linux
- Vim
- FZF
description: I spend a lot of time on the command line, and honestly believe that I am more productive because of that. Here are some of the tools that help me out on a day to day basis.
date: 2020-03-20
---
The terminal is often seen as a "hipster" tool that people only use to make their lives difficult. I disagree with that sentiment. I can honestly say that I am more productive due to the efficiency boost I get from using command line tools.
That is not to say I do everything in the terminal, I don't. There are graphical tools that I rely on and I would struggle to do a days work without. I am a proponent of people using the tools that they want to use. The purpose of this post is to shed light on some of the tools I use, not to evangelise you into using them.
Many of these tools come with a learning curve. However, in my case, the time I put into learning them has been more than made up for in the increased productivity I've enjoyed since learning them.
## The shell
This is often overlooked, but is arguably the most important tool that you use in the terminal. It is the program that you interact with when you first open a terminal. The most common shell on unix-y operating systems is [Bash](https://www.gnu.org/software/bash/). I don't want this to become a comparison of different shells, most of them are similar in function, but I would encourage terminal users to learn the basics of their shell as a programming language. Being able to `cd` into folder is not really much faster than double clicking on a folder in your GUI file manager. File manipulation on the terminal gets much faster when you can say create 3 folders called `folder1`, `folder2` and `folder3`, then in each of those folders create files called `file1`, `file2` and `file3`.
```bash
mkdir folder{1..3}
touch folder{1..3}/file{1..3}
```
This results in
```
.
├── folder1/
│   ├── file1
│   ├── file2
│   └── file3
├── folder2/
│   ├── file1
│   ├── file2
│   └── file3
└── folder3/
├── file1
├── file2
└── file3
```
Or, how about pinging a list of ip addresses 3 times, and outputting the result to file with a sensible name.
```bash
for ip in "10.0.0.1" "10.0.0.2" "10.0.0.3" "10.0.0.4" ; do
ping -c 3 "$ip" > "$ip.ping"
done
```
This results in
```
.
├── 10.0.0.1.ping
├── 10.0.0.2.ping
├── 10.0.0.3.ping
└── 10.0.0.4.ping
```
Using your shell as a programming language could easily make a blog post by itself. When that is paired with the fact that you can write these as scripts in separate files (shell scripts) and call them on demand as needed makes this the single largest productivity boost for me. A lot of common tasks can be automated with shell scripts. Here are a few of my favourite:
* [Organise photos into portrait or landscape folders](https://git.jonathanh.co.uk/Jab2870/dotfiles/blob/master/bin/.bin/images/lanPortImages)
* [Add up the length of all audio tracks in a folder](https://git.jonathanh.co.uk/Jab2870/dotfiles/blob/master/bin/.bin/albumDuration)
* [Encode text in multiple different ways at once](https://git.jonathanh.co.uk/Jab2870/dotfiles/blob/master/bin/.bin/encode)
## Vim
[Vim](https://www.vim.org/) is the text editor I use, and I use it a lot. For everything from programming, emails, letters, reports and this blog. I don't do this to punish myself, as many assume, but because it makes my life easier.
There is a misconception that Vim is hard to use. In truth, anyone can use Vim. Type `vim <filename>` into your terminal to open a file, push the `i` key to enter insert mode and you should be able to do what you need. The arrow keys will move your cursor and the letter keys will type letters. There is nothing groundbreaking there. Apart from mouse support, you have a basic text editor that anyone can use. This obviously isn't the whole story, but I hope it goes some way to shift the misconception that it is hard. Rather than hard, Vim is powerful. It has many features that can help you, but you don't need to learn all of them before you start.
Modal editing is probably Vim's distinguishing feature. The concept is simple: change what the keyboard keys do based on the mode we are in. Our smartphones all do something similar: when we are typing a text message, we see a full keyboard; if we are typing a phone number, we just get numbers. Modal editing in Vim is very similar. Obviously, it can't change the keys on the keyboard, but it can change what they do.
![Android keyboard modes. Qwerty on the left, number on the right](/assets/tools/android-keyboard-modes.png "Android Keyboard Modes")
Vim has 4 main modes:
* Insert mode - this is the mode that is most like all other text editors, if you push the `a` key, an `a` is inserted where your cursor is
* Normal mode - this is the mode you land in when you first enter Vim. It is optimised for moving around the document and editing specific parts of a document
* Visual mode - this mode allows you to select text to perform actions on
* Command mode - this mode if for running more complex commands on either a document or a selection.
If you only take one thing away from this blog post, I hope it is this: Vim isn't as hard as a lot of Vim users would like you to think it is.
If your interest has been peaked, I would encourage you to open a terminal and type `vimtutor`. It will walk you through some of the basics of normal mode and insert mode. From there, you should be able to start using Vim at a basic level. You can then pick things up from reading Vim's excellent help pages or, if you prefer, there are hundreds of blog posts and videos on the internet that walk you through some of the more advanced aspects. Remember, you don't need to master all of Vim's features to use Vim.
## Colemak
I spend a lot of my day typing, and I hope to continue to be typing for some time to come. [Colemak](https://colemak.com/) is an alternative keyboard layout that is designed to be faster and more ergonomic than Qwerty. It is not strictly a terminal tool, but since the keyboard is the primary method of input when using terminal tools, I feel I can justify including it.
If you are an English speaker, you are probably using a Qwerty keyboard, named because the top row spells qwerty with the first 6 keys. It was designed when typists used typewriters and was sold as an improvement over alphabetical keyboards in 1873! One of its improvements over the old keyboard was that the layout made the typewriter's "arms" jam far less often. My keyboard today doesn't have arms to jam, so using a keyboard layout that was designed to reduce jams seems pointless.
If you haven't ever looked into alternative keyboard layouts, I should warn you - it is quite a rabbit hole. The three most common non-querty layouts are, at the time of writing:
* [Dvorak](https://www.dvorak-keyboard.com/)
* [Colemak](https://colemak.com)
* [Workman](https://workmanlayout.org/)
## FZF
This is a relatively recent find for me. However, it has soon became one of the tools I use most. It follows perfectly one of the key Unix philosophies of doing one thing, and doing it well. In its most simple form, [FZF](https://github.com/junegunn/fzf) is a tool for searching through a list interactively. It is powerful because the list you give it, can be anything. Here are a few examples I use it for:
![Searching through files](/assets/tools/fzf-files.gif)
![Search through wordlists](/assets/tools/fzf-wordlists.gif)
![Killing Processes](/assets/tools/fzf-process.gif)
![[Completing hashcat modes](/blog/fuzzy-search-hashcat-modes.html)](/assets/fuzzy-search-hashcat-modes/hashcat-fzf.gif)
## FD
[Find](https://www.gnu.org/software/findutils/manual/html_mono/find.html) is one of those tools on Unix-y systems that most people only scratch the surface of. It is incredibly powerful, but for most use cases, it is overkill.
Enter, [FD](https://github.com/sharkdp/fd). It doesn't do everything find does, but it does do what most people use find for. As an added bonus, [it is fast](https://github.com/sharkdp/fd#benchmark).
As an example, lets say we were looking for a file who's name contains the word "test", although we can't remember if it is capital or lowercase.
**Find**
```bash
find -iname '*test*'
```
**Fd**
```bash
fd test
```
So, not only is the command shorter, it defaults to a "smart case" search. Basically, if you type in all lowercase, it will do a case-insensitive search; if you put a capital in there, it assumes you know the case and will search case-sensitively.
Also, by default, it will ignore files in your `.gitignore` and hidden files / directories. Of course, you can change this functionality but for 90% of use cases, this is helpful.
## RipGrep
As you might have guessed, [RipGrep](https://github.com/BurntSushi/ripgrep) is to [grep](https://www.gnu.org/software/grep/), what FD is to find. A fast alternative with defaults that are more helpful 90% of the time.
Similarly to Fd, RipGrep has smart case, interprets `.gitignore` files and will ignore hidden files.
## Bat
Anyone who has used the terminal a couple of times knows about the `cat` command. [Bat](https://github.com/sharkdp/bat) is very similar to cat, but with a few big advantages.
![Bat with syntax highlighting](/assets/tools/bat.png)
If you are not piping the output to another file, Bat will syntax highlight your code, it will add line numbers and, if you're in a git repo, it will show you which lines have changed since your last commit.
---
This is by no means an extensive list. There are many tools I use that I have not listed here, but these are the ones that have made the biggest difference to my workflow.