PappyProxy/README.md

The Pappy Proxy

Introduction

The Pappy (Proxy Attack Proxy ProxY) Proxy is an intercepting proxy for performing web application security testing. Its features are often similar, or straight up rippoffs from Burp Suite. However, Burp Suite is neither open source nor a command line tool, thus making a proxy like Pappy inevitable. The project is still in its early stages, so there are bugs and not a ton of features, but it should be ready for the bigtime soon (I'm already trying to use it as a replacement for Burp Suite).

Contributing

I am taking any and all feature requests. If you've used Burp and had any inconvenience with it, tell me about it and I'll do everything in my power to make sure Pappy doesn't have those issues. Or even better, if you want Burp to do something that it doesn't already, let me know so that I can use it to stomp them into the dust improve my project.

If you're brave and want to try and contribute code, please let me know. Right now the codebase is a giant clusterfun which I have refactored a few times already, but I would be more than happy to find a stable part of the codebase that you can contribute to.

How to Use It

Installation

Installation requires pip or some other command that can handle a setup.py with requirements. Once the requirements are installed, you can run the pappy.py script to run the proxy. You're on your own to link it somewhere in your PATH.

$ git clone https://github.com/roglew/pappy-proxy.git
$ cd pappy-proxy
$ pip install -e .

Quickstart

Pappy projects take up an entire directory. While a full directory may seem like a dumb idea compared to storing everything in a zip file, future releases will do fun stuff like generate attack scripts or other files that need to be used by other programs on a regular basis. To start a project, do something like:

$ mkdir test_project
$ cd test_project 
$ /path/to/pappy.py
Copying default config to directory
Updating schema to version 1
Proxy is listening on port 8000
itsPappyTime> exit
$ ls
data.db            project_config.json  project_config.pyc
$ 

And that's it! The proxy will by default be running on port 8000 and bound to localhost (to keep the hackers out). You can modify the port/interface in config.json. You can list all your intercepted requests with ls, view a full request with vfq <reqid> or view a full response with vfs <reqid>. No you can't delete them yet. I'm working on it.

Adding The CA Cert to Your Browser

In order for Pappy to view data sent using HTTPS, you need to add a generated CA cert (certificate.crt) to your browser. Certificates are generated using the gencerts command and are by default stored in the same directory as pappy.py. This allows Pappy to act as a CA and MITM HTTPS connections. I believe that Firefox and Chrome ignore keychain/system certs, so you will have to install the CA cert to the browsers instead of (or in addition to) adding the cert to your keychain.

Firefox

You can add the CA cert to Firefox by going to Preferences -> Advanced -> View Certificates -> Authorities -> Import and selecting the certificate.crt file in the certs directory.

Chrome

You can add the CA cert to Chrome by going to Settings -> Show advanced settings -> HTTPS/SSL -> Manage Certificates -> Authorities -> Import and selecting the certificate.crt file in the certs directory.

Safari

For Safari (on macs, obviously), you need to add the CA cert to your system keychain. You can do this by double clicking on the CA cert and following the prompts.

Internet Explorer

I didn't search too hard for instructions on this and I don't own a Windows machine to try this, so if you have trouble, hit me up and I'll see if I can help and add real instructions. According to Google you can double-click the cert to install it to the system, or you can do Tools -> Content -> Certificates -> Trusted Root Certificates -> Import

Configuration

Configuration for each project is done in the config.json file. The file is a JSON-formatted dictionary that contains settings for the proxy. The following fields can be used to configure the proxy:

Key	Value
data_file	The file where requests and images will be stored
debug_dir (optional)	Where connection debug info should be stored. If not present, debug info is not saved to a file.
cert_dir	Where the CA cert and the private key for the CA cert are stored
proxy_listeners	A list of dicts which describe which ports the proxy will listen on. Each item is a dict with "port" and "interface" values which determine which port and interface to listen on. For example, if port=8000 and the interface is 127.0.0.1, the proxy will only accept connections from localhost on port 8000. To accept connections from anywhere, set the interface to 0.0.0.0.

The following tokens will also be replaced with values:

Token	Replaced with
{PAPPYDIR}	The directory where Pappy's files are stored

Generating Pappy's CA Cert

In order to intercept and modify requests to sites that use HTTPS, you have to generate and install CA certs to your browser. You can do this by running the gencerts command in Pappy. By default, certs are stored in the same directory as Pappy's script files. However, you can change where Pappy will look for the private key file in the config file. In addition, you can give the gencerts command an argument to have it put the generated certs in a different directory.

Command	Description
gencerts [/path/to/put/certs/in]	Generate a CA cert that can be added to your browser to let Pappy decrypt HTTPS traffic. Also generates the private key for that cert in the same directory.

Browsing Recorded Requests/Responses

The following commands can be used to view requests and responses

Command	Aliases	Description
`ls [a	]`	list, ls
vfq <id> [u]	view_full_request, vfq	[V]iew [F]ull Re[Q]uest, prints the full request including headers and data. If 'u' is given as an additional argument, it will print the unmangled version of the request.
vhq <id> [u]	view_request_headers, vhq	[V]iew [H]eaders of a Re[Q]uest. Prints just the headers of a request. If 'u' is given as an additional argument, it will print the unmangled version of the request.
vfs <id> [u]	view_full_response, vfs	[V]iew [F]ull Re[S]ponse, prints the full response associated with a request including headers and data. If 'u' is given as an additional argument, it will print the unmangled version of the response.
vhs <id> [u]	view_response_headers, vhs	[V]iew [H]eaders of a Re[S]ponse. Prints just the headers of a response associated with a request. If 'u' is given as an additional argument, it will print the unmangled version of the response.

The table shown will have the following columns:

Label	Description
ID	The request ID of that request. Used to identify the request for other commands.
Method	The method(/http verb) for the request
Host	The host that the request was sent to
Path	The path of the request
S-Code	The status code of the response
Req Len	The length of the data submitted
Rsp Len	The length of the data returned in the response
Time	The time in seconds it took to complete the request
Mngl	If the request or response were mangled with the interceptor. If the request was mangled, the column will show 'q'. If the response was mangled, the column will show 's'. If both were mangled, it will show 'q/s'.

Context

The context is a set of filters that define which requests are considered "active". Only requests in the current context are displayed with ls, and eventually contexts will be how Pappy will manage requests for group operations. By default, the context includes every single request that passes through the proxy. You can limit down the current context by applying filters. Filters apply rules such as "the response code must equal 500" or "the host must contain google.com". Once you apply one or more filters, only requests/responses which pass every active filter will be a part of the current context.

Command	Aliases	Description
fl <filter string>	filter, fl	Add a filter that limits which requests are included in the current context. See the Filter String section for how to create a filter string
fc	filter_clear, fc	Clears the filters and resets the context to contain all requests and responses. Ignores scope
fls	filter_list, fls	Print the filters that make up the current context

Filter Strings

Filter strings define a condition that a request/response pair must pass to be part of a context. Most filter strings have the following format:

<field> <comparer> <value>

Where <field> is some part of the request/response, <comparer> is some comparison to <value>. Also if you prefix a comparer with 'n' it turns it into a negation. For example, if you wanted a filter that only matches requests to target.org, you could use the following filter string:

host is target.org

field = "host"
comparer = "is"
value = "target.org"

For fields that are a list of key/value pairs (headers, get params, post params, and cookies) you can use the following format:

<field> <comparer1> <value1>[ <comparer2> <value2>]

This is a little more complicated. If you don't give comparer2/value2, the filter will pass any pair where the key or the value matches comparer1 and value1. If you do give comparer2/value2, the key must match comparer1/value1 and the value must match comparer2/value2 For example:

Filter A:
    cookie contains Session

Filter B:
    cookie contains Session contains 456

Filter C:
    cookie ncontains Ultra

Cookie: SuperSession=abc123
Matches A and C but not B

Cookie: UltraSession=abc123456
Matches both A and B but not C

List of fields

Field Name	Aliases	Description	Format
all	all	The entire request represented as one string	String
host	host, domain, hs, dm	The target host (ie www.target.com)	String
path	path, pt	The path of the url (ie /path/to/secrets.php)	String
body	body, data, bd, dt	The body (data section) of either the request or the response	String
verb	verb, vb	The HTTP verb of the request (ie GET, POST)	String
param	param, pm	Either the get or post parameters	Key/Value
header	header, hd	An HTTP header (ie User-Agent, Basic-Authorization) in the request or response	Key/Value
rawheaders	rawheaders, rh	The entire header section (as one string) of either the head or the response	String
sentcookie	sentcookie, sck	A cookie sent in a request	Key/Value
setcookie	setcookie, stck	A cookie set by a response	Key/Value
statuscode	statuscode, sc, responsecode	The response code of the response	Numeric

List of comparers

Field Name	Aliases	Description
is	is	Exact string match
contains	contains, ct	A contain B is true if B is a substring of A
containsr	containsr, ctr	A containr B is true if A matches regexp B (NOT IMPLEMENTED)
exists	exists, ex	A exists B if A is not an empty string (likely buggy)
Leq	Leq, L=	A Leq B if A's length equals B (B must be a number)
Lgt	Lgt, L>	A Lgt B if A's length is greater than B (B must be a number )
Llt	Llt, L<	A Llt B if A's length is less than B (B must be a number)
eq	eq, =	A eq B if A = B (A and B must be a number)
gt	gt, >	A gt B if A > B (A and B must be a number)
lt	lt, <	A lt B if A < B (A and B must be a number)

Scope

Scope is a set of rules to define whether Pappy should mess with a request. You define the scope by setting the context to what you want the scope to be and running scope_save. The scope is saved in data.db and is automatically restored when using the same project directory.

Any requests which don't match all the filters in the scope will be passed straight to the browser and will not be caught by the interceptor or recorded in the database. This is useful to make sure you don't accidentally do something like log in to your email through the proxy and have your plaintext username/password stored and accidentally shown to your coworkers.

Command	Aliases	Description
scope_save	scope_save	Set the current context to be the scope
sr	scope_reset, sr	Set the current context to the scope
scope_delete	scope_delete	Clear the scope (everything's in scope!)
scope_list	scope_list, sls	List all the filters that are applied to the scope

Interceptor

This feature is like Burp's proxy with "Intercept Mode" turned on, except it's not turned on unless you explicitly turn it on. When the proxy gets a request while in intercept mode, it lets you edit it with vim before it forwards it to the server. In addition, it can stop responses from the server and let you edit them with vim before they get forwarded to the browser. When you run the command, you can pass request and/or response as arguments to say whether you would like to intercept requests and/or responses. Only in-scope requests/responses will be intercepted (see Scope section)

To forward a request, edit it, save the file, then quit.

Command	Aliases	Description
ic <requests, responses, request, response, req, rsp>+	intercept, ic	Begins interception mode. Press enter to leave interception mode and return to the command prompt. Pass in request to intercept requests, response to intercept responses, or both to intercept both.

Intercept both requests and responses:
> ic requests responses
> ic req rsp

Intercept just requests:
> ic requests
> ic req

Intercept just responses:
> ic responses
> ic rsp

Be totally useless:
> ic

Repeater

This feature is like Burp's repeater (yes, really). You choose a request and Pappy will open vim in a split window with your request on the left and the original response on the right. You can make changes to the request and then run ":RepeaterSubmitBuffer" to submit the modified request. The response will be displayed on the right. This command is bound to <leader>f by default, but you can rebind it in your vimrc (I think, dunno if vim will complain if it's undefined). This command will submit whatever buffer your cursor is in, so make sure it's in the request buffer.

When you're done with repeater, run ":qa!" to avoid having to save changes to nonexistent files.

Command	Aliases	Description
rp <id>	repeater, rp	Open the specified request in the repeater

Vim Command	Keybinding	Action
RepeaterSubmitBuffer	<leader>f	Submit the current buffer, split the windows vertically, and show the result in the right window

Logging

You can watch in real-time what requests are going through the proxy. Verbosisty defaults to 1 which just states when connections are made/lost and some information on what is happening. If verbosity is set to 3, it includes all the data which is sent through the proxy and processed. It will print the raw response from the server, what it decodes it to, etc. Even if you don't run this command, all the information is stored in the dubug directory (the directory is cleared every start though!)

Command	Description
log [verbosity]	View the log at the given verbosity. Default verbosity is 1 which just shows connections being made/lost and some other info, verbosity 3 shows full requests/responses as they pass through and are processed by the proxy