Version 0.2.0
This commit is contained in:
Rob Glew
parent
312b985229
commit
26376eaaec
43 changed files with 4699 additions and 2668 deletions
232
README.md
232
README.md
|
|
@ -4,7 +4,7 @@ The Pappy Proxy
|
|||
|
||||
Introduction
|
||||
------------
|
||||
The Pappy (**P**roxy **A**ttack **P**roxy **P**rox**Y**) Proxy is an intercepting proxy for performing web application security testing. Its features are often similar, or straight up rippoffs from [Burp Suite](https://portswigger.net/burp/). However, Burp Suite is neither open source nor a command line tool, thus making a proxy like Pappy inevitable. The project is still in its early stages, so there are bugs and only the bare minimum features, but it should be able to do some cool stuff soon (I'm already using it for real work).
|
||||
The Pappy (**P**roxy **A**ttack **P**roxy **P**rox**Y**) Proxy is an intercepting proxy for performing web application security testing. Its features are often similar, or straight up rippoffs from [Burp Suite](https://portswigger.net/burp/). However, Burp Suite is neither open source nor a command line tool, thus making a proxy like Pappy inevitable. The project is still in its early stages, so there are bugs and only the bare minimum features, but it can already do some cool stuff.
|
||||
|
||||
Contributing
|
||||
------------
|
||||
|
|
@ -12,6 +12,8 @@ Contributing
|
|||
|
||||
If you're brave and want to try and contribute code, please let me know. Right now the codebase is kind of rough and I have refactored it a few times already, but I would be more than happy to find a stable part of the codebase that you can contribute to.
|
||||
|
||||
Another option is to try writing a plugin. It might be a bit easier than contributing code and plugins are extremely easy to integrate as a core feature. So you can also contribute by writing a plugin and letting me know about it. You can find out more by looking at [the official plugin docs](https://roglew.github.io/pappy-proxy/pappyplugins.html).
|
||||
|
||||
How to Use It
|
||||
=============
|
||||
|
||||
|
|
@ -26,7 +28,7 @@ $ pip install .
|
|||
|
||||
Quickstart
|
||||
----------
|
||||
Pappy projects take up an entire directory. Any generated scripts, exported responses, etc. will be placed in the current directory so it's good to give your project a directory of its own. To start a project, do something like:
|
||||
Pappy projects take up an entire directory. Any generated scripts, exported responses, plugin data, etc. will be placed in the current directory so it's good to give your project a directory of its own. To start a project, do something like:
|
||||
|
||||
```
|
||||
$ mkdir test_project
|
||||
|
|
@ -34,7 +36,7 @@ $ cd test_project
|
|||
$ pappy
|
||||
Copying default config to directory
|
||||
Proxy is listening on port 8000
|
||||
itsPappyTime> exit
|
||||
pappy> exit
|
||||
$ ls
|
||||
data.db project_config.json
|
||||
$
|
||||
|
|
@ -51,7 +53,7 @@ Example:
|
|||
$ pappy -l
|
||||
Temporary datafile is /tmp/tmpw4mGv2
|
||||
Proxy is listening on port 8000
|
||||
itsPappyTime> quit
|
||||
pappy> quit
|
||||
Deleting temporary datafile
|
||||
$
|
||||
```
|
||||
|
|
@ -173,6 +175,7 @@ The context is a set of filters that define which requests are considered "activ
|
|||
|:--------|:------------|:---|
|
||||
| `f <filter string>` | filter, fl, f |Add a filter that limits which requests are included in the current context. See the Filter String section for how to create a filter string |
|
||||
| `fc` | filter_clear, fc | Clears the filters and resets the context to contain all requests and responses. Ignores scope |
|
||||
| `fu` | filter_up, fu | Removes the most recently applied filter |
|
||||
| `fls` | filter_list, fls | Print the filters that make up the current context |
|
||||
| `filter_prune` | filter_prune | Delete all the requests that aren't in the current context from the data file |
|
||||
|
||||
|
|
@ -259,6 +262,14 @@ Matches both A and B but not C
|
|||
| gt | gt | A gt B if A > B (A and B must be a number) |
|
||||
| lt | lt | A lt B if A < B (A and B must be a number) |
|
||||
|
||||
### Special form filters
|
||||
A few filters don't conform to the field, comparer, value format. You can still negate these.
|
||||
|
||||
| Format | Aliases | Description |
|
||||
|:--|:--|:--|
|
||||
| before <reqid> | before, bf, b4 | Filters out any request that is not before the given request. Filters out any request without a time. |
|
||||
| after <reqid> | after, af | Filters out any request that is not before the given request. Filters out any request without a time. |
|
||||
|
||||
Scope
|
||||
-----
|
||||
Scope is a set of rules to define whether Pappy should mess with a request. You define the scope by setting the context to what you want the scope to be and running `scope_save`. The scope is saved in the data file and is automatically restored when using the same project directory.
|
||||
|
|
@ -346,7 +357,7 @@ $ ls -l
|
|||
In this case we have a `blank`, `hackthensa`, `testgen`, and `test` macro. A macro script is any python script that defines a `run_macro(args)` function and a `MACRO_NAME` variable. For example, a simple macro would be:
|
||||
|
||||
```
|
||||
--- macro_print.py
|
||||
### macro_print.py
|
||||
|
||||
MACRO_NAME = 'Print Macro'
|
||||
|
||||
|
|
@ -362,19 +373,19 @@ You can place this macro in your project directory then load and run it from Pap
|
|||
```
|
||||
$ pappy
|
||||
Proxy is listening on port 8000
|
||||
itsPappyTime> lma
|
||||
pappy> lma
|
||||
Loaded "<Macro Test Macro (tm/test)>"
|
||||
Loaded "<Macro Macro 6494496 (testgen)>"
|
||||
Loaded "<Macro Print Macro (print)>"
|
||||
Loaded "<Macro Hack the NSA (htnsa/hackthensa)>"
|
||||
Loaded "<Macro Macro 62449408 (blank)>"
|
||||
itsPappyTime> rma print
|
||||
pappy> rma print
|
||||
Hello, Pappy!
|
||||
itsPappyTime> rma print NSA
|
||||
pappy> rma print NSA
|
||||
Hello, NSA!
|
||||
itsPappyTime> rma print Idiot Slayer
|
||||
pappy> rma print Idiot Slayer
|
||||
Hello, Idiot!
|
||||
itsPappyTime> rma print "Idiot Slayer"
|
||||
pappy> rma print "Idiot Slayer"
|
||||
Hello, Idiot Slayer!
|
||||
```
|
||||
|
||||
|
|
@ -387,16 +398,16 @@ You can also generate macros that have Pappy `Request` objects created with the
|
|||
```
|
||||
$ pappy
|
||||
Proxy is listening on port 8000
|
||||
itsPappyTime> ls
|
||||
pappy> ls
|
||||
ID Verb Host Path S-Code Req Len Rsp Len Time Mngl
|
||||
5 GET vitaly.sexy /esr1.jpg 200 OK 0 17653 -- --
|
||||
4 GET vitaly.sexy /netscape.gif 200 OK 0 1135 -- --
|
||||
3 GET vitaly.sexy /construction.gif 200 OK 0 28366 -- --
|
||||
2 GET vitaly.sexy /vitaly2.jpg 200 OK 0 2034003 -- --
|
||||
1 GET vitaly.sexy / 200 OK 0 1201 -- --
|
||||
itsPappyTime> gma sexy 1
|
||||
pappy> gma sexy 1
|
||||
Wrote script to macro_sexy.py
|
||||
itsPappyTime> quit
|
||||
pappy> quit
|
||||
$ cat macro_sexy.py
|
||||
from pappyproxy.http import Request, get_request, post_request
|
||||
|
||||
|
|
@ -430,7 +441,7 @@ def run_macro(args):
|
|||
pass
|
||||
```
|
||||
|
||||
If you enter in a value for `SHORT_NAME`, you can use it as a shortcut to run that macro. So if in a macro you set `SHORT_NAME='tm'` you can run it by running `itsPappyTime> rma tm`.
|
||||
If you enter in a value for `SHORT_NAME`, you can use it as a shortcut to run that macro. So if in a macro you set `SHORT_NAME='tm'` you can run it by running `pappy> rma tm`.
|
||||
|
||||
| Command | Aliases | Description |
|
||||
|:--------|:--------|:------------|
|
||||
|
|
@ -441,9 +452,9 @@ If you enter in a value for `SHORT_NAME`, you can use it as a shortcut to run th
|
|||
|
||||
### Request Objects
|
||||
|
||||
The main method of interacting with the proxy is through `Request` objects. You can submit a request with `req.sumbit()` and save it to the data file with `req.save()`. The objects also have attributes which can be used to modify the request in a high-level way. Unfortunately, I haven't gotten around to writing full docs on the API and it's still changing every once in a while so I apologize if I pull the carpet out from underneath you.
|
||||
The main method of interacting with the proxy is through `Request` objects. You can submit a request with `req.sumbit()` and save it to the data file with `req.save()`. The objects also have attributes which can be used to modify the request in a high-level way. You can see the [full documentation](https://roglew.github.io/pappy-proxy/pappyproxy.html#module-pappyproxy.http) for more details on using these objects.
|
||||
|
||||
Dict-like objects are represented with a custom class called a `RepeatableDict`. I haven't gotten around to writing docs on it yet, so just interact with it like a dict and don't be surprised if it's missing some methods you would expect a dict to have.
|
||||
Dict-like objects are represented with a custom class called a `RepeatableDict`. Again, look at the docs for details. For the most part, you can interact with it like a normal dictionary, but don't be surprised if it's missing some methods you would expect.
|
||||
|
||||
Here is a quick list of attributes that you can use with `Request` objects:
|
||||
|
||||
|
|
@ -500,12 +511,30 @@ Like I said, these interfaces are prone to change and will probably crash when y
|
|||
|
||||
### Useful Functions
|
||||
|
||||
There are also a few functions which could be useful for creating requests in macros.
|
||||
There are also a few functions which could be useful for creating requests in macros. It's worth pointing out that `request_by_id` is useful for passing request objects as arguments. For example, here is a macro that lets you resubmit a request with the Google Bot user agent:
|
||||
|
||||
```
|
||||
## macro_googlebot.py
|
||||
|
||||
from pappyproxy.http import Request, get_request, post_request, request_by_id
|
||||
from pappyproxy.context import set_tag
|
||||
from pappyproxy.iter import *
|
||||
|
||||
MACRO_NAME = 'Submit as Google'
|
||||
SHORT_NAME = ''
|
||||
|
||||
def run_macro(args):
|
||||
req = request_by_id(args[0])
|
||||
req.headers['User-Agent'] = "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
|
||||
req.submit()
|
||||
req.save()
|
||||
```
|
||||
|
||||
| Function | Description |
|
||||
|:--|:--|
|
||||
| get_request(url, url_params={}) | Returns a Request object that contains a GET request to the given url with the given url params |
|
||||
| post_request(url, post_params={}, url_params={}) | Returns a Request object that contains a POST request to the given url with the given url and post params |
|
||||
| request_by_id(reqid) | Get a request object from its id. |
|
||||
|
||||
Intercepting Macros
|
||||
-------------------
|
||||
|
|
@ -556,6 +585,46 @@ def mangle_request(request):
|
|||
return r
|
||||
```
|
||||
|
||||
In addition, you can use an `init(args)` function to get arguments from the command line. If no arguments are passed, args will be an empty list. Here is an example macro that does a search and replace:
|
||||
|
||||
```
|
||||
## int_replace.py
|
||||
|
||||
MACRO_NAME = 'Find and Replace'
|
||||
SHORT_NAME = ''
|
||||
runargs = []
|
||||
|
||||
def init(args):
|
||||
global runargs
|
||||
runargs = args
|
||||
|
||||
def mangle_request(request):
|
||||
global runargs
|
||||
if len(runargs) < 2:
|
||||
return request
|
||||
request.body = request.body.replace(runargs[0], runargs[1])
|
||||
return request
|
||||
|
||||
def mangle_response(request):
|
||||
global runargs
|
||||
if len(runargs) < 2:
|
||||
return request.response
|
||||
request.response.body = request.response.body.replace(runargs[0], runargs[1])
|
||||
return request.response
|
||||
```
|
||||
|
||||
You can use this macro to do any search and replace that you want. For example, if you wanted to replace "Google" with "Skynet", you can run the macro like this:
|
||||
|
||||
```
|
||||
pappy> lma
|
||||
Loaded "<InterceptingMacro Find and Replace (replace)>"
|
||||
pappy> rim replace Google Skynet
|
||||
"Find and Replace" started
|
||||
pappy>
|
||||
```
|
||||
|
||||
Now every site that you visit will be a little bit more accurate.
|
||||
|
||||
### Enabling/Disabling Intercepting Macros
|
||||
You can use the following commands to start/stop intercepting macros
|
||||
|
||||
|
|
@ -563,8 +632,8 @@ You can use the following commands to start/stop intercepting macros
|
|||
|:--------|:--------|:------------|
|
||||
| `lma [dir]` | `load_macros`, `lma` | Load macros from a directory. If `dir` is not given, use the current directory (the project directory) |
|
||||
| `rim <macro name>` | `run_int_macro`, `rim` | Run an intercepting macro. Similarly to normal macros you can use the name, short name, or file name of the macro. |
|
||||
| `sim <macro name>` | `stop_int_macro`, `sim` | Stop an intercepting macro. |
|
||||
| `lim` | `list_int_macros`, `lim` | List all enabled/disabled intercepting macros |
|
||||
| `sim <macro name> [args]` | `stop_int_macro`, `sim` | Stop an intercepting macro. If arguments are given, they will be passed to the macro's `init(args)` function if it exists. |
|
||||
| `lim` | `list_int_macros`, `lsim` | List all enabled/disabled intercepting macros |
|
||||
| `gima <name>` | `generate_int_macro`, `gima` | Generate an intercepting macro with the given name. |
|
||||
|
||||
Logging
|
||||
|
|
@ -575,8 +644,8 @@ You can watch in real-time what requests are going through the proxy. Verbosisty
|
|||
|:--------|:------------|
|
||||
| `log [verbosity]` | View the log at the given verbosity. Default verbosity is 1 which just shows connections being made/lost and some other info, verbosity 3 shows full requests/responses as they pass through and are processed by the proxy |
|
||||
|
||||
Additional Commands
|
||||
-------------------
|
||||
Additional Commands and Features
|
||||
--------------------------------
|
||||
This is a list of other random stuff you can do that isn't categorized under anything else. These are mostly commands that I found that I needed while doing a test and just added. They likely don't do a ton of error checking.
|
||||
|
||||
| Command | Aliases | Description |
|
||||
|
|
@ -584,3 +653,124 @@ This is a list of other random stuff you can do that isn't categorized under any
|
|||
| `dump_response <reqid> [filename]` | `dump_response` | Dumps the data from the response to the given filename (useful for images, .swf, etc). If no filename is given, it uses the name given in the path. |
|
||||
| `export <req|rsp> <reqid>` | `export` | Writes either the full request or response to a file in the current directory. |
|
||||
|
||||
### Response streaming
|
||||
|
||||
If you don't have any intercepting macros running, Pappy will forward data to the browser as it gets it. However, if you're trying to mangle messages/responses, Pappy will need to download the entire message first.
|
||||
|
||||
Plugins
|
||||
-------
|
||||
Note that this section is a very quick overview of plugins. For a full description of how to write them, please see [the official docs](https://roglew.github.io/pappy-proxy/pappyplugins.html).
|
||||
|
||||
It is also possible to write plugins which are reusable across projects. Plugins are simply Python scripts located in `~/.pappy/plugins`. Plugins are able to create new console commands and maintain state throughout a Pappy session. They can access the same API as macros, but the plugin system is designed to allow you to create general purpose commands as compared to macros which are meant to be project-specific scripts. Still, it may not be a bad idea to try building a macro to do something in a quick and dirty way before writing a plugin since plugins are more complicated to write.
|
||||
|
||||
A simple hello world plugin could be something like:
|
||||
|
||||
```
|
||||
## hello.py
|
||||
import shlex
|
||||
|
||||
def hello_world(line):
|
||||
if line:
|
||||
args = shlex.split(line)
|
||||
print 'Hello, %s!' % (', '.join(args))
|
||||
else:
|
||||
print "Hello, world!"
|
||||
|
||||
###############
|
||||
## Plugin hooks
|
||||
|
||||
def load_cmds(cmd):
|
||||
cmd.set_cmds({
|
||||
'hello': (hello_world, None),
|
||||
})
|
||||
cmd.add_aliases([
|
||||
('hello', 'hlo'),
|
||||
('hello', 'ho'),
|
||||
])
|
||||
```
|
||||
|
||||
You can also create commands which support autocomplete:
|
||||
|
||||
```
|
||||
import shlex
|
||||
|
||||
_AUTOCOMPLETE_NAMES = ['alice', 'allie', 'sarah', 'mallory', 'slagathor']
|
||||
|
||||
def hello_world(line):
|
||||
if line:
|
||||
args = shlex.split(line)
|
||||
print 'Hello, %s!' % (', '.join(args))
|
||||
else:
|
||||
print "Hello, world!"
|
||||
|
||||
def complete_hello_world(text, line, begidx, endidx):
|
||||
return [n for n in _AUTOCOMPLETE_NAMES if n.startswith(text)]
|
||||
|
||||
###############
|
||||
## Plugin hooks
|
||||
|
||||
def load_cmds(cmd):
|
||||
cmd.set_cmds({
|
||||
'hello': (hello_world, complete_hello_world),
|
||||
})
|
||||
cmd.add_aliases([
|
||||
('hello', 'hlo'),
|
||||
])
|
||||
```
|
||||
|
||||
Then when you run Pappy you can use the ``hello`` command:
|
||||
|
||||
```
|
||||
$ pappy -l
|
||||
Temporary datafile is /tmp/tmpBOXyJ3
|
||||
Proxy is listening on port 8000
|
||||
pappy> ho
|
||||
Hello, world!
|
||||
pappy> ho foo bar baz
|
||||
Hello, foo, bar, baz!
|
||||
pappy> ho foo bar "baz lihtyur"
|
||||
Hello, foo, bar, baz lihtyur!
|
||||
pappy>
|
||||
```
|
||||
|
||||
### Should I Write a Plugin or a Macro?
|
||||
|
||||
A lot of the time, you can get away with writing a macro. However, you may consider writing a plugin if:
|
||||
|
||||
* You find yourself copying one macro to multiple projects
|
||||
* You want to write a general tool that can be applied to any website
|
||||
* You need to maintain state during the Pappy session
|
||||
|
||||
My guess is that if you need one quick thing for a project, you're better off writing a macro first and seeing if you end up using it in future projects. Then if you find yourself needing it a lot, write a plugin for it. You may also consider keeping a `mine.py` plugin where you can write out commands that you use regularly but may not be worth creating a dedicated plugin for.
|
||||
|
||||
FAQ
|
||||
---
|
||||
|
||||
### I still like Burp, but Pappy looks interesting, can I use both?
|
||||
Yes! If you don't want to go completely over to Pappy yet, you can configure Burp to use Pappy as an upstream proxy server. That way, traffic will go through both Burp and Pappy and you can use whichever you want to do your testing.
|
||||
|
||||
How to have Burp forward traffic through Pappy:
|
||||
|
||||
1. Open Burp
|
||||
2. Go to `Options -> Connections -> Upstream Proxy Servers`
|
||||
3. Click `Add`
|
||||
4. Leave `Destination Host` blank, but put `127.0.0.1` in `Proxy Host` and `8000` into `Port` (assuming you're using the default listener)
|
||||
5. Configure your browser to use Burp as a proxy
|
||||
|
||||
### Why does my request have an id of `--`?!?!
|
||||
You can't do anything with a request/response until it is decoded and saved to disk. In between the time when a request is decoded and when it's saved to disk, it will have an ID of `--`. So just wait a little bit and it will get an ID you can use.
|
||||
|
||||
Changelog
|
||||
---------
|
||||
The boring part of the readme
|
||||
|
||||
* 0.2.0
|
||||
* Lots of refactoring
|
||||
* Plugins
|
||||
* Bugfixes probably
|
||||
* Change prompt to make Pappy look more professional (but it will always be pappy time in your heart, I promise)
|
||||
* Create changelog
|
||||
* Add response streaming if no intercepting macros are active
|
||||
* 0.1.1
|
||||
* Start using sane versioning system
|
||||
* Did proxy things
|
||||
Loading…
Add table
Add a link
Reference in a new issue