If you enter in a value for `SHORT_NAME`, you can use it as a shortcut to run that macro. So if in a macro you set `SHORT_NAME='tm'` you can run it by running `pappy> rma tm`.
If you enter in a value for `SHORT_NAME`, you can use it as a shortcut to run that macro. So if in a macro you set `SHORT_NAME='tm'` you can run it by running `pappy> rma tm`.
Remember, you can use the wildcard to generate a macro with all in-context requests:
```
# Generate a macro with all in-context requests
pappy> gma allreqs *
```
### Passing Arguments to Macros
### Passing Arguments to Macros
When you run the macro, any additional command line arguments will be passed to the run_macro function in the `args` argument. For example, if you run your macro using
When you run the macro, any additional command line arguments will be passed to the run_macro function in the `args` argument. For example, if you run your macro using
@ -802,6 +812,7 @@ def run_macro(args):
| get_request(url, url_params={}) | Returns a Request object that contains a GET request to the given url with the given url params |
| get_request(url, url_params={}) | Returns a Request object that contains a GET request to the given url with the given url params |
| post_request(url, post_params={}, url_params={}) | Returns a Request object that contains a POST request to the given url with the given url and post params |
| post_request(url, post_params={}, url_params={}) | Returns a Request object that contains a POST request to the given url with the given url and post params |
| request_by_id(reqid) | Get a request object from its id. |
| request_by_id(reqid) | Get a request object from its id. |
| main_context_ids() | Returns a list of the IDs that are in the current context. Use this for macros that need to act on every in-context request. For example, it can be used in a macro to resubmit a set of requests. |
Intercepting Macros
Intercepting Macros
-------------------
-------------------
@ -903,6 +914,77 @@ You can use the following commands to start/stop intercepting macros
| `lim` | `list_int_macros`, `lsim` | List all enabled/disabled intercepting macros |
| `lim` | `list_int_macros`, `lsim` | List all enabled/disabled intercepting macros |
| `gima <name>` | `generate_int_macro`, `gima` | Generate an intercepting macro with the given name. |
| `gima <name>` | `generate_int_macro`, `gima` | Generate an intercepting macro with the given name. |
Macro Templates
---------------
Pappy also includes some other templates for generating macros. They can be generated with the `gtma` command. You can then modify the generated macros to do what you want. For example, you could modify the resubmit macro to get a new session token before submitting each request. Using a template can save you from writing boilerplate for commonly created macros.
Examples:
```
# The same as gma foo 1,2,3
pappy> gtma foo macro 1,2,3
Wrote script to macro_foo.py
# Generate a macro that resubmits all in-context requests
pappy> gtma suball resubmit
Wrote script to macro_suball.py
# Generate an intercepting macro that modifies headers as they pass through the proxy
pappy> gtma headers modheader
Wrote script to int_headers.py
```
Command information:
| Command | Aliases | Description |
|:--------|:--------|:------------|
| `gtma <name> <template name> [template arguments]` | `generate_template_macro`, `gtma` | Generate a macro using a template. |
Available macro templates:
| Name | Arguments | Description |
|:-----|:----------|:------------|
| `macro` | `[reqids]` | The template used to generate macros from request IDs. |
| `intmacro` | None | The template used to generate an intercepting macro. |
| `modheader` | None | Create an intercepting macro that modifies a header in the request or response. |
| `resubmit` | None | Create a macro that resubmits all in-context requests. Includes commented out code to maintain session state using a cookie jar. |
Resubmitting Groups of Requests
-------------------------------
You can use the `submit` request to resubmit requests. It is suggested that you use this command with a heavy use of filters and using the wildcard (`*`) to submit all in-context requests. Be careful submitting everything in context, remember, if you have to Ctl-C out you will close Pappy and lose all in-memory requests!
| Command | Aliases | Description |
|:--------|:--------|:------------|
| `submit reqids [-m] [-u] [-p] [-c [COOKIES [COOKIES ...]]] [-d [HEADERS [HEADERS ...]]]` | `submit` | Submit a given set of requests. Request IDs must be passed in as the first argument. The wildcard (`*`) selector can be very useful. Resubmitted requests are given a `resubmitted` tag. See the arguments section for information on the arguments. |
### Useful Filters For Selecting Requests to Resubmit
* `before` and `after` to select requests in a time range. You can use the `after` filter on the most recent request, browse the site, then use the `before` filter to select a continuous browsing session.
* `verb` if you only want to select GET requests
* `path ct logout` to avoid logging out
### Arguments
There are a few simple parameters you can pass to the command to modify requests. These behave like normal command parameters in the terminal. If you need something more complex (ie getting CSRF tokens, refreshing the session token, reacting to Set-Cookie headers, etc.) you should consider writing a macro and using the `main_context_ids` function to get in-context IDs then iterating over them and handling them however you want.
| Argument | Description |
|:---------|:------------|
| `-c <cookie>=<val>` | Modify a cookie on each request before submitting. Can pass more than one pair to the flag to modify more than one cookie. Does not encode the cookie values in any way. |
| `-d <header>=<val>` | Modify a header on each request before submitting. Can pass more than one pair to the flag to modify more than one header. |
| `-m` | Store requests in memory instead of saving to the data file. |
| `-u` | Only submit one request per endpoint. Will count requests with the same path but different url params as *different* endpoints. |
| `-p` | Only submit one request per endpoint. Will count requests with the same path but different url params as *the same* endpoints. |
Examples:
```
# Resubmit all in-context requests with the SESSIONID cookie set to 1234 and SESSIONSTATE set to {'admin'='true'}
# Resubmit all in-context requests with the User-Agent header set to "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" then store them in memory
You can watch in real-time what requests are going through the proxy. Verbosisty defaults to 1 which just states when connections are made/lost and some information on what is happening. If verbosity is set to 3, it includes all the data which is sent through the proxy and processed. It will print the raw response from the server, what it decodes it to, etc. Even if you don't run this command, all the information is stored in the dubug directory (the directory is cleared every start though!)
You can watch in real-time what requests are going through the proxy. Verbosisty defaults to 1 which just states when connections are made/lost and some information on what is happening. If verbosity is set to 3, it includes all the data which is sent through the proxy and processed. It will print the raw response from the server, what it decodes it to, etc. Even if you don't run this command, all the information is stored in the dubug directory (the directory is cleared every start though!)
@ -1110,6 +1192,72 @@ Or if you’re going to YOLO it do the same thing then listen on port 80/443 dir
Pappy will automatically use this host to make the connection and forward the request to the new server.
Pappy will automatically use this host to make the connection and forward the request to the new server.
Project File Encryption
-----------------------
Pappy includes some basic features for automatically compressing and encrypting your project directory with a password. However, before I go into details on how to do this, I need to make one thing clear.
**Don't rely on Pappy to encrypt confidential information. Use a dedicated encryption product to encrypt your project directory instead.**
Other commercial and large open source crypto projects have had a much larger number of people look at their crypto implementations and are less likely to have errors in their implementation. However, for cases where you don't need enterprise level security or if you just want your project stored in a single password-protected file instead of a directory, Pappy's got you covered.
Here is how Pappy's project encryption works:
* Open a project by running Pappy with the `-c` flag
* Pappy creates a `crypt/` directory in the current directory and changes the working directory into it
* Do work as normal. You can use other tools in the created `crypt/` directory
* When you quit Pappy, the file is compressed and encrypted with the provided password
* The project directory is deleted
Unfortunately, if Pappy hard crashes the files will not be cleaned up. However, if you start Pappy and it notices a `crypt/` directory, it will attempt to use it as the project directory and create a new encrypted project file upon exiting.
@ -35,8 +35,6 @@ Anyways, here's some ideas for things you could implement:
Make it easy to pass a request to SQLMap to check for SQLi. Make sure you can configure which fields you do/don't want tested and by default just give either "yes it looks like SQLi" or "no it doesn't look like SQLi"
Make it easy to pass a request to SQLMap to check for SQLi. Make sure you can configure which fields you do/don't want tested and by default just give either "yes it looks like SQLi" or "no it doesn't look like SQLi"
* Additional macro templates
* Additional macro templates
Write some commands for generating additional types of macros. For example let people generate an intercepting macro that does search/replace or modifies a header. Save as much typing as possible for common actions.
Write some commands for generating additional types of macros. For example let people generate an intercepting macro that does search/replace or modifies a header. Save as much typing as possible for common actions.
* Show requests/responses real-time as they go through the proxy
Let people watch requests as they pass through the proxy. It's fine to implement this as an intercepting macro since people watching the requests aren't going to notice response streaming being disabled.
* Vim plugin to make editing HTTP messages easier
* Vim plugin to make editing HTTP messages easier
Implement some functionality to make editing HTTP messages easier. It would be great to have a plugin to automatically add to vim when using the interceptor/repeater to make editing requests easier. Look at burp's request editor and try to implement anything you miss from it.
Implement some functionality to make editing HTTP messages easier. It would be great to have a plugin to automatically add to vim when using the interceptor/repeater to make editing requests easier. Look at burp's request editor and try to implement anything you miss from it.
@ -272,10 +272,10 @@ Using defer.inlineCallbacks With a Command
..note::
..note::
This tutorial won't tell you how to use inlineCallbacks in general. Type "twisted inline callbacks" into google to figure out what they are. This is mainly just a reminder to use the ``crochet`` wrapper for console commands and warning you that some functions may return deferreds that you may have to deal with.
This tutorial won't tell you how to use inlineCallbacks in general. Type "twisted inline callbacks" into google to figure out what they are. This is mainly just a reminder to use the ``crochet`` wrapper for console commands and warning you that some functions may return deferreds that you may have to deal with.
Since you're writing a plugin, you'll probably be using functions which return a deferred. And to keep things readable, you'll want to use the ``defer.inlineCallbacks`` function wrapper. Unfortunately, you can't bind async functions to commands. Luckily, there's a library called `crochet <https://pypi.python.org/pypi/crochet>`_ which lets you add another wrapper to the function that lets it be used like a blocking function. Rather than talking about it, let's write a plugin to call :func:`pappyproxy.console.load_reqlist` to print out some requests' hosts. Let's start by pretending it's a normal function::
Since you're writing a plugin, you'll probably be using functions which return a deferred. And to keep things readable, you'll want to use the ``defer.inlineCallbacks`` function wrapper. Unfortunately, you can't bind async functions to commands. Luckily, there's a library called `crochet <https://pypi.python.org/pypi/crochet>`_ which lets you add another wrapper to the function that lets it be used like a blocking function. Rather than talking about it, let's write a plugin to call :func:`pappyproxy.util.load_reqlist` to print out some requests' hosts. Let's start by pretending it's a normal function::
import shlex
import shlex
from pappyproxy.console import load_reqlist
from pappyproxy.util import load_reqlist
def print_hosts(line):
def print_hosts(line):
args = shlex.split(line)
args = shlex.split(line)
@ -309,10 +309,10 @@ And we run it::
iteration over non-sequence
iteration over non-sequence
pappy>
pappy>
Iteration over a non-sequence? what? Well, :func:`pappyproxy.console.load_reqlist` doesn't actually return a list of requests. It returns a deferred which returns a list of requests. I'm not going into the details (look up some stuff on using inline callbacks with Twisted if you want more info), but the way to fix it is to slap an ``inlineCallbacks`` wrapper on the function and ``yield`` the result of the function. Now it looks like this::
Iteration over a non-sequence? what? Well, :func:`pappyproxy.util.load_reqlist` doesn't actually return a list of requests. It returns a deferred which returns a list of requests. I'm not going into the details (look up some stuff on using inline callbacks with Twisted if you want more info), but the way to fix it is to slap an ``inlineCallbacks`` wrapper on the function and ``yield`` the result of the function. Now it looks like this::
import shlex
import shlex
from pappyproxy.console import load_reqlist
from pappyproxy.util import load_reqlist
from twisted.internet import defer
from twisted.internet import defer
@defer.inlineCallbacks
@defer.inlineCallbacks
@ -336,7 +336,7 @@ However, the console assumes that any functions it calls will be blocking. As a
import shlex
import shlex
import crochet
import crochet
from pappyproxy.console import load_reqlist
from pappyproxy.util import load_reqlist
from twisted.internet import defer
from twisted.internet import defer
@crochet.wait_for(timeout=None)
@crochet.wait_for(timeout=None)
@ -394,7 +394,7 @@ Here is an example plugin for storing the user-agent (if it exists) in the ``plu
import shlex
import shlex
from twisted.internet import defer
from twisted.internet import defer
from pappyproxy.console import load_reqlist
from pappyproxy.util import load_reqlist
from pappyproxy.plugin import main_context
from pappyproxy.plugin import main_context
from pappyproxy.util import PappyException
from pappyproxy.util import PappyException
@ -435,8 +435,7 @@ Here is an example plugin for storing the user-agent (if it exists) in the ``plu
Useful Functions
Useful Functions
----------------
----------------
* Load a request by id: :func:`pappyproxy.http.Request.load_request`
See :mod:`pappyproxy.plugin` and :mod:`pappyproxy.util` for useful functions
* Create a filter from a filter string: :func:`pappyproxy.context.Filter.from_filter_string`
parser.add_argument('-m','--inmem',action='store_true',help='Store resubmitted requests in memory without storing them in the data file')
parser.add_argument('-u','--unique',action='store_true',help='Only resubmit one request per endpoint (different URL parameters are different endpoints)')
parser.add_argument('-p','--uniquepath',action='store_true',help='Only resubmit one request per endpoint (ignoring URL parameters)')
parser.add_argument('-c','--cookies',nargs='*',help='Apply a cookie to requests before submitting')
parser.add_argument('-d','--headers',nargs='*',help='Apply a header to requests before submitting')
args=parser.parse_args(shlex.split(line))
headers={}
cookies={}
ifargs.headers:
forhinargs.headers:
k,v=h.split('=',1)
headers[k]=v
ifargs.cookies:
forcinargs.cookies:
k,v=c.split('=',1)
cookies[k]=v
ifargs.uniqueandargs.uniquepath:
raisePappyException('Both -u and -p cannot be given as arguments')
Rob Glew
9 years ago