v0.2.7

`--stream` can be used to enable streaming also with `--pretty` and to ensure
a more frequent output flushing.

.gitignore

@@ -2,3 +2,8 @@ dist
 httpie.egg-info
 build
 *.pyc
+.tox
+README.html
+.coverage
+htmlcov
+

.travis.yml

@@ -0,0 +1,9 @@
+language: python
+python:
+  - 2.6
+  - 2.7
+  - pypy
+  - 3.2
+script: python setup.py test
+install:
+  - pip install . --use-mirrors

AUTHORS.rst

@@ -0,0 +1,29 @@
+==============
+HTTPie authors
+==============
+
+* `Jakub Roztocil <https://github.com/jkbr>`_
+
+
+Patches and ideas
+-----------------
+
+* `Hank Gay <https://github.com/gthank>`_
+* `Jake Basile <https://github.com/jakebasile>`_
+* `Vladimir Berkutov <https://github.com/dair-targ>`_
+* `Jakob Kramer <https://github.com/gandaro>`_
+* `Chris Faulkner <https://github.com/faulkner>`_
+* `Alen Mujezinovic <https://github.com/flashingpumpkin>`_
+* `Praful Mathur <https://github.com/tictactix>`_
+* `Marc Abramowitz <https://github.com/msabramo>`_
+* `Ismail Badawi <https://github.com/isbadawi>`_
+* `Laurent Bachelier <https://github.com/laurentb>`_
+* `Isman Firmansyah <https://github.com/iromli>`_
+* `Simon Olofsson <https://github.com/simono>`_
+* `Churkin Oleg <https://github.com/Bahus>`_
+* `Jökull Sólberg Auðunsson <https://github.com/jokull>`_
+* `Matthew M. Boedicker <https://github.com/mmb>`_
+* `marblar <https://github.com/marblar>`_
+* `Tomek Wójcik <https://github.com/tomekwojcik>`_
+* `Davey Shafik <https://github.com/dshafik>`_
+* `cido <https://github.com/cido>`_

LICENSE

@@ -0,0 +1,26 @@
+Copyright © 2012 Jakub Roztocil <jakub@roztocil.name>
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+    1. Redistributions of source code must retain the above copyright notice,
+       this list of conditions and the following disclaimer.
+
+    2. Redistributions in binary form must reproduce the above copyright
+       notice, this list of conditions and the following disclaimer in the
+       documentation and/or other materials provided with the distribution.
+
+    3. Neither the name of The author nor the names of its contributors may
+       be used to endorse or promote products derived from this software
+       without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE AUTHOR AND CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

MANIFEST.in

@@ -0,0 +1 @@
+include README.rst LICENSE

README.md

@@ -1,77 +0,0 @@
-## HTTPie: cURL for humans
-
-HTTPie is a CLI frontend for [python-requests](http://python-requests.org) built out of frustration. It provides an `http` command that can be used to easily issue HTTP requests. It is meant to be used by humans to interact with HTTP-based APIs and web servers. The response headers are colorized and the body is syntax-highlighed if its `Content-Type` is known to [Pygments](http://pygments.org/) (unless the output is redirected).
-
-![httpie](https://github.com/jkbr/httpie/raw/master/httpie.png)
-
-
-### Installation
-
-    pip install httpie
-
-
-### Usage
-
-    http [flags] METHOD URL [header:value | data-field-name=value]*
-
-The default request `Content-Type` is `application/json` and data fields are automatically serialized as a JSON `Object`, so this:
-
-    http PATCH api.example.com/person/1 X-API-Token:123 name=John email=john@example.org
-
-Will issue the following request:
-
-    PATCH /person/1 HTTP/1.1
-    User-Agent: HTTPie/0.1
-    X-API-Token: 123
-    Content-Type: application/json; charset=utf-8
-
-    {"name": "John", "email": "john@example.org"}
-
-You can use the `--form` flag to set `Content-Type` and serialize the data as `application/x-www-form-urlencoded`.
-
-The data to be sent can also be passed via `stdin`:
-
-    http PUT api.example.com/person/1 X-API-Token:123 < person.json
-
-Most of the flags mirror the arguments you would use with `requests.request`. See `http -h`:
-
-    $ http -h
-    usage: http [-h] [--json | --form] [--traceback] [--ugly] [--headers | --body]
-                [--auth AUTH] [--verify VERIFY] [--proxy PROXY]
-                [--allow-redirects] [--file PATH] [--timeout TIMEOUT]
-                method URL [item [item ...]]
-
-    HTTPie - cURL for humans.
-
-    positional arguments:
-      method                HTTP method to be used for the request (GET, POST,
-                            PUT, DELETE, PATCH, ...).
-      URL                   Protocol defaults to http:// if the URL does not
-                            include it.
-      item                  HTTP header (key:value) or data field (key=value)
-
-    optional arguments:
-      -h, --help            show this help message and exit
-      --json, -j            Serialize data items as a JSON object and set Content-
-                            Type to application/json, if not specified.
-      --form, -f            Serialize data items as form values and set Content-
-                            Type to application/x-www-form-urlencoded, if not
-                            specified.
-      --traceback           Print a full exception traceback should one be raised
-                            by `requests`.
-      --ugly, -u            Do not prettify the response.
-      --headers, -t         Print only the response headers.
-      --body, -b            Print only the response body.
-      --auth AUTH, -a AUTH  username:password
-      --verify VERIFY       Set to "yes" to check the host's SSL certificate. You
-                            can also pass the path to a CA_BUNDLE file for private
-                            certs. You can also set the REQUESTS_CA_BUNDLE
-                            environment variable.
-      --proxy PROXY         String mapping protocol to the URL of the proxy (e.g.
-                            http:foo.bar:3128).
-      --allow-redirects     Set this flag if full redirects are allowed (e.g. re-
-                            POST-ing of data at new ``Location``)
-      --file PATH           File to multipart upload
-      --timeout TIMEOUT     Float describes the timeout of the request (Use
-                            socket.setdefaulttimeout() as fallback).
-

README.rst

@@ -0,0 +1,986 @@
+***********************
+HTTPie: cURL for Humans
+***********************
+
+v0.2.7
+
+HTTPie is a **command line HTTP client** whose goal is to make CLI interaction
+with HTTP-based services as **human-friendly** as possible. It provides a
+simple ``http`` command that allows for sending arbitrary HTTP requests with a
+simple and natural syntax, and displays colorized responses. HTTPie can be used
+for **testing, debugging**, and generally **interacting** with HTTP servers.
+
+
+.. image:: https://github.com/jkbr/httpie/raw/master/httpie.png
+    :alt: HTTPie compared to cURL
+    :width: 835
+    :height: 835
+
+
+HTTPie is written in Python, and under the hood it uses the excellent
+`Requests`_ and `Pygments`_ libraries.
+
+
+**Table of Contents**
+
+
+.. contents::
+    :local:
+    :depth: 1
+    :backlinks: none
+
+
+
+=============
+Main Features
+=============
+
+* Expressive and intuitive syntax
+* Formatted and colorized terminal output
+* Built-in JSON support
+* Forms and file uploads
+* HTTPS and authorization
+* Arbitrary request data
+* Custom headers
+* Python 2.6 and Python 3 support
+* Linux, Mac OS X and Windows support
+* Documentation
+* Test coverage
+
+
+============
+Installation
+============
+
+The latest **stable version** of HTTPie can always be installed or updated
+to via `pip`_ (prefered)
+or ``easy_install``:
+
+.. code-block:: bash
+
+    $ pip install -U httpie
+
+
+.. code-block:: bash
+
+    $ easy_install httpie
+
+
+Or, you can install the **development version** directly from GitHub:
+
+
+.. image:: https://secure.travis-ci.org/jkbr/httpie.png
+    :target: http://travis-ci.org/jkbr/httpie
+    :alt: Build Status of the master branch
+
+
+.. code-block:: bash
+
+    $ pip install -U https://github.com/jkbr/httpie/tarball/master
+
+
+There are also packages available for `Ubuntu`_, `Debian`_, and possibly other
+Linux distributions as well.
+
+
+===========
+Quick Start
+===========
+
+
+Hello World:
+
+
+.. code-block:: bash
+
+    $ http httpie.org
+
+
+Synopsis:
+
+.. code-block:: bash
+
+    $ http [flags] [METHOD] URL [ITEM [ITEM]]
+
+
+See also ``http --help``.
+
+
+--------
+Examples
+--------
+
+
+Send a ``HEAD`` request:
+
+.. code-block:: bash
+
+    $ http HEAD example.org
+
+
+Submit a form:
+
+.. code-block:: bash
+
+    $ http --form POST example.org hello=World
+
+
+Send a ``PUT`` request with a custom header and some JSON data:
+
+.. code-block:: bash
+
+    $ http PUT example.org X-API-Token:123 name='David Bowie'
+
+See the request that is being sent:
+
+.. code-block:: bash
+
+    $ http --verbose example.org
+
+
+Use `Github API`_ to post a comment on an issue:
+
+.. code-block:: bash
+
+    $ http -a USERNAME POST https://api.github.com/repos/jkbr/httpie/issues/83/comments body='HTTPie is awesome!'
+
+
+Upload a file:
+
+.. code-block:: bash
+
+    $ http example.org < file.json
+
+
+Download a file:
+
+.. code-block:: bash
+
+    $ http example.org/file > file
+
+============
+HTTP Method
+============
+
+The name of the HTTP method comes right before the URL argument:
+
+.. code-block:: bash
+
+    $ http DELETE example.org/todos/7
+
+
+It makes the command look similar to the actual ``Request-Line`` that is sent:
+
+.. code-block:: http
+
+    DELETE /todos/7 HTTP/1.1
+
+
+When the ``METHOD`` argument is **omitted** from the command, HTTPie defaults to
+either ``GET`` or ``POST``. This depends on whether you are sending
+some data:
+
+.. code-block:: bash
+
+    $ http example.org/todos text='Check out HTTPie'
+
+
+.. code-block:: http
+
+    POST /todos HTTP/1.1
+
+
+, or no data at all:
+
+.. code-block:: bash
+
+    $ http example.org/todos
+
+
+.. code-block:: http
+
+    GET /todos HTTP/1.1
+
+
+===========
+Request URL
+===========
+
+The only information HTTPie needs to perform a request is a URL.
+The default scheme is, somewhat unsurprisingly, ``http://``,
+and can be omitted from the argument – ``http example.org`` works just fine.
+
+If find yourself manually constructing URLs with **querystring parameters**
+on the terminal, you may appreciate the ``param==value`` syntax for appending
+URL parameters so that you don't have to worry about escaping the ``&``
+separators. To search for ``HTTPie`` on Google Images you could use this
+command:
+
+.. code-block:: bash
+
+    $ http GET www.google.com search==HTTPie tbm==isch
+
+
+.. code-block:: http
+
+    GET /?search=HTTPie&tbm=isch HTTP/1.1
+
+
+=============
+Request Items
+=============
+
+There are five different *request item* types that provide a
+convenient mechanism for specifying HTTP headers, simple JSON and
+form data, files, and URL parameters.
+
+They are key/value pairs specified after the URL. All have in
+common that they become part of the actual request that is sent and that
+their type is distinguished only by the separator used:
+``:``, ``=``, ``:=``, ``@``, and ``==``.
+
++-----------------------+-----------------------------------------------------+
+| Item Type             | Description                                         |
++=======================+=====================================================+
+| HTTP Headers          | Arbitrary HTTP header, e.g. ``X-API-Token:123``.    |
+| ``Name:Value``        |                                                     |
++-----------------------+-----------------------------------------------------+
+| URL parameters        | Appends the given name/value pair as a query        |
+| ``name==value``       | string parameter to the URL.                        |
+|                       | The ``==`` separator is used                        |
++-----------------------+-----------------------------------------------------+
+| Data Fields           | Request data fields to be serialized as a JSON      |
+| ``field=value``       | object (default), or to be form encoded (``--form`` |
+|                       | / ``-f``).                                          |
++-----------------------+-----------------------------------------------------+
+| Raw JSON fields       | Useful when sending JSON and one or                 |
+| ``field:=json``       | more fields need to be a ``Boolean``, ``Number``,   |
+|                       | nested ``Object``, or an ``Array``,  e.g.,          |
+|                       | ``meals:='["ham","spam"]'`` or ``pies:=[1,2,3]``    |
+|                       | (note the quotes).                                  |
++-----------------------+-----------------------------------------------------+
+| Files                 | Only available with ``-f`` / ``--form``.            |
+| ``field@/dir/file``   | For example ``screenshot@~/Pictures/img.png``.      |
+|                       | The presence of a file field results                |
+|                       | in a ``multipart/form-data`` request.               |
++-----------------------+-----------------------------------------------------+
+
+You can use ``\`` to escape characters that shouldn't be used as separators
+(or parts thereof). e.g., ``foo\==bar`` will become a data key/value
+pair (``foo=`` and ``bar``) instead of a URL parameter.
+
+No that data fields aren't the only way to specify request data,
+`redirected input`_ allows passing arbitrary data to be sent with the request.
+
+
+====
+JSON
+====
+
+JSON is the *lingua franca* of modern web services and it is also the
+**default content type** HTTPie uses:
+
+If your command includes some data items, they are serialized as a JSON
+object by default. HTTPie also automatically sets the following headers,
+both of which can be overwritten:
+
+================    =======================================
+``Content-Type``    ``application/json; charset=utf-8``
+``Accept``          ``application/json``
+================    =======================================
+
+You can use ``--json`` / ``-j`` to set ``Accept`` to ``application/json``
+regardless of whether you are sending data (it's a shortcut for using setting
+the header via the usual header notation –
+``http url Accept:application/json``).
+
+Simple example:
+
+.. code-block:: bash
+
+    $ http PUT example.org name=John email=john@example.org
+
+.. code-block:: http
+
+    PUT / HTTP/1.1
+    Accept: application/json
+    Accept-Encoding: identity, deflate, compress, gzip
+    Content-Type: application/json; charset=utf-8
+    Host: example.org
+    User-Agent: HTTPie/0.2.7dev
+
+    {
+        "name": "John",
+        "email": "john@example.org"
+    }
+
+
+Non-string fields use the ``:=`` separator, which allows you to embed raw JSON
+into the resulting object:
+
+.. code-block:: bash
+
+    $ http PUT api.example.com/person/1 name=John age:=29 married:=false hobbies:='["http", "pies"]'
+
+
+.. code-block:: http
+
+    PUT /person/1 HTTP/1.1
+    Accept: application/json
+    Content-Type: application/json; charset=utf-8
+    Host: api.example.com
+    User-Agent: HTTPie/0.2.7dev
+
+    {
+        "age": 29,
+        "hobbies": [
+            "http",
+            "pies"
+        ],
+        "married": false,
+        "name": "John"
+    }
+
+
+
+=====
+Forms
+=====
+
+Submitting forms is very similar to sending `JSON`_ requests. Often the only
+difference is in adding the ``--form`` / ``-f`` option, which ensures that
+data fields are serialized and ``Content-Type`` is set to
+``application/x-www-form-urlencoded; charset=utf-8``.
+
+-------------
+Regular Forms
+-------------
+
+.. code-block:: bash
+
+    $ http --form POST api.example.org/person/1 name='John Smith' email=john@example.org
+
+
+.. code-block:: http
+
+    POST /person/1 HTTP/1.1
+    User-Agent: HTTPie/0.2.7dev
+    Content-Type: application/x-www-form-urlencoded; charset=utf-8
+
+    name=John+Smith&email=john%40example.org
+
+
+-----------------
+File Upload Forms
+-----------------
+
+When one or more file fields are present, the content type is
+``multipart/form-data``:
+
+.. code-block:: bash
+
+    $ http -f POST example.com/jobs name='John Smith' cv@~/Documents/cv.pdf
+
+
+The request above is the same as if the following HTML form were
+submitted:
+
+.. code-block:: html
+
+    <form enctype="multipart/form-data" method="post" action="http://example.com/jobs">
+        <input type="text" name="name" />
+        <input type="file" name="cv" />
+    </form>
+
+
+============
+HTTP Headers
+============
+
+To set custom headers you can use the ``Header:Value`` notation:
+
+.. code-block:: bash
+
+    $ http example.org  User-Agent:Bacon/1.0  Cookie:valued-visitor=yes  X-Foo:Bar  Referer:http://httpie.org/
+
+
+.. code-block:: http
+
+    GET / HTTP/1.1
+    Accept: */*
+    Accept-Encoding: identity, deflate, compress, gzip
+    Cookie: valued-visitor=yes
+    Host: example.org
+    Referer: http://httpie.org/
+    User-Agent: Bacon/1.0
+    X-Foo: Bar
+
+
+There are a couple of default headers that HTTPie sets, but they can easily
+be overwritten:
+
+.. code-block:: http
+
+    GET / HTTP/1.1
+    Accept: */*
+    Accept-Encoding: identity, deflate, compress, gzip
+    User-Agent: HTTPie/<version>
+    Host: <taken-from-URL>
+
+
+====
+Auth
+====
+
+The currently supported authorization schemes are Basic and Digest (more to
+come). There are two flags that control authorization:
+
+===================     ======================================================
+``--auth, -a``          Pass a ``username:password`` pair as
+                        the argument. Or, if you only specify a username
+                        (``-a username``), you'll be prompted for
+                        the password before the request is sent.
+                        To send a an empty password, pass ``username:``.
+
+``--auth-type``         Specify the auth mechanism. Possible values are
+                        ``basic`` and ``digest``. The default value is
+                        ``basic`` so it can often be omitted.
+===================     ======================================================
+
+
+Basic auth:
+
+
+.. code-block:: bash
+
+    $ http -a username:password example.org
+
+
+Digest auth:
+
+
+.. code-block:: bash
+
+    $ http --auth-type=digest -a username:password example.org
+
+
+With password prompt:
+
+.. code-block:: bash
+
+    $ http -a username example.org
+
+
+==============
+Output Options
+==============
+
+By default, HTTPie outputs the whole response message (headers as well as the
+body).
+
+You can control what should be printed via several options:
+
+=================   =====================================================
+``--headers, -h``   Only the response headers are printed.
+``--body, -b``      Only the response body is printed.
+``--verbose, -v``   Print the whole HTTP exchange (request and response).
+``--print, -p``     Selects parts of the HTTP exchange.
+=================   =====================================================
+
+``--verbose`` can often be useful for debugging the request and generating
+documentation examples:
+
+.. code-block:: bash
+
+    $ http --verbose PUT httpbin.org/put hello=world
+    PUT /put HTTP/1.1
+    Accept: application/json
+    Accept-Encoding: identity, deflate, compress, gzip
+    Content-Type: application/json; charset=utf-8
+    Host: httpbin.org
+    User-Agent: HTTPie/0.2.7dev
+
+    {
+        "hello": "world"
+    }
+
+
+    HTTP/1.1 200 OK
+    Connection: keep-alive
+    Content-Length: 477
+    Content-Type: application/json
+    Date: Sun, 05 Aug 2012 00:25:23 GMT
+    Server: gunicorn/0.13.4
+
+    {
+        […]
+    }
+
+
+All the other options are just a shortcut for ``--print`` / ``-p``.
+It accepts a string of characters each of which represents a specific part of
+the HTTP exchange:
+
+==========  ==================
+Character   Stands for
+==========  ==================
+``H``       Request headers.
+``B``       Request body.
+``h``       Response headers.
+``b``       Response body.
+==========  ==================
+
+Print both the request and response headers:
+
+.. code-block:: bash
+
+    $ http --print=Hh PUT httpbin.org/put hello=world
+
+
+-------------------------
+Conditional Body Download
+-------------------------
+
+As an optimization, the response body is downloaded from the server
+only if it's part of the output. This is similar to performing a ``HEAD``
+request, except that it applies to any HTTP method you use.
+
+Let's say that there is an API that returns the whole resource when it is
+updated, but you are only interested in the response headers to see the
+status code after the update:
+
+.. code-block:: bash
+
+    $ http --headers PATCH example.org/Really-Huge-Resource name='New Name'
+
+
+Since we are only printing the HTTP headers here, the connection to server
+is closed as soon as all the response headers have been received.
+Therefore, bandwidth and time isn't wasted downloading the body
+which you don't care about.
+
+The response headers are downloaded always, even if they are not part of
+the output
+
+
+================
+Redirected Input
+================
+
+**A universal method for passing request data is through redirected** ``stdin``
+(standard input). Such data is buffered and then with no further processing
+used as the request body. There are multiple useful ways to use piping:
+
+Redirect from a file:
+
+.. code-block:: bash
+
+    $ http PUT example.com/person/1 X-API-Token:123 < person.json
+
+
+Or the output of another program:
+
+.. code-block:: bash
+
+    $ grep /var/log/httpd/error_log '401 Unauthorized' | http POST example.org/intruders
+
+
+You can use ``echo`` for simple data:
+
+.. code-block:: bash
+
+    $ echo '{"name": "John"}' | http PATCH example.com/person/1 X-API-Token:123
+
+
+You can even pipe web services together using HTTPie:
+
+.. code-block:: bash
+
+    $ http GET https://api.github.com/repos/jkbr/httpie | http POST httpbin.org/post
+
+
+You can use ``cat`` to enter multiline data on the terminal:
+
+.. code-block:: bash
+
+    $ cat | http POST example.com⏎
+    <paste>
+    ^D
+
+
+.. code-block:: bash
+
+    $ cat | http POST example.com/todos Content-Type:text/plain⏎
+    - buy milk
+    - call parents
+    ^D
+
+
+On OS X, you can send the contents of the clipboard with ``pbpaste``:
+
+.. code-block:: bash
+
+    $ pbpaste | http PUT example.com
+
+
+Passing data through ``stdin`` cannot be combined with data fields specified
+on the command line.
+
+
+-------------------------
+Body Data From a Filename
+-------------------------
+
+**An alternative to redirected** ``stdin`` is specifying a filename (as
+``@/path/to/file``) whose content is used as if it came from ``stdin``.
+
+It has the advantage that **the** ``Content-Type``
+**header will automatically be set** to the appropriate value based on the
+filename extension. For example, the following request sends the
+verbatim contents of that XML file with ``Content-Type: application/xml``:
+
+.. code-block:: bash
+
+    $ http PUT httpbin.org/put @/data/file.xml
+
+
+=================
+Terminal Output
+=================
+
+HTTPie does several things by default to make its terminal output easy to read.
+
+
+---------------------
+Colors and Formatting
+---------------------
+
+Syntax highlighting is applied to HTTP headers and bodies (where it makes
+sense). Also, the following formatting is used:
+
+* HTTP headers are sorted by name.
+* JSON data is indented, sorted by keys, and unicode escapes are converted
+  to the characters they represent.
+
+Colorizing and formatting can be disabled with ``--ugly, -u``.
+
+
+-----------
+Binary data
+-----------
+
+Binary data is suppressed for terminal output, which makes it safe to perform
+requests to URLs send back binary data. Binary data is suppressed also in
+redirected, but prettified output. The connection is closed as soon as we know
+that the response body is binary,
+
+.. code-block:: bash
+
+    http example.org/File.mov
+
+
+You will immediately see something like this:
+
+.. code-block:: http
+
+    HTTP/1.1 200 OK
+    Accept-Ranges: bytes
+    Content-Encoding: gzip
+    Content-Type: video/quicktime
+    Transfer-Encoding: chunked
+
+    +-----------------------------------------+
+    | NOTE: binary data not shown in terminal |
+    +-----------------------------------------+
+
+
+=================
+Redirected Output
+=================
+
+HTTPie uses **different defaults** for redirected output than for
+`terminal output`_:
+
+* Formatting and colors aren't applied (unless ``--pretty`` is set).
+* Only the response body is printed (unless one of the `output options`_ is set).
+* Also, binary data isn't suppressed.
+
+The reason is to make piping HTTPie's output to another programs and
+downloading files work with no extra flags. Most of the time, only the raw
+response body is of an interest when the output is redirected.
+
+Download a file:
+
+.. code-block:: bash
+
+    $ http example.org/Movie.mov > Movie.mov
+
+
+Download an image of Octocat, resize it using ImageMagick, upload it elsewhere:
+
+.. code-block:: bash
+
+    $ http octodex.github.com/images/original.jpg | convert - -resize 25% -  | http example.org/Octocats
+
+
+Force colorizing and formatting, and show both the request and response in
+``less`` pager:
+
+.. code-block:: bash
+
+    $ http --pretty --verbose example.org | less -R
+
+
+==================
+Streamed Responses
+==================
+
+Responses are downloaded and printed in chunks, which allows for streaming
+and large file downloads without using too much RAM. However, when
+`colors and formatting`_ are applied, the whole response is buffered and only
+then processed at once.
+
+
+You can use the ``--stream, -S`` flag to make two things happen:
+
+1. The output is flushed in **much smaller chunks** without any buffering,
+   which makes HTTPie behave kind of like ``tail -f`` for URLs.
+
+2. Streaming becomes enabled even when the output is prettified: It will be
+   applied to **each line** of the response and flushed immediately. This makes
+   it possible to have a nice output of long-lived requests, such as one
+   to the Twitter streaming API.
+
+
+Prettified streamed response:
+
+.. code-block:: bash
+
+    $ http --stream -f -a YOUR-TWITTER-NAME https://stream.twitter.com/1/statuses/filter.json track='Justin Bieber'
+
+
+Streamed output by small chunks:
+
+.. code-block:: bash
+
+    # Send each new tweet (JSON object) mentioning "Apple" to another
+    # server as soon as it arrives from the Twitter streaming API:
+    $ http --stream -f -a YOUR-TWITTER-NAME https://stream.twitter.com/1/statuses/filter.json track=Apple \
+    | while read tweet; do echo "$tweet" | http POST example.org/tweets ; done
+
+
+=========
+Scripting
+=========
+
+When using HTTPie from **shell scripts**, it can be handy to set the
+``--check-status`` flag. It instructs HTTPie to exit with an error if the
+HTTP status is one of ``3xx``, ``4xx``, or ``5xx``. The exit status will
+be ``3`` (unless ``--allow-redirects`` is set), ``4``, or ``5``,
+respectively:
+
+.. code-block:: bash
+
+    #!/bin/bash
+
+    if http --check-status HEAD example.org/health &> /dev/null; then
+        echo 'OK!'
+    else
+        case $? in
+            3) echo 'Unexpected HTTP 3xx Redirection!' ;;
+            4) echo 'HTTP 4xx Client Error!' ;;
+            5) echo 'HTTP 5xx Server Error!' ;;
+            *) echo 'Other Error!' ;;
+        esac
+    fi
+
+
+================
+Interface Design
+================
+
+The syntax of the command arguments closely corresponds to the actual HTTP
+requests sent over the wire. It has the advantage  that it's easy to remember
+and read. It is often possible to translate an HTTP request to an HTTPie
+argument list just by inlining the request elements. For example, compare this
+HTTP request:
+
+.. code-block:: http
+
+    POST /collection HTTP/1.1
+    X-API-Key: 123
+    User-Agent: Bacon/1.0
+    Content-Type: application/x-www-form-urlencoded
+
+    name=value&name2=value2
+
+
+with the HTTPie command that sends it:
+
+.. code-block:: bash
+
+    $ http -f POST example.org/collection \
+      X-API-Key:123 \
+      User-Agent:Bacon/1.0 \
+      name=value \
+      name2=value2
+
+
+Notice that both the order of elements and the syntax is very similar,
+and that only a small portion of the command is used to control HTTPie and
+doesn't directly correspond to any part of the request (here it's only ``-f``
+asking HTTPie to send a form request).
+
+The two modes, ``--pretty, -p`` (default for terminal) and ``--ugly, -u``
+(default for redirected output), allow for both user-friendly interactive use
+and usage from scripts, where HTTPie serves as a generic HTTP client.
+
+
+==========
+Contribute
+==========
+
+Bug reports and code and documentation patches are greatly appretiated. You can
+also help by using the development version of HTTPie and reporting any bugs you
+might encounter.
+
+Before working on a new feature or a bug, please browse the `existing issues`_
+to see whether it has been previously discussed. If the change in question
+is a bigger one, it's always good to discuss before your starting working on
+it.
+
+Then fork and clone `the repository`_.
+
+It's very useful to point the ``http`` command to your local branch during
+development. To do so, install HTTPie with ``pip`` in editable mode:
+
+.. code-block:: bash
+
+    $ pip install --upgrade --force-reinstall --editable .
+
+
+Please run the existing suite of tests before a pull request is submitted:
+
+.. code-block:: bash
+
+    python setup.py test
+
+
+`Tox`_ can also be used to conveniently run tests in all of the
+`supported Python environments`_:
+
+.. code-b®lock:: bash
+
+    # Install tox
+    pip install tox
+
+    # Run tests
+    tox
+
+
+Don't forget to add yourself to `AUTHORS`_.
+
+
+=======
+Authors
+=======
+
+`Jakub Roztocil`_  (`@jakubroztocil`_) created HTTPie and `these fine people`_
+have contributed.
+
+
+=======
+Licence
+=======
+
+Please see `LICENSE`_.
+
+
+=========
+Changelog
+=========
+
+* `0.2.8dev`_
+* `0.2.7`_ (2012-08-07)
+    * Compatibility with Requests 0.13.6.
+    * Streamed terminal output. ``--stream`` / ``-S`` can be used to enable
+      streaming also with ``--pretty`` and to ensure a more frequent output
+      flushing.
+    * Support for efficient large file downloads.
+    * Sort headers by name (unless ``--ugly``).
+    * Response body is fetched only when needed (e.g., not with ``--headers``).
+    * Improved content type matching.
+    * Updated Solarized color scheme.
+    * Windows: Added ``--output FILE`` to store output into a file
+      (piping results in corrupted data on Windows).
+    * Proper handling of binary requests and responses.
+    * Fixed printing of ``multipart/form-data`` requests.
+    * Renamed ``--traceback`` to ``--debug``.
+* `0.2.6`_ (2012-07-26)
+    * The short option for ``--headers`` is now ``-h`` (``-t`` has been
+      removed, for usage use ``--help``).
+    * Form data and URL parameters can have multiple fields with the same name
+      (e.g.,``http -f url a=1 a=2``).
+    * Added ``--check-status`` to exit with an error on HTTP 3xx, 4xx and
+      5xx (3, 4, and 5, respectively).
+    * If the output is piped to another program or redirected to a file,
+      the default behaviour is to only print the response body.
+      (It can still be overwritten via the ``--print`` flag.)
+    * Improved highlighting of HTTP headers.
+    * Added query string parameters (``param==value``).
+    * Added support for terminal colors under Windows.
+* `0.2.5`_ (2012-07-17)
+    * Unicode characters in prettified JSON now don't get escaped for
+      improved readability.
+    * --auth now prompts for a password if only a username provided.
+    * Added support for request payloads from a file path with automatic
+      ``Content-Type`` (``http URL @/path``).
+    * Fixed missing query string when displaying the request headers via
+      ``--verbose``.
+    * Fixed Content-Type for requests with no data.
+* `0.2.2`_ (2012-06-24)
+    * The ``METHOD`` positional argument can now be omitted (defaults to
+      ``GET``, or to ``POST`` with data).
+    * Fixed --verbose --form.
+    * Added support for `Tox`_.
+* `0.2.1`_ (2012-06-13)
+    * Added compatibility with ``requests-0.12.1``.
+    * Dropped custom JSON and HTTP lexers in favor of the ones newly included
+      in ``pygments-1.5``.
+* `0.2.0`_ (2012-04-25)
+    * Added Python 3 support.
+    * Added the ability to print the HTTP request as well as the response
+      (see ``--print`` and ``--verbose``).
+    * Added support for Digest authentication.
+    * Added file upload support
+      (``http -f POST file_field_name@/path/to/file``).
+    * Improved syntax highlighting for JSON.
+    * Added support for field name escaping.
+    * Many bug fixes.
+* `0.1.6`_ (2012-03-04)
+
+
+.. _Requests: http://python-requests.org
+.. _Pygments: http://pygments.org/
+.. _pip: http://www.pip-installer.org/en/latest/index.html
+.. _Tox: http://tox.testrun.org
+.. _Github API: http://developer.github.com/v3/issues/comments/#create-a-comment
+.. _supported Python environments: https://github.com/jkbr/httpie/blob/master/tox.ini
+.. _Ubuntu: http://packages.ubuntu.com/httpie
+.. _Debian: http://packages.debian.org/httpie
+.. _the repository: https://github.com/jkbr/httpie
+.. _these fine people: https://github.com/jkbr/httpie/contributors
+.. _Jakub Roztocil: http://roztocil.name
+.. _@jakubroztocil: https://twitter.com/jakubroztocil
+.. _existing issues: https://github.com/jkbr/httpie/issues?state=open
+.. _0.1.6: https://github.com/jkbr/httpie/compare/0.1.4...0.1.6
+.. _0.2.0: https://github.com/jkbr/httpie/compare/0.1.6...0.2.0
+.. _0.2.1: https://github.com/jkbr/httpie/compare/0.2.0...0.2.1
+.. _0.2.2: https://github.com/jkbr/httpie/compare/0.2.1...0.2.2
+.. _0.2.5: https://github.com/jkbr/httpie/compare/0.2.2...0.2.5
+.. _0.2.6: https://github.com/jkbr/httpie/compare/0.2.5...0.2.6
+.. _0.2.7: https://github.com/jkbr/httpie/compare/0.2.5...0.2.7
+.. _0.2.8dev: https://github.com/jkbr/httpie/compare/0.2.7...master
+.. _README for stable version: https://github.com/jkbr/httpie/tree/0.2.6#readme
+.. _AUTHORS: https://github.com/jkbr/httpie/blob/master/AUTHORS.rst
+.. _LICENSE: https://github.com/jkbr/httpie/blob/master/LICENSE

httpie/__init__.py

@@ -3,5 +3,14 @@ HTTPie - cURL for humans.
 
 """
 __author__ = 'Jakub Roztocil'
-__version__ = '0.1.4'
+__version__ = '0.2.7'
 __licence__ = 'BSD'
+
+
+class EXIT:
+    OK = 0
+    ERROR = 1
+    # Used only when requested:
+    ERROR_HTTP_3XX = 3
+    ERROR_HTTP_4XX = 4
+    ERROR_HTTP_5XX = 5

httpie/__main__.py

@@ -0,0 +1,10 @@
+#!/usr/bin/env python
+"""The main entry point. Invoke as `http' or `python -m httpie'.
+
+"""
+import sys
+from .core import main
+
+
+if __name__ == '__main__':
+    sys.exit(main())

httpie/cli.py

@@ -0,0 +1,274 @@
+"""CLI arguments definition.
+
+NOTE: the CLI interface may change before reaching v1.0.
+
+"""
+import argparse
+
+from requests.compat import is_windows
+
+from . import __doc__
+from . import __version__
+from .output import AVAILABLE_STYLES, DEFAULT_STYLE
+from .input import (Parser, AuthCredentialsArgType, KeyValueArgType,
+                    PRETTIFY_STDOUT_TTY_ONLY,
+                    SEP_PROXY, SEP_CREDENTIALS, SEP_GROUP_ITEMS,
+                    OUT_REQ_HEAD, OUT_REQ_BODY, OUT_RESP_HEAD,
+                    OUT_RESP_BODY, OUTPUT_OPTIONS)
+
+
+def _(text):
+    """Normalize whitespace."""
+    return ' '.join(text.strip().split())
+
+
+parser = Parser(description='%s <http://httpie.org>' % __doc__.strip())
+parser.add_argument('--version', action='version', version=__version__)
+
+
+# Content type.
+#############################################
+
+group_type = parser.add_mutually_exclusive_group(required=False)
+group_type.add_argument(
+    '--json', '-j', action='store_true',
+    help=_('''
+        (default) Data items from the command
+        line are serialized as a JSON object.
+        The Content-Type and Accept headers
+        are set to application/json (if not specified).
+    ''')
+)
+group_type.add_argument(
+    '--form', '-f', action='store_true',
+    help=_('''
+        Data items from the command line are serialized as form fields.
+        The Content-Type is set to application/x-www-form-urlencoded
+        (if not specified).
+        The presence of any file fields results
+        into a multipart/form-data request.
+     ''')
+)
+
+
+# Output options.
+#############################################
+
+
+parser.add_argument(
+    '--output', '-o', type=argparse.FileType('w+b'),
+    metavar='FILE',
+    help= argparse.SUPPRESS if not is_windows else _(
+        '''
+        Save output to FILE.
+        This option is a replacement for piping output to FILE,
+        which would on Windows result into corrupted data
+        being saved.
+
+        '''
+    )
+)
+
+prettify = parser.add_mutually_exclusive_group(required=False)
+prettify.add_argument(
+    '--pretty', dest='prettify', action='store_true',
+    default=PRETTIFY_STDOUT_TTY_ONLY,
+    help=_('''
+        If stdout is a terminal, the response is prettified
+        by default (colorized and indented if it is JSON).
+        This flag ensures prettifying even when stdout is redirected.
+    ''')
+)
+prettify.add_argument(
+    '--ugly', '-u', dest='prettify', action='store_false',
+    help=_('''
+        Do not prettify the response.
+    ''')
+)
+
+output_options = parser.add_mutually_exclusive_group(required=False)
+output_options.add_argument('--print', '-p', dest='output_options',
+    help=_('''
+        String specifying what the output should contain:
+        "{request_headers}" stands for the request headers,  and
+        "{request_body}" for the request body.
+        "{response_headers}" stands for the response headers and
+        "{response_body}" for response the body.
+        The default behaviour is "hb" (i.e., the response
+        headers and body is printed), if standard output is not redirected.
+        If the output is piped to another program or to a file,
+        then only the body is printed by default.
+    '''.format(
+        request_headers=OUT_REQ_HEAD,
+        request_body=OUT_REQ_BODY,
+        response_headers=OUT_RESP_HEAD,
+        response_body=OUT_RESP_BODY,
+    ))
+)
+output_options.add_argument(
+    '--verbose', '-v', dest='output_options',
+    action='store_const', const=''.join(OUTPUT_OPTIONS),
+    help=_('''
+        Print the whole request as well as the response.
+        Shortcut for --print={0}.
+    '''.format(''.join(OUTPUT_OPTIONS)))
+)
+output_options.add_argument(
+    '--headers', '-h', dest='output_options',
+    action='store_const', const=OUT_RESP_HEAD,
+    help=_('''
+        Print only the response headers.
+        Shortcut for --print={0}.
+    '''.format(OUT_RESP_HEAD))
+)
+output_options.add_argument(
+    '--body', '-b', dest='output_options',
+    action='store_const', const=OUT_RESP_BODY,
+    help=_('''
+        Print only the response body.
+        Shortcut for --print={0}.
+    '''.format(OUT_RESP_BODY))
+)
+
+parser.add_argument(
+    '--style', '-s', dest='style', default=DEFAULT_STYLE, metavar='STYLE',
+    choices=AVAILABLE_STYLES,
+    help=_('''
+        Output coloring style, one of %s. Defaults to "%s".
+        For this option to work properly, please make sure that the
+        $TERM environment variable is set to "xterm-256color" or similar
+        (e.g., via `export TERM=xterm-256color' in your ~/.bashrc).
+    ''') % (', '.join(sorted(AVAILABLE_STYLES)), DEFAULT_STYLE)
+)
+
+parser.add_argument('--stream', '-S', action='store_true', default=False, help=_(
+    '''
+    Always stream the output by line, i.e., behave like `tail -f'.
+
+    Without --stream and with --pretty (either set or implied),
+    HTTPie fetches the whole response before it outputs the processed data.
+
+    Set this option when you want to continuously display a prettified
+    long-lived response, such as one from the Twitter streaming API.
+
+    It is useful also without --pretty: It ensures that the output is flushed
+    more often and in smaller chunks.
+
+    '''
+))
+parser.add_argument(
+    '--check-status', default=False, action='store_true',
+    help=_('''
+        By default, HTTPie exits with 0 when no network or other fatal
+        errors occur.
+
+        This flag instructs HTTPie to also check the HTTP status code and
+        exit with an error if the status indicates one.
+
+        When the server replies with a 4xx (Client Error) or 5xx
+        (Server Error) status code, HTTPie exits with 4 or 5 respectively.
+        If the response is a 3xx (Redirect) and --allow-redirects
+        hasn't been set, then the exit status is 3.
+
+        Also an error message is written to stderr if stdout is redirected.
+
+    ''')
+)
+
+# ``requests.request`` keyword arguments.
+parser.add_argument(
+    '--auth', '-a',
+    type=AuthCredentialsArgType(SEP_CREDENTIALS),
+    help=_('''
+        username:password.
+        If only the username is provided (-a username),
+        HTTPie will prompt for the password.
+    '''),
+)
+
+parser.add_argument(
+    '--auth-type', choices=['basic', 'digest'], default='basic',
+    help=_('''
+        The authentication mechanism to be used.
+        Defaults to "basic".
+    ''')
+)
+
+parser.add_argument(
+    '--verify', default='yes',
+    help=_('''
+        Set to "no" to skip checking the host\'s SSL certificate.
+        You can also pass the  path to a CA_BUNDLE
+        file for private certs. You can also set
+        the REQUESTS_CA_BUNDLE  environment variable.
+        Defaults to "yes".
+    ''')
+)
+parser.add_argument(
+    '--proxy', default=[], action='append',
+    type=KeyValueArgType(SEP_PROXY),
+    help=_('''
+        String mapping protocol to the URL of the proxy
+        (e.g. http:foo.bar:3128).
+    ''')
+)
+parser.add_argument(
+    '--allow-redirects', default=False, action='store_true',
+    help=_('''
+        Set this flag if full redirects are allowed
+        (e.g. re-POST-ing of data at new ``Location``)
+    ''')
+)
+parser.add_argument(
+    '--timeout', type=float,
+    help=_('''
+        Float describes the timeout of the request
+        (Use socket.setdefaulttimeout() as fallback).
+    ''')
+)
+parser.add_argument(
+    '--debug', action='store_true', default=False,
+    help=_('''
+        Prints exception traceback should one occur and other
+        information useful for debugging HTTPie itself.
+    ''')
+)
+
+
+# Positional arguments.
+#############################################
+
+parser.add_argument(
+    'method', metavar='METHOD',
+    nargs='?',
+    default=None,
+    help=_('''
+        The HTTP method to be used for the request
+        (GET, POST, PUT, DELETE, PATCH, ...).
+        If this argument is omitted, then HTTPie
+        will guess the HTTP method. If there is some
+        data to be sent, then it will be POST, otherwise GET.
+    ''')
+)
+parser.add_argument(
+    'url', metavar='URL',
+    help=_('''
+        The protocol defaults to http:// if the
+        URL does not include one.
+    ''')
+)
+parser.add_argument(
+    'items', nargs='*',
+    metavar='ITEM',
+    type=KeyValueArgType(*SEP_GROUP_ITEMS),
+    help=_('''
+        A key-value pair whose type is defined by the
+        separator used. It can be an HTTP header (header:value),
+        a data field to be used in the request body (field_name=value),
+        a raw JSON data field (field_name:=value),
+        a query parameter (name==value),
+        or a file field (field_name@/path/to/file).
+        You can use a backslash to escape a colliding
+        separator in the field name.
+    ''')
+)

httpie/core.py

@@ -0,0 +1,144 @@
+"""This module provides the main functionality of HTTPie.
+
+Invocation flow:
+
+    1. Read, validate and process the input (args, `stdin`).
+    2. Create and send a request.
+    3. Stream, and possibly process and format, the requested parts
+       of the request-response exchange.
+    4. Simultaneously write to `stdout`
+    5. Exit.
+
+"""
+import sys
+import json
+import errno
+
+import requests
+import requests.auth
+from requests.compat import str
+
+from .cli import parser
+from .models import Environment
+from .output import output_stream, write
+from . import EXIT
+
+
+FORM = 'application/x-www-form-urlencoded; charset=utf-8'
+JSON = 'application/json; charset=utf-8'
+
+
+def get_response(args):
+    """Send the request and return a `request.Response`."""
+
+    auto_json = args.data and not args.form
+    if args.json or auto_json:
+        if 'Content-Type' not in args.headers and args.data:
+            args.headers['Content-Type'] = JSON
+
+        if 'Accept' not in args.headers:
+            # Default Accept to JSON as well.
+            args.headers['Accept'] = 'application/json'
+
+        if isinstance(args.data, dict):
+            # If not empty, serialize the data `dict` parsed from arguments.
+            # Otherwise set it to `None` avoid sending "{}".
+            args.data = json.dumps(args.data) if args.data else None
+
+    elif args.form:
+        if not args.files and 'Content-Type' not in args.headers:
+            # If sending files, `requests` will set
+            # the `Content-Type` for us.
+            args.headers['Content-Type'] = FORM
+
+    credentials = None
+    if args.auth:
+        credentials = {
+            'basic': requests.auth.HTTPBasicAuth,
+            'digest': requests.auth.HTTPDigestAuth,
+        }[args.auth_type](args.auth.key, args.auth.value)
+
+    return requests.request(
+        prefetch=False,
+        method=args.method.lower(),
+        url=args.url,
+        headers=args.headers,
+        data=args.data,
+        verify={'yes': True, 'no': False}.get(args.verify, args.verify),
+        timeout=args.timeout,
+        auth=credentials,
+        proxies=dict((p.key, p.value) for p in args.proxy),
+        files=args.files,
+        allow_redirects=args.allow_redirects,
+        params=args.params,
+    )
+
+
+def get_exist_status(code, allow_redirects=False):
+    """Translate HTTP status code to exit status."""
+    if 300 <= code <= 399 and not allow_redirects:
+        # Redirect
+        return EXIT.ERROR_HTTP_3XX
+    elif 400 <= code <= 499:
+        # Client Error
+        return EXIT.ERROR_HTTP_4XX
+    elif 500 <= code <= 599:
+        # Server Error
+        return EXIT.ERROR_HTTP_5XX
+    else:
+        return EXIT.OK
+
+
+def main(args=sys.argv[1:], env=Environment()):
+    """Run the main program and write the output to ``env.stdout``.
+
+    Return exit status.
+
+    """
+
+    def error(msg, *args):
+        msg = msg % args
+        env.stderr.write('\nhttp: error: %s\n' % msg)
+
+    debug = '--debug' in args
+    status = EXIT.OK
+
+    try:
+        args = parser.parse_args(args=args, env=env)
+        response = get_response(args)
+
+        if args.check_status:
+            status = get_exist_status(response.status_code,
+                                      args.allow_redirects)
+            if status and not env.stdout_isatty:
+                error('%s %s', response.raw.status, response.raw.reason)
+
+        stream = output_stream(args, env, response.request, response)
+
+        try:
+            write(stream=stream,
+                  outfile=env.stdout,
+                  flush=env.stdout_isatty or args.stream)
+
+        except IOError as e:
+            if not debug and e.errno == errno.EPIPE:
+                # Ignore broken pipes unless --debug.
+                env.stderr.write('\n')
+            else:
+                raise
+
+    except (KeyboardInterrupt, SystemExit):
+        if debug:
+            raise
+        env.stderr.write('\n')
+        status = EXIT.ERROR
+
+    except Exception as e:
+        # TODO: distinguish between expected and unexpected errors.
+        #       network errors vs. bugs, etc.
+        if debug:
+            raise
+        error('%s: %s', type(e).__name__, str(e))
+        status = EXIT.ERROR
+
+    return status

httpie/httpie.py

@@ -1,183 +0,0 @@
-#!/usr/bin/env python
-import os
-import sys
-import json
-import argparse
-from collections import namedtuple
-import requests
-from requests.structures import CaseInsensitiveDict
-from . import pretty
-from . import __version__ as version
-from . import __doc__ as doc
-
-
-DEFAULT_UA = 'HTTPie/%s' % version
-SEP_COMMON = ':'
-SEP_DATA = '='
-TYPE_FORM = 'application/x-www-form-urlencoded; charset=utf-8'
-TYPE_JSON = 'application/json; charset=utf-8'
-
-
-KeyValue = namedtuple('KeyValue', ['key', 'value', 'sep'])
-
-
-class KeyValueType(object):
-
-    def __init__(self, separators):
-        self.separators = separators
-
-    def __call__(self, string):
-        found = dict((string.find(sep), sep)
-                     for sep in self.separators
-                     if string.find(sep) != -1)
-
-        if not found:
-            raise argparse.ArgumentTypeError(
-                '"%s" is not a valid value' % string)
-        sep = found[min(found.keys())]
-        key, value = string.split(sep, 1)
-        return KeyValue(key=key, value=value, sep=sep)
-
-
-parser = argparse.ArgumentParser(
-    description=doc.strip())
-
-
-# Content type.
-group_type = parser.add_mutually_exclusive_group(required=False)
-group_type.add_argument('--json', '-j', action='store_true',
-                   help='Serialize data items as a JSON object and set'
-                        ' Content-Type to application/json, if not specified.')
-group_type.add_argument('--form', '-f', action='store_true',
-                   help='Serialize data items as form values and set'
-                        ' Content-Type to application/x-www-form-urlencoded,'
-                        ' if not specified.')
-
-# Output options.
-parser.add_argument('--traceback', action='store_true', default=False,
-                  help='Print a full exception traceback should one'
-                       ' be raised by `requests`.')
-parser.add_argument('--ugly', '-u', help='Do not prettify the response.',
-                     dest='prettify', action='store_false', default=True)
-group_only = parser.add_mutually_exclusive_group(required=False)
-group_only.add_argument('--headers', '-t', dest='print_body',
-                        action='store_false', default=True,
-                        help='Print only the response headers.')
-group_only.add_argument('--body', '-b', dest='print_headers',
-                        action='store_false', default=True,
-                        help='Print only the response body.')
-
-
-# ``requests.request`` keyword arguments.
-parser.add_argument('--auth', '-a', help='username:password',
-                    type=KeyValueType(SEP_COMMON))
-parser.add_argument('--verify',
-                    help='Set to "yes" to check the host\'s SSL certificate.'
-                         ' You can also pass the  path to a CA_BUNDLE'
-                         ' file for private certs. You can also set '
-                         'the REQUESTS_CA_BUNDLE  environment variable.')
-parser.add_argument('--proxy', default=[], action='append',
-                    type=KeyValueType(SEP_COMMON),
-                    help='String mapping protocol to the URL of the proxy'
-                         ' (e.g. http:foo.bar:3128).')
-parser.add_argument('--allow-redirects', default=False, action='store_true',
-                    help='Set this flag if full redirects are allowed'
-                         ' (e.g. re-POST-ing of data at new ``Location``)')
-parser.add_argument('--file', metavar='PATH', type=argparse.FileType(),
-                    default=[], action='append',
-                    help='File to multipart upload')
-parser.add_argument('--timeout', type=float,
-                    help='Float describes the timeout of the request'
-                         ' (Use socket.setdefaulttimeout() as fallback).')
-
-# Positional arguments.
-parser.add_argument('method',
-                    help='HTTP method to be used for the request'
-                         ' (GET, POST, PUT, DELETE, PATCH, ...).')
-parser.add_argument('url', metavar='URL',
-                    help='Protocol defaults to http:// if the'
-                         ' URL does not include it.')
-parser.add_argument('items', metavar='item', nargs='*',
-                    type=KeyValueType([SEP_COMMON, SEP_DATA]),
-                    help='HTTP header (key:value) or data field (key=value)')
-
-
-def main():
-    args = parser.parse_args()
-
-    # Parse request headers and data from the command line.
-    headers = CaseInsensitiveDict()
-    headers['User-Agent'] = DEFAULT_UA
-    data = {}
-    for item in args.items:
-        if item.sep == SEP_COMMON:
-            target = headers
-        else:
-            if not sys.stdin.isatty():
-                parser.error('Request body (stdin) and request '
-                            'data (key=value) cannot be mixed.')
-            target = data
-        target[item.key] = item.value
-
-    if not sys.stdin.isatty():
-        data = sys.stdin.read()
-
-    # JSON/Form content type.
-    if args.json or (not args.form and data):
-        if sys.stdin.isatty():
-            data = json.dumps(data)
-        if 'Content-Type' not in headers and (data or args.json):
-            headers['Content-Type'] = TYPE_JSON
-    elif 'Content-Type' not in headers:
-        headers['Content-Type'] = TYPE_FORM
-
-    # Fire the request.
-    try:
-        response = requests.request(
-            method=args.method.lower(),
-            url=args.url if '://' in args.url else 'http://%s' % args.url,
-            headers=headers,
-            data=data,
-            verify=True if args.verify == 'yes' else args.verify,
-            timeout=args.timeout,
-            auth=(args.auth.key, args.auth.value) if args.auth else None,
-            proxies=dict((p.key, p.value) for p in args.proxy),
-            files=dict((os.path.basename(f.name), f) for f in args.file),
-        )
-    except (KeyboardInterrupt, SystemExit) as e:
-        sys.stderr.write('\n')
-        sys.exit(1)
-    except Exception as e:
-        if args.traceback:
-            raise
-        sys.stderr.write(str(e.message) + '\n')
-        sys.exit(1)
-
-    # Display the response.
-    encoding = response.encoding or 'ISO-8859-1'
-    original = response.raw._original_response
-    status_line, headers, body = (
-        u'HTTP/{version} {status} {reason}'.format(
-            version='.'.join(str(original.version)),
-            status=original.status, reason=original.reason,
-        ),
-        str(original.msg).decode(encoding),
-        response.content.decode(encoding) if response.content else u''
-    )
-
-    if args.prettify and sys.stdout.isatty():
-        if args.print_headers:
-            status_line = pretty.prettify_http(status_line).strip()
-            headers = pretty.prettify_http(headers)
-        if args.print_body:
-            body = pretty.prettify_body(body,
-                response.headers['content-type'])
-
-    if args.print_headers:
-        print status_line
-        print headers
-    if args.print_body:
-        print body
-
-if __name__ == '__main__':
-    main()

httpie/input.py

@@ -0,0 +1,457 @@
+"""Parsing and processing of CLI input (args, auth credentials, files, stdin).
+
+"""
+import os
+import sys
+import re
+import json
+import argparse
+import mimetypes
+import getpass
+from io import BytesIO
+
+try:
+    from collections import OrderedDict
+except ImportError:
+    OrderedDict = dict
+
+from requests.structures import CaseInsensitiveDict
+from requests.compat import str, urlparse
+
+from . import __version__
+
+
+HTTP_POST = 'POST'
+HTTP_GET = 'GET'
+HTTP = 'http://'
+HTTPS = 'https://'
+
+
+# Various separators used in args
+SEP_HEADERS = ':'
+SEP_CREDENTIALS = ':'
+SEP_PROXY = ':'
+SEP_DATA = '='
+SEP_DATA_RAW_JSON = ':='
+SEP_FILES = '@'
+SEP_QUERY = '=='
+
+# Separators that become request data
+SEP_GROUP_DATA_ITEMS = frozenset([
+    SEP_DATA,
+    SEP_DATA_RAW_JSON,
+    SEP_FILES
+])
+
+# Separators allowed in ITEM arguments
+SEP_GROUP_ITEMS = frozenset([
+    SEP_HEADERS,
+    SEP_QUERY,
+    SEP_DATA,
+    SEP_DATA_RAW_JSON,
+    SEP_FILES
+])
+
+
+# Output options
+OUT_REQ_HEAD = 'H'
+OUT_REQ_BODY = 'B'
+OUT_RESP_HEAD = 'h'
+OUT_RESP_BODY = 'b'
+
+OUTPUT_OPTIONS = frozenset([
+    OUT_REQ_HEAD,
+    OUT_REQ_BODY,
+    OUT_RESP_HEAD,
+    OUT_RESP_BODY
+])
+
+
+# Defaults
+OUTPUT_OPTIONS_DEFAULT = OUT_RESP_HEAD + OUT_RESP_BODY
+OUTPUT_OPTIONS_DEFAULT_STDOUT_REDIRECTED = OUT_RESP_BODY
+PRETTIFY_STDOUT_TTY_ONLY = object()
+DEFAULT_UA = 'HTTPie/%s' % __version__
+
+
+class Parser(argparse.ArgumentParser):
+    """Adds additional logic to `argparse.ArgumentParser`.
+
+    Handles all input (CLI args, file args, stdin), applies defaults,
+    and performs extra validation.
+
+    """
+
+    def __init__(self, *args, **kwargs):
+        kwargs['add_help'] = False
+        super(Parser, self).__init__(*args, **kwargs)
+        # Help only as --help (-h is used for --headers).
+        self.add_argument('--help',
+            action='help', default=argparse.SUPPRESS,
+            help=argparse._('show this help message and exit'))
+
+    #noinspection PyMethodOverriding
+    def parse_args(self, env, args=None, namespace=None):
+
+        self.env = env
+
+        if env.is_windows and not env.stdout_isatty:
+            self.error('Output redirection is not supported on Windows.'
+                  ' Please use `--output FILE\' instead.')
+
+        args = super(Parser, self).parse_args(args, namespace)
+
+        if args.output:
+            env.stdout = args.output
+            env.stdout_isatty = False
+
+        self._process_output_options(args, env)
+        self._guess_method(args, env)
+        self._parse_items(args)
+
+        if not env.stdin_isatty:
+            self._body_from_file(args, env.stdin)
+
+        if not (args.url.startswith(HTTP) or args.url.startswith(HTTPS)):
+            scheme = HTTPS if env.progname == 'https' else HTTP
+            args.url = scheme + args.url
+
+        if args.auth and not args.auth.has_password():
+            # Stdin already read (if not a tty) so it's save to prompt.
+            args.auth.prompt_password(urlparse(args.url).netloc)
+
+        if args.prettify == PRETTIFY_STDOUT_TTY_ONLY:
+            args.prettify = env.stdout_isatty
+        elif args.prettify and env.is_windows:
+            self.error('Only terminal output can be prettified on Windows.')
+
+        return args
+
+    def _print_message(self, message, file=None):
+        # Sneak in our stderr/stdout.
+        file = {
+            sys.stdout: self.env.stdout,
+            sys.stderr: self.env.stderr,
+            None: self.env.stderr
+        }.get(file, file)
+
+        super(Parser, self)._print_message(message, file)
+
+    def _body_from_file(self, args, fd):
+        """There can only be one source of request data.
+
+        Bytes are always read.
+
+        """
+        if args.data:
+            self.error('Request body (from stdin or a file) and request '
+                       'data (key=value) cannot be mixed.')
+        args.data = getattr(fd, 'buffer', fd).read()
+
+    def _guess_method(self, args, env):
+        """Set `args.method` if not specified to either POST or GET
+        based on whether the request has data or not.
+
+        """
+        if args.method is None:
+            # Invoked as `http URL'.
+            assert not args.items
+            if not env.stdin_isatty:
+                args.method = HTTP_POST
+            else:
+                args.method = HTTP_GET
+
+        # FIXME: False positive, e.g., "localhost" matches but is a valid URL.
+        elif not re.match('^[a-zA-Z]+$', args.method):
+            # Invoked as `http URL item+'. The URL is now in `args.method`
+            # and the first ITEM is now incorrectly in `args.url`.
+            try:
+                # Parse the URL as an ITEM and store it as the first ITEM arg.
+                args.items.insert(
+                    0, KeyValueArgType(*SEP_GROUP_ITEMS).__call__(args.url))
+
+            except argparse.ArgumentTypeError as e:
+                if args.debug:
+                    raise
+                self.error(e.message)
+
+            else:
+                # Set the URL correctly
+                args.url = args.method
+                # Infer the method
+                has_data = not env.stdin_isatty or any(
+                    item.sep in SEP_GROUP_DATA_ITEMS for item in args.items)
+                args.method = HTTP_POST if has_data else HTTP_GET
+
+    def _parse_items(self, args):
+        """Parse `args.items` into `args.headers`, `args.data`,
+        `args.`, and `args.files`.
+
+        """
+        args.headers = CaseInsensitiveDict()
+        args.headers['User-Agent'] = DEFAULT_UA
+        args.data = ParamDict() if args.form else OrderedDict()
+        args.files = OrderedDict()
+        args.params = ParamDict()
+
+        try:
+            parse_items(items=args.items,
+                        headers=args.headers,
+                        data=args.data,
+                        files=args.files,
+                        params=args.params)
+        except ParseError as e:
+            if args.debug:
+                raise
+            self.error(e.message)
+
+        if args.files and not args.form:
+            # `http url @/path/to/file`
+            file_fields = list(args.files.keys())
+            if file_fields != ['']:
+                self.error(
+                    'Invalid file fields (perhaps you meant --form?): %s'
+                    % ','.join(file_fields))
+
+            fn, fd = args.files['']
+            args.files = {}
+            self._body_from_file(args, fd)
+            if 'Content-Type' not in args.headers:
+                mime, encoding = mimetypes.guess_type(fn, strict=False)
+                if mime:
+                    content_type = mime
+                    if encoding:
+                        content_type = '%s; charset=%s' % (mime, encoding)
+                    args.headers['Content-Type'] = content_type
+
+    def _process_output_options(self, args, env):
+        """Apply defaults to output options or validate the provided ones.
+
+        The default output options are stdout-type-sensitive.
+
+        """
+        if not args.output_options:
+            args.output_options = (OUTPUT_OPTIONS_DEFAULT if env.stdout_isatty
+                                else OUTPUT_OPTIONS_DEFAULT_STDOUT_REDIRECTED)
+
+        unknown = set(args.output_options) - OUTPUT_OPTIONS
+        if unknown:
+            self.error('Unknown output options: %s' % ','.join(unknown))
+
+
+class ParseError(Exception):
+    pass
+
+
+class KeyValue(object):
+    """Base key-value pair parsed from CLI."""
+
+    def __init__(self, key, value, sep, orig):
+        self.key = key
+        self.value = value
+        self.sep = sep
+        self.orig = orig
+
+    def __eq__(self, other):
+        return self.__dict__ == other.__dict__
+
+
+class KeyValueArgType(object):
+    """A key-value pair argument type used with `argparse`.
+
+    Parses a key-value arg and constructs a `KeyValue` instance.
+    Used for headers, form data, and other key-value pair types.
+
+    """
+
+    key_value_class = KeyValue
+
+    def __init__(self, *separators):
+        self.separators = separators
+
+    def __call__(self, string):
+        """Parse `string` and return `self.key_value_class()` instance.
+
+        The best of `self.separators` is determined (first found, longest).
+        Back slash escaped characters aren't considered as separators
+        (or parts thereof). Literal back slash characters have to be escaped
+        as well (r'\\').
+
+        """
+
+        class Escaped(str):
+            """Represents an escaped character."""
+
+        def tokenize(s):
+            """Tokenize `s`. There are only two token types - strings
+            and escaped characters:
+
+            >>> tokenize(r'foo\=bar\\baz')
+            ['foo', Escaped('='), 'bar', Escaped('\\'), 'baz']
+
+            """
+            tokens = ['']
+            esc = False
+            for c in s:
+                if esc:
+                    tokens.extend([Escaped(c), ''])
+                    esc = False
+                else:
+                    if c == '\\':
+                        esc = True
+                    else:
+                        tokens[-1] += c
+            return tokens
+
+        tokens = tokenize(string)
+
+        # Sorting by length ensures that the longest one will be
+        # chosen as it will overwrite any shorter ones starting
+        # at the same position in the `found` dictionary.
+        separators = sorted(self.separators, key=len)
+
+        for i, token in enumerate(tokens):
+
+            if isinstance(token, Escaped):
+                continue
+
+            found = {}
+            for sep in separators:
+                pos = token.find(sep)
+                if pos != -1:
+                    found[pos] = sep
+
+            if found:
+                # Starting first, longest separator found.
+                sep = found[min(found.keys())]
+
+                key, value = token.split(sep, 1)
+
+                # Any preceding tokens are part of the key.
+                key = ''.join(tokens[:i]) + key
+
+                # Any following tokens are part of the value.
+                value += ''.join(tokens[i + 1:])
+
+                break
+
+        else:
+            raise argparse.ArgumentTypeError(
+                '"%s" is not a valid value' % string)
+
+        return self.key_value_class(
+            key=key, value=value, sep=sep, orig=string)
+
+
+class AuthCredentials(KeyValue):
+    """Represents parsed credentials."""
+
+    def _getpass(self, prompt):
+        # To allow mocking.
+        return getpass.getpass(prompt)
+
+    def has_password(self):
+        return self.value is not None
+
+    def prompt_password(self, host):
+        try:
+            self.value = self._getpass(
+                'http: password for %s@%s: ' % (self.key, host))
+        except (EOFError, KeyboardInterrupt):
+            sys.stderr.write('\n')
+            sys.exit(0)
+
+
+class AuthCredentialsArgType(KeyValueArgType):
+    """A key-value arg type that parses credentials."""
+
+    key_value_class = AuthCredentials
+
+    def __call__(self, string):
+        """Parse credentials from `string`.
+
+        ("username" or "username:password").
+
+        """
+        try:
+            return super(AuthCredentialsArgType, self).__call__(string)
+        except argparse.ArgumentTypeError:
+            # No password provided, will prompt for it later.
+            return self.key_value_class(
+                key=string,
+                value=None,
+                sep=SEP_CREDENTIALS,
+                orig=string
+            )
+
+
+class ParamDict(OrderedDict):
+    """Multi-value dict for URL parameters and form data."""
+
+    #noinspection PyMethodOverriding
+    def __setitem__(self, key, value):
+        """ If `key` is assigned more than once, `self[key]` holds a
+        `list` of all the values.
+
+        This allows having multiple fields with the same name in form
+        data and URL params.
+
+        """
+        # NOTE: Won't work when used for form data with multiple values
+        # for a field and a file field is present:
+        # https://github.com/kennethreitz/requests/issues/737
+        if key not in self:
+            super(ParamDict, self).__setitem__(key, value)
+        else:
+            if not isinstance(self[key], list):
+                super(ParamDict, self).__setitem__(key, [self[key]])
+            self[key].append(value)
+
+
+def parse_items(items, data=None, headers=None, files=None, params=None):
+    """Parse `KeyValue` `items` into `data`, `headers`, `files`,
+    and `params`.
+
+    """
+    if headers is None:
+        headers = CaseInsensitiveDict()
+    if data is None:
+        data = OrderedDict()
+    if files is None:
+        files = OrderedDict()
+    if params is None:
+        params = ParamDict()
+
+    for item in items:
+
+        value = item.value
+        key = item.key
+
+        if item.sep == SEP_HEADERS:
+            target = headers
+        elif item.sep == SEP_QUERY:
+            target = params
+        elif item.sep == SEP_FILES:
+            try:
+                with open(os.path.expanduser(value), 'rb') as f:
+                    value = (os.path.basename(value),
+                             BytesIO(f.read()))
+            except IOError as e:
+                raise ParseError(
+                    'Invalid argument "%s": %s' % (item.orig, e))
+            target = files
+
+        elif item.sep in [SEP_DATA, SEP_DATA_RAW_JSON]:
+            if item.sep == SEP_DATA_RAW_JSON:
+                try:
+                    value = json.loads(item.value)
+                except ValueError:
+                    raise ParseError('"%s" is not valid JSON' % item.orig)
+            target = data
+
+        else:
+            raise TypeError(item)
+
+        target[key] = value
+
+    return headers, data, files, params

httpie/models.py

@@ -0,0 +1,184 @@
+import os
+import sys
+from requests.compat import urlparse, is_windows, bytes, str
+
+
+class Environment(object):
+    """Holds information about the execution context.
+
+    Groups various aspects of the environment in a changeable object
+    and allows for mocking.
+
+    """
+
+    #noinspection PyUnresolvedReferences
+    is_windows = is_windows
+
+    progname = os.path.basename(sys.argv[0])
+    if progname not in ['http', 'https']:
+        progname = 'http'
+
+    if is_windows:
+        import colorama.initialise
+        colorama.initialise.init()
+
+    stdin_isatty = sys.stdin.isatty()
+    stdin = sys.stdin
+    stdout_isatty = sys.stdout.isatty()
+    stdout = sys.stdout
+    stderr = sys.stderr
+
+    # Can be set to 0 to disable colors completely.
+    colors = 256 if '256color' in os.environ.get('TERM', '') else 88
+
+    def __init__(self, **kwargs):
+        assert all(hasattr(type(self), attr)
+                   for attr in kwargs.keys())
+        self.__dict__.update(**kwargs)
+
+
+class HTTPMessage(object):
+    """Abstract class for HTTP messages."""
+
+    def __init__(self, orig):
+        self._orig = orig
+
+    def iter_body(self, chunk_size):
+        """Return an iterator over the body."""
+        raise NotImplementedError()
+
+    def iter_lines(self, chunk_size):
+        """Return an iterator over the body yielding (`line`, `line_feed`)."""
+        raise NotImplementedError()
+
+    @property
+    def headers(self):
+        """Return a `str` with the message's headers."""
+        raise NotImplementedError()
+
+    @property
+    def encoding(self):
+        """Return a `str` with the message's encoding, if known."""
+        raise NotImplementedError()
+
+    @property
+    def body(self):
+        """Return a `bytes` with the message's body."""
+        raise NotImplementedError()
+
+    @property
+    def content_type(self):
+        """Return the message content type."""
+        ct = self._orig.headers.get('Content-Type', '')
+        if isinstance(ct, bytes):
+            ct = ct.decode()
+        return ct
+
+
+class HTTPResponse(HTTPMessage):
+    """A :class:`requests.models.Response` wrapper."""
+
+    def iter_body(self, chunk_size=1):
+        return self._orig.iter_content(chunk_size=chunk_size)
+
+    def iter_lines(self, chunk_size):
+        for line in self._orig.iter_lines(chunk_size):
+            yield line, b'\n'
+
+    @property
+    def headers(self):
+        original = self._orig.raw._original_response
+        status_line = 'HTTP/{version} {status} {reason}'.format(
+             version='.'.join(str(original.version)),
+             status=original.status,
+             reason=original.reason
+         )
+        headers = str(original.msg)
+        return '\n'.join([status_line, headers]).strip()
+
+    @property
+    def encoding(self):
+        return self._orig.encoding or 'utf8'
+
+    @property
+    def body(self):
+        # Only now the response body is fetched.
+        # Shouldn't be touched unless the body is actually needed.
+        return self._orig.content
+
+
+class HTTPRequest(HTTPMessage):
+    """A :class:`requests.models.Request` wrapper."""
+
+    def iter_body(self, chunk_size):
+        yield self.body
+
+    def iter_lines(self, chunk_size):
+        yield self.body, b''
+
+    @property
+    def headers(self):
+        """Return Request-Line"""
+        url = urlparse(self._orig.url)
+
+        # Querystring
+        qs = ''
+        if url.query or self._orig.params:
+            qs = '?'
+            if url.query:
+                qs += url.query
+            # Requests doesn't make params part of ``request.url``.
+            if self._orig.params:
+                if url.query:
+                    qs += '&'
+                #noinspection PyUnresolvedReferences
+                qs += type(self._orig)._encode_params(self._orig.params)
+
+        # Request-Line
+        request_line = '{method} {path}{query} HTTP/1.1'.format(
+            method=self._orig.method,
+            path=url.path or '/',
+            query=qs
+        )
+
+        headers = dict(self._orig.headers)
+
+        if 'Host' not in headers:
+            headers['Host'] = urlparse(self._orig.url).netloc
+
+        headers = ['%s: %s' % (name, value)
+                   for name, value in headers.items()]
+
+        headers.insert(0, request_line)
+
+        return '\n'.join(headers).strip()
+
+    @property
+    def encoding(self):
+        return 'utf8'
+
+    @property
+    def body(self):
+        """Reconstruct and return the original request body bytes."""
+        if self._orig.files:
+            # TODO: would be nice if we didn't need to encode the files again
+            # FIXME: Also the boundary header doesn't match the one used.
+            for fn, fd in self._orig.files.values():
+                # Rewind the files as they have already been read before.
+                fd.seek(0)
+            body, _ = self._orig._encode_files(self._orig.files)
+        else:
+            try:
+                body = self._orig.data
+            except AttributeError:
+                # requests < 0.12.1
+                body = self._orig._enc_data
+
+            if isinstance(body, dict):
+                #noinspection PyUnresolvedReferences
+                body = type(self._orig)._encode_params(body)
+
+            if isinstance(body, str):
+                body = body.encode('utf8')
+
+        return body

httpie/output.py

@@ -0,0 +1,476 @@
+"""Output streaming, processing and formatting.
+
+"""
+import json
+from functools import partial
+from itertools import chain
+
+import pygments
+from pygments import token, lexer
+from pygments.styles import get_style_by_name, STYLE_MAP
+from pygments.lexers import get_lexer_for_mimetype, get_lexer_by_name
+from pygments.formatters.terminal import TerminalFormatter
+from pygments.formatters.terminal256 import Terminal256Formatter
+from pygments.util import ClassNotFound
+from requests.compat import is_windows
+
+from .solarized import Solarized256Style
+from .models import HTTPRequest, HTTPResponse, Environment
+from .input import (OUT_REQ_BODY, OUT_REQ_HEAD,
+                    OUT_RESP_HEAD, OUT_RESP_BODY)
+
+
+# Colors on Windows via colorama aren't that great and fruity
+# seems to give the best result there.
+DEFAULT_STYLE = 'solarized' if not is_windows else 'fruity'
+
+#noinspection PySetFunctionToLiteral
+AVAILABLE_STYLES = set([DEFAULT_STYLE]) | set(STYLE_MAP.keys())
+
+
+BINARY_SUPPRESSED_NOTICE = (
+    b'\n'
+    b'+-----------------------------------------+\n'
+    b'| NOTE: binary data not shown in terminal |\n'
+    b'+-----------------------------------------+'
+)
+
+
+class BinarySuppressedError(Exception):
+    """An error indicating that the body is binary and won't be written,
+     e.g., for terminal output)."""
+
+    message = BINARY_SUPPRESSED_NOTICE
+
+
+###############################################################################
+# Output Streams
+###############################################################################
+
+
+def write(stream, outfile, flush):
+    """Write the output stream."""
+    try:
+        # Writing bytes so we use the buffer interface (Python 3).
+        buf = outfile.buffer
+    except AttributeError:
+        buf = outfile
+
+    for chunk in stream:
+        buf.write(chunk)
+        if flush:
+            outfile.flush()
+
+
+def output_stream(args, env, request, response):
+    """Build and return a chain of iterators over the `request`-`response`
+    exchange each of which yields `bytes` chunks.
+
+    """
+
+    Stream = make_stream(env, args)
+
+    req_h = OUT_REQ_HEAD in args.output_options
+    req_b = OUT_REQ_BODY in args.output_options
+    resp_h = OUT_RESP_HEAD in args.output_options
+    resp_b = OUT_RESP_BODY  in args.output_options
+
+    req = req_h or req_b
+    resp = resp_h or resp_b
+
+    output = []
+
+    if req:
+        output.append(Stream(
+            msg=HTTPRequest(request),
+            with_headers=req_h,
+            with_body=req_b))
+
+    if req and resp:
+        output.append([b'\n\n\n'])
+
+    if resp:
+        output.append(Stream(
+            msg=HTTPResponse(response),
+            with_headers=resp_h,
+            with_body=resp_b))
+
+    if env.stdout_isatty:
+        output.append([b'\n\n'])
+
+    return chain(*output)
+
+
+def make_stream(env, args):
+    """Pick the right stream type based on `env` and `args`.
+    Wrap it in a partial with the type-specific args so that
+    we don't need to think what stream we are dealing with.
+
+    """
+    if not env.stdout_isatty and not args.prettify:
+        Stream = partial(
+            RawStream,
+            chunk_size=RawStream.CHUNK_SIZE_BY_LINE
+                       if args.stream
+                       else RawStream.CHUNK_SIZE)
+    elif args.prettify:
+        Stream = partial(
+            PrettyStream if args.stream else BufferedPrettyStream,
+            processor=OutputProcessor(env, pygments_style=args.style),
+            env=env)
+    else:
+        Stream = partial(EncodedStream, env=env)
+
+    return Stream
+
+
+class BaseStream(object):
+    """Base HTTP message stream class."""
+
+    def __init__(self, msg, with_headers=True, with_body=True):
+        """
+        :param msg: a :class:`models.HTTPMessage` subclass
+        :param with_headers: if `True`, headers will be included
+        :param with_body: if `True`, body will be included
+
+        """
+        self.msg = msg
+        self.with_headers = with_headers
+        self.with_body = with_body
+
+    def _headers(self):
+        """Return the headers' bytes."""
+        return self.msg.headers.encode('ascii')
+
+    def _body(self):
+        """Return an iterator over the message body."""
+        raise NotImplementedError()
+
+    def __iter__(self):
+        """Return an iterator over `self.msg`."""
+        if self.with_headers:
+            yield self._headers()
+
+        if self.with_body:
+            it = self._body()
+
+            try:
+                if self.with_headers:
+                    # Yield the headers/body separator only if needed.
+                    chunk = next(it)
+                    if chunk:
+                        yield b'\n\n'
+                        yield chunk
+
+                for chunk in it:
+                    yield chunk
+
+            except BinarySuppressedError as e:
+                if self.with_headers:
+                    yield b'\n'
+                yield e.message
+
+
+class RawStream(BaseStream):
+    """The message is streamed in chunks with no processing."""
+
+    CHUNK_SIZE = 1024 * 100
+    CHUNK_SIZE_BY_LINE = 1024 * 5
+
+    def __init__(self, chunk_size=CHUNK_SIZE, **kwargs):
+        super(RawStream, self).__init__(**kwargs)
+        self.chunk_size = chunk_size
+
+    def _body(self):
+        return self.msg.iter_body(self.chunk_size)
+
+
+class EncodedStream(BaseStream):
+    """Encoded HTTP message stream.
+
+    The message bytes are converted to an encoding suitable for
+    `self.env.stdout`. Unicode errors are replaced and binary data
+    is suppressed. The body is always streamed by line.
+
+    """
+    CHUNK_SIZE = 1024 * 5
+    def __init__(self, env=Environment(), **kwargs):
+
+        super(EncodedStream, self).__init__(**kwargs)
+
+        if env.stdout_isatty:
+            # Use the encoding supported by the terminal.
+            output_encoding = getattr(env.stdout, 'encoding', None)
+        else:
+            # Preserve the message encoding.
+            output_encoding = self.msg.encoding
+
+        # Default to utf8 when unsure.
+        self.output_encoding = output_encoding or 'utf8'
+
+    def _body(self):
+
+        for line, lf in self.msg.iter_lines(self.CHUNK_SIZE):
+
+            if b'\0' in line:
+                raise BinarySuppressedError()
+
+            yield line.decode(self.msg.encoding)\
+                      .encode(self.output_encoding, 'replace') + lf
+
+
+class PrettyStream(EncodedStream):
+    """In addition to :class:`EncodedStream` behaviour, this stream applies
+    content processing.
+
+    Useful for long-lived HTTP responses that stream by lines
+    such as the Twitter streaming API.
+
+    """
+
+    CHUNK_SIZE = 1024 * 5
+
+    def __init__(self, processor, **kwargs):
+        super(PrettyStream, self).__init__(**kwargs)
+        self.processor = processor
+
+    def _headers(self):
+        return self.processor.process_headers(
+            self.msg.headers).encode(self.output_encoding)
+
+    def _body(self):
+        for line, lf in self.msg.iter_lines(self.CHUNK_SIZE):
+            if b'\0' in line:
+                raise BinarySuppressedError()
+            yield self._process_body(line) + lf
+
+    def _process_body(self, chunk):
+        return (self.processor
+                    .process_body(
+                        chunk.decode(self.msg.encoding, 'replace'),
+                        self.msg.content_type)
+                    .encode(self.output_encoding, 'replace'))
+
+
+class BufferedPrettyStream(PrettyStream):
+    """The same as :class:`PrettyStream` except that the body is fully
+    fetched before it's processed.
+
+    Suitable regular HTTP responses.
+
+    """
+
+    CHUNK_SIZE = 1024 * 10
+
+    def _body(self):
+
+        #noinspection PyArgumentList
+        # Read the whole body before prettifying it,
+        # but bail out immediately if the body is binary.
+        body = bytearray()
+        for chunk in self.msg.iter_body(self.CHUNK_SIZE):
+            if b'\0' in chunk:
+                raise BinarySuppressedError()
+            body.extend(chunk)
+
+        yield self._process_body(body)
+
+
+###############################################################################
+# Processing
+###############################################################################
+
+class HTTPLexer(lexer.RegexLexer):
+    """Simplified HTTP lexer for Pygments.
+
+    It only operates on headers and provides a stronger contrast between
+    their names and values than the original one bundled with Pygments
+    (:class:`pygments.lexers.text import HttpLexer`), especially when
+    Solarized color scheme is used.
+
+    """
+    name = 'HTTP'
+    aliases = ['http']
+    filenames = ['*.http']
+    tokens = {
+        'root': [
+            # Request-Line
+            (r'([A-Z]+)( +)([^ ]+)( +)(HTTP)(/)(\d+\.\d+)',
+             lexer.bygroups(
+                token.Name.Function,
+                token.Text,
+                token.Name.Namespace,
+                token.Text,
+                token.Keyword.Reserved,
+                token.Operator,
+                token.Number
+             )),
+            # Response Status-Line
+            (r'(HTTP)(/)(\d+\.\d+)( +)(\d{3})( +)(.+)',
+             lexer.bygroups(
+                 token.Keyword.Reserved,  # 'HTTP'
+                 token.Operator,  # '/'
+                 token.Number,  # Version
+                 token.Text,
+                 token.Number,  # Status code
+                 token.Text,
+                 token.Name.Exception,  # Reason
+             )),
+            # Header
+            (r'(.*?)( *)(:)( *)(.+)', lexer.bygroups(
+                token.Name.Attribute, # Name
+                token.Text,
+                token.Operator,  # Colon
+                token.Text,
+                token.String  # Value
+            ))
+    ]}
+
+
+class BaseProcessor(object):
+    """Base, noop output processor class."""
+
+    enabled = True
+
+    def __init__(self, env, **kwargs):
+        """
+        :param env:
+            an class:`Environment` instance
+        :param kwargs:
+            additional keyword argument that some processor might require.
+
+        """
+        self.env = env
+        self.kwargs = kwargs
+
+    def process_headers(self, headers):
+        """Return processed `headers`
+
+        :param headers:
+            The headers as text.
+
+        """
+        return headers
+
+    def process_body(self, content, content_type, subtype):
+        """Return processed `content`.
+
+        :param content:
+            The body content as text
+
+        :param content_type:
+            Full content type, e.g., 'application/atom+xml'.
+
+        :param subtype:
+            E.g. 'xml'.
+
+        """
+        return content
+
+
+class JSONProcessor(BaseProcessor):
+    """JSON body processor."""
+
+    def process_body(self, content, content_type, subtype):
+        if subtype == 'json':
+            try:
+                # Indent the JSON data, sort keys by name, and
+                # avoid unicode escapes to improve readability.
+                content = json.dumps(json.loads(content),
+                                     sort_keys=True,
+                                     ensure_ascii=False,
+                                     indent=4)
+            except ValueError:
+                # Invalid JSON but we don't care.
+                pass
+        return content
+
+
+class PygmentsProcessor(BaseProcessor):
+    """A processor that applies syntax-highlighting using Pygments
+    to the headers, and to the body as well if its content type is recognized.
+
+    """
+    def __init__(self, *args, **kwargs):
+        super(PygmentsProcessor, self).__init__(*args, **kwargs)
+
+        # Cache that speeds up when we process streamed body by line.
+        self.lexers_by_type = {}
+
+        if not self.env.colors:
+            self.enabled = False
+            return
+
+        try:
+            style = get_style_by_name(self.kwargs['pygments_style'])
+        except ClassNotFound:
+            style = Solarized256Style
+
+        if self.env.is_windows or self.env.colors == 256:
+            fmt_class = Terminal256Formatter
+        else:
+            fmt_class = TerminalFormatter
+        self.formatter = fmt_class(style=style)
+
+    def process_headers(self, headers):
+        return pygments.highlight(
+            headers, HTTPLexer(), self.formatter).strip()
+
+    def process_body(self, content, content_type, subtype):
+        try:
+            lexer = self.lexers_by_type.get(content_type)
+            if not lexer:
+                try:
+                    lexer = get_lexer_for_mimetype(content_type)
+                except ClassNotFound:
+                    lexer = get_lexer_by_name(subtype)
+                self.lexers_by_type[content_type] = lexer
+        except ClassNotFound:
+            pass
+        else:
+            content = pygments.highlight(content, lexer, self.formatter)
+        return content.strip()
+
+
+class HeadersProcessor(BaseProcessor):
+    """Sorts headers by name retaining relative order of multiple headers
+    with the same name.
+
+    """
+    def process_headers(self, headers):
+        lines = headers.splitlines()
+        headers = sorted(lines[1:], key=lambda h: h.split(':')[0])
+        return '\n'.join(lines[:1] + headers)
+
+
+class OutputProcessor(object):
+    """A delegate class that invokes the actual processors."""
+
+    installed_processors = [
+        JSONProcessor,
+        HeadersProcessor,
+        PygmentsProcessor
+    ]
+
+    def __init__(self, env, **kwargs):
+        processors = [
+            cls(env, **kwargs)
+            for cls in self.installed_processors
+        ]
+        self.processors = [p for p in processors if p.enabled]
+
+    def process_headers(self, headers):
+        for processor in self.processors:
+            headers = processor.process_headers(headers)
+        return headers
+
+    def process_body(self, content, content_type):
+        # e.g., 'application/atom+xml'
+        content_type = content_type.split(';')[0]
+        # e.g., 'xml'
+        subtype = content_type.split('/')[-1].split('+')[-1]
+
+        for processor in self.processors:
+            content = processor.process_body(content, content_type, subtype)
+
+        return content

httpie/pretty.py

@@ -1,55 +0,0 @@
-import json
-from functools import partial
-import pygments
-from pygments.lexers import get_lexer_for_mimetype
-from pygments.formatters.terminal256 import Terminal256Formatter
-from pygments.lexer import RegexLexer, bygroups
-from pygments import token
-from . import solarized
-
-
-TYPE_JS = 'application/javascript'
-
-
-class HTTPLexer(RegexLexer):
-    name = 'HTTP'
-    aliases = ['http']
-    filenames = ['*.http']
-    tokens = {
-        'root': [
-            (r'\s+', token.Text),
-            (r'(HTTP/[\d.]+\s+)(\d+)(\s+.+)', bygroups(
-             token.Operator, token.Number, token.String)),
-            (r'(.*?:)(.+)',  bygroups(token.Name, token.String))
-    ]}
-
-
-highlight = partial(pygments.highlight,
-                    formatter=Terminal256Formatter(
-                        style=solarized.SolarizedStyle))
-highlight_http = partial(highlight, lexer=HTTPLexer())
-
-
-def prettify_http(headers):
-    return highlight_http(headers)
-
-
-def prettify_body(content, content_type):
-    content_type = content_type.split(';')[0]
-    if 'json' in content_type:
-        content_type = TYPE_JS
-        try:
-            # Indent JSON
-            content = json.dumps(json.loads(content),
-                                sort_keys=True, indent=4)
-        except Exception:
-            pass
-    try:
-        lexer = get_lexer_for_mimetype(content_type)
-        content = highlight(code=content, lexer=lexer)
-        if content:
-            content = content[:-1]
-    except Exception:
-        pass
-    return content
-

httpie/solarized.py

@@ -1,73 +1,57 @@
-"""
+# -*- coding: utf-8 -*-
-A Pygments_ style based on the dark background variant of Solarized_.
-
-.. _Pygments: http://pygments.org/
-.. _Solarized: http://ethanschoonover.com/solarized
-
-Copyright (c) 2011 Hank Gay
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.
-
 
 """
+    solarized256
+    ------------
+
+    A Pygments style inspired by Solarized's 256 color mode.
+
+    :copyright: (c) 2011 by Hank Gay, (c) 2012 by John Mastro.
+    :license: BSD, see LICENSE for more details.
+"""
+
 from pygments.style import Style
-from pygments.token import Token, Comment, Name, Keyword, Generic, Number, Operator, String
+from pygments.token import Token, Comment, Name, Keyword, Generic, Number, \
+    Operator, String
+
+BASE03  = "#1c1c1c"
+BASE02  = "#262626"
+BASE01  = "#4e4e4e"
+BASE00  = "#585858"
+BASE0   = "#808080"
+BASE1   = "#8a8a8a"
+BASE2   = "#d7d7af"
+BASE3   = "#ffffd7"
+YELLOW  = "#af8700"
+ORANGE  = "#d75f00"
+RED     = "#af0000"
+MAGENTA = "#af005f"
+VIOLET  = "#5f5faf"
+BLUE    = "#0087ff"
+CYAN    = "#00afaf"
+GREEN   = "#5f8700"
 
 
-BASE03 = '#002B36'
+class Solarized256Style(Style):
-BASE02 = '#073642'
-BASE01 = '#586E75'
-BASE00 = '#657B83'
-BASE0 = '#839496'
-BASE1 = '#93A1A1'
-BASE2 = '#EEE8D5'
-BASE3 = '#FDF6E3'
-YELLOW = '#B58900'
-ORANGE = '#CB4B16'
-RED = '#DC322F'
-MAGENTA = '#D33682'
-VIOLET = '#6C71C4'
-BLUE = '#268BD2'
-CYAN = '#2AA198'
-GREEN = '#859900'
-
-
-class SolarizedStyle(Style):
     background_color = BASE03
     styles = {
         Keyword: GREEN,
         Keyword.Constant: ORANGE,
         Keyword.Declaration: BLUE,
-        #Keyword.Namespace
+        Keyword.Namespace: ORANGE,
         #Keyword.Pseudo
         Keyword.Reserved: BLUE,
         Keyword.Type: RED,
 
         #Name
         Name.Attribute: BASE1,
-        Name.Builtin: YELLOW,
+        Name.Builtin: BLUE,
         Name.Builtin.Pseudo: BLUE,
         Name.Class: BLUE,
         Name.Constant: ORANGE,
         Name.Decorator: BLUE,
         Name.Entity: ORANGE,
-        Name.Exception: ORANGE,
+        Name.Exception: YELLOW,
         Name.Function: BLUE,
         #Name.Label
         #Name.Namespace
@@ -83,10 +67,10 @@ class SolarizedStyle(Style):
         String: CYAN,
         String.Backtick: BASE01,
         String.Char: CYAN,
-        String.Doc: BASE1,
+        String.Doc: CYAN,
         #String.Double
-        String.Escape: ORANGE,
+        String.Escape: RED,
-        String.Heredoc: BASE1,
+        String.Heredoc: CYAN,
         #String.Interpol
         #String.Other
         String.Regex: RED,
@@ -99,8 +83,8 @@ class SolarizedStyle(Style):
         #Number.Integer.Long
         #Number.Oct
 
-        Operator: GREEN,
+        Operator: BASE1,
-        #Operator.Word
+        Operator.Word: GREEN,
 
         #Punctuation: ORANGE,
 

setup.py

@@ -1,12 +1,60 @@
+import os
+import sys
 from setuptools import setup
 import httpie
 
 
-setup(name='httpie',version=httpie.__version__,
+if sys.argv[-1] == 'test':
+    status = os.system('python tests/tests.py')
+    sys.exit(1 if status > 127 else status)
+
+
+requirements = [
+    # Debian has only requests==0.10.1 and httpie.deb depends on that.
+    'requests>=0.10.1',
+    'Pygments>=1.5'
+]
+if sys.version_info[:2] in ((2, 6), (3, 1)):
+    # argparse has been added in Python 3.2 / 2.7
+    requirements.append('argparse>=1.2.1')
+if 'win32' in str(sys.platform).lower():
+    # Terminal colors for Windows
+    requirements.append('colorama>=0.2.4')
+
+
+setup(
+    name='httpie',
+    version=httpie.__version__,
     description=httpie.__doc__.strip(),
-    url='https://github.com/jkbr/httpie',
+    long_description=open('README.rst').read(),
+    url='http://httpie.org/',
+    download_url='https://github.com/jkbr/httpie',
     author=httpie.__author__,
+    author_email='jakub@roztocil.name',
     license=httpie.__licence__,
     packages=['httpie'],
-    entry_points={'console_scripts': ['http = httpie.httpie:main']},
+    entry_points={
-    install_requires=['requests>=0.10.4', 'Pygments>=1.4'])
+        'console_scripts': [
+            'http = httpie.__main__:main',
+        ],
+    },
+    install_requires=requirements,
+    classifiers=[
+        'Development Status :: 5 - Production/Stable',
+        'Programming Language :: Python',
+        'Programming Language :: Python :: 2.6',
+        'Programming Language :: Python :: 2.7',
+        'Programming Language :: Python :: 3.1',
+        'Programming Language :: Python :: 3.2',
+        'Environment :: Console',
+        'Intended Audience :: Developers',
+        'Intended Audience :: System Administrators',
+        'License :: OSI Approved :: BSD License',
+        'Topic :: Internet :: WWW/HTTP',
+        'Topic :: Software Development',
+        'Topic :: System :: Networking',
+        'Topic :: Terminals',
+        'Topic :: Text Processing',
+        'Topic :: Utilities'
+    ],
+)

tests/fixtures/file.txt

@@ -0,0 +1 @@
+__test_file_content__

tests/fixtures/file2.txt

@@ -0,0 +1 @@
+__test_file_content__

tox.ini

@@ -0,0 +1,19 @@
+# Tox (http://tox.testrun.org/) is a tool for running tests
+# in multiple virtualenvs. This configuration file will run the
+# test suite on all supported python versions. To use it, "pip install tox"
+# and then run "tox" from this directory.
+
+[tox]
+envlist = py26, py27, py32, pypy
+
+[testenv]
+commands = {envpython} setup.py test
+
+[testenv:py26]
+deps = argparse
+
+[testenv:py30]
+deps = argparse
+
+[testenv:py31]
+deps = argparse