Fixed changelog link.

Expunge user:pass@... from Host header.

.gitignore

@@ -6,4 +6,5 @@ build
 README.html
 .coverage
 htmlcov
-
+.idea
+.DS_Store

.travis.yml

@@ -3,7 +3,7 @@ python:
   - 2.6
   - 2.7
   - pypy
-  - 3.2
+  - 3.3
 script: python setup.py test
 install:
   - pip install . --use-mirrors

AUTHORS.rst

@@ -8,6 +8,7 @@ HTTPie authors
 Patches and ideas
 -----------------
 
+* `Cláudia T. Delgado <https://github.com/claudiatd>`_ (logo)
 * `Hank Gay <https://github.com/gthank>`_
 * `Jake Basile <https://github.com/jakebasile>`_
 * `Vladimir Berkutov <https://github.com/dair-targ>`_
@@ -27,3 +28,5 @@ Patches and ideas
 * `Tomek Wójcik <https://github.com/tomekwojcik>`_
 * `Davey Shafik <https://github.com/dshafik>`_
 * `cido <https://github.com/cido>`_
+* `Justin Bonnar <https://github.com/jargonjustin>`_
+* `Nathan LaFreniere <https://github.com/nlf>`_

README.rst

@@ -1,12 +1,11 @@
-***********************
-HTTPie: cURL for Humans
-***********************
+****************************************
+HTTPie: a CLI, cURL-like tool for humans
+****************************************
 
-v0.2.8-alpha (`stable version`_)
 
-HTTPie is a **command line HTTP client** whose goal is to make CLI interaction
+HTTPie is a **command line HTTP client**. Its goal is to make CLI interaction
 with web services as **human-friendly** as possible. It provides a
-simple ``http`` command that allows for sending arbitrary HTTP requests with a
+simple ``http`` command that allows for sending arbitrary HTTP requests using a
 simple and natural syntax, and displays colorized responses. HTTPie can be used
 for **testing, debugging**, and generally **interacting** with HTTP servers.
 
@@ -15,10 +14,18 @@ for **testing, debugging**, and generally **interacting** with HTTP servers.
     :alt: HTTPie compared to cURL
     :width: 835
     :height: 835
+    :align: center
 
 
+------
+
+
+.. image:: https://raw.github.com/claudiatd/httpie-artwork/master/images/httpie_logo_simple.png
+    :alt: HTTPie logo
+    :align: center
+
 HTTPie is written in Python, and under the hood it uses the excellent
-`Requests`_ for HTTP and `Pygments`_ for colorizing.
+`Requests`_ and `Pygments`_ libraries.
 
 
 **Table of Contents**
@@ -30,7 +37,6 @@ HTTPie is written in Python, and under the hood it uses the excellent
     :backlinks: none
 
 
-
 =============
 Main Features
 =============
@@ -42,6 +48,8 @@ Main Features
 * HTTPS, proxies, and authentication
 * Arbitrary request data
 * Custom headers
+* Persistent sessions
+* Wget-like downloads
 * Python 2.6, 2.7 and 3.x support
 * Linux, Mac OS X and Windows support
 * Documentation
@@ -58,9 +66,11 @@ or ``easy_install``:
 
 .. code-block:: bash
 
-    $ pip install -U httpie
+    $ pip install --upgrade httpie
 
 
+Alternatively:
+
 .. code-block:: bash
 
     $ easy_install httpie
@@ -76,12 +86,12 @@ Or, you can install the **development version** directly from GitHub:
 
 .. code-block:: bash
 
-    $ pip install -U https://github.com/jkbr/httpie/tarball/master
+    $ pip install --upgrade https://github.com/jkbr/httpie/tarball/master
 
 
 There are also packages available for `Ubuntu`_, `Debian`_, and possibly other
-Linux distributions as well. However, they may be a significant delay between
-releases and package updates.
+Linux distributions as well. However, there may be a significant delay between
+official HTTPie releases and package updates.
 
 
 =====
@@ -111,7 +121,6 @@ See also ``http --help``.
 Examples
 --------
 
-
 Custom `HTTP method`_, `HTTP headers`_ and `JSON`_ data:
 
 .. code-block:: bash
@@ -126,14 +135,16 @@ Submitting `forms`_:
     $ http -f POST example.org hello=World
 
 
-See the request that is being sent using on of the `output options`_:
+See the request that is being sent using one of the `output options`_:
 
 .. code-block:: bash
 
     $ http -v example.org
 
 
-Use `Github API`_ to post a comment on an issue with `authentication`_:
+Use `Github API`_ to post a comment on an
+`issue <https://github.com/jkbr/httpie/issues/83>`_
+with `authentication`_:
 
 .. code-block:: bash
 
@@ -154,10 +165,34 @@ Download a file and save it via `redirected output`_:
     $ http example.org/file > file
 
 
+Download a file ``wget`` style:
+
+.. code-block:: bash
+
+    $ http --download example.org/file
+
+Use named `sessions`_ to make certain aspects or the communication persistent
+between requests to the same host:
+
+.. code-block:: bash
+
+    $ http --session=logged-in -a username:password httpbin.org/get API-Key:123
+
+    $ http --session=logged-in httpbin.org/headers
+
+
+Set a custom ``Host`` header to work around missing DNS records:
+
+.. code-block:: bash
+
+    $ http localhost:8000 Host:example.com
+
+..
+
 --------
 
 *What follows is a detailed documentation. It covers the command syntax,
-advances usage, and also features additional examples.*
+advanced usage, and also features additional examples.*
 
 
 ============
@@ -190,6 +225,42 @@ The only information HTTPie needs to perform a request is a URL.
 The default scheme is, somewhat unsurprisingly, ``http://``,
 and can be omitted from the argument – ``http example.org`` works just fine.
 
+Additionally, curl-like shorthand for localhost is supported.
+This means that, for example ``:3000`` would expand to ``http://localhost:3000``
+If the port is omitted, then port 80 is assumed.
+
+.. code-block:: bash
+
+    $ http :/foo
+
+
+.. code-block:: http
+
+    GET /foo HTTP/1.1
+    Host: localhost
+
+
+.. code-block:: bash
+
+    $ http :3000/bar
+
+
+.. code-block:: http
+
+    GET /bar HTTP/1.1
+    Host: localhost:3000
+
+
+.. code-block:: bash
+
+    $ http :
+
+
+.. code-block:: http
+
+    GET / HTTP/1.1
+    Host: localhost
+
 If find yourself manually constructing URLs with **querystring parameters**
 on the terminal, you may appreciate the ``param==value`` syntax for appending
 URL parameters so that you don't have to worry about escaping the ``&``
@@ -210,14 +281,15 @@ command:
 Request Items
 =============
 
-There are five different *request item* types that provide a
+There are a few different *request item* types that provide a
 convenient mechanism for specifying HTTP headers, simple JSON and
 form data, files, and URL parameters.
 
 They are key/value pairs specified after the URL. All have in
 common that they become part of the actual request that is sent and that
 their type is distinguished only by the separator used:
-``:``, ``=``, ``:=``, ``@``, and ``==``.
+``:``, ``=``, ``:=``, ``==``, ``@``, ``=@``, and ``:=@``. The ones with an
+``@`` expect a file path as value.
 
 +-----------------------+-----------------------------------------------------+
 | Item Type             | Description                                         |
@@ -230,27 +302,30 @@ their type is distinguished only by the separator used:
 |                       | The ``==`` separator is used                        |
 +-----------------------+-----------------------------------------------------+
 | Data Fields           | Request data fields to be serialized as a JSON      |
-| ``field=value``       | object (default), or to be form encoded (``--form`` |
-|                       | / ``-f``).                                          |
+| ``field=value``,      | object (default), or to be form-encoded             |
+| ``field=@file.txt``   | (``--form, -f``).                                   |
 +-----------------------+-----------------------------------------------------+
 | Raw JSON fields       | Useful when sending JSON and one or                 |
-| ``field:=json``       | more fields need to be a ``Boolean``, ``Number``,   |
-|                       | nested ``Object``, or an ``Array``,  e.g.,          |
+| ``field:=json``,      | more fields need to be a ``Boolean``, ``Number``,   |
+| ``field:=@file.json`` | nested ``Object``, or an ``Array``,  e.g.,          |
 |                       | ``meals:='["ham","spam"]'`` or ``pies:=[1,2,3]``    |
 |                       | (note the quotes).                                  |
 +-----------------------+-----------------------------------------------------+
-| Files                 | Only available with ``-f`` / ``--form``.            |
+| Form File Fields      | Only available with ``--form, -f``.                 |
 | ``field@/dir/file``   | For example ``screenshot@~/Pictures/img.png``.      |
 |                       | The presence of a file field results                |
 |                       | in a ``multipart/form-data`` request.               |
 +-----------------------+-----------------------------------------------------+
 
 You can use ``\`` to escape characters that shouldn't be used as separators
-(or parts thereof). e.g., ``foo\==bar`` will become a data key/value
+(or parts thereof). For instance, ``foo\==bar`` will become a data key/value
 pair (``foo=`` and ``bar``) instead of a URL parameter.
 
-Note that data fields aren't the only way to specify request data,
-`redirected input`_ allows passing arbitrary data to be sent with the request.
+You can also quote values, e.g. ``foo="bar baz"``.
+
+Note that data fields aren't the only way to specify request data:
+`Redirected input`_ allows for passing arbitrary data to be sent with the
+request.
 
 
 ====
@@ -269,7 +344,7 @@ both of which can be overwritten:
 ``Accept``          ``application/json``
 ================    =======================================
 
-You can use ``--json`` / ``-j`` to explicitly set ``Accept``
+You can use ``--json, -j`` to explicitly set ``Accept``
 to ``application/json`` regardless of whether you are sending data
 (it's a shortcut for setting the header via the usual header notation –
 ``http url Accept:application/json``).
@@ -287,7 +362,6 @@ Simple example:
     Accept-Encoding: identity, deflate, compress, gzip
     Content-Type: application/json; charset=utf-8
     Host: example.org
-    User-Agent: HTTPie/0.2.7dev
 
     {
         "name": "John",
@@ -296,11 +370,16 @@ Simple example:
 
 
 Non-string fields use the ``:=`` separator, which allows you to embed raw JSON
-into the resulting object:
+into the resulting object. Text and raw JSON files can also be embedded into
+fields using ``=@`` and ``:=@``:
 
 .. code-block:: bash
 
-    $ http PUT api.example.com/person/1 name=John age:=29 married:=false hobbies:='["http", "pies"]'
+    $ http PUT api.example.com/person/1 \
+        name=John \
+        age:=29 married:=false hobbies:='["http", "pies"]' \  # Raw JSON
+        description=@about-john.txt \   # Embed text file
+        bookmarks:=@bookmarks.json      # Embed JSON file
 
 
 .. code-block:: http
@@ -309,7 +388,6 @@ into the resulting object:
     Accept: application/json
     Content-Type: application/json; charset=utf-8
     Host: api.example.com
-    User-Agent: HTTPie/0.2.7dev
 
     {
         "age": 29,
@@ -317,8 +395,12 @@ into the resulting object:
             "http",
             "pies"
         ],
+        "description": "John is a nice guy who likes pies.",
         "married": false,
-        "name": "John"
+        "name": "John",
+        "bookmarks": {
+            "HTTPie": "http://httpie.org",
+        }
     }
 
 
@@ -334,12 +416,12 @@ Forms
 =====
 
 Submitting forms is very similar to sending `JSON`_ requests. Often the only
-difference is in adding the ``--form`` / ``-f`` option, which ensures that
+difference is in adding the ``--form, -f`` option, which ensures that
 data fields are serialized as, and ``Content-Type`` is set to,
 ``application/x-www-form-urlencoded; charset=utf-8``.
 
-It is possible to make form data the implicit content type via the `config`_
-file.
+It is possible to make form data the implicit content type instead of JSON
+via the `config`_ file.
 
 
 -------------
@@ -348,16 +430,15 @@ Regular Forms
 
 .. code-block:: bash
 
-    $ http --form POST api.example.org/person/1 name='John Smith' email=john@example.org
+    $ http --form POST api.example.org/person/1 name='John Smith' email=john@example.org cv=@~/Documents/cv.txt
 
 
 .. code-block:: http
 
     POST /person/1 HTTP/1.1
-    User-Agent: HTTPie/0.2.7dev
     Content-Type: application/x-www-form-urlencoded; charset=utf-8
 
-    name=John+Smith&email=john%40example.org
+    name=John+Smith&email=john%40example.org&cv=John's+CV+...
 
 
 -----------------
@@ -382,6 +463,9 @@ submitted:
         <input type="file" name="cv" />
     </form>
 
+Note that ``@`` is used to simulate a file upload form field, whereas
+``=@`` just embeds the file content as a regular text field value.
+
 
 ============
 HTTP Headers
@@ -391,7 +475,7 @@ To set custom headers you can use the ``Header:Value`` notation:
 
 .. code-block:: bash
 
-    $ http example.org  User-Agent:Bacon/1.0  Cookie:valued-visitor=yes  X-Foo:Bar  Referer:http://httpie.org/
+    $ http example.org  User-Agent:Bacon/1.0  'Cookie:valued-visitor=yes;foo=bar'  X-Foo:Bar  Referer:http://httpie.org/
 
 
 .. code-block:: http
@@ -399,7 +483,7 @@ To set custom headers you can use the ``Header:Value`` notation:
     GET / HTTP/1.1
     Accept: */*
     Accept-Encoding: identity, deflate, compress, gzip
-    Cookie: valued-visitor=yes
+    Cookie: valued-visitor=yes;foo=bar
     Host: example.org
     Referer: http://httpie.org/
     User-Agent: Bacon/1.0
@@ -424,8 +508,8 @@ Any of the default headers can be overwritten.
 Authentication
 ==============
 
-The currently supported authentication schemes are Basic and Digest (more to
-come). There are two flags that control authentication:
+The currently supported authentication schemes are Basic and Digest
+(see `auth plugins`_ for more). There are two flags that control authentication:
 
 ===================     ======================================================
 ``--auth, -a``          Pass a ``username:password`` pair as
@@ -433,13 +517,16 @@ come). There are two flags that control authentication:
                         (``-a username``), you'll be prompted for
                         the password before the request is sent.
                         To send a an empty password, pass ``username:``.
+                        The ``username:password@hostname`` URL syntax is
+                        supported as well (but credentials passed via ``-a``
+                        have higher priority).
 
 ``--auth-type``         Specify the auth mechanism. Possible values are
                         ``basic`` and ``digest``. The default value is
                         ``basic`` so it can often be omitted.
 ===================     ======================================================
 
-Authorization information from ``.netrc`` is respected as well.
+
 
 Basic auth:
 
@@ -464,22 +551,46 @@ With password prompt:
     $ http -a username example.org
 
 
+Authorization information from your ``~/.netrc`` file is honored as well:
+
+.. code-block:: bash
+
+    $ cat ~/.netrc
+    machine httpbin.org
+    login httpie
+    password test
+
+    $ http httpbin.org/basic-auth/httpie/test
+    HTTP/1.1 200 OK
+    [...]
+
+
+------------
+Auth Plugins
+------------
+
+* `httpie-oauth <https://github.com/jkbr/httpie-oauth>`_: OAuth
+* `httpie-ntlm <https://github.com/jkbr/httpie-ntlm>`_: NTLM (NT LAN Manager)
+* `httpie-negotiate <https://github.com/ndzou/httpie-negotiate>`_: SPNEGO (GSS Negotiate)
+
+
 =======
 Proxies
 =======
 
-You can specify proxies to be used through the ``--proxy`` argument:
+You can specify proxies to be used through the ``--proxy`` argument for each
+protocol (which is included in the value in case of redirects across protocols):
 
 .. code-block:: bash
 
-    http --proxy=http:10.10.1.10:3128 --https:10.10.1.10:1080 example.org
+    $ http --proxy=http:http://10.10.1.10:3128 --proxy=https:https://10.10.1.10:1080 example.org
 
 
 With Basic authentication:
 
 .. code-block:: bash
 
-    http --proxy=http:http://user:pass@10.10.1.10:3128 example.org
+    $ http --proxy=http:http://user:pass@10.10.1.10:3128 example.org
 
 You can also configure proxies by environment variables ``HTTP_PROXY`` and
 ``HTTPS_PROXY``, and the underlying Requests library will pick them up as well.
@@ -490,8 +601,8 @@ In your ``~/.bash_profile``:
 
 .. code-block:: bash
 
- export HTTP_PROXY=10.10.1.10:3128
- export HTTPS_PROXY=10.10.1.10:1080
+ export HTTP_PROXY=http://10.10.1.10:3128
+ export HTTPS_PROXY=https://10.10.1.10:1080
  export NO_PROXY=localhost,example.com
 
 
@@ -551,7 +662,7 @@ documentation examples:
     }
 
 
-All the other options are just a shortcut for ``--print`` / ``-p``.
+All the other options are just a shortcut for ``--print, -p``.
 It accepts a string of characters each of which represents a specific part of
 the HTTP exchange:
 
@@ -658,7 +769,16 @@ On OS X, you can send the contents of the clipboard with ``pbpaste``:
 
 
 Passing data through ``stdin`` cannot be combined with data fields specified
-on the command line.
+on the command line:
+
+
+.. code-block:: bash
+
+    $ echo 'data' | http POST example.org more=data   # This is invalid
+
+
+To prevent HTTPie from reading ``stdin`` data you can use the
+``--ignore-stdin`` option.
 
 
 -------------------------
@@ -700,6 +820,7 @@ Also, the following formatting is applied:
 * HTTP headers are sorted by name.
 * JSON data is indented, sorted by keys, and unicode escapes are converted
   to the characters they represent.
+* XML data is indented for better readability.
 
 One of these options can be used to control output processing:
 
@@ -723,7 +844,7 @@ that the response body is binary,
 
 .. code-block:: bash
 
-    http example.org/Movie.mov
+    $ http example.org/Movie.mov
 
 
 You will nearly instantly see something like this:
@@ -781,6 +902,76 @@ Force colorizing and formatting, and show both the request and the response in
 The ``-R`` flag tells ``less`` to interpret color escape sequences included
 HTTPie`s output.
 
+You can create a shortcut for invoking HTTPie with colorized and paged output
+by adding the following to your ``~/.bash_profile``:
+
+.. code-block:: bash
+
+    function httpless {
+        # `httpless example.org'
+        http --pretty=all --print=hb "$@" | less -R;
+    }
+
+
+=============
+Download Mode
+=============
+
+HTTPie features a download mode in which it acts similarly to ``wget``.
+
+When enabled using the ``--download, -d`` flag, response headers are printed to
+the terminal (``stderr``), and a progress bar is shown while the response body
+is being saved to a file.
+
+.. code-block:: bash
+
+    $ http --download https://github.com/jkbr/httpie/tarball/master
+
+.. code-block:: http
+
+    HTTP/1.1 200 OK
+    Connection: keep-alive
+    Content-Disposition: attachment; filename=jkbr-httpie-0.4.1-33-gfc4f70a.tar.gz
+    Content-Length: 505530
+    Content-Type: application/x-gzip
+    Server: GitHub.com
+    Vary: Accept-Encoding
+
+    Downloading 494.89 kB to "jkbr-httpie-0.4.1-33-gfc4f70a.tar.gz"
+    /  21.01% 104.00 kB   47.55 kB/s  0:00:08 ETA
+
+
+If not provided via ``--output, -o``, the output filename will be determined
+from ``Content-Disposition`` (if available), or from the URL and
+``Content-Type``. If the guessed filename already exists, HTTPie adds a unique
+suffix to it.
+
+You can also redirect the response body to another program while the response
+headers and progress are still shown in the terminal:
+
+.. code-block:: bash
+
+    $ http -d https://github.com/jkbr/httpie/tarball/master |  tar zxf -
+
+
+If ``--output, -o`` is specified, you can resume a partial download using the
+``--continue, -c`` option. This only works with servers that support
+``Range`` requests and ``206 Partial Content`` responses. If the server doesn't
+support that, the whole file will simply be downloaded:
+
+.. code-block:: bash
+
+    $ http -dco file.zip example.org/file
+
+Other notes:
+
+* The ``--download`` option only changes how the response body is treated.
+* You can still set custom headers, use sessions, ``--verbose, -v``, etc.
+* ``--download`` always implies ``--follow`` (redirects are followed).
+* HTTPie exits with status code ``1`` (error) if the body hasn't been fully
+  downloaded.
+* ``Accept-Encoding`` cannot be set with ``--download``.
+
 
 ==================
 Streamed Responses
@@ -819,54 +1010,80 @@ Streamed output by small chunks alá ``tail -f``:
     $ http --stream -f -a YOUR-TWITTER-NAME https://stream.twitter.com/1/statuses/filter.json track=Apple \
     | while read tweet; do echo "$tweet" | http POST example.org/tweets ; done
 
-
-
 ========
 Sessions
 ========
 
-HTTPie supports named, per-host sessions, where custom headers, authorization,
-and cookies (manually specified or sent by the server) persist between requests:
+By default, every request is completely independent of any previous ones.
+HTTPie also supports persistent sessions, where custom headers (except for the
+ones starting with ``Content-`` or ``If-``), authorization, and cookies
+(manually specified or sent by the server) persist between requests
+to the same host.
+
+--------------
+Named Sessions
+--------------
+
+Create a new session named ``user1`` for ``example.org``:
 
 .. code-block:: bash
 
-    $ http --session user1 -a user1:password example.org X-Foo:Bar
+    $ http --session=user1 -a user1:password example.org X-Foo:Bar
 
-Now you can refer to the session by its name:
+Now you can refer to the session by its name, and the previously used
+authorization and HTTP headers will automatically be set:
 
 .. code-block:: bash
 
-    $ http --session user1 example.org
+    $ http --session=user1 example.org
 
-
-To switch to another session simple pass a different name:
+To create or reuse a different session, simple specify a different name:
 
 .. code-block:: bash
 
-    $ http --session user2 -a user2:password example.org X-Bar:Foo
+    $ http --session=user2 -a user2:password example.org X-Bar:Foo
 
 To use a session without updating it from the request/response exchange
 once it is created, specify the session name via
 ``--session-read-only=SESSION_NAME`` instead.
 
-You can view and manipulate existing sessions via the ``httpie`` management
-command, see ``httpie --help``.
-
-Sessions are stored as JSON in ``~/.httpie/sessions/<host>/<name>.json``
+Named sessions' data is stored in JSON files in the directory
+``~/.httpie/sessions/<host>/<name>.json``
 (``%APPDATA%\httpie\sessions\<host>\<name>.json`` on Windows).
 
-See also `config`_.
+------------------
+Anonymous Sessions
+------------------
+
+Instead of a name, you can also directly specify a path to a session file. This
+allows for sessions to be re-used across multiple hosts:
+
+.. code-block:: bash
+
+    $ http --session=/tmp/session.json example.org
+    $ http --session=/tmp/session.json admin.example.org
+    $ http --session=~/.httpie/sessions/another.example.org/test.json example.org
+    $ http --session-read-only=/tmp/session.json example.org
+
+
+**Warning:** All session data, including credentials, cookie data,
+and custom headers are stored in plain text.
+
+Note that session files can also be created and edited manually in a text
+editor; they are plain JSON.
+
+See also `Config`_.
 
 
 ======
 Config
 ======
 
-HTTPie provides a simple configuration file containing a JSON
-object with the following keys:
+HTTPie uses a simple configuration file that contains a JSON object with the
+following keys:
 
 =========================     =================================================
-``__version__``               HTTPie automatically sets this to its version.
+``__meta__``                  HTTPie automatically stores some metadata here.
                               Do not change.
 
 ``implicit_content_type``     A ``String`` specifying the implicit content type
@@ -876,10 +1093,24 @@ object with the following keys:
 
 ``default_options``           An ``Array`` (by default empty) of options
                               that should be applied to every request.
+
+                              For instance, you can use this option to change
+                              the default style and output options:
+                              ``"default_options": ["--style=fruity", "--body"]``
+
+                              Another useful default option is
+                              ``"--session=default"`` to make HTTPie always
+                              use `sessions`_.
+
+                              Default options from config file can be unset
+                              for a particular invocation via
+                              ``--no-OPTION`` arguments passed on the
+                              command line (e.g., ``--no-style``
+                              or ``--no-session``).
 =========================     =================================================
 
-The default location is ``~/.httpie/config.json``
-(``%APPDATA%\httpie\config.json`` on Windows).
+The default location of the configuration file is ``~/.httpie/config.json``
+(or ``%APPDATA%\httpie\config.json`` on Windows).
 
 The config directory location can be changed by setting the
 ``HTTPIE_CONFIG_DIR`` environment variable.
@@ -893,14 +1124,18 @@ When using HTTPie from **shell scripts**, it can be handy to set the
 ``--check-status`` flag. It instructs HTTPie to exit with an error if the
 HTTP status is one of ``3xx``, ``4xx``, or ``5xx``. The exit status will
 be ``3`` (unless ``--follow`` is set), ``4``, or ``5``,
-respectively. Also, the ``--timeout`` option allows to overwrite the default
-30s timeout:
+respectively.
+
+The ``--ignore-stdin`` option prevents HTTPie from reading data from ``stdin``,
+which is usually not desirable during non-interactive invocations.
+
+Also, the ``--timeout`` option allows to overwrite the default 30s timeout:
 
 .. code-block:: bash
 
     #!/bin/bash
 
-    if http --timeout=2.5 --check-status HEAD example.org/health &> /dev/null; then
+    if http --check-status --ignore-stdin --timeout=2.5 HEAD example.org/health &> /dev/null; then
         echo 'OK!'
     else
         case $? in
@@ -1003,15 +1238,19 @@ Please run the existing suite of tests before a pull request is submitted:
 
 Don't forget to add yourself to `AUTHORS.rst`_.
 
+=======
+Logo
+=======
+
+See `claudiatd/httpie-artwork`_
 
 =======
 Authors
 =======
 
-`Jakub Roztocil`_  (`@jakubroztocil`_) created HTTPie and `these fine people`_
+`Jakub Roztocil`_  (`@jkbrzt`_) created HTTPie and `these fine people`_
 have contributed.
 
-
 =======
 Licence
 =======
@@ -1025,9 +1264,41 @@ Changelog
 
 *You can click a version name to see a diff with the previous one.*
 
-* `0.2.8-alpha`_
+* `0.9.0-dev`_
+* `0.8.0`_ (2014-01-25)
+    * Added ``field=@file.txt`` and ``field:=@file.json`` for embedding
+      the contents of text and JSON files into request data.
+    * Added curl-style shorthand for localhost.
+    * Fixed request ``Host`` header value output so that it doesn't contain
+      credentials, if included in the URL.
+* `0.7.1`_ (2013-09-24)
+    * Added ``--ignore-stdin``.
+    * Added support for auth plugins.
+    * Improved ``--help`` output.
+    * Improved ``Content-Disposition`` parsing for ``--download`` mode.
+    * Update to Requests 2.0.0
+* `0.6.0`_ (2013-06-03)
+    * XML data is now formatted.
+    * ``--session`` and ``--session-read-only`` now also accept paths to
+      session files (eg. ``http --session=/tmp/session.json example.org``).
+* `0.5.1`_ (2013-05-13)
+    * ``Content-*`` and ``If-*`` request headers are not stored in sessions
+      anymore as they are request-specific.
+* `0.5.0`_ (2013-04-27)
+    * Added a `download mode`_ via ``--download``.
+    * Bugfixes.
+* `0.4.1`_ (2013-02-26)
+    * Fixed ``setup.py``.
+* `0.4.0`_ (2013-02-22)
+    * Python 3.3 compatibility.
+    * Requests >= v1.0.4 compatibility.
+    * Added support for credentials in URL.
+    * Added ``--no-option`` for every ``--option`` to be config-friendly.
+    * Mutually exclusive arguments can be specified multiple times. The
+      last value is used.
+* `0.3.0`_ (2012-09-21)
     * Allow output redirection on Windows.
-    * Added config file.
+    * Added configuration file.
     * Added persistent session support.
     * Renamed ``--allow-redirects`` to ``--follow``.
     * Improved the usability of ``http --help``.
@@ -1040,7 +1311,7 @@ Changelog
       ``--ugly`` has bee removed in favor of ``--pretty=none``.
 * `0.2.7`_ (2012-08-07)
     * Compatibility with Requests 0.13.6.
-    * Streamed terminal output. ``--stream`` / ``-S`` can be used to enable
+    * Streamed terminal output. ``--stream, -S`` can be used to enable
       streaming also with ``--pretty`` and to ensure a more frequent output
       flushing.
     * Support for efficient large file downloads.
@@ -1097,6 +1368,14 @@ Changelog
 * `0.1.6`_ (2012-03-04)
 
 
+------------
+
+
+.. image:: https://d2weczhvl823v0.cloudfront.net/jkbr/httpie/trend.png
+    :target: https://bitdeli.com/free
+    :alt: Bitdeli Badge
+
+
 .. _Requests: http://python-requests.org
 .. _Pygments: http://pygments.org/
 .. _pip: http://www.pip-installer.org/en/latest/index.html
@@ -1107,9 +1386,10 @@ Changelog
 .. _Debian: http://packages.debian.org/httpie
 .. _the repository: https://github.com/jkbr/httpie
 .. _these fine people: https://github.com/jkbr/httpie/contributors
-.. _Jakub Roztocil: http://roztocil.name
-.. _@jakubroztocil: https://twitter.com/jakubroztocil
+.. _Jakub Roztocil: http://subtleapps.com
+.. _@jkbrzt: https://twitter.com/jkbrzt
 .. _existing issues: https://github.com/jkbr/httpie/issues?state=open
+.. _claudiatd/httpie-artwork: https://github.com/claudiatd/httpie-artwork
 .. _0.1.6: https://github.com/jkbr/httpie/compare/0.1.4...0.1.6
 .. _0.2.0: https://github.com/jkbr/httpie/compare/0.1.6...0.2.0
 .. _0.2.1: https://github.com/jkbr/httpie/compare/0.2.0...0.2.1
@@ -1117,7 +1397,14 @@ Changelog
 .. _0.2.5: https://github.com/jkbr/httpie/compare/0.2.2...0.2.5
 .. _0.2.6: https://github.com/jkbr/httpie/compare/0.2.5...0.2.6
 .. _0.2.7: https://github.com/jkbr/httpie/compare/0.2.5...0.2.7
-.. _0.2.8-alpha: https://github.com/jkbr/httpie/compare/0.2.7...master
-.. _stable version: https://github.com/jkbr/httpie/tree/0.2.7#readme
+.. _0.3.0: https://github.com/jkbr/httpie/compare/0.2.7...0.3.0
+.. _0.4.0: https://github.com/jkbr/httpie/compare/0.3.0...0.4.0
+.. _0.4.1: https://github.com/jkbr/httpie/compare/0.4.0...0.4.1
+.. _0.5.0: https://github.com/jkbr/httpie/compare/0.4.1...0.5.0
+.. _0.5.1: https://github.com/jkbr/httpie/compare/0.5.0...0.5.1
+.. _0.6.0: https://github.com/jkbr/httpie/compare/0.5.1...0.6.0
+.. _0.7.1: https://github.com/jkbr/httpie/compare/0.6.0...0.7.1
+.. _0.8.0: https://github.com/jkbr/httpie/compare/0.7.1...0.8.0
+.. _0.9.0-dev: https://github.com/jkbr/httpie/compare/0.8.0...master
 .. _AUTHORS.rst: https://github.com/jkbr/httpie/blob/master/AUTHORS.rst
 .. _LICENSE: https://github.com/jkbr/httpie/blob/master/LICENSE

httpie/__init__.py

@@ -1,13 +1,14 @@
 """
-HTTPie - cURL for humans.
+HTTPie - a CLI, cURL-like tool for humans.
 
 """
 __author__ = 'Jakub Roztocil'
-__version__ = '0.2.8-alpha'
+__version__ = '0.8.0'
 __licence__ = 'BSD'
 
 
-class EXIT:
+class ExitStatus:
+    """Exit status code constants."""
     OK = 0
     ERROR = 1
     ERROR_TIMEOUT = 2

httpie/cli.py

@@ -3,207 +3,292 @@
 NOTE: the CLI interface may change before reaching v1.0.
 
 """
-from argparse import FileType, OPTIONAL, ZERO_OR_MORE, SUPPRESS
-
-from requests.compat import is_windows
+from textwrap import dedent, wrap
+#noinspection PyCompatibility
+from argparse import (RawDescriptionHelpFormatter, FileType,
+                      OPTIONAL, ZERO_OR_MORE, SUPPRESS)
 
 from . import __doc__
 from . import __version__
-from .config import DEFAULT_CONFIG_DIR
+from .plugins.builtin import BuiltinAuthPlugin
+from .plugins import plugin_manager
+from .sessions import DEFAULT_SESSIONS_DIR
 from .output import AVAILABLE_STYLES, DEFAULT_STYLE
 from .input import (Parser, AuthCredentialsArgType, KeyValueArgType,
-                    SEP_PROXY, SEP_CREDENTIALS, SEP_GROUP_ITEMS,
+                    SEP_PROXY, SEP_CREDENTIALS, SEP_GROUP_ALL_ITEMS,
                     OUT_REQ_HEAD, OUT_REQ_BODY, OUT_RESP_HEAD,
-                    OUT_RESP_BODY, OUTPUT_OPTIONS,
-                    PRETTY_MAP, PRETTY_STDOUT_TTY_ONLY)
+                    OUT_RESP_BODY, OUTPUT_OPTIONS, OUTPUT_OPTIONS_DEFAULT,
+                    PRETTY_MAP, PRETTY_STDOUT_TTY_ONLY, SessionNameValidator)
 
 
-def _(text):
-    """Normalize whitespace."""
-    return ' '.join(text.strip().split())
+class HTTPieHelpFormatter(RawDescriptionHelpFormatter):
+    """A nicer help formatter.
 
+    Help for arguments can be indented and contain new lines.
+    It will be de-dented and arguments in the help
+    will be separated by a blank line for better readability.
+
+
+    """
+    def __init__(self, max_help_position=6, *args, **kwargs):
+        # A smaller indent for args help.
+        kwargs['max_help_position'] = max_help_position
+        super(HTTPieHelpFormatter, self).__init__(*args, **kwargs)
+
+    def _split_lines(self, text, width):
+        text = dedent(text).strip() + '\n\n'
+        return text.splitlines()
 
 parser = Parser(
+    formatter_class=HTTPieHelpFormatter,
     description='%s <http://httpie.org>' % __doc__.strip(),
-    epilog=_('''
-        Suggestions and bug reports are greatly appreciated:
+    epilog=dedent("""
+    For every --OPTION there is also a --no-OPTION that reverts OPTION
+    to its default value.
+
+    Suggestions and bug reports are greatly appreciated:
+
         https://github.com/jkbr/httpie/issues
-    ''')
+
+    """)
 )
 
 
-
-###############################################################################
+#######################################################################
 # Positional arguments.
-###############################################################################
+#######################################################################
 
 positional = parser.add_argument_group(
     title='Positional arguments',
-    description=_('''
-        These arguments come after any flags and in the
-        order they are listed here. Only URL is required.'''
-    )
+    description=dedent("""
+    These arguments come after any flags and in the order they are listed here.
+    Only URL is required.
+
+    """)
 )
 positional.add_argument(
-    'method', metavar='METHOD',
+    'method',
+    metavar='METHOD',
     nargs=OPTIONAL,
     default=None,
-    help=_('''
-        The HTTP method to be used for the request
-        (GET, POST, PUT, DELETE, PATCH, ...).
-        If this argument is omitted, then HTTPie
-        will guess the HTTP method. If there is some
-        data to be sent, then it will be POST, otherwise GET.
-    ''')
+    help="""
+    The HTTP method to be used for the request (GET, POST, PUT, DELETE, ...).
+
+    This argument can be omitted in which case HTTPie will use POST if there
+    is some data to be sent, otherwise GET:
+
+        $ http example.org               # => GET
+        $ http example.org hello=world   # => POST
+
+    """
 )
 positional.add_argument(
-    'url', metavar='URL',
-    help=_('''
-        The protocol defaults to http:// if the
-        URL does not include one.
-    ''')
+    'url',
+    metavar='URL',
+    help="""
+    The scheme defaults to 'http://' if the URL does not include one.
+
+    You can also use a shorthand for localhost
+
+        $ http :3000                    # => http://localhost:3000
+        $ http :/foo                    # => http://localhost/foo
+
+    """
 )
 positional.add_argument(
-    'items', metavar='REQUEST ITEM',
+    'items',
+    metavar='REQUEST_ITEM',
     nargs=ZERO_OR_MORE,
-    type=KeyValueArgType(*SEP_GROUP_ITEMS),
-    help=_('''
-        A key-value pair whose type is defined by the
-        separator used. It can be an HTTP header (header:value),
-        a data field to be used in the request body (field_name=value),
-        a raw JSON data field (field_name:=value),
-        a query parameter (name==value),
-        or a file field (field_name@/path/to/file).
-        You can use a backslash to escape a colliding
-        separator in the field name.
-    ''')
+    type=KeyValueArgType(*SEP_GROUP_ALL_ITEMS),
+    help=r"""
+    Optional key-value pairs to be included in the request. The separator used
+    determines the type:
+
+    ':' HTTP headers:
+
+        Referer:http://httpie.org  Cookie:foo=bar  User-Agent:bacon/1.0
+
+    '==' URL parameters to be appended to the request URI:
+
+        search==httpie
+
+    '=' Data fields to be serialized into a JSON object (with --json, -j)
+        or form data (with --form, -f):
+
+        name=HTTPie  language=Python  description='CLI HTTP client'
+
+    ':=' Non-string JSON data fields (only with --json, -j):
+
+        awesome:=true  amount:=42  colors:='["red", "green", "blue"]'
+
+    '@' Form file fields (only with --form, -f):
+
+        cs@~/Documents/CV.pdf
+
+    '=@' A data field like '=', but takes a file path and embeds its content:
+
+         essay=@Documents/essay.txt
+
+    ':=@' A raw JSON field like ':=', but takes a file path and embeds its content:
+
+        package:=@./package.json
+
+    You can use a backslash to escape a colliding separator in the field name:
+
+        field-name-with\:colon=value
+
+    """
 )
 
 
-###############################################################################
+#######################################################################
 # Content type.
-###############################################################################
+#######################################################################
 
 content_type = parser.add_argument_group(
     title='Predefined content types',
     description=None
-).add_mutually_exclusive_group(required=False)
+)
 
 content_type.add_argument(
-    '--json', '-j', action='store_true',
-    help=_('''
-        (default) Data items from the command
-        line are serialized as a JSON object.
-        The Content-Type and Accept headers
-        are set to application/json (if not specified).
-    ''')
+    '--json', '-j',
+    action='store_true',
+    help="""
+    (default) Data items from the command line are serialized as a JSON object.
+    The Content-Type and Accept headers are set to application/json
+    (if not specified).
+
+    """
 )
 content_type.add_argument(
-    '--form', '-f', action='store_true',
-    help=_('''
-        Data items from the command line are serialized as form fields.
-        The Content-Type is set to application/x-www-form-urlencoded
-        (if not specified).
-        The presence of any file fields results
-        in a multipart/form-data request.
-     ''')
+    '--form', '-f',
+    action='store_true',
+    help="""
+    Data items from the command line are serialized as form fields.
+
+    The Content-Type is set to application/x-www-form-urlencoded (if not
+    specified). The presence of any file fields results in a
+    multipart/form-data request.
+
+    """
 )
 
 
-###############################################################################
+#######################################################################
 # Output processing
-###############################################################################
+#######################################################################
 
 output_processing = parser.add_argument_group(title='Output processing')
 
 output_processing.add_argument(
-    '--output', '-o', type=FileType('w+b'),
-    metavar='FILE',
-    help= SUPPRESS if not is_windows else _(
-        '''
-        Save output to FILE.
-        This option is a replacement for piping output to FILE,
-        which would on Windows result in corrupted data
-        being saved.
+    '--pretty',
+    dest='prettify',
+    default=PRETTY_STDOUT_TTY_ONLY,
+    choices=sorted(PRETTY_MAP.keys()),
+    help="""
+    Controls output processing. The value can be "none" to not prettify
+    the output (default for redirected output), "all" to apply both colors
+    and formatting (default for terminal output), "colors", or "format".
 
-        '''
+    """
+)
+output_processing.add_argument(
+    '--style', '-s',
+    dest='style',
+    metavar='STYLE',
+    default=DEFAULT_STYLE,
+    choices=AVAILABLE_STYLES,
+    help="""
+    Output coloring style (default is "{default}"). One of:
+
+{available}
+
+    For this option to work properly, please make sure that the $TERM
+    environment variable is set to "xterm-256color" or similar
+    (e.g., via `export TERM=xterm-256color' in your ~/.bashrc).
+
+    """
+    .format(
+        default=DEFAULT_STYLE,
+        available='\n'.join(
+            '{0: >20}'.format(line.strip())
+            for line in
+            wrap(' '.join(sorted(AVAILABLE_STYLES)), 60)
+        ),
     )
 )
-output_processing.add_argument(
-    '--pretty', dest='prettify', default=PRETTY_STDOUT_TTY_ONLY,
-    choices=sorted(PRETTY_MAP.keys()),
-    help=_('''
-        Controls output processing. The value can be "none" to not prettify
-        the output (default for redirected output), "all" to apply both colors
-        and formatting
-        (default for terminal output), "colors", or "format".
-    ''')
-)
-output_processing.add_argument(
-    '--style', '-s', dest='style', default=DEFAULT_STYLE, metavar='STYLE',
-    choices=AVAILABLE_STYLES,
-    help=_('''
-        Output coloring style. One of %s. Defaults to "%s".
-        For this option to work properly, please make sure that the
-        $TERM environment variable is set to "xterm-256color" or similar
-        (e.g., via `export TERM=xterm-256color' in your ~/.bashrc).
-    ''') % (', '.join(sorted(AVAILABLE_STYLES)), DEFAULT_STYLE)
-)
 
 
-
-###############################################################################
+#######################################################################
 # Output options
-###############################################################################
+#######################################################################
 output_options = parser.add_argument_group(title='Output options')
 
-output_print = output_options.add_mutually_exclusive_group(required=False)
-output_print.add_argument('--print', '-p', dest='output_options',
+output_options.add_argument(
+    '--print', '-p',
+    dest='output_options',
     metavar='WHAT',
-    help=_('''
-        String specifying what the output should contain:
-        "{request_headers}" stands for the request headers,  and
-        "{request_body}" for the request body.
-        "{response_headers}" stands for the response headers and
-        "{response_body}" for response the body.
-        The default behaviour is "hb" (i.e., the response
-        headers and body is printed), if standard output is not redirected.
-        If the output is piped to another program or to a file,
-        then only the body is printed by default.
-    '''.format(
-        request_headers=OUT_REQ_HEAD,
-        request_body=OUT_REQ_BODY,
-        response_headers=OUT_RESP_HEAD,
-        response_body=OUT_RESP_BODY,
-    ))
+    help="""
+    String specifying what the output should contain:
+
+        '{req_head}' request headers
+        '{req_body}' request body
+        '{res_head}' response headers
+        '{res_body}' response body
+
+    The default behaviour is '{default}' (i.e., the response headers and body
+    is printed), if standard output is not redirected. If the output is piped
+    to another program or to a file, then only the response body is printed
+    by default.
+
+    """
+    .format(
+        req_head=OUT_REQ_HEAD,
+        req_body=OUT_REQ_BODY,
+        res_head=OUT_RESP_HEAD,
+        res_body=OUT_RESP_BODY,
+        default=OUTPUT_OPTIONS_DEFAULT,
+    )
 )
-output_print.add_argument(
-    '--verbose', '-v', dest='output_options',
-    action='store_const', const=''.join(OUTPUT_OPTIONS),
-    help=_('''
-        Print the whole request as well as the response.
-        Shortcut for --print={0}.
-    '''.format(''.join(OUTPUT_OPTIONS)))
+output_options.add_argument(
+    '--verbose', '-v',
+    dest='output_options',
+    action='store_const',
+    const=''.join(OUTPUT_OPTIONS),
+    help="""
+    Print the whole request as well as the response. Shortcut for --print={0}.
+
+    """
+    .format(''.join(OUTPUT_OPTIONS))
 )
-output_print.add_argument(
-    '--headers', '-h', dest='output_options',
-    action='store_const', const=OUT_RESP_HEAD,
-    help=_('''
-        Print only the response headers.
-        Shortcut for --print={0}.
-    '''.format(OUT_RESP_HEAD))
+output_options.add_argument(
+    '--headers', '-h',
+    dest='output_options',
+    action='store_const',
+    const=OUT_RESP_HEAD,
+    help="""
+    Print only the response headers. Shortcut for --print={0}.
+
+    """
+    .format(OUT_RESP_HEAD)
 )
-output_print.add_argument(
-    '--body', '-b', dest='output_options',
-    action='store_const', const=OUT_RESP_BODY,
-    help=_('''
-        Print only the response body.
-        Shortcut for --print={0}.
-    '''.format(OUT_RESP_BODY))
+output_options.add_argument(
+    '--body', '-b',
+    dest='output_options',
+    action='store_const',
+    const=OUT_RESP_BODY,
+    help="""
+    Print only the response body. Shortcut for --print={0}.
+
+    """
+    .format(OUT_RESP_BODY)
 )
 
-output_options.add_argument('--stream', '-S', action='store_true', default=False,
-    help=_('''
+output_options.add_argument(
+    '--stream', '-S',
+    action='store_true',
+    default=False,
+    help="""
     Always stream the output by line, i.e., behave like `tail -f'.
 
     Without --stream and with --pretty (either set or implied),
@@ -215,138 +300,249 @@ output_options.add_argument('--stream', '-S', action='store_true', default=False
     It is useful also without --pretty: It ensures that the output is flushed
     more often and in smaller chunks.
 
-    '''
-))
+    """
+)
+output_options.add_argument(
+    '--output', '-o',
+    type=FileType('a+b'),
+    dest='output_file',
+    metavar='FILE',
+    help="""
+    Save output to FILE. If --download is set, then only the response body is
+    saved to the file. Other parts of the HTTP exchange are printed to stderr.
+
+    """
+
+)
+
+output_options.add_argument(
+    '--download', '-d',
+    action='store_true',
+    default=False,
+    help="""
+    Do not print the response body to stdout. Rather, download it and store it
+    in a file. The filename is guessed unless specified with --output
+    [filename]. This action is similar to the default behaviour of wget.
+
+    """
+)
+
+output_options.add_argument(
+    '--continue', '-c',
+    dest='download_resume',
+    action='store_true',
+    default=False,
+    help="""
+    Resume an interrupted download. Note that the --output option needs to be
+    specified as well.
+
+    """
+)
 
 
-###############################################################################
+#######################################################################
 # Sessions
-###############################################################################
+#######################################################################
+
 sessions = parser.add_argument_group(title='Sessions')\
                  .add_mutually_exclusive_group(required=False)
 
+session_name_validator = SessionNameValidator(
+    'Session name contains invalid characters.'
+)
+
 sessions.add_argument(
-    '--session', metavar='SESSION_NAME',
-    help=_('''
-    Create, or reuse and update a session.
-    Withing a session, custom headers, auth credential, as well as any
-    cookies sent by the server persist between requests.
-    You can use the `httpie' management command to manipulate
-    and inspect existing sessions. See `httpie --help'.
-    ''')
+    '--session',
+    metavar='SESSION_NAME_OR_PATH',
+    type=session_name_validator,
+    help="""
+    Create, or reuse and update a session. Within a session, custom headers,
+    auth credential, as well as any cookies sent by the server persist between
+    requests.
+
+    Session files are stored in:
+
+        {session_dir}/<HOST>/<SESSION_NAME>.json.
+
+    """
+    .format(session_dir=DEFAULT_SESSIONS_DIR)
 )
 sessions.add_argument(
-    '--session-read-only', metavar='SESSION_NAME',
-    help=_('''
-        Create or reuse a session, but do not update it once saved.
-    ''')
+    '--session-read-only',
+    metavar='SESSION_NAME_OR_PATH',
+    type=session_name_validator,
+    help="""
+    Create or read a session without updating it form the request/response
+    exchange.
+
+    """
 )
 
 
-###############################################################################
+#######################################################################
 # Authentication
-###############################################################################
+#######################################################################
+
 # ``requests.request`` keyword arguments.
 auth = parser.add_argument_group(title='Authentication')
 auth.add_argument(
-    '--auth', '-a', metavar='USER[:PASS]',
+    '--auth', '-a',
+    metavar='USER[:PASS]',
     type=AuthCredentialsArgType(SEP_CREDENTIALS),
-    help=_('''
-        If only the username is provided (-a username),
-        HTTPie will prompt for the password.
-    '''),
+    help="""
+    If only the username is provided (-a username), HTTPie will prompt
+    for the password.
+
+    """,
 )
 
+_auth_plugins = plugin_manager.get_auth_plugins()
 auth.add_argument(
-    '--auth-type', choices=['basic', 'digest'], default='basic',
-    help=_('''
-        The authentication mechanism to be used.
-        Defaults to "basic".
-    ''')
+    '--auth-type',
+    choices=[plugin.auth_type for plugin in _auth_plugins],
+    default=_auth_plugins[0].auth_type,
+    help="""
+    The authentication mechanism to be used. Defaults to "{default}".
+
+    {types}
+
+    """
+    .format(default=_auth_plugins[0].auth_type, types='\n    '.join(
+        '"{type}": {name}{package}{description}'.format(
+            type=plugin.auth_type,
+            name=plugin.name,
+            package=(
+                '' if issubclass(plugin, BuiltinAuthPlugin)
+                else ' (provided by %s)' % plugin.package_name
+            ),
+            description=(
+                '' if not plugin.description else
+                '\n      ' + ('\n      '.join(wrap(plugin.description)))
+            )
+        )
+        for plugin in _auth_plugins
+    )),
 )
 
 
-
+#######################################################################
 # Network
-#############################################
+#######################################################################
 
 network = parser.add_argument_group(title='Network')
 
 network.add_argument(
-    '--proxy', default=[], action='append', metavar='PROTOCOL:HOST',
+    '--proxy',
+    default=[],
+    action='append',
+    metavar='PROTOCOL:PROXY_URL',
     type=KeyValueArgType(SEP_PROXY),
-    help=_('''
-        String mapping protocol to the URL of the proxy
-        (e.g. http:foo.bar:3128). You can specify multiple
-        proxies with different protocols.
-    ''')
+    help="""
+    String mapping protocol to the URL of the proxy
+    (e.g. http:http://foo.bar:3128). You can specify multiple proxies with
+    different protocols.
+
+    """
 )
 network.add_argument(
-    '--follow', default=False, action='store_true',
-    help=_('''
-        Set this flag if full redirects are allowed
-        (e.g. re-POST-ing of data at new ``Location``)
-    ''')
+    '--follow',
+    default=False,
+    action='store_true',
+    help="""
+    Set this flag if full redirects are allowed (e.g. re-POST-ing of data at
+    new Location).
+
+    """
 )
 network.add_argument(
-    '--verify', default='yes',
-    help=_('''
-        Set to "no" to skip checking the host\'s SSL certificate.
-        You can also pass the  path to a CA_BUNDLE
-        file for private certs. You can also set
-        the REQUESTS_CA_BUNDLE  environment variable.
-        Defaults to "yes".
-    ''')
+    '--verify',
+    default='yes',
+    help="""
+    Set to "no" to skip checking the host's SSL certificate. You can also pass
+    the  path to a CA_BUNDLE file for private certs. You can also set the
+    REQUESTS_CA_BUNDLE  environment variable. Defaults to "yes".
+
+    """
 )
 
 network.add_argument(
-    '--timeout', type=float, default=30, metavar='SECONDS',
-    help=_('''
-        The connection timeout of the request in seconds.
-        The default value is 30 seconds.
-    ''')
+    '--timeout',
+    type=float,
+    default=30,
+    metavar='SECONDS',
+    help="""
+    The connection timeout of the request in seconds. The default value is
+    30 seconds.
+
+    """
 )
 network.add_argument(
-    '--check-status', default=False, action='store_true',
-    help=_('''
-        By default, HTTPie exits with 0 when no network or other fatal
-        errors occur.
+    '--check-status',
+    default=False,
+    action='store_true',
+    help="""
+    By default, HTTPie exits with 0 when no network or other fatal errors
+    occur. This flag instructs HTTPie to also check the HTTP status code and
+    exit with an error if the status indicates one.
 
-        This flag instructs HTTPie to also check the HTTP status code and
-        exit with an error if the status indicates one.
+    When the server replies with a 4xx (Client Error) or 5xx (Server Error)
+    status code, HTTPie exits with 4 or 5 respectively. If the response is a
+    3xx (Redirect) and --follow hasn't been set, then the exit status is 3.
+    Also an error message is written to stderr if stdout is redirected.
 
-        When the server replies with a 4xx (Client Error) or 5xx
-        (Server Error) status code, HTTPie exits with 4 or 5 respectively.
-        If the response is a 3xx (Redirect) and --follow
-        hasn't been set, then the exit status is 3.
-
-        Also an error message is written to stderr if stdout is redirected.
-
-    ''')
+    """
 )
 
 
-###############################################################################
+#######################################################################
 # Troubleshooting
-###############################################################################
+#######################################################################
 
 troubleshooting = parser.add_argument_group(title='Troubleshooting')
 
+troubleshooting.add_argument(
+    '--ignore-stdin',
+    action='store_true',
+    default=False,
+    help="""
+    Do not attempt to read stdin.
+
+    """
+)
 troubleshooting.add_argument(
     '--help',
-    action='help', default=SUPPRESS,
-    help='Show this help message and exit'
-)
-troubleshooting.add_argument('--version', action='version', version=__version__)
-troubleshooting.add_argument(
-    '--traceback', action='store_true', default=False,
-    help='Prints exception traceback should one occur.'
+    action='help',
+    default=SUPPRESS,
+    help="""
+    Show this help message and exit.
+
+    """
 )
 troubleshooting.add_argument(
-    '--debug', action='store_true', default=False,
-    help=_('''
-        Prints exception traceback should one occur, and also other
-        information that is useful for debugging HTTPie itself and
-        for bug reports.
-    ''')
+    '--version',
+    action='version',
+    version=__version__,
+    help="""
+    Show version and exit.
+
+    """
+)
+troubleshooting.add_argument(
+    '--traceback',
+    action='store_true',
+    default=False,
+    help="""
+    Prints exception traceback should one occur.
+
+    """
+)
+troubleshooting.add_argument(
+    '--debug',
+    action='store_true',
+    default=False,
+    help="""
+    Prints exception traceback should one occur, and also other information
+    that is useful for debugging HTTPie itself and for reporting bugs.
+
+    """
 )

httpie/client.py

@@ -3,11 +3,10 @@ import sys
 from pprint import pformat
 
 import requests
-import requests.auth
-from requests.defaults import defaults
 
 from . import sessions
 from . import __version__
+from .plugins import plugin_manager
 
 
 FORM = 'application/x-www-form-urlencoded; charset=utf-8'
@@ -21,51 +20,61 @@ def get_response(args, config_dir):
     requests_kwargs = get_requests_kwargs(args)
 
     if args.debug:
-        sys.stderr.write(
-            '\n>>> requests.request(%s)\n\n' % pformat(requests_kwargs))
+        sys.stderr.write('\n>>> requests.request(%s)\n\n'
+                         % pformat(requests_kwargs))
 
     if not args.session and not args.session_read_only:
-        return requests.request(**requests_kwargs)
+        response = requests.request(**requests_kwargs)
     else:
-        return sessions.get_response(
+        response = sessions.get_response(
+            args=args,
             config_dir=config_dir,
-            name=args.session or args.session_read_only,
-            request_kwargs=requests_kwargs,
+            session_name=args.session or args.session_read_only,
+            requests_kwargs=requests_kwargs,
             read_only=bool(args.session_read_only),
         )
 
+    return response
+
 
 def get_requests_kwargs(args):
     """Translate our `args` into `requests.request` keyword arguments."""
 
-    base_headers = defaults['base_headers'].copy()
-    base_headers['User-Agent'] = DEFAULT_UA
+    implicit_headers = {
+        'User-Agent': DEFAULT_UA
+    }
 
     auto_json = args.data and not args.form
+    # FIXME: Accept is set to JSON with `http url @./file.txt`.
     if args.json or auto_json:
-        base_headers['Accept'] = 'application/json'
-        if args.data:
-            base_headers['Content-Type'] = JSON
+        implicit_headers['Accept'] = 'application/json'
+        if args.json or (auto_json and args.data):
+            implicit_headers['Content-Type'] = JSON
 
         if isinstance(args.data, dict):
-            # If not empty, serialize the data `dict` parsed from arguments.
-            # Otherwise set it to `None` avoid sending "{}".
-            args.data = json.dumps(args.data) if args.data else None
+            if args.data:
+                args.data = json.dumps(args.data)
+            else:
+                # We need to set data to an empty string to prevent requests
+                # from assigning an empty list to `response.request.data`.
+                args.data = ''
 
     elif args.form and not args.files:
         # If sending files, `requests` will set
         # the `Content-Type` for us.
-        base_headers['Content-Type'] = FORM
+        implicit_headers['Content-Type'] = FORM
+
+    for name, value in implicit_headers.items():
+        if name not in args.headers:
+            args.headers[name] = value
 
     credentials = None
     if args.auth:
-        credentials = {
-            'basic': requests.auth.HTTPBasicAuth,
-            'digest': requests.auth.HTTPDigestAuth,
-        }[args.auth_type](args.auth.key, args.auth.value)
+        auth_plugin = plugin_manager.get_auth_plugin(args.auth_type)()
+        credentials = auth_plugin.get_auth(args.auth.key, args.auth.value)
 
     kwargs = {
-        'prefetch': False,
+        'stream': True,
         'method': args.method.lower(),
         'url': args.url,
         'headers': args.headers,
@@ -80,9 +89,6 @@ def get_requests_kwargs(args):
         'files': args.files,
         'allow_redirects': args.follow,
         'params': args.params,
-        'config': {
-            'base_headers': base_headers
-        }
     }
 
     return kwargs

httpie/compat.py

@@ -0,0 +1,19 @@
+"""
+Python 2/3 compatibility.
+
+"""
+#noinspection PyUnresolvedReferences
+from requests.compat import (
+    is_windows,
+    bytes,
+    str,
+    is_py3,
+    is_py26,
+)
+
+try:
+    #noinspection PyUnresolvedReferences,PyCompatibility
+    from urllib.parse import urlsplit
+except ImportError:
+    #noinspection PyUnresolvedReferences,PyCompatibility
+    from urlparse import urlsplit

httpie/config.py

@@ -3,7 +3,7 @@ import json
 import errno
 
 from . import __version__
-from requests.compat import  is_windows
+from .compat import is_windows
 
 
 DEFAULT_CONFIG_DIR = os.environ.get(
@@ -16,8 +16,9 @@ DEFAULT_CONFIG_DIR = os.environ.get(
 class BaseConfigDict(dict):
 
     name = None
-    help = None
-    directory=DEFAULT_CONFIG_DIR
+    helpurl = None
+    about = None
+    directory = DEFAULT_CONFIG_DIR
 
     def __init__(self, directory=None, *args, **kwargs):
         super(BaseConfigDict, self).__init__(*args, **kwargs)
@@ -27,18 +28,24 @@ class BaseConfigDict(dict):
     def __getattr__(self, item):
         return self[item]
 
-    @property
-    def path(self):
-        try:
-            os.makedirs(self.directory, mode=0o700)
-        except OSError as e:
-            if e.errno != errno.EEXIST:
-                raise
+    def _get_path(self):
+        """Return the config file path without side-effects."""
         return os.path.join(self.directory, self.name + '.json')
 
+    @property
+    def path(self):
+        """Return the config file path creating basedir, if needed."""
+        path = self._get_path()
+        try:
+            os.makedirs(os.path.dirname(path), mode=0o700)
+        except OSError as e:
+            if e.errno != errno.EEXIST:
+                raise
+        return path
+
     @property
     def is_new(self):
-        return not os.path.exists(self.path)
+        return not os.path.exists(self._get_path())
 
     def load(self):
         try:
@@ -56,7 +63,15 @@ class BaseConfigDict(dict):
                 raise
 
     def save(self):
-        self['__version__'] = __version__
+        self['__meta__'] = {
+            'httpie': __version__
+        }
+        if self.helpurl:
+            self['__meta__']['help'] = self.helpurl
+
+        if self.about:
+            self['__meta__']['about'] = self.about
+
         with open(self.path, 'w') as f:
             json.dump(self, f, indent=4, sort_keys=True, ensure_ascii=True)
             f.write('\n')
@@ -72,6 +87,8 @@ class BaseConfigDict(dict):
 class Config(BaseConfigDict):
 
     name = 'config'
+    helpurl = 'https://github.com/jkbr/httpie#config'
+    about = 'HTTPie configuration file'
 
     DEFAULTS = {
         'implicit_content_type': 'json',

httpie/core.py

@@ -14,31 +14,32 @@ import sys
 import errno
 
 import requests
-from requests.compat import str, is_py3
 from httpie import __version__ as httpie_version
 from requests import __version__ as requests_version
 from pygments import __version__ as pygments_version
 
-from .cli import parser
+from .compat import str, is_py3
 from .client import get_response
+from .downloads import Download
 from .models import Environment
-from .output import output_stream, write, write_with_colors_win_p3k
-from . import EXIT
+from .output import build_output_stream, write, write_with_colors_win_py3
+from . import ExitStatus
+from .plugins import plugin_manager
 
 
-def get_exist_status(code, follow=False):
-    """Translate HTTP status code to exit status."""
-    if 300 <= code <= 399 and not follow:
+def get_exit_status(http_status, follow=False):
+    """Translate HTTP status code to exit status code."""
+    if 300 <= http_status <= 399 and not follow:
         # Redirect
-        return EXIT.ERROR_HTTP_3XX
-    elif 400 <= code <= 499:
+        return ExitStatus.ERROR_HTTP_3XX
+    elif 400 <= http_status <= 499:
         # Client Error
-        return EXIT.ERROR_HTTP_4XX
-    elif 500 <= code <= 599:
+        return ExitStatus.ERROR_HTTP_4XX
+    elif 500 <= http_status <= 599:
         # Server Error
-        return EXIT.ERROR_HTTP_5XX
+        return ExitStatus.ERROR_HTTP_5XX
     else:
-        return EXIT.OK
+        return ExitStatus.OK
 
 
 def print_debug_info(env):
@@ -54,70 +55,117 @@ def print_debug_info(env):
 def main(args=sys.argv[1:], env=Environment()):
     """Run the main program and write the output to ``env.stdout``.
 
-    Return exit status.
+    Return exit status code.
 
     """
+    plugin_manager.load_installed_plugins()
+    from .cli import parser
+
     if env.config.default_options:
         args = env.config.default_options + args
 
-    def error(msg, *args):
+    def error(msg, *args, **kwargs):
         msg = msg % args
-        env.stderr.write('\nhttp: error: %s\n' % msg)
+        level = kwargs.get('level', 'error')
+        env.stderr.write('\nhttp: %s: %s\n' % (level, msg))
 
     debug = '--debug' in args
     traceback = debug or '--traceback' in args
-    status = EXIT.OK
+    exit_status = ExitStatus.OK
 
     if debug:
         print_debug_info(env)
         if args == ['--debug']:
-            sys.exit(EXIT.OK)
+            return exit_status
+
+    download = None
 
     try:
         args = parser.parse_args(args=args, env=env)
 
+        if args.download:
+            args.follow = True  # --download implies --follow.
+            download = Download(
+                output_file=args.output_file,
+                progress_file=env.stderr,
+                resume=args.download_resume
+            )
+            download.pre_request(args.headers)
+
         response = get_response(args, config_dir=env.config.directory)
 
-        if args.check_status:
-            status = get_exist_status(response.status_code,
-                                      args.follow)
-            if status and not env.stdout_isatty:
-                error('%s %s', response.raw.status, response.raw.reason)
+        if args.check_status or download:
 
-        stream = output_stream(args, env, response.request, response)
+            exit_status = get_exit_status(
+                http_status=response.status_code,
+                follow=args.follow
+            )
+
+            if not env.stdout_isatty and exit_status != ExitStatus.OK:
+                error('HTTP %s %s',
+                      response.raw.status,
+                      response.raw.reason,
+                      level='warning')
 
         write_kwargs = {
-            'stream': stream,
+            'stream': build_output_stream(
+                args, env, response.request, response),
+
+            # This will in fact be `stderr` with `--download`
             'outfile': env.stdout,
+
             'flush': env.stdout_isatty or args.stream
         }
+
         try:
+
             if env.is_windows and is_py3 and 'colors' in args.prettify:
-                write_with_colors_win_p3k(**write_kwargs)
+                write_with_colors_win_py3(**write_kwargs)
             else:
                 write(**write_kwargs)
 
+            if download and exit_status == ExitStatus.OK:
+                # Response body download.
+                download_stream, download_to = download.start(response)
+                write(
+                    stream=download_stream,
+                    outfile=download_to,
+                    flush=False,
+                )
+                download.finish()
+                if download.interrupted:
+                    exit_status = ExitStatus.ERROR
+                    error('Incomplete download: size=%d; downloaded=%d' % (
+                        download.status.total_size,
+                        download.status.downloaded
+                    ))
+
         except IOError as e:
             if not traceback and e.errno == errno.EPIPE:
                 # Ignore broken pipes unless --traceback.
                 env.stderr.write('\n')
             else:
                 raise
-
     except (KeyboardInterrupt, SystemExit):
         if traceback:
             raise
         env.stderr.write('\n')
-        status = EXIT.ERROR
+        exit_status = ExitStatus.ERROR
+
     except requests.Timeout:
-        status = EXIT.ERROR_TIMEOUT
+        exit_status = ExitStatus.ERROR_TIMEOUT
         error('Request timed out (%ss).', args.timeout)
+
     except Exception as e:
-        # TODO: distinguish between expected and unexpected errors.
-        #       network errors vs. bugs, etc.
+        # TODO: Better distinction between expected and unexpected errors.
+        #       Network errors vs. bugs, etc.
         if traceback:
             raise
         error('%s: %s', type(e).__name__, str(e))
-        status = EXIT.ERROR
+        exit_status = ExitStatus.ERROR
 
-    return status
+    finally:
+        if download and not download.finished:
+            download.failed()
+
+    return exit_status

httpie/downloads.py

@@ -0,0 +1,427 @@
+# coding=utf-8
+"""
+Download mode implementation.
+
+"""
+from __future__ import division
+import os
+import re
+import sys
+import mimetypes
+import threading
+from time import sleep, time
+from mailbox import Message
+
+from .output import RawStream
+from .models import HTTPResponse
+from .utils import humanize_bytes
+from .compat import urlsplit
+
+
+PARTIAL_CONTENT = 206
+
+
+CLEAR_LINE = '\r\033[K'
+PROGRESS = (
+    '{percentage: 6.2f} %'
+    ' {downloaded: >10}'
+    ' {speed: >10}/s'
+    ' {eta: >8} ETA'
+)
+PROGRESS_NO_CONTENT_LENGTH = '{downloaded: >10} {speed: >10}/s'
+SUMMARY = 'Done. {downloaded} in {time:0.5f}s ({speed}/s)\n'
+SPINNER = '|/-\\'
+
+
+class ContentRangeError(ValueError):
+    pass
+
+
+def parse_content_range(content_range, resumed_from):
+    """
+    Parse and validate Content-Range header.
+
+    <http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html>
+
+    :param content_range: the value of a Content-Range response header
+                          eg. "bytes 21010-47021/47022"
+    :param resumed_from: first byte pos. from the Range request header
+    :return: total size of the response body when fully downloaded.
+
+    """
+    if content_range is None:
+        raise ContentRangeError('Missing Content-Range')
+
+    pattern = (
+        '^bytes (?P<first_byte_pos>\d+)-(?P<last_byte_pos>\d+)'
+        '/(\*|(?P<instance_length>\d+))$'
+    )
+    match = re.match(pattern, content_range)
+
+    if not match:
+        raise ContentRangeError(
+            'Invalid Content-Range format %r' % content_range)
+
+    content_range_dict = match.groupdict()
+    first_byte_pos = int(content_range_dict['first_byte_pos'])
+    last_byte_pos = int(content_range_dict['last_byte_pos'])
+    instance_length = (
+        int(content_range_dict['instance_length'])
+        if content_range_dict['instance_length']
+        else None
+    )
+
+    # "A byte-content-range-spec with a byte-range-resp-spec whose
+    # last- byte-pos value is less than its first-byte-pos value,
+    # or whose instance-length value is less than or equal to its
+    # last-byte-pos value, is invalid. The recipient of an invalid
+    # byte-content-range- spec MUST ignore it and any content
+    # transferred along with it."
+    if (first_byte_pos >= last_byte_pos
+            or (instance_length is not None
+                and instance_length <= last_byte_pos)):
+        raise ContentRangeError(
+            'Invalid Content-Range returned: %r' % content_range)
+
+    if (first_byte_pos != resumed_from
+        or (instance_length is not None
+            and last_byte_pos + 1 != instance_length)):
+        # Not what we asked for.
+        raise ContentRangeError(
+            'Unexpected Content-Range returned (%r)'
+            ' for the requested Range ("bytes=%d-")'
+            % (content_range, resumed_from)
+        )
+
+    return last_byte_pos + 1
+
+
+def filename_from_content_disposition(content_disposition):
+    """
+    Extract and validate filename from a Content-Disposition header.
+
+    :param content_disposition: Content-Disposition value
+    :return: the filename if present and valid, otherwise `None`
+
+    """
+    # attachment; filename=jkbr-httpie-0.4.1-20-g40bd8f6.tar.gz
+
+    msg = Message('Content-Disposition: %s' % content_disposition)
+    filename = msg.get_filename()
+    if filename:
+        # Basic sanitation.
+        filename = os.path.basename(filename).lstrip('.').strip()
+        if filename:
+            return filename
+
+
+def filename_from_url(url, content_type):
+    fn = urlsplit(url).path.rstrip('/')
+    fn = os.path.basename(fn) if fn else 'index'
+    if '.' not in fn and content_type:
+        content_type = content_type.split(';')[0]
+        if content_type == 'text/plain':
+            # mimetypes returns '.ksh'
+            ext = '.txt'
+        else:
+            ext = mimetypes.guess_extension(content_type)
+
+        if ext == '.htm':  # Python 3
+            ext = '.html'
+
+        if ext:
+            fn += ext
+
+    return fn
+
+
+def get_unique_filename(fn, exists=os.path.exists):
+    attempt = 0
+    while True:
+        suffix = '-' + str(attempt) if attempt > 0 else ''
+        if not exists(fn + suffix):
+            return fn + suffix
+        attempt += 1
+
+
+class Download(object):
+
+    def __init__(self, output_file=None,
+                 resume=False, progress_file=sys.stderr):
+        """
+        :param resume: Should the download resume if partial download
+                       already exists.
+        :type resume: bool
+
+        :param output_file: The file to store response body in. If not
+                            provided, it will be guessed from the response.
+        :type output_file: file
+
+        :param progress_file: Where to report download progress.
+        :type progress_file: file
+
+        """
+        self._output_file = output_file
+        self._resume = resume
+        self._resumed_from = 0
+        self.finished = False
+
+        self.status = Status()
+        self._progress_reporter = ProgressReporterThread(
+            status=self.status,
+            output=progress_file
+        )
+
+    def pre_request(self, request_headers):
+        """Called just before the HTTP request is sent.
+
+        Might alter `request_headers`.
+
+        :type request_headers: dict
+
+        """
+        # Disable content encoding so that we can resume, etc.
+        request_headers['Accept-Encoding'] = None
+        if self._resume:
+            bytes_have = os.path.getsize(self._output_file.name)
+            if bytes_have:
+                # Set ``Range`` header to resume the download
+                # TODO: Use "If-Range: mtime" to make sure it's fresh?
+                request_headers['Range'] = 'bytes=%d-' % bytes_have
+                self._resumed_from = bytes_have
+
+    def start(self, response):
+        """
+        Initiate and return a stream for `response` body  with progress
+        callback attached. Can be called only once.
+
+        :param response: Initiated response object with headers already fetched
+        :type response: requests.models.Response
+
+        :return: RawStream, output_file
+
+        """
+        assert not self.status.time_started
+
+        try:
+            total_size = int(response.headers['Content-Length'])
+        except (KeyError, ValueError, TypeError):
+            total_size = None
+
+        if self._output_file:
+            if self._resume and response.status_code == PARTIAL_CONTENT:
+                total_size = parse_content_range(
+                    response.headers.get('Content-Range'),
+                    self._resumed_from
+                )
+
+            else:
+                self._resumed_from = 0
+                try:
+                    self._output_file.seek(0)
+                    self._output_file.truncate()
+                except IOError:
+                    pass  # stdout
+        else:
+            # TODO: Should the filename be taken from response.history[0].url?
+            # Output file not specified. Pick a name that doesn't exist yet.
+            fn = None
+            if 'Content-Disposition' in response.headers:
+                fn = filename_from_content_disposition(
+                    response.headers['Content-Disposition'])
+            if not fn:
+                fn = filename_from_url(
+                    url=response.url,
+                    content_type=response.headers.get('Content-Type'),
+                )
+            self._output_file = open(get_unique_filename(fn), mode='a+b')
+
+        self.status.started(
+            resumed_from=self._resumed_from,
+            total_size=total_size
+        )
+
+        stream = RawStream(
+            msg=HTTPResponse(response),
+            with_headers=False,
+            with_body=True,
+            on_body_chunk_downloaded=self.chunk_downloaded,
+            chunk_size=1024 * 8
+        )
+
+        self._progress_reporter.output.write(
+            'Downloading %sto "%s"\n' % (
+                (humanize_bytes(total_size) + ' '
+                 if total_size is not None
+                 else ''),
+                self._output_file.name
+            )
+        )
+        self._progress_reporter.start()
+
+        return stream, self._output_file
+
+    def finish(self):
+        assert not self.finished
+        self.finished = True
+        self.status.finished()
+
+    def failed(self):
+        self._progress_reporter.stop()
+
+    @property
+    def interrupted(self):
+        return (
+            self.finished
+            and self.status.total_size
+            and self.status.total_size != self.status.downloaded
+        )
+
+    def chunk_downloaded(self, chunk):
+        """
+        A download progress callback.
+
+        :param chunk: A chunk of response body data that has just
+                      been downloaded and written to the output.
+        :type chunk: bytes
+
+        """
+        self.status.chunk_downloaded(len(chunk))
+
+
+class Status(object):
+    """Holds details about the downland status."""
+
+    def __init__(self):
+        self.downloaded = 0
+        self.total_size = None
+        self.resumed_from = 0
+        self.time_started = None
+        self.time_finished = None
+
+    def started(self, resumed_from=0, total_size=None):
+        assert self.time_started is None
+        if total_size is not None:
+            self.total_size = total_size
+        self.downloaded = self.resumed_from = resumed_from
+        self.time_started = time()
+
+    def chunk_downloaded(self, size):
+        assert self.time_finished is None
+        self.downloaded += size
+
+    @property
+    def has_finished(self):
+        return self.time_finished is not None
+
+    def finished(self):
+        assert self.time_started is not None
+        assert self.time_finished is None
+        self.time_finished = time()
+
+
+class ProgressReporterThread(threading.Thread):
+    """
+    Reports download progress based on its status.
+
+    Uses threading to periodically update the status (speed, ETA, etc.).
+
+    """
+    def __init__(self, status, output, tick=.1, update_interval=1):
+        """
+
+        :type status: Status
+        :type output: file
+        """
+        super(ProgressReporterThread, self).__init__()
+        self.status = status
+        self.output = output
+        self._tick = tick
+        self._update_interval = update_interval
+        self._spinner_pos = 0
+        self._status_line = ''
+        self._prev_bytes = 0
+        self._prev_time = time()
+        self._should_stop = threading.Event()
+
+    def stop(self):
+        """Stop reporting on next tick."""
+        self._should_stop.set()
+
+    def run(self):
+        while not self._should_stop.is_set():
+            if self.status.has_finished:
+                self.sum_up()
+                break
+
+            self.report_speed()
+            sleep(self._tick)
+
+    def report_speed(self):
+
+        now = time()
+
+        if now - self._prev_time >= self._update_interval:
+            downloaded = self.status.downloaded
+            try:
+                speed = ((downloaded - self._prev_bytes)
+                         / (now - self._prev_time))
+            except ZeroDivisionError:
+                speed = 0
+
+            if not self.status.total_size:
+                self._status_line = PROGRESS_NO_CONTENT_LENGTH.format(
+                    downloaded=humanize_bytes(downloaded),
+                    speed=humanize_bytes(speed),
+                )
+            else:
+                try:
+                    percentage = downloaded / self.status.total_size * 100
+                except ZeroDivisionError:
+                    percentage = 0
+
+                if not speed:
+                    eta = '-:--:--'
+                else:
+                    s = int((self.status.total_size - downloaded) / speed)
+                    h, s = divmod(s, 60 * 60)
+                    m, s = divmod(s, 60)
+                    eta = '{0}:{1:0>2}:{2:0>2}'.format(h, m, s)
+
+                self._status_line = PROGRESS.format(
+                    percentage=percentage,
+                    downloaded=humanize_bytes(downloaded),
+                    speed=humanize_bytes(speed),
+                    eta=eta,
+                )
+
+            self._prev_time = now
+            self._prev_bytes = downloaded
+
+        self.output.write(
+            CLEAR_LINE
+            + ' '
+            + SPINNER[self._spinner_pos]
+            + ' '
+            + self._status_line
+        )
+        self.output.flush()
+
+        self._spinner_pos = (self._spinner_pos + 1
+                             if self._spinner_pos + 1 != len(SPINNER)
+                             else 0)
+
+    def sum_up(self):
+        actually_downloaded = (self.status.downloaded
+                               - self.status.resumed_from)
+        time_taken = self.status.time_finished - self.status.time_started
+
+        self.output.write(CLEAR_LINE)
+        self.output.write(SUMMARY.format(
+            downloaded=humanize_bytes(actually_downloaded),
+            total=(self.status.total_size
+                   and humanize_bytes(self.status.total_size)),
+            speed=humanize_bytes(actually_downloaded / time_taken),
+            time=time_taken,
+        ))
+        self.output.flush()

httpie/input.py

@@ -8,15 +8,20 @@ import json
 import mimetypes
 import getpass
 from io import BytesIO
-from argparse import ArgumentParser, ArgumentTypeError
+#noinspection PyCompatibility
+from argparse import ArgumentParser, ArgumentTypeError, ArgumentError
 
 try:
     from collections import OrderedDict
 except ImportError:
     OrderedDict = dict
 
+# TODO: Use MultiDict for headers once added to `requests`.
+# https://github.com/jkbr/httpie/issues/130
 from requests.structures import CaseInsensitiveDict
-from requests.compat import str, urlparse
+
+from .compat import urlsplit, str
+from .sessions import VALID_SESSION_NAME_PATTERN
 
 
 HTTP_POST = 'POST'
@@ -32,22 +37,40 @@ SEP_PROXY = ':'
 SEP_DATA = '='
 SEP_DATA_RAW_JSON = ':='
 SEP_FILES = '@'
+SEP_DATA_EMBED_FILE = '=@'
+SEP_DATA_EMBED_RAW_JSON_FILE = ':=@'
 SEP_QUERY = '=='
 
 # Separators that become request data
 SEP_GROUP_DATA_ITEMS = frozenset([
     SEP_DATA,
     SEP_DATA_RAW_JSON,
-    SEP_FILES
+    SEP_FILES,
+    SEP_DATA_EMBED_FILE,
+    SEP_DATA_EMBED_RAW_JSON_FILE
+])
+
+# Separators for items whose value is a filename to be embedded
+SEP_GROUP_DATA_EMBED_ITEMS = frozenset([
+    SEP_DATA_EMBED_FILE,
+    SEP_DATA_EMBED_RAW_JSON_FILE,
+])
+
+# Separators for raw JSON items
+SEP_GROUP_RAW_JSON_ITEMS = frozenset([
+    SEP_DATA_RAW_JSON,
+    SEP_DATA_EMBED_RAW_JSON_FILE,
 ])
 
 # Separators allowed in ITEM arguments
-SEP_GROUP_ITEMS = frozenset([
+SEP_GROUP_ALL_ITEMS = frozenset([
     SEP_HEADERS,
     SEP_QUERY,
     SEP_DATA,
     SEP_DATA_RAW_JSON,
-    SEP_FILES
+    SEP_FILES,
+    SEP_DATA_EMBED_FILE,
+    SEP_DATA_EMBED_RAW_JSON_FILE,
 ])
 
 
@@ -95,37 +118,43 @@ class Parser(ArgumentParser):
     def parse_args(self, env, args=None, namespace=None):
 
         self.env = env
+        self.args, no_options = super(Parser, self)\
+            .parse_known_args(args, namespace)
 
-        args = super(Parser, self).parse_args(args, namespace)
+        if self.args.debug:
+            self.args.traceback = True
 
-        if not args.json and env.config.implicit_content_type == 'form':
-            args.form = True
+        # Arguments processing and environment setup.
+        self._apply_no_options(no_options)
+        self._apply_config()
+        self._validate_download_options()
+        self._setup_standard_streams()
+        self._process_output_options()
+        self._process_pretty_options()
+        self._guess_method()
+        self._parse_items()
+        if not self.args.ignore_stdin and not env.stdin_isatty:
+            self._body_from_file(self.env.stdin)
+        if not (self.args.url.startswith((HTTP, HTTPS))):
+            # Default to 'https://' if invoked as `https args`.
+            scheme = HTTPS if self.env.progname == 'https' else HTTP
 
-        if args.debug:
-            args.traceback = True
+            # See if we're using curl style shorthand for localhost (:3000/foo)
+            shorthand = re.match(r'^:(?!:)(\d*)(/?.*)$', self.args.url)
+            if shorthand:
+                port = shorthand.group(1)
+                rest = shorthand.group(2)
+                self.args.url = scheme + 'localhost'
+                if port:
+                    self.args.url += ':' + port
+                self.args.url += rest
+            else:
+                self.args.url = scheme + self.args.url
+        self._process_auth()
 
-        if args.output:
-            env.stdout = args.output
-            env.stdout_isatty = False
-
-        self._process_output_options(args, env)
-        self._process_pretty_options(args, env)
-        self._guess_method(args, env)
-        self._parse_items(args)
-
-        if not env.stdin_isatty:
-            self._body_from_file(args, env.stdin)
-
-        if not (args.url.startswith(HTTP) or args.url.startswith(HTTPS)):
-            scheme = HTTPS if env.progname == 'https' else HTTP
-            args.url = scheme + args.url
-
-        if args.auth and not args.auth.has_password():
-            # Stdin already read (if not a tty) so it's save to prompt.
-            args.auth.prompt_password(urlparse(args.url).netloc)
-
-        return args
+        return self.args
 
+    # noinspection PyShadowingBuiltins
     def _print_message(self, message, file=None):
         # Sneak in our stderr/stdout.
         file = {
@@ -136,113 +165,232 @@ class Parser(ArgumentParser):
 
         super(Parser, self)._print_message(message, file)
 
-    def _body_from_file(self, args, fd):
+    def _setup_standard_streams(self):
+        """
+        Modify `env.stdout` and `env.stdout_isatty` based on args, if needed.
+
+        """
+        if not self.env.stdout_isatty and self.args.output_file:
+            self.error('Cannot use --output, -o with redirected output.')
+
+        # FIXME: Come up with a cleaner solution.
+        if self.args.download:
+
+            if not self.env.stdout_isatty:
+                # Use stdout as tge download output file.
+                self.args.output_file = self.env.stdout
+
+            # With `--download`, we write everything that would normally go to
+            # `stdout` to `stderr` instead. Let's replace the stream so that
+            # we don't have to use many `if`s throughout the codebase.
+            # The response body will be treated separately.
+            self.env.stdout = self.env.stderr
+            self.env.stdout_isatty = self.env.stderr_isatty
+
+        elif self.args.output_file:
+            # When not `--download`ing, then `--output` simply replaces
+            # `stdout`. The file is opened for appending, which isn't what
+            # we want in this case.
+            self.args.output_file.seek(0)
+            self.args.output_file.truncate()
+
+            self.env.stdout = self.args.output_file
+            self.env.stdout_isatty = False
+
+    def _apply_config(self):
+        if (not self.args.json
+                and self.env.config.implicit_content_type == 'form'):
+            self.args.form = True
+
+    def _process_auth(self):
+        """
+        If only a username provided via --auth, then ask for a password.
+        Or, take credentials from the URL, if provided.
+
+        """
+        url = urlsplit(self.args.url)
+
+        if self.args.auth:
+            if not self.args.auth.has_password():
+                # Stdin already read (if not a tty) so it's save to prompt.
+                if self.args.ignore_stdin:
+                    self.error('Unable to prompt for passwords because'
+                               ' --ignore-stdin is set.')
+                self.args.auth.prompt_password(url.netloc)
+
+        elif url.username is not None:
+            # Handle http://username:password@hostname/
+            username, password = url.username, url.password
+            self.args.auth = AuthCredentials(
+                key=username,
+                value=password,
+                sep=SEP_CREDENTIALS,
+                orig=SEP_CREDENTIALS.join([username, password])
+            )
+
+    def _apply_no_options(self, no_options):
+        """For every `--no-OPTION` in `no_options`, set `args.OPTION` to
+        its default value. This allows for un-setting of options, e.g.,
+        specified in config.
+
+        """
+        invalid = []
+
+        for option in no_options:
+            if not option.startswith('--no-'):
+                invalid.append(option)
+                continue
+
+            # --no-option => --option
+            inverted = '--' + option[5:]
+            for action in self._actions:
+                if inverted in action.option_strings:
+                    setattr(self.args, action.dest, action.default)
+                    break
+            else:
+                invalid.append(option)
+
+        if invalid:
+            msg = 'unrecognized arguments: %s'
+            self.error(msg % ' '.join(invalid))
+
+    def _body_from_file(self, fd):
         """There can only be one source of request data.
 
         Bytes are always read.
 
         """
-        if args.data:
+        if self.args.data:
             self.error('Request body (from stdin or a file) and request '
                        'data (key=value) cannot be mixed.')
-        args.data = getattr(fd, 'buffer', fd).read()
+        self.args.data = getattr(fd, 'buffer', fd).read()
 
-    def _guess_method(self, args, env):
+    def _guess_method(self):
         """Set `args.method` if not specified to either POST or GET
         based on whether the request has data or not.
 
         """
-        if args.method is None:
+        if self.args.method is None:
             # Invoked as `http URL'.
-            assert not args.items
-            if not env.stdin_isatty:
-                args.method = HTTP_POST
+            assert not self.args.items
+            if not self.args.ignore_stdin and not self.env.stdin_isatty:
+                self.args.method = HTTP_POST
             else:
-                args.method = HTTP_GET
+                self.args.method = HTTP_GET
 
         # FIXME: False positive, e.g., "localhost" matches but is a valid URL.
-        elif not re.match('^[a-zA-Z]+$', args.method):
+        elif not re.match('^[a-zA-Z]+$', self.args.method):
             # Invoked as `http URL item+'. The URL is now in `args.method`
             # and the first ITEM is now incorrectly in `args.url`.
             try:
                 # Parse the URL as an ITEM and store it as the first ITEM arg.
-                args.items.insert(
-                    0, KeyValueArgType(*SEP_GROUP_ITEMS).__call__(args.url))
+                self.args.items.insert(
+                    0,
+                    KeyValueArgType(*SEP_GROUP_ALL_ITEMS).__call__(self.args.url)
+                )
 
             except ArgumentTypeError as e:
-                if args.traceback:
+                if self.args.traceback:
                     raise
                 self.error(e.message)
 
             else:
                 # Set the URL correctly
-                args.url = args.method
+                self.args.url = self.args.method
                 # Infer the method
-                has_data = not env.stdin_isatty or any(
-                    item.sep in SEP_GROUP_DATA_ITEMS for item in args.items)
-                args.method = HTTP_POST if has_data else HTTP_GET
+                has_data = (
+                    (not self.args.ignore_stdin and
+                     not self.env.stdin_isatty) or any(
+                        item.sep in SEP_GROUP_DATA_ITEMS
+                        for item in self.args.items
+                    )
+                )
+                self.args.method = HTTP_POST if has_data else HTTP_GET
 
-    def _parse_items(self, args):
-        """Parse `args.items` into `args.headers`, `args.data`,
-        `args.`, and `args.files`.
+    def _parse_items(self):
+        """Parse `args.items` into `args.headers`, `args.data`, `args.params`,
+         and `args.files`.
 
         """
-        args.headers = CaseInsensitiveDict()
-        args.data = ParamDict() if args.form else OrderedDict()
-        args.files = OrderedDict()
-        args.params = ParamDict()
+        self.args.headers = CaseInsensitiveDict()
+        self.args.data = ParamDict() if self.args.form else OrderedDict()
+        self.args.files = OrderedDict()
+        self.args.params = ParamDict()
 
         try:
-            parse_items(items=args.items,
-                        headers=args.headers,
-                        data=args.data,
-                        files=args.files,
-                        params=args.params)
+            parse_items(items=self.args.items,
+                        headers=self.args.headers,
+                        data=self.args.data,
+                        files=self.args.files,
+                        params=self.args.params)
         except ParseError as e:
-            if args.traceback:
+            if self.args.traceback:
                 raise
             self.error(e.message)
 
-        if args.files and not args.form:
+        if self.args.files and not self.args.form:
             # `http url @/path/to/file`
-            file_fields = list(args.files.keys())
+            file_fields = list(self.args.files.keys())
             if file_fields != ['']:
                 self.error(
                     'Invalid file fields (perhaps you meant --form?): %s'
                     % ','.join(file_fields))
 
-            fn, fd = args.files['']
-            args.files = {}
-            self._body_from_file(args, fd)
-            if 'Content-Type' not in args.headers:
+            fn, fd = self.args.files['']
+            self.args.files = {}
+
+            self._body_from_file(fd)
+
+            if 'Content-Type' not in self.args.headers:
                 mime, encoding = mimetypes.guess_type(fn, strict=False)
                 if mime:
                     content_type = mime
                     if encoding:
                         content_type = '%s; charset=%s' % (mime, encoding)
-                    args.headers['Content-Type'] = content_type
+                    self.args.headers['Content-Type'] = content_type
 
-    def _process_output_options(self, args, env):
-        """Apply defaults to output options or validate the provided ones.
+    def _process_output_options(self):
+        """Apply defaults to output options, or validate the provided ones.
 
         The default output options are stdout-type-sensitive.
 
         """
-        if not args.output_options:
-            args.output_options = (OUTPUT_OPTIONS_DEFAULT if env.stdout_isatty
-                                else OUTPUT_OPTIONS_DEFAULT_STDOUT_REDIRECTED)
+        if not self.args.output_options:
+            self.args.output_options = (
+                OUTPUT_OPTIONS_DEFAULT
+                if self.env.stdout_isatty
+                else OUTPUT_OPTIONS_DEFAULT_STDOUT_REDIRECTED
+            )
 
-        unknown = set(args.output_options) - OUTPUT_OPTIONS
-        if unknown:
-            self.error('Unknown output options: %s' % ','.join(unknown))
+        unknown_output_options = set(self.args.output_options) - OUTPUT_OPTIONS
+        if unknown_output_options:
+            self.error(
+                'Unknown output options: %s' % ','.join(unknown_output_options)
+            )
 
-    def _process_pretty_options(self, args, env):
-        if args.prettify == PRETTY_STDOUT_TTY_ONLY:
-            args.prettify = PRETTY_MAP['all' if env.stdout_isatty else 'none']
-        elif args.prettify and env.is_windows:
+        if self.args.download and OUT_RESP_BODY in self.args.output_options:
+            # Response body is always downloaded with --download and it goes
+            # through a different routine, so we remove it.
+            self.args.output_options = str(
+                set(self.args.output_options) - set(OUT_RESP_BODY))
+
+    def _process_pretty_options(self):
+        if self.args.prettify == PRETTY_STDOUT_TTY_ONLY:
+            self.args.prettify = PRETTY_MAP[
+                'all' if self.env.stdout_isatty else 'none']
+        elif self.args.prettify and self.env.is_windows:
             self.error('Only terminal output can be colorized on Windows.')
         else:
-            args.prettify = PRETTY_MAP[args.prettify]
+            # noinspection PyTypeChecker
+            self.args.prettify = PRETTY_MAP[self.args.prettify]
+
+    def _validate_download_options(self):
+        if not self.args.download:
+            if self.args.download_resume:
+                self.error('--continue only works with --download')
+        if self.args.download_resume and not (
+                self.args.download and self.args.output_file):
+            self.error('--continue requires --output to be specified')
 
 
 class ParseError(Exception):
@@ -262,6 +410,19 @@ class KeyValue(object):
         return self.__dict__ == other.__dict__
 
 
+class SessionNameValidator(object):
+
+    def __init__(self, error_message):
+        self.error_message = error_message
+
+    def __call__(self, value):
+        # Session name can be a path or just a name.
+        if (os.path.sep not in value
+                and not VALID_SESSION_NAME_PATTERN.search(value)):
+            raise ArgumentError(None, self.error_message)
+        return value
+
+
 class KeyValueArgType(object):
     """A key-value pair argument type used with `argparse`.
 
@@ -292,8 +453,8 @@ class KeyValueArgType(object):
             """Tokenize `s`. There are only two token types - strings
             and escaped characters:
 
-            >>> tokenize(r'foo\=bar\\baz')
-            ['foo', Escaped('='), 'bar', Escaped('\\'), 'baz']
+            tokenize(r'foo\=bar\\baz')
+            => ['foo', Escaped('='), 'bar', Escaped('\\'), 'baz']
 
             """
             tokens = ['']
@@ -403,9 +564,6 @@ class ParamDict(OrderedDict):
         data and URL params.
 
         """
-        # NOTE: Won't work when used for form data with multiple values
-        # for a field and a file field is present:
-        # https://github.com/kennethreitz/requests/issues/737
         if key not in self:
             super(ParamDict, self).__setitem__(key, value)
         else:
@@ -429,9 +587,7 @@ def parse_items(items, data=None, headers=None, files=None, params=None):
         params = ParamDict()
 
     for item in items:
-
         value = item.value
-        key = item.key
 
         if item.sep == SEP_HEADERS:
             target = headers
@@ -443,21 +599,34 @@ def parse_items(items, data=None, headers=None, files=None, params=None):
                     value = (os.path.basename(value),
                              BytesIO(f.read()))
             except IOError as e:
-                raise ParseError(
-                    'Invalid argument "%s": %s' % (item.orig, e))
+                raise ParseError('"%s": %s' % (item.orig, e))
             target = files
 
-        elif item.sep in [SEP_DATA, SEP_DATA_RAW_JSON]:
-            if item.sep == SEP_DATA_RAW_JSON:
+        elif item.sep in SEP_GROUP_DATA_ITEMS:
+
+            if item.sep in SEP_GROUP_DATA_EMBED_ITEMS:
                 try:
-                    value = json.loads(item.value)
-                except ValueError:
-                    raise ParseError('"%s" is not valid JSON' % item.orig)
+                    with open(os.path.expanduser(value), 'rb') as f:
+                        value = f.read().decode('utf8')
+                except IOError as e:
+                    raise ParseError('"%s": %s' % (item.orig, e))
+                except UnicodeDecodeError:
+                    raise ParseError(
+                        '"%s": cannot embed the content of "%s",'
+                        ' not a UTF8 or ASCII-encoded text file'
+                        % (item.orig, item.value)
+                    )
+
+            if item.sep in SEP_GROUP_RAW_JSON_ITEMS:
+                try:
+                    value = json.loads(value)
+                except ValueError as e:
+                    raise ParseError('"%s": %s' % (item.orig, e))
             target = data
 
         else:
             raise TypeError(item)
 
-        target[key] = value
+        target[item.key] = value
 
     return headers, data, files, params

httpie/manage.py

@@ -1,30 +0,0 @@
-"""
-Provides the `httpie' management command.
-
-Note that the main `http' command points to `httpie.__main__.main()`.
-
-"""
-import argparse
-
-from . import sessions
-from . import __version__
-
-
-parser = argparse.ArgumentParser(
-    description='The HTTPie management command.',
-    version=__version__
-)
-subparsers = parser.add_subparsers()
-
-
-# Only sessions as of now.
-sessions.add_commands(subparsers)
-
-
-def main():
-    args = parser.parse_args()
-    args.command(args)
-
-
-if __name__ == '__main__':
-    main()

httpie/models.py

@@ -1,8 +1,8 @@
 import os
 import sys
-from requests.compat import urlparse, is_windows, bytes, str
 
 from .config import DEFAULT_CONFIG_DIR, Config
+from .compat import urlsplit, is_windows, bytes, str
 
 
 class Environment(object):
@@ -13,30 +13,33 @@ class Environment(object):
 
     """
 
-    #noinspection PyUnresolvedReferences
     is_windows = is_windows
 
     progname = os.path.basename(sys.argv[0])
     if progname not in ['http', 'https']:
         progname = 'http'
 
-    stdin_isatty = sys.stdin.isatty()
-    stdin = sys.stdin
-    stdout_isatty = sys.stdout.isatty()
-
     config_dir = DEFAULT_CONFIG_DIR
 
-    if stdout_isatty and is_windows:
-        from colorama.initialise import wrap_stream
-        stdout = wrap_stream(sys.stdout, convert=None,
-                             strip=None, autoreset=True, wrap=True)
-    else:
-        stdout = sys.stdout
-    stderr = sys.stderr
-
     # Can be set to 0 to disable colors completely.
     colors = 256 if '256color' in os.environ.get('TERM', '') else 88
 
+    stdin = sys.stdin
+    stdin_isatty = sys.stdin.isatty()
+
+    stdout_isatty = sys.stdout.isatty()
+    stderr_isatty = sys.stderr.isatty()
+    if is_windows:
+        # noinspection PyUnresolvedReferences
+        from colorama.initialise import wrap_stream
+        stdout = wrap_stream(sys.stdout, convert=None,
+                             strip=None, autoreset=True, wrap=True)
+        stderr = wrap_stream(sys.stderr, convert=None,
+                             strip=None, autoreset=True, wrap=True)
+    else:
+        stdout = sys.stdout
+        stderr = sys.stderr
+
     def __init__(self, **kwargs):
         assert all(hasattr(type(self), attr)
                    for attr in kwargs.keys())
@@ -85,10 +88,7 @@ class HTTPMessage(object):
     @property
     def content_type(self):
         """Return the message content type."""
-        ct = self._orig.headers.get('Content-Type', '')
-        if isinstance(ct, bytes):
-            ct = ct.decode()
-        return ct
+        return self._orig.headers.get('Content-Type', '')
 
 
 class HTTPResponse(HTTPMessage):
@@ -100,6 +100,7 @@ class HTTPResponse(HTTPMessage):
     def iter_lines(self, chunk_size):
         return ((line, b'\n') for line in self._orig.iter_lines(chunk_size))
 
+    #noinspection PyProtectedMember
     @property
     def headers(self):
         original = self._orig.raw._original_response
@@ -143,33 +144,18 @@ class HTTPRequest(HTTPMessage):
 
     @property
     def headers(self):
-        """Return Request-Line"""
-        url = urlparse(self._orig.url)
+        url = urlsplit(self._orig.url)
 
-        # Querystring
-        qs = ''
-        if url.query or self._orig.params:
-            qs = '?'
-            if url.query:
-                qs += url.query
-            # Requests doesn't make params part of ``request.url``.
-            if self._orig.params:
-                if url.query:
-                    qs += '&'
-                #noinspection PyUnresolvedReferences
-                qs += type(self._orig)._encode_params(self._orig.params)
-
-        # Request-Line
         request_line = '{method} {path}{query} HTTP/1.1'.format(
             method=self._orig.method,
             path=url.path or '/',
-            query=qs
+            query='?' + url.query if url.query else ''
         )
 
         headers = dict(self._orig.headers)
 
         if 'Host' not in headers:
-            headers['Host'] = urlparse(self._orig.url).netloc
+            headers['Host'] = url.netloc.split('@')[-1]
 
         headers = ['%s: %s' % (name, value)
                    for name, value in headers.items()]
@@ -184,26 +170,8 @@ class HTTPRequest(HTTPMessage):
 
     @property
     def body(self):
-        """Reconstruct and return the original request body bytes."""
-        if self._orig.files:
-            # TODO: would be nice if we didn't need to encode the files again
-            # FIXME: Also the boundary header doesn't match the one used.
-            for fn, fd in self._orig.files.values():
-                # Rewind the files as they have already been read before.
-                fd.seek(0)
-            body, _ = self._orig._encode_files(self._orig.files)
-        else:
-            try:
-                body = self._orig.data
-            except AttributeError:
-                # requests < 0.12.1
-                body = self._orig._enc_data
-
-            if isinstance(body, dict):
-                #noinspection PyUnresolvedReferences
-                body = type(self._orig)._encode_params(body)
-
-            if isinstance(body, str):
-                body = body.encode('utf8')
-
-        return body
+        body = self._orig.body
+        if isinstance(body, str):
+            # Happens with JSON/form request data parsed from the command line.
+            body = body.encode('utf8')
+        return body or b''

httpie/output.py

@@ -2,6 +2,7 @@
 
 """
 import json
+import xml.dom.minidom
 from functools import partial
 from itertools import chain
 
@@ -12,14 +13,17 @@ from pygments.lexers import get_lexer_for_mimetype, get_lexer_by_name
 from pygments.formatters.terminal import TerminalFormatter
 from pygments.formatters.terminal256 import Terminal256Formatter
 from pygments.util import ClassNotFound
-from requests.compat import is_windows
 
+from .compat import is_windows
 from .solarized import Solarized256Style
 from .models import HTTPRequest, HTTPResponse, Environment
 from .input import (OUT_REQ_BODY, OUT_REQ_HEAD,
                     OUT_RESP_HEAD, OUT_RESP_BODY)
 
 
+# The default number of spaces to indent when pretty printing
+DEFAULT_INDENT = 4
+
 # Colors on Windows via colorama don't look that
 # great and fruity seems to give the best result there.
 AVAILABLE_STYLES = set(STYLE_MAP.keys())
@@ -61,7 +65,7 @@ def write(stream, outfile, flush):
             outfile.flush()
 
 
-def write_with_colors_win_p3k(stream, outfile, flush):
+def write_with_colors_win_py3(stream, outfile, flush):
     """Like `write`, but colorized chunks are written as text
     directly to `outfile` to ensure it gets processed by colorama.
     Applies only to Windows with Python 3 and colorized terminal output.
@@ -78,23 +82,21 @@ def write_with_colors_win_p3k(stream, outfile, flush):
             outfile.flush()
 
 
-def output_stream(args, env, request, response):
+def build_output_stream(args, env, request, response):
     """Build and return a chain of iterators over the `request`-`response`
     exchange each of which yields `bytes` chunks.
 
     """
 
-    Stream = make_stream(env, args)
-
     req_h = OUT_REQ_HEAD in args.output_options
     req_b = OUT_REQ_BODY in args.output_options
     resp_h = OUT_RESP_HEAD in args.output_options
     resp_b = OUT_RESP_BODY in args.output_options
-
     req = req_h or req_b
     resp = resp_h or resp_b
 
     output = []
+    Stream = get_stream_type(env, args)
 
     if req:
         output.append(Stream(
@@ -120,7 +122,7 @@ def output_stream(args, env, request, response):
     return chain(*output)
 
 
-def make_stream(env, args):
+def get_stream_type(env, args):
     """Pick the right stream type based on `env` and `args`.
     Wrap it in a partial with the type-specific args so that
     we don't need to think what stream we are dealing with.
@@ -147,37 +149,42 @@ def make_stream(env, args):
 
 
 class BaseStream(object):
-    """Base HTTP message stream class."""
+    """Base HTTP message output stream class."""
 
-    def __init__(self, msg, with_headers=True, with_body=True):
+    def __init__(self, msg, with_headers=True, with_body=True,
+                 on_body_chunk_downloaded=None):
         """
         :param msg: a :class:`models.HTTPMessage` subclass
         :param with_headers: if `True`, headers will be included
         :param with_body: if `True`, body will be included
 
         """
+        assert with_headers or with_body
         self.msg = msg
         self.with_headers = with_headers
         self.with_body = with_body
+        self.on_body_chunk_downloaded = on_body_chunk_downloaded
 
-    def _headers(self):
+    def _get_headers(self):
         """Return the headers' bytes."""
         return self.msg.headers.encode('ascii')
 
-    def _body(self):
+    def _iter_body(self):
         """Return an iterator over the message body."""
         raise NotImplementedError()
 
     def __iter__(self):
         """Return an iterator over `self.msg`."""
         if self.with_headers:
-            yield self._headers()
+            yield self._get_headers()
             yield b'\r\n\r\n'
 
         if self.with_body:
             try:
-                for chunk in self._body():
+                for chunk in self._iter_body():
                     yield chunk
+                    if self.on_body_chunk_downloaded:
+                        self.on_body_chunk_downloaded(chunk)
             except BinarySuppressedError as e:
                 if self.with_headers:
                     yield b'\n'
@@ -188,13 +195,13 @@ class RawStream(BaseStream):
     """The message is streamed in chunks with no processing."""
 
     CHUNK_SIZE = 1024 * 100
-    CHUNK_SIZE_BY_LINE = 1024 * 5
+    CHUNK_SIZE_BY_LINE = 1
 
     def __init__(self, chunk_size=CHUNK_SIZE, **kwargs):
         super(RawStream, self).__init__(**kwargs)
         self.chunk_size = chunk_size
 
-    def _body(self):
+    def _iter_body(self):
         return self.msg.iter_body(self.chunk_size)
 
 
@@ -206,7 +213,7 @@ class EncodedStream(BaseStream):
     is suppressed. The body is always streamed by line.
 
     """
-    CHUNK_SIZE = 1024 * 5
+    CHUNK_SIZE = 1
 
     def __init__(self, env=Environment(), **kwargs):
 
@@ -222,7 +229,7 @@ class EncodedStream(BaseStream):
         # Default to utf8 when unsure.
         self.output_encoding = output_encoding or 'utf8'
 
-    def _body(self):
+    def _iter_body(self):
 
         for line, lf in self.msg.iter_lines(self.CHUNK_SIZE):
 
@@ -242,17 +249,17 @@ class PrettyStream(EncodedStream):
 
     """
 
-    CHUNK_SIZE = 1024 * 5
+    CHUNK_SIZE = 1
 
     def __init__(self, processor, **kwargs):
         super(PrettyStream, self).__init__(**kwargs)
         self.processor = processor
 
-    def _headers(self):
+    def _get_headers(self):
         return self.processor.process_headers(
             self.msg.headers).encode(self.output_encoding)
 
-    def _body(self):
+    def _iter_body(self):
         for line, lf in self.msg.iter_lines(self.CHUNK_SIZE):
             if b'\0' in line:
                 raise BinarySuppressedError()
@@ -261,8 +268,9 @@ class PrettyStream(EncodedStream):
     def _process_body(self, chunk):
         return (self.processor
                     .process_body(
-                        chunk.decode(self.msg.encoding, 'replace'),
-                        self.msg.content_type)
+                        content=chunk.decode(self.msg.encoding, 'replace'),
+                        content_type=self.msg.content_type,
+                        encoding=self.msg.encoding)
                     .encode(self.output_encoding, 'replace'))
 
 
@@ -276,9 +284,8 @@ class BufferedPrettyStream(PrettyStream):
 
     CHUNK_SIZE = 1024 * 10
 
-    def _body(self):
+    def _iter_body(self):
 
-        #noinspection PyArgumentList
         # Read the whole body before prettifying it,
         # but bail out immediately if the body is binary.
         body = bytearray()
@@ -365,12 +372,13 @@ class BaseProcessor(object):
         """
         return headers
 
-    def process_body(self, content, content_type, subtype):
+    def process_body(self, content, content_type, subtype, encoding):
         """Return processed `content`.
 
         :param content: The body content as text
         :param content_type: Full content type, e.g., 'application/atom+xml'.
         :param subtype: E.g. 'xml'.
+        :param encoding: The original content encoding.
 
         """
         return content
@@ -379,7 +387,7 @@ class BaseProcessor(object):
 class JSONProcessor(BaseProcessor):
     """JSON body processor."""
 
-    def process_body(self, content, content_type, subtype):
+    def process_body(self, content, content_type, subtype, encoding):
         if subtype == 'json':
             try:
                 # Indent the JSON data, sort keys by name, and
@@ -387,13 +395,29 @@ class JSONProcessor(BaseProcessor):
                 content = json.dumps(json.loads(content),
                                      sort_keys=True,
                                      ensure_ascii=False,
-                                     indent=4)
+                                     indent=DEFAULT_INDENT)
             except ValueError:
                 # Invalid JSON but we don't care.
                 pass
         return content
 
 
+class XMLProcessor(BaseProcessor):
+    """XML body processor."""
+    # TODO: tests
+
+    def process_body(self, content, content_type, subtype, encoding):
+        if subtype == 'xml':
+            try:
+                # Pretty print the XML
+                doc = xml.dom.minidom.parseString(content.encode(encoding))
+                content = doc.toprettyxml(indent=' ' * DEFAULT_INDENT)
+            except xml.parsers.expat.ExpatError:
+                # Ignore invalid XML errors (skips attempting to pretty print)
+                pass
+        return content
+
+
 class PygmentsProcessor(BaseProcessor):
     """A processor that applies syntax-highlighting using Pygments
     to the headers, and to the body as well if its content type is recognized.
@@ -425,7 +449,7 @@ class PygmentsProcessor(BaseProcessor):
         return pygments.highlight(
             headers, HTTPLexer(), self.formatter).strip()
 
-    def process_body(self, content, content_type, subtype):
+    def process_body(self, content, content_type, subtype, encoding):
         try:
             lexer = self.lexers_by_type.get(content_type)
             if not lexer:
@@ -458,7 +482,8 @@ class OutputProcessor(object):
     installed_processors = {
         'format': [
             HeadersProcessor,
-            JSONProcessor
+            JSONProcessor,
+            XMLProcessor
         ],
         'colors': [
             PygmentsProcessor
@@ -484,13 +509,18 @@ class OutputProcessor(object):
             headers = processor.process_headers(headers)
         return headers
 
-    def process_body(self, content, content_type):
+    def process_body(self, content, content_type, encoding):
         # e.g., 'application/atom+xml'
         content_type = content_type.split(';')[0]
         # e.g., 'xml'
         subtype = content_type.split('/')[-1].split('+')[-1]
 
         for processor in self.processors:
-            content = processor.process_body(content, content_type, subtype)
+            content = processor.process_body(
+                content,
+                content_type,
+                subtype,
+                encoding
+            )
 
         return content

httpie/plugins/__init__.py

@@ -0,0 +1,9 @@
+from .base import AuthPlugin
+from .manager import PluginManager
+from .builtin import BasicAuthPlugin, DigestAuthPlugin
+
+
+plugin_manager = PluginManager()
+plugin_manager.register(BasicAuthPlugin)
+plugin_manager.register(DigestAuthPlugin)
+

httpie/plugins/base.py

@@ -0,0 +1,28 @@
+class AuthPlugin(object):
+    """
+    Base auth plugin class.
+
+    See <https://github.com/jkbr/httpie-ntlm> for an example auth plugin.
+
+    """
+
+    # The value that should be passed to --auth-type
+    # to use this auth plugin. Eg. "my-auth"
+    auth_type = None
+
+    # The name of the plugin, eg. "My auth".
+    name = None
+
+    # Optional short description. Will be be shown in the help
+    # under --auth-type.
+    description = None
+
+    # This be set automatically once the plugin has been loaded.
+    package_name = None
+
+    def get_auth(self, username, password):
+        """
+        Return a ``requests.auth.AuthBase`` subclass instance.
+
+        """
+        raise NotImplementedError()

httpie/plugins/builtin.py

@@ -0,0 +1,26 @@
+import requests.auth
+
+from .base import AuthPlugin
+
+
+class BuiltinAuthPlugin(AuthPlugin):
+
+    package_name = '(builtin)'
+
+
+class BasicAuthPlugin(BuiltinAuthPlugin):
+
+    name = 'Basic HTTP auth'
+    auth_type = 'basic'
+
+    def get_auth(self, username, password):
+        return requests.auth.HTTPBasicAuth(username, password)
+
+
+class DigestAuthPlugin(BuiltinAuthPlugin):
+
+    name = 'Digest HTTP auth'
+    auth_type = 'digest'
+
+    def get_auth(self, username, password):
+        return requests.auth.HTTPDigestAuth(username, password)

httpie/plugins/manager.py

@@ -0,0 +1,35 @@
+from pkg_resources import iter_entry_points
+
+
+ENTRY_POINT_NAMES = [
+    'httpie.plugins.auth.v1'
+]
+
+
+class PluginManager(object):
+
+    def __init__(self):
+        self._plugins = []
+
+    def __iter__(self):
+        return iter(self._plugins)
+
+    def register(self, plugin):
+        self._plugins.append(plugin)
+
+    def get_auth_plugins(self):
+        return list(self._plugins)
+
+    def get_auth_plugin_mapping(self):
+        return dict((plugin.auth_type, plugin) for plugin in self)
+
+    def get_auth_plugin(self, auth_type):
+        return self.get_auth_plugin_mapping()[auth_type]
+
+    def load_installed_plugins(self):
+
+        for entry_point_name in ENTRY_POINT_NAMES:
+            for entry_point in iter_entry_points(entry_point_name):
+                plugin = entry_point.load()
+                plugin.package_name = entry_point.dist.key
+                self.register(entry_point.load())

httpie/sessions.py

@@ -1,115 +1,120 @@
 """Persistent, JSON-serialized sessions.
 
 """
+import re
 import os
-import sys
-import glob
-import errno
-import codecs
-import shutil
-import subprocess
 
 import requests
-from requests.compat import urlparse
 from requests.cookies import RequestsCookieJar, create_cookie
-from requests.auth import HTTPBasicAuth, HTTPDigestAuth
-from argparse import OPTIONAL
 
+from .compat import urlsplit
 from .config import BaseConfigDict, DEFAULT_CONFIG_DIR
-from .output import PygmentsProcessor
+from httpie.plugins import plugin_manager
 
 
 SESSIONS_DIR_NAME = 'sessions'
+DEFAULT_SESSIONS_DIR = os.path.join(DEFAULT_CONFIG_DIR, SESSIONS_DIR_NAME)
+VALID_SESSION_NAME_PATTERN = re.compile('^[a-zA-Z0-9_.-]+$')
+# Request headers starting with these prefixes won't be stored in sessions.
+# They are specific to each request.
+# http://en.wikipedia.org/wiki/List_of_HTTP_header_fields#Requests
+SESSION_IGNORED_HEADER_PREFIXES = ['Content-', 'If-']
 
 
-def get_response(name, request_kwargs, config_dir, read_only=False):
+def get_response(session_name, requests_kwargs, config_dir, args,
+                 read_only=False):
     """Like `client.get_response`, but applies permanent
     aspects of the session to the request.
 
     """
-    sessions_dir = os.path.join(config_dir, SESSIONS_DIR_NAME)
-    host = Host(
-        root_dir=sessions_dir,
-        name=request_kwargs['headers'].get('Host', None)
-             or urlparse(request_kwargs['url']).netloc.split('@')[-1]
-    )
+    if os.path.sep in session_name:
+        path = os.path.expanduser(session_name)
+    else:
+        hostname = (
+            requests_kwargs['headers'].get('Host', None)
+            or urlsplit(requests_kwargs['url']).netloc.split('@')[-1]
+        )
 
-    session = Session(host, name)
+        assert re.match('^[a-zA-Z0-9_.:-]+$', hostname)
+
+        # host:port => host_port
+        hostname = hostname.replace(':', '_')
+        path = os.path.join(config_dir,
+                            SESSIONS_DIR_NAME,
+                            hostname,
+                            session_name + '.json')
+
+    session = Session(path)
     session.load()
 
-    # Update session headers with the request headers.
-    session['headers'].update(request_kwargs.get('headers', {}))
-    # Use the merged headers for the request
-    request_kwargs['headers'] = session['headers']
+    request_headers = requests_kwargs.get('headers', {})
+    requests_kwargs['headers'] = dict(session.headers, **request_headers)
+    session.update_headers(request_headers)
 
-    auth = request_kwargs.get('auth', None)
-    if auth:
-        session.auth = auth
+    if args.auth:
+        session.auth = {
+            'type': args.auth_type,
+            'username': args.auth.key,
+            'password': args.auth.value,
+        }
     elif session.auth:
-        request_kwargs['auth'] = session.auth
+        requests_kwargs['auth'] = session.auth
+
+    requests_session = requests.Session()
+    requests_session.cookies = session.cookies
 
-    rsession = requests.Session(cookies=session.cookies)
     try:
-        response = rsession.request(**request_kwargs)
+        response = requests_session.request(**requests_kwargs)
     except Exception:
         raise
     else:
         # Existing sessions with `read_only=True` don't get updated.
         if session.is_new or not read_only:
-            session.cookies = rsession.cookies
+            session.cookies = requests_session.cookies
             session.save()
         return response
 
 
-class Host(object):
-    """A host is a per-host directory on the disk containing sessions files."""
-
-    def __init__(self, name, root_dir=DEFAULT_CONFIG_DIR):
-        self.name = name
-        self.root_dir = root_dir
-
-    def __iter__(self):
-        """Return a iterator yielding `(session_name, session_path)`."""
-        for fn in sorted(glob.glob1(self.path, '*.json')):
-            yield os.path.splitext(fn)[0], os.path.join(self.path, fn)
-
-    def delete(self):
-        shutil.rmtree(self.path)
-
-    @property
-    def path(self):
-        # Name will include ':' if a port is specified, which is invalid
-        # on windows. DNS does not allow '_' in a domain, or for it to end
-        # in a number (I think?)
-        path = os.path.join(self.root_dir, self.name.replace(':', '_'))
-        try:
-            os.makedirs(path, mode=0o700)
-        except OSError as e:
-            if e.errno != errno.EEXIST:
-                raise
-        return path
-
-    @classmethod
-    def all(cls):
-        """Return a generator yielding a host at a time."""
-        for name in sorted(glob.glob1(SESSIONS_DIR, '*')):
-            if os.path.isdir(os.path.join(SESSIONS_DIR, name)):
-                yield Host(name)
-
-
 class Session(BaseConfigDict):
-    """"""
+    helpurl = 'https://github.com/jkbr/httpie#sessions'
+    about = 'HTTPie session file'
 
-    def __init__(self, host, name, *args, **kwargs):
+    def __init__(self, path, *args, **kwargs):
         super(Session, self).__init__(*args, **kwargs)
-        self.host = host
-        self.name = name
+        self._path = path
         self['headers'] = {}
         self['cookies'] = {}
+        self['auth'] = {
+            'type': None,
+            'username': None,
+            'password': None
+        }
+
+    def _get_path(self):
+        return self._path
+
+    def update_headers(self, request_headers):
+        """
+        Update the session headers with the request ones while ignoring
+        certain name prefixes.
+
+        :type request_headers: dict
+
+        """
+        for name, value in request_headers.items():
+
+            if name == 'User-Agent' and value.startswith('HTTPie/'):
+                continue
+
+            for prefix in SESSION_IGNORED_HEADER_PREFIXES:
+                if name.lower().startswith(prefix.lower()):
+                    break
+            else:
+                self['headers'][name] = value
 
     @property
-    def directory(self):
-        return self.host.path
+    def headers(self):
+        return self['headers']
 
     @property
     def cookies(self):
@@ -122,108 +127,27 @@ class Session(BaseConfigDict):
 
     @cookies.setter
     def cookies(self, jar):
-        excluded = [
-            '_rest', 'name', 'port_specified',
-            'domain_specified', 'domain_initial_dot',
-            'path_specified', 'comment', 'comment_url'
-        ]
+        """
+        :type jar: CookieJar
+        """
+        # http://docs.python.org/2/library/cookielib.html#cookie-objects
+        stored_attrs = ['value', 'path', 'secure', 'expires']
         self['cookies'] = {}
-        for host in jar._cookies.values():
-            for path in host.values():
-                for name, cookie in path.items():
-                    cookie_dict = {}
-                    for k, v in cookie.__dict__.items():
-                        if k not in excluded:
-                            cookie_dict[k] = v
-                    self['cookies'][name] = cookie_dict
+        for cookie in jar:
+            self['cookies'][cookie.name] = dict(
+                (attname, getattr(cookie, attname))
+                for attname in stored_attrs
+            )
 
     @property
     def auth(self):
         auth = self.get('auth', None)
-        if not auth:
-            return None
-        Auth = {'basic': HTTPBasicAuth,
-                'digest': HTTPDigestAuth}[auth['type']]
-        return Auth(auth['username'], auth['password'])
+        if not auth or not auth['type']:
+            return
+        auth_plugin = plugin_manager.get_auth_plugin(auth['type'])()
+        return auth_plugin.get_auth(auth['username'], auth['password'])
 
     @auth.setter
-    def auth(self, cred):
-        self['auth'] = {
-            'type': {HTTPBasicAuth: 'basic',
-                     HTTPDigestAuth: 'digest'}[type(cred)],
-            'username': cred.username,
-            'password': cred.password,
-        }
-
-
-def list_command(args):
-    if args.host:
-        for name, path in Host(args.host):
-            print(name + ' [' + path + ']')
-    else:
-        for host in Host.all():
-            print(host.name)
-            for name, path in host:
-                print(' ' + name + ' [' + path + ']')
-
-
-def show_command(args):
-    path = Session(Host(args.host), args.name).path
-    if not os.path.exists(path):
-        sys.stderr.write('Session "%s" does not exist [%s].\n'
-        % (args.name, path))
-        sys.exit(1)
-
-    with codecs.open(path, encoding='utf8') as f:
-        print(path + ':\n')
-        proc = PygmentsProcessor()
-        print(proc.process_body(f.read(), 'application/json', 'json'))
-        print('')
-
-
-def delete_command(args):
-    host = Host(args.host)
-    if not args.name:
-        host.delete()
-    else:
-        Session(host, args.name).delete()
-
-
-def edit_command(args):
-    editor = os.environ.get('EDITOR', None)
-    if not editor:
-        sys.stderr.write(
-            'You need to configure the environment variable EDITOR.\n')
-        sys.exit(1)
-    command = editor.split()
-    command.append(Session(Host(args.host), args.name).path)
-    subprocess.call(command)
-
-
-def add_commands(subparsers):
-
-    # List
-    list_ = subparsers.add_parser('session-list', help='list sessions')
-    list_.set_defaults(command=list_command)
-    list_.add_argument('host', nargs=OPTIONAL)
-
-    # Show
-    show = subparsers.add_parser('session-show', help='show a session')
-    show.set_defaults(command=show_command)
-    show.add_argument('host')
-    show.add_argument('name')
-
-    # Edit
-    edit = subparsers.add_parser(
-        'session-edit', help='edit a session in $EDITOR')
-    edit.set_defaults(command=edit_command)
-    edit.add_argument('host')
-    edit.add_argument('name')
-
-    # Delete
-    delete = subparsers.add_parser('session-delete', help='delete a session')
-    delete.set_defaults(command=delete_command)
-    delete.add_argument('host')
-    delete.add_argument('name', nargs=OPTIONAL,
-        help='The name of the session to be deleted.'
-             ' If not specified, all host sessions are deleted.')
+    def auth(self, auth):
+        assert set(['type', 'username', 'password']) == set(auth.keys())
+        self['auth'] = auth

httpie/utils.py

@@ -0,0 +1,46 @@
+from __future__ import division
+
+
+def humanize_bytes(n, precision=2):
+    # Author: Doug Latornell
+    # Licence: MIT
+    # URL: http://code.activestate.com/recipes/577081/
+    """Return a humanized string representation of a number of bytes.
+
+    Assumes `from __future__ import division`.
+
+    >>> humanize_bytes(1)
+    '1 byte'
+    >>> humanize_bytes(1024)
+    '1.0 kB'
+    >>> humanize_bytes(1024 * 123)
+    '123.0 kB'
+    >>> humanize_bytes(1024 * 12342)
+    '12.1 MB'
+    >>> humanize_bytes(1024 * 12342, 2)
+    '12.05 MB'
+    >>> humanize_bytes(1024 * 1234, 2)
+    '1.21 MB'
+    >>> humanize_bytes(1024 * 1234 * 1111, 2)
+    '1.31 GB'
+    >>> humanize_bytes(1024 * 1234 * 1111, 1)
+    '1.3 GB'
+
+    """
+    abbrevs = [
+        (1 << 50, 'PB'),
+        (1 << 40, 'TB'),
+        (1 << 30, 'GB'),
+        (1 << 20, 'MB'),
+        (1 << 10, 'kB'),
+        (1, 'B')
+    ]
+
+    if n == 1:
+        return '1 B'
+
+    for factor, suffix in abbrevs:
+        if n >= factor:
+            break
+
+    return '%.*f %s' % (precision, n / factor, suffix)

requirements-dev.txt

@@ -0,0 +1,3 @@
+tox
+git+git://github.com/kennethreitz/httpbin.git@7c96875e87a448f08fb1981e85eb79e77d592d98
+docutils

setup.py

@@ -1,6 +1,5 @@
 import os
 import sys
-import re
 import codecs
 from setuptools import setup
 import httpie
@@ -12,25 +11,23 @@ if sys.argv[-1] == 'test':
 
 
 requirements = [
-    # Debian has only requests==0.10.1 and httpie.deb depends on that.
-    'requests>=0.10.1',
+    'requests>=2.0.0',
     'Pygments>=1.5'
 ]
-if sys.version_info[:2] in ((2, 6), (3, 1)):
-    # argparse has been added in Python 3.2 / 2.7
+try:
+    #noinspection PyUnresolvedReferences
+    import argparse
+except ImportError:
     requirements.append('argparse>=1.2.1')
+
 if 'win32' in str(sys.platform).lower():
     # Terminal colors for Windows
     requirements.append('colorama>=0.2.4')
 
 
 def long_description():
-    """Pre-process the README so that PyPi can render it properly."""
     with codecs.open('README.rst', encoding='utf8') as f:
-        rst = f.read()
-    code_block = '(:\n\n)?\.\. code-block::.*'
-    rst = re.sub(code_block, '::', rst)
-    return rst
+        return f.read()
 
 
 setup(
@@ -43,11 +40,10 @@ setup(
     author=httpie.__author__,
     author_email='jakub@roztocil.name',
     license=httpie.__licence__,
-    packages=['httpie'],
+    packages=['httpie', 'httpie.plugins'],
     entry_points={
         'console_scripts': [
             'http = httpie.__main__:main',
-            'httpie = httpie.manage:main',
         ],
     },
     install_requires=requirements,

tests/fixtures/test.json

@@ -0,0 +1,3 @@
+{
+    "hello": "world"
+}

tests/tests.py

@@ -23,19 +23,27 @@ import subprocess
 import os
 import sys
 import json
+#noinspection PyCompatibility
 import argparse
 import tempfile
 import unittest
 import shutil
+import time
+from requests.structures import CaseInsensitiveDict
 
 try:
+    #noinspection PyCompatibility
     from urllib.request import urlopen
 except ImportError:
+    # noinspection PyUnresolvedReferences
+    #noinspection PyCompatibility
     from urllib2 import urlopen
 try:
     from unittest import skipIf, skip
 except ImportError:
     skip = lambda msg: lambda self: None
+
+    # noinspection PyUnusedLocal
     def skipIf(cond, reason):
         def decorator(test_method):
             if cond:
@@ -44,7 +52,6 @@ except ImportError:
         return decorator
 
 from requests import __version__ as requests_version
-from requests.compat import is_windows, is_py26, bytes, str
 
 
 #################################################################
@@ -55,12 +62,22 @@ from requests.compat import is_windows, is_py26, bytes, str
 TESTS_ROOT = os.path.abspath(os.path.dirname(__file__))
 sys.path.insert(0, os.path.realpath(os.path.join(TESTS_ROOT, '..')))
 
-from httpie import EXIT
+from httpie import ExitStatus
 from httpie import input
+from httpie.cli import parser
 from httpie.models import Environment
 from httpie.core import main
 from httpie.output import BINARY_SUPPRESSED_NOTICE
 from httpie.input import ParseError
+from httpie.compat import is_windows, is_py26, bytes, str
+from httpie.downloads import (
+    parse_content_range,
+    filename_from_content_disposition,
+    filename_from_url,
+    get_unique_filename,
+    ContentRangeError,
+    Download,
+)
 
 
 CRLF = '\r\n'
@@ -85,19 +102,27 @@ def patharg(path):
 FILE_PATH = os.path.join(TESTS_ROOT, 'fixtures', 'file.txt')
 FILE2_PATH = os.path.join(TESTS_ROOT, 'fixtures', 'file2.txt')
 BIN_FILE_PATH = os.path.join(TESTS_ROOT, 'fixtures', 'file.bin')
+JSON_FILE_PATH = os.path.join(TESTS_ROOT, 'fixtures', 'test.json')
 
 FILE_PATH_ARG = patharg(FILE_PATH)
 FILE2_PATH_ARG = patharg(FILE2_PATH)
 BIN_FILE_PATH_ARG = patharg(BIN_FILE_PATH)
+JSON_FILE_PATH_ARG = patharg(JSON_FILE_PATH)
 
 with open(FILE_PATH) as f:
+    # Strip because we don't want new lines in the data so that we can
+    # easily count occurrences also when embedded in JSON (where the new
+    # line would be escaped).
     FILE_CONTENT = f.read().strip()
 with open(BIN_FILE_PATH, 'rb') as f:
     BIN_FILE_CONTENT = f.read()
+with open(JSON_FILE_PATH, 'rb') as f:
+    JSON_FILE_CONTENT = f.read()
 
 
 def httpbin(path):
-    return HTTPBIN_URL + path
+    url = HTTPBIN_URL + path
+    return url
 
 
 def mk_config_dir():
@@ -130,6 +155,7 @@ class TestEnvironment(Environment):
         if self.delete_config_dir:
             self._shutil.rmtree(self.config_dir)
 
+
 def has_docutils():
     try:
         #noinspection PyUnresolvedReferences
@@ -138,6 +164,7 @@ def has_docutils():
     except ImportError:
         return False
 
+
 def get_readme_errors():
     p = subprocess.Popen([
         'rst2pseudoxml.py',
@@ -152,6 +179,8 @@ def get_readme_errors():
 
 class BytesResponse(bytes):
     stderr = json = exit_status = None
+
+
 class StrResponse(str):
     stderr = json = exit_status = None
 
@@ -175,20 +204,25 @@ def http(*args, **kwargs):
     if not env:
         env = kwargs['env'] = TestEnvironment()
 
+    stdout = env.stdout
+    stderr = env.stderr
     try:
 
         try:
             exit_status = main(args=['--traceback'] + list(args), **kwargs)
+            if '--download' in args:
+                # Let the progress reporter thread finish.
+                time.sleep(.5)
         except Exception:
-            sys.stderr.write(env.stderr.read())
+            sys.stderr.write(stderr.read())
             raise
         except SystemExit:
-            exit_status = EXIT.ERROR
+            exit_status = ExitStatus.ERROR
 
-        env.stdout.seek(0)
-        env.stderr.seek(0)
+        stdout.seek(0)
+        stderr.seek(0)
 
-        output = env.stdout.read()
+        output = stdout.read()
         try:
             r = StrResponse(output.decode('utf8'))
         except UnicodeDecodeError:
@@ -210,18 +244,20 @@ def http(*args, **kwargs):
                         except ValueError:
                             pass
 
-        r.stderr = env.stderr.read()
+        r.stderr = stderr.read()
         r.exit_status = exit_status
 
         return r
 
     finally:
-        env.stdout.close()
-        env.stderr.close()
+        stdout.close()
+        stderr.close()
 
 
 class BaseTestCase(unittest.TestCase):
 
+    maxDiff = 100000
+
     if is_py26:
         def assertIn(self, member, container, msg=None):
             self.assertTrue(member in container, msg)
@@ -233,11 +269,15 @@ class BaseTestCase(unittest.TestCase):
             self.assertEqual(set(d1.keys()), set(d2.keys()), msg)
             self.assertEqual(sorted(d1.values()), sorted(d2.values()), msg)
 
+    def assertIsNone(self, obj, msg=None):
+        self.assertEqual(obj, None, msg=msg)
+
 
 #################################################################
 # High-level tests using httpbin.
 #################################################################
 
+
 class HTTPieTest(BaseTestCase):
 
     def test_GET(self):
@@ -261,7 +301,7 @@ class HTTPieTest(BaseTestCase):
             'foo=bar'
         )
         self.assertIn(OK, r)
-        self.assertIn('"foo": "bar"', r)
+        self.assertIn(r'\"foo\": \"bar\"', r)
 
     def test_POST_JSON_data(self):
         r = http(
@@ -270,7 +310,7 @@ class HTTPieTest(BaseTestCase):
             'foo=bar'
         )
         self.assertIn(OK, r)
-        self.assertIn('"foo": "bar"', r)
+        self.assertIn(r'\"foo\": \"bar\"', r)
 
     def test_POST_form(self):
         r = http(
@@ -429,7 +469,9 @@ class AutoContentTypeAndAcceptHeadersTest(BaseTestCase):
         )
         self.assertIn(OK, r)
         self.assertEqual(r.json['headers']['Accept'], 'application/json')
-        self.assertFalse(r.json['headers'].get('Content-Type'))
+        # Make sure Content-Type gets set even with no data.
+        # https://github.com/jkbr/httpie/issues/137
+        self.assertIn('application/json', r.json['headers']['Content-Type'])
 
     def test_GET_explicit_JSON_explicit_headers(self):
         r = http(
@@ -512,7 +554,7 @@ class ImplicitHTTPMethodTest(BaseTestCase):
             'hello=world'
         )
         self.assertIn(OK, r)
-        self.assertIn('"hello": "world"', r)
+        self.assertIn(r'\"hello\": \"world\"', r)
 
     def test_implicit_POST_form(self):
         r = http(
@@ -651,8 +693,8 @@ class VerboseFlagTest(BaseTestCase):
             'baz=bar'
         )
         self.assertIn(OK, r)
-        #noinspection PyUnresolvedReferences
-        self.assertEqual(r.count('"baz": "bar"'), 2)
+        self.assertIn('"baz": "bar"', r)  # request
+        self.assertIn(r'\"baz\": \"bar\"', r)  # response
 
 
 class MultipartFormDataFileUploadTest(BaseTestCase):
@@ -776,6 +818,7 @@ class RequestBodyFromFilePathTest(BaseTestCase):
     """
     def test_request_body_from_file_by_path(self):
         r = http(
+            '--verbose',
             'POST',
             httpbin('/post'),
             '@' + FILE_PATH_ARG
@@ -838,8 +881,8 @@ class AuthTest(BaseTestCase):
             httpbin('/digest-auth/auth/user/password')
         )
         self.assertIn(OK, r)
-        self.assertIn('"authenticated": true', r)
-        self.assertIn('"user": "user"', r)
+        self.assertIn(r'"authenticated": true', r)
+        self.assertIn(r'"user": "user"', r)
 
     def test_password_prompt(self):
 
@@ -856,6 +899,31 @@ class AuthTest(BaseTestCase):
         self.assertIn('"authenticated": true', r)
         self.assertIn('"user": "user"', r)
 
+    def test_credentials_in_url(self):
+        url = httpbin('/basic-auth/user/password')
+        url = 'http://user:password@' + url.split('http://', 1)[1]
+        r = http(
+            'GET',
+            url
+        )
+        self.assertIn(OK, r)
+        self.assertIn('"authenticated": true', r)
+        self.assertIn('"user": "user"', r)
+
+    def test_credentials_in_url_auth_flag_has_priority(self):
+        """When credentials are passed in URL and via -a at the same time,
+         then the ones from -a are used."""
+        url = httpbin('/basic-auth/user/password')
+        url = 'http://user:wrong_password@' + url.split('http://', 1)[1]
+        r = http(
+            '--auth=user:password',
+            'GET',
+            url
+        )
+        self.assertIn(OK, r)
+        self.assertIn('"authenticated": true', r)
+        self.assertIn('"user": "user"', r)
+
 
 class ExitStatusTest(BaseTestCase):
 
@@ -865,7 +933,7 @@ class ExitStatusTest(BaseTestCase):
             httpbin('/status/200')
         )
         self.assertIn(OK, r)
-        self.assertEqual(r.exit_status, EXIT.OK)
+        self.assertEqual(r.exit_status, ExitStatus.OK)
 
     def test_error_response_exits_0_without_check_status(self):
         r = http(
@@ -873,7 +941,7 @@ class ExitStatusTest(BaseTestCase):
             httpbin('/status/500')
         )
         self.assertIn('HTTP/1.1 500', r)
-        self.assertEqual(r.exit_status, EXIT.OK)
+        self.assertEqual(r.exit_status, ExitStatus.OK)
         self.assertTrue(not r.stderr)
 
     def test_timeout_exit_status(self):
@@ -882,7 +950,7 @@ class ExitStatusTest(BaseTestCase):
             'GET',
             httpbin('/delay/1')
         )
-        self.assertEqual(r.exit_status, EXIT.ERROR_TIMEOUT)
+        self.assertEqual(r.exit_status, ExitStatus.ERROR_TIMEOUT)
 
     def test_3xx_check_status_exits_3_and_stderr_when_stdout_redirected(self):
         r = http(
@@ -893,7 +961,7 @@ class ExitStatusTest(BaseTestCase):
             env=TestEnvironment(stdout_isatty=False,)
         )
         self.assertIn('HTTP/1.1 301', r)
-        self.assertEqual(r.exit_status, EXIT.ERROR_HTTP_3XX)
+        self.assertEqual(r.exit_status, ExitStatus.ERROR_HTTP_3XX)
         self.assertIn('301 moved permanently', r.stderr.lower())
 
     @skipIf(requests_version == '0.13.6',
@@ -907,7 +975,7 @@ class ExitStatusTest(BaseTestCase):
         )
         # The redirect will be followed so 200 is expected.
         self.assertIn('HTTP/1.1 200 OK', r)
-        self.assertEqual(r.exit_status, EXIT.OK)
+        self.assertEqual(r.exit_status, ExitStatus.OK)
 
     def test_4xx_check_status_exits_4(self):
         r = http(
@@ -916,7 +984,7 @@ class ExitStatusTest(BaseTestCase):
             httpbin('/status/401')
         )
         self.assertIn('HTTP/1.1 401', r)
-        self.assertEqual(r.exit_status, EXIT.ERROR_HTTP_4XX)
+        self.assertEqual(r.exit_status, ExitStatus.ERROR_HTTP_4XX)
         # Also stderr should be empty since stdout isn't redirected.
         self.assertTrue(not r.stderr)
 
@@ -927,7 +995,7 @@ class ExitStatusTest(BaseTestCase):
             httpbin('/status/500')
         )
         self.assertIn('HTTP/1.1 500', r)
-        self.assertEqual(r.exit_status, EXIT.ERROR_HTTP_5XX)
+        self.assertEqual(r.exit_status, ExitStatus.ERROR_HTTP_5XX)
 
 
 class WindowsOnlyTests(BaseTestCase):
@@ -980,7 +1048,8 @@ class StreamTest(BaseTestCase):
         #self.assertIn(OK_COLOR, r)
 
     def test_encoded_stream(self):
-        """Test that --stream works with non-prettified redirected terminal output."""
+        """Test that --stream works with non-prettified
+        redirected terminal output."""
         with open(BIN_FILE_PATH, 'rb') as f:
             r = http(
                 '--pretty=none',
@@ -998,7 +1067,8 @@ class StreamTest(BaseTestCase):
         #self.assertIn(OK, r)
 
     def test_redirected_stream(self):
-        """Test that --stream works with non-prettified redirected terminal output."""
+        """Test that --stream works with non-prettified
+        redirected terminal output."""
         with open(BIN_FILE_PATH, 'rb') as f:
             r = http(
                 '--pretty=none',
@@ -1017,12 +1087,35 @@ class StreamTest(BaseTestCase):
         self.assertIn(BIN_FILE_CONTENT, r)
 
 
+class IgnoreStdinTest(BaseTestCase):
+
+    def test_ignore_stdin(self):
+        with open(FILE_PATH) as f:
+            r = http(
+                '--ignore-stdin',
+                '--verbose',
+                httpbin('/get'),
+                env=TestEnvironment(stdin=f, stdin_isatty=False)
+            )
+        self.assertIn(OK, r)
+        self.assertIn('GET /get HTTP', r)  # Don't default to POST.
+        self.assertNotIn(FILE_CONTENT, r)  # Don't send stdin data.
+
+    def test_ignore_stdin_cannot_prompt_password(self):
+        r = http(
+            '--ignore-stdin',
+            '--auth=username-without-password',
+            httpbin('/get'),
+        )
+        self.assertEqual(r.exit_status, ExitStatus.ERROR)
+        self.assertIn('because --ignore-stdin', r.stderr)
+
+
 class LineEndingsTest(BaseTestCase):
     """Test that CRLF is properly used in headers and
     as the headers/body separator."""
 
     def _validate_crlf(self, msg):
-        #noinspection PyUnresolvedReferences
         lines = iter(msg.splitlines(True))
         for header in lines:
             if header == CRLF:
@@ -1057,7 +1150,7 @@ class LineEndingsTest(BaseTestCase):
             'GET',
             httpbin('/get')
         )
-        self.assertEqual(r.exit_status,0)
+        self.assertEqual(r.exit_status, 0)
         self._validate_crlf(r)
 
     def test_CRLF_ugly_request(self):
@@ -1087,11 +1180,7 @@ class ItemParsingTest(BaseTestCase):
 
     def setUp(self):
         self.key_value_type = input.KeyValueArgType(
-            input.SEP_HEADERS,
-            input.SEP_QUERY,
-            input.SEP_DATA,
-            input.SEP_DATA_RAW_JSON,
-            input.SEP_FILES,
+            *input.SEP_GROUP_ALL_ITEMS
         )
 
     def test_invalid_items(self):
@@ -1110,6 +1199,8 @@ class ItemParsingTest(BaseTestCase):
             # files
             self.key_value_type('bar\\@baz@%s' % FILE_PATH_ARG)
         ])
+        # `requests.structures.CaseInsensitiveDict` => `dict`
+        headers = dict(headers._store.values())
         self.assertDictEqual(headers, {
             'foo:bar': 'baz',
             'jack@jill': 'hill',
@@ -1136,24 +1227,99 @@ class ItemParsingTest(BaseTestCase):
             self.key_value_type('eh:'),
             self.key_value_type('ed='),
             self.key_value_type('bool:=true'),
-            self.key_value_type('test-file@%s' % FILE_PATH_ARG),
+            self.key_value_type('file@' + FILE_PATH_ARG),
             self.key_value_type('query==value'),
+            self.key_value_type('string-embed=@' + FILE_PATH_ARG),
+            self.key_value_type('raw-json-embed:=@' + JSON_FILE_PATH_ARG),
         ])
+
+        # Parsed headers
+        # `requests.structures.CaseInsensitiveDict` => `dict`
+        headers = dict(headers._store.values())
         self.assertDictEqual(headers, {
             'header': 'value',
             'eh': ''
         })
-        self.assertDictEqual(data, {
+
+        # Parsed data
+        raw_json_embed = data.pop('raw-json-embed')
+        self.assertDictEqual(raw_json_embed, json.loads(
+            JSON_FILE_CONTENT.decode('utf8')))
+        data['string-embed'] = data['string-embed'].strip()
+        self.assertDictEqual(dict(data), {
             "ed": "",
             "string": "value",
             "bool": True,
             "list": ["a", 1, {}, False],
             "obj": {"a": "b"},
+            "string-embed": FILE_CONTENT,
         })
+
+        # Parsed query string parameters
         self.assertDictEqual(params, {
             'query': 'value',
         })
-        self.assertIn('test-file', files)
+
+        # Parsed file fields
+        self.assertIn('file', files)
+        self.assertEqual(files['file'][1].read().strip().decode('utf8'),
+                         FILE_CONTENT)
+
+
+class CLIParserTestCase(unittest.TestCase):
+
+    def test_expand_localhost_shorthand(self):
+        args = parser.parse_args(args=[':'], env=TestEnvironment())
+
+        self.assertEqual(args.url, 'http://localhost')
+
+    def test_expand_localhost_shorthand_with_slash(self):
+        args = parser.parse_args(args=[':/'], env=TestEnvironment())
+
+        self.assertEqual(args.url, 'http://localhost/')
+
+    def test_expand_localhost_shorthand_with_port(self):
+        args = parser.parse_args(args=[':3000'], env=TestEnvironment())
+
+        self.assertEqual(args.url, 'http://localhost:3000')
+
+    def test_expand_localhost_shorthand_with_path(self):
+        args = parser.parse_args(args=[':/path'], env=TestEnvironment())
+
+        self.assertEqual(args.url, 'http://localhost/path')
+
+    def test_expand_localhost_shorthand_with_port_and_slash(self):
+        args = parser.parse_args(args=[':3000/'], env=TestEnvironment())
+
+        self.assertEqual(args.url, 'http://localhost:3000/')
+
+    def test_expand_localhost_shorthand_with_port_and_path(self):
+        args = parser.parse_args(args=[':3000/path'], env=TestEnvironment())
+
+        self.assertEqual(args.url, 'http://localhost:3000/path')
+
+    def test_dont_expand_shorthand_ipv6_as_shorthand(self):
+        args = parser.parse_args(args=['::1'], env=TestEnvironment())
+
+        self.assertEqual(args.url, 'http://::1')
+
+    def test_dont_expand_longer_ipv6_as_shorthand(self):
+        args = parser.parse_args(
+            args=['::ffff:c000:0280'],
+            env=TestEnvironment()
+        )
+        self.assertEqual(args.url, 'http://::ffff:c000:0280')
+
+    def test_dont_expand_full_ipv6_as_shorthand(self):
+        args = parser.parse_args(
+            args=['0000:0000:0000:0000:0000:0000:0000:0001'],
+            env=TestEnvironment()
+        )
+
+        self.assertEqual(
+            args.url,
+            'http://0000:0000:0000:0000:0000:0000:0000:0001'
+        )
 
 
 class ArgumentParserTestCase(unittest.TestCase):
@@ -1162,78 +1328,113 @@ class ArgumentParserTestCase(unittest.TestCase):
         self.parser = input.Parser()
 
     def test_guess_when_method_set_and_valid(self):
-        args = argparse.Namespace()
-        args.method = 'GET'
-        args.url = 'http://example.com/'
-        args.items = []
+        self.parser.args = argparse.Namespace()
+        self.parser.args.method = 'GET'
+        self.parser.args.url = 'http://example.com/'
+        self.parser.args.items = []
+        self.parser.args.ignore_stdin = False
 
-        self.parser._guess_method(args, TestEnvironment())
+        self.parser.env = TestEnvironment()
 
-        self.assertEqual(args.method, 'GET')
-        self.assertEqual(args.url, 'http://example.com/')
-        self.assertEqual(args.items, [])
+        self.parser._guess_method()
+
+        self.assertEqual(self.parser.args.method, 'GET')
+        self.assertEqual(self.parser.args.url, 'http://example.com/')
+        self.assertEqual(self.parser.args.items, [])
 
     def test_guess_when_method_not_set(self):
-        args = argparse.Namespace()
-        args.method = None
-        args.url = 'http://example.com/'
-        args.items = []
 
-        self.parser._guess_method(args, TestEnvironment())
+        self.parser.args = argparse.Namespace()
+        self.parser.args.method = None
+        self.parser.args.url = 'http://example.com/'
+        self.parser.args.items = []
+        self.parser.args.ignore_stdin = False
+        self.parser.env = TestEnvironment()
 
-        self.assertEqual(args.method, 'GET')
-        self.assertEqual(args.url, 'http://example.com/')
-        self.assertEqual(args.items, [])
+        self.parser._guess_method()
+
+        self.assertEqual(self.parser.args.method, 'GET')
+        self.assertEqual(self.parser.args.url, 'http://example.com/')
+        self.assertEqual(self.parser.args.items, [])
 
     def test_guess_when_method_set_but_invalid_and_data_field(self):
-        args = argparse.Namespace()
-        args.method = 'http://example.com/'
-        args.url = 'data=field'
-        args.items = []
+        self.parser.args = argparse.Namespace()
+        self.parser.args.method = 'http://example.com/'
+        self.parser.args.url = 'data=field'
+        self.parser.args.items = []
+        self.parser.args.ignore_stdin = False
+        self.parser.env = TestEnvironment()
+        self.parser._guess_method()
 
-        self.parser._guess_method(args, TestEnvironment())
-
-        self.assertEqual(args.method, 'POST')
-        self.assertEqual(args.url, 'http://example.com/')
+        self.assertEqual(self.parser.args.method, 'POST')
+        self.assertEqual(self.parser.args.url, 'http://example.com/')
         self.assertEqual(
-            args.items,
+            self.parser.args.items,
             [input.KeyValue(
                 key='data', value='field', sep='=', orig='data=field')])
 
     def test_guess_when_method_set_but_invalid_and_header_field(self):
-        args = argparse.Namespace()
-        args.method = 'http://example.com/'
-        args.url = 'test:header'
-        args.items = []
+        self.parser.args = argparse.Namespace()
+        self.parser.args.method = 'http://example.com/'
+        self.parser.args.url = 'test:header'
+        self.parser.args.items = []
+        self.parser.args.ignore_stdin = False
 
-        self.parser._guess_method(args, TestEnvironment())
+        self.parser.env = TestEnvironment()
 
-        self.assertEqual(args.method, 'GET')
-        self.assertEqual(args.url, 'http://example.com/')
+        self.parser._guess_method()
+
+        self.assertEqual(self.parser.args.method, 'GET')
+        self.assertEqual(self.parser.args.url, 'http://example.com/')
         self.assertEqual(
-            args.items,
+            self.parser.args.items,
             [input.KeyValue(
                 key='test', value='header', sep=':', orig='test:header')])
 
     def test_guess_when_method_set_but_invalid_and_item_exists(self):
-        args = argparse.Namespace()
-        args.method = 'http://example.com/'
-        args.url = 'new_item=a'
-        args.items = [
+        self.parser.args = argparse.Namespace()
+        self.parser.args.method = 'http://example.com/'
+        self.parser.args.url = 'new_item=a'
+        self.parser.args.items = [
             input.KeyValue(
                 key='old_item', value='b', sep='=', orig='old_item=b')
         ]
+        self.parser.args.ignore_stdin = False
 
-        self.parser._guess_method(args, TestEnvironment())
+        self.parser.env = TestEnvironment()
 
-        self.assertEqual(args.items, [
+        self.parser._guess_method()
+
+        self.assertEqual(self.parser.args.items, [
             input.KeyValue(
                 key='new_item', value='a', sep='=', orig='new_item=a'),
-            input.KeyValue(key
-                ='old_item', value='b', sep='=', orig='old_item=b'),
+            input.KeyValue(
+                key='old_item', value='b', sep='=', orig='old_item=b'),
         ])
 
 
+class TestNoOptions(BaseTestCase):
+
+    def test_valid_no_options(self):
+        r = http(
+            '--verbose',
+            '--no-verbose',
+            'GET',
+            httpbin('/get')
+        )
+        self.assertNotIn('GET /get HTTP/1.1', r)
+
+    def test_invalid_no_options(self):
+        r = http(
+            '--no-war',
+            'GET',
+            httpbin('/get')
+        )
+        self.assertEqual(r.exit_status, 1)
+        self.assertIn('unrecognized arguments: --no-war', r.stderr)
+        self.assertNotIn('GET /get HTTP/1.1', r)
+
+
 class READMETest(BaseTestCase):
 
     @skipIf(not has_docutils(), 'docutils not installed')
@@ -1267,7 +1468,7 @@ class SessionTest(BaseTestCase):
         shutil.rmtree(self.config_dir)
 
     def test_session_create(self):
-        # Verify that the has been created
+        # Verify that the session has been created.
         r = http(
             '--session=test',
             'GET',
@@ -1280,6 +1481,26 @@ class SessionTest(BaseTestCase):
         self.assertEqual(r.json['headers']['Cookie'], 'hello=world')
         self.assertIn('Basic ', r.json['headers']['Authorization'])
 
+    def test_session_ignored_header_prefixes(self):
+        r = http(
+            '--session=test',
+            'GET',
+            httpbin('/get'),
+            'Content-Type: text/plain',
+            'If-Unmodified-Since: Sat, 29 Oct 1994 19:43:31 GMT',
+            env=self.env
+        )
+        self.assertIn(OK, r)
+
+        r2 = http(
+            '--session=test',
+            'GET',
+            httpbin('/get')
+        )
+        self.assertIn(OK, r2)
+        self.assertNotIn('Content-Type', r2.json['headers'])
+        self.assertNotIn('If-Unmodified-Since', r2.json['headers'])
+
     def test_session_update(self):
         # Get a response to a request from the original session.
         r1 = http(
@@ -1316,7 +1537,7 @@ class SessionTest(BaseTestCase):
         self.assertNotEqual(r1.json['headers']['Authorization'],
                             r3.json['headers']['Authorization'])
 
-    def test_session_only(self):
+    def test_session_read_only(self):
         # Get a response from the original session.
         r1 = http(
             '--session=test',
@@ -1348,10 +1569,187 @@ class SessionTest(BaseTestCase):
         )
         self.assertIn(OK, r3)
 
+        # Origin can differ on Travis.
+        del r1.json['origin'], r3.json['origin']
+
         # Should be the same as before r2.
         self.assertDictEqual(r1.json, r3.json)
 
+    def test_session_by_path(self):
+        session_path = os.path.join(self.config_dir, 'session-by-path.json')
+
+        r1 = http(
+            '--session=' + session_path,
+            'GET',
+            httpbin('/get'),
+            'Foo:Bar',
+            env=self.env
+        )
+        self.assertIn(OK, r1)
+
+        r2 = http(
+            '--session=' + session_path,
+            'GET',
+            httpbin('/get'),
+            env=self.env
+        )
+        self.assertIn(OK, r2)
+
+        self.assertEqual(r2.json['headers']['Foo'], 'Bar')
+
+
+class DownloadUtilsTest(BaseTestCase):
+
+    def test_Content_Range_parsing(self):
+
+        parse = parse_content_range
+
+        self.assertEqual(parse('bytes 100-199/200', 100), 200)
+        self.assertEqual(parse('bytes 100-199/*', 100), 200)
+
+        # missing
+        self.assertRaises(ContentRangeError, parse, None, 100)
+
+        # syntax error
+        self.assertRaises(ContentRangeError, parse, 'beers 100-199/*', 100)
+
+        # unexpected range
+        self.assertRaises(ContentRangeError, parse, 'bytes 100-199/*', 99)
+
+        # invalid instance-length
+        self.assertRaises(ContentRangeError, parse, 'bytes 100-199/199', 100)
+
+        # invalid byte-range-resp-spec
+        self.assertRaises(ContentRangeError, parse, 'bytes 100-99/199', 100)
+
+        # invalid byte-range-resp-spec
+        self.assertRaises(ContentRangeError, parse, 'bytes 100-100/*', 100)
+
+    def test_Content_Disposition_parsing(self):
+        parse = filename_from_content_disposition
+        self.assertEqual(
+            parse('attachment; filename=hello-WORLD_123.txt'),
+            'hello-WORLD_123.txt'
+        )
+        self.assertEqual(
+            parse('attachment; filename=".hello-WORLD_123.txt"'),
+            'hello-WORLD_123.txt'
+        )
+        self.assertEqual(
+            parse('attachment; filename="white space.txt"'),
+            'white space.txt'
+        )
+        self.assertEqual(
+            parse(r'attachment; filename="\"quotes\".txt"'),
+            '"quotes".txt'
+        )
+        self.assertEqual(parse('attachment; filename=/etc/hosts'), 'hosts')
+        self.assertIsNone(parse('attachment; filename='))
+
+    def test_filename_from_url(self):
+        self.assertEqual(filename_from_url(
+            url='http://example.org/foo',
+            content_type='text/plain'
+        ), 'foo.txt')
+        self.assertEqual(filename_from_url(
+            url='http://example.org/foo',
+            content_type='text/html; charset=utf8'
+        ), 'foo.html')
+        self.assertEqual(filename_from_url(
+            url='http://example.org/foo',
+            content_type=None
+        ), 'foo')
+        self.assertEqual(filename_from_url(
+            url='http://example.org/foo',
+            content_type='x-foo/bar'
+        ), 'foo')
+
+    def test_unique_filename(self):
+
+        def make_exists(unique_on_attempt=0):
+            # noinspection PyUnresolvedReferences,PyUnusedLocal
+            def exists(filename):
+                if exists.attempt == unique_on_attempt:
+                    return False
+                exists.attempt += 1
+                return True
+            exists.attempt = 0
+            return exists
+
+        self.assertEqual(
+            get_unique_filename('foo.bar', exists=make_exists()),
+            'foo.bar'
+        )
+        self.assertEqual(
+            get_unique_filename('foo.bar', exists=make_exists(1)),
+            'foo.bar-1'
+        )
+        self.assertEqual(
+            get_unique_filename('foo.bar', exists=make_exists(10)),
+            'foo.bar-10'
+        )
+
+
+class Response(object):
+
+    # noinspection PyDefaultArgument
+    def __init__(self, url, headers={}, status_code=200):
+        self.url = url
+        self.headers = CaseInsensitiveDict(headers)
+        self.status_code = status_code
+
+
+# noinspection PyTypeChecker
+class DownloadTest(BaseTestCase):
+    # TODO: more tests
+
+    def test_actual_download(self):
+        url = httpbin('/robots.txt')
+        body = urlopen(url).read().decode()
+        r = http(
+            '--download',
+            url,
+            env=TestEnvironment(
+                stdin_isatty=True,
+                stdout_isatty=False
+            )
+        )
+        self.assertIn('Downloading', r.stderr)
+        self.assertIn('[K', r.stderr)
+        self.assertIn('Done', r.stderr)
+        self.assertEqual(body, r)
+
+    def test_download_with_Content_Length(self):
+        download = Download(output_file=open(os.devnull, 'w'))
+        download.start(Response(
+            url=httpbin('/'),
+             headers={'Content-Length': 10}
+        ))
+        time.sleep(1.1)
+        download.chunk_downloaded(b'12345')
+        time.sleep(1.1)
+        download.chunk_downloaded(b'12345')
+        download.finish()
+        self.assertFalse(download.interrupted)
+
+    def test_download_no_Content_Length(self):
+        download = Download(output_file=open(os.devnull, 'w'))
+        download.start(Response(url=httpbin('/')))
+        time.sleep(1.1)
+        download.chunk_downloaded(b'12345')
+        download.finish()
+        self.assertFalse(download.interrupted)
+
+    def test_download_interrupted(self):
+        download = Download(output_file=open(os.devnull, 'w'))
+        download.start(Response(
+            url=httpbin('/'),
+            headers={'Content-Length': 5}
+        ))
+        download.chunk_downloaded(b'1234')
+        download.finish()
+        self.assertTrue(download.interrupted)
+
 
 if __name__ == '__main__':
-    #noinspection PyCallingNonCallable
     unittest.main()

tox.ini

@@ -4,16 +4,10 @@
 # and then run "tox" from this directory.
 
 [tox]
-envlist = py26, py27, py32, pypy
+envlist = py26, py27, py33, pypy
 
 [testenv]
 commands = {envpython} setup.py test
 
 [testenv:py26]
 deps = argparse
-
-[testenv:py30]
-deps = argparse
-
-[testenv:py31]
-deps = argparse