mirror of
https://github.com/httpie/cli.git
synced 2025-08-14 02:28:29 +02:00
Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f7e62336db |
3
.gitignore
vendored
3
.gitignore
vendored
@ -6,5 +6,4 @@ build
|
||||
README.html
|
||||
.coverage
|
||||
htmlcov
|
||||
.idea
|
||||
.DS_Store
|
||||
|
||||
|
||||
@ -3,7 +3,7 @@ python:
|
||||
- 2.6
|
||||
- 2.7
|
||||
- pypy
|
||||
- 3.3
|
||||
- 3.2
|
||||
script: python setup.py test
|
||||
install:
|
||||
- pip install . --use-mirrors
|
||||
|
||||
@ -8,7 +8,6 @@ HTTPie authors
|
||||
Patches and ideas
|
||||
-----------------
|
||||
|
||||
* `Cláudia T. Delgado <https://github.com/claudiatd>`_ (logo)
|
||||
* `Hank Gay <https://github.com/gthank>`_
|
||||
* `Jake Basile <https://github.com/jakebasile>`_
|
||||
* `Vladimir Berkutov <https://github.com/dair-targ>`_
|
||||
@ -28,4 +27,3 @@ Patches and ideas
|
||||
* `Tomek Wójcik <https://github.com/tomekwojcik>`_
|
||||
* `Davey Shafik <https://github.com/dshafik>`_
|
||||
* `cido <https://github.com/cido>`_
|
||||
* `Justin Bonnar <https://github.com/jargonjustin>`_
|
||||
|
||||
290
README.rst
290
README.rst
@ -1,11 +1,12 @@
|
||||
****************************************
|
||||
HTTPie: a CLI, cURL-like tool for humans
|
||||
****************************************
|
||||
***********************
|
||||
HTTPie: cURL for Humans
|
||||
***********************
|
||||
|
||||
v0.3.0
|
||||
|
||||
HTTPie is a **command line HTTP client**. Its goal is to make CLI interaction
|
||||
HTTPie is a **command line HTTP client** whose goal is to make CLI interaction
|
||||
with web services as **human-friendly** as possible. It provides a
|
||||
simple ``http`` command that allows for sending arbitrary HTTP requests using a
|
||||
simple ``http`` command that allows for sending arbitrary HTTP requests with a
|
||||
simple and natural syntax, and displays colorized responses. HTTPie can be used
|
||||
for **testing, debugging**, and generally **interacting** with HTTP servers.
|
||||
|
||||
@ -14,18 +15,10 @@ for **testing, debugging**, and generally **interacting** with HTTP servers.
|
||||
:alt: HTTPie compared to cURL
|
||||
:width: 835
|
||||
:height: 835
|
||||
:align: center
|
||||
|
||||
|
||||
------
|
||||
|
||||
|
||||
.. image:: https://raw.github.com/claudiatd/httpie-artwork/master/images/httpie_logo_simple.png
|
||||
:alt: HTTPie logo
|
||||
:align: center
|
||||
|
||||
HTTPie is written in Python, and under the hood it uses the excellent
|
||||
`Requests`_ and `Pygments`_ libraries.
|
||||
`Requests`_ for HTTP and `Pygments`_ for colorizing.
|
||||
|
||||
|
||||
**Table of Contents**
|
||||
@ -37,6 +30,7 @@ HTTPie is written in Python, and under the hood it uses the excellent
|
||||
:backlinks: none
|
||||
|
||||
|
||||
|
||||
=============
|
||||
Main Features
|
||||
=============
|
||||
@ -49,7 +43,6 @@ Main Features
|
||||
* Arbitrary request data
|
||||
* Custom headers
|
||||
* Persistent sessions
|
||||
* Wget-like downloads
|
||||
* Python 2.6, 2.7 and 3.x support
|
||||
* Linux, Mac OS X and Windows support
|
||||
* Documentation
|
||||
@ -66,11 +59,9 @@ or ``easy_install``:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install --upgrade httpie
|
||||
$ pip install -U httpie
|
||||
|
||||
|
||||
Alternatively:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ easy_install httpie
|
||||
@ -86,12 +77,12 @@ Or, you can install the **development version** directly from GitHub:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ pip install --upgrade https://github.com/jkbr/httpie/tarball/master
|
||||
$ pip install -U https://github.com/jkbr/httpie/tarball/master
|
||||
|
||||
|
||||
There are also packages available for `Ubuntu`_, `Debian`_, and possibly other
|
||||
Linux distributions as well. However, there may be a significant delay between
|
||||
official HTTPie releases and package updates.
|
||||
Linux distributions as well. However, they may be a significant delay between
|
||||
releases and package updates.
|
||||
|
||||
|
||||
=====
|
||||
@ -136,16 +127,14 @@ Submitting `forms`_:
|
||||
$ http -f POST example.org hello=World
|
||||
|
||||
|
||||
See the request that is being sent using one of the `output options`_:
|
||||
See the request that is being sent using on of the `output options`_:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http -v example.org
|
||||
|
||||
|
||||
Use `Github API`_ to post a comment on an
|
||||
`issue <https://github.com/jkbr/httpie/issues/83>`_
|
||||
with `authentication`_:
|
||||
Use `Github API`_ to post a comment on an issue with `authentication`_:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@ -166,34 +155,10 @@ Download a file and save it via `redirected output`_:
|
||||
$ http example.org/file > file
|
||||
|
||||
|
||||
Download a file ``wget`` style:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http --download example.org/file
|
||||
|
||||
Use named `sessions`_ to make certain aspects or the communication persistent
|
||||
between requests to the same host:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http --session=logged-in -a username:password httpbin.org/get API-Key:123
|
||||
|
||||
$ http --session=logged-in httpbin.org/headers
|
||||
|
||||
|
||||
Set a custom ``Host`` header to work around missing DNS records:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http localhost:8000 Host:example.com
|
||||
|
||||
..
|
||||
|
||||
--------
|
||||
|
||||
*What follows is a detailed documentation. It covers the command syntax,
|
||||
advanced usage, and also features additional examples.*
|
||||
advances usage, and also features additional examples.*
|
||||
|
||||
|
||||
============
|
||||
@ -266,8 +231,8 @@ their type is distinguished only by the separator used:
|
||||
| | The ``==`` separator is used |
|
||||
+-----------------------+-----------------------------------------------------+
|
||||
| Data Fields | Request data fields to be serialized as a JSON |
|
||||
| ``field=value`` | object (default), or to be form encoded |
|
||||
| | (``--form, -f``). |
|
||||
| ``field=value`` | object (default), or to be form encoded (``--form`` |
|
||||
| | / ``-f``). |
|
||||
+-----------------------+-----------------------------------------------------+
|
||||
| Raw JSON fields | Useful when sending JSON and one or |
|
||||
| ``field:=json`` | more fields need to be a ``Boolean``, ``Number``, |
|
||||
@ -275,19 +240,18 @@ their type is distinguished only by the separator used:
|
||||
| | ``meals:='["ham","spam"]'`` or ``pies:=[1,2,3]`` |
|
||||
| | (note the quotes). |
|
||||
+-----------------------+-----------------------------------------------------+
|
||||
| Files | Only available with ``--form, -f``. |
|
||||
| Files | Only available with ``-f`` / ``--form``. |
|
||||
| ``field@/dir/file`` | For example ``screenshot@~/Pictures/img.png``. |
|
||||
| | The presence of a file field results |
|
||||
| | in a ``multipart/form-data`` request. |
|
||||
+-----------------------+-----------------------------------------------------+
|
||||
|
||||
You can use ``\`` to escape characters that shouldn't be used as separators
|
||||
(or parts thereof). For instance, ``foo\==bar`` will become a data key/value
|
||||
(or parts thereof). e.g., ``foo\==bar`` will become a data key/value
|
||||
pair (``foo=`` and ``bar``) instead of a URL parameter.
|
||||
|
||||
Note that data fields aren't the only way to specify request data:
|
||||
`Redirected input`_ allows for passing arbitrary data to be sent with the
|
||||
request.
|
||||
Note that data fields aren't the only way to specify request data,
|
||||
`redirected input`_ allows passing arbitrary data to be sent with the request.
|
||||
|
||||
|
||||
====
|
||||
@ -306,7 +270,7 @@ both of which can be overwritten:
|
||||
``Accept`` ``application/json``
|
||||
================ =======================================
|
||||
|
||||
You can use ``--json, -j`` to explicitly set ``Accept``
|
||||
You can use ``--json`` / ``-j`` to explicitly set ``Accept``
|
||||
to ``application/json`` regardless of whether you are sending data
|
||||
(it's a shortcut for setting the header via the usual header notation –
|
||||
``http url Accept:application/json``).
|
||||
@ -324,6 +288,7 @@ Simple example:
|
||||
Accept-Encoding: identity, deflate, compress, gzip
|
||||
Content-Type: application/json; charset=utf-8
|
||||
Host: example.org
|
||||
User-Agent: HTTPie/0.2.7dev
|
||||
|
||||
{
|
||||
"name": "John",
|
||||
@ -345,6 +310,7 @@ into the resulting object:
|
||||
Accept: application/json
|
||||
Content-Type: application/json; charset=utf-8
|
||||
Host: api.example.com
|
||||
User-Agent: HTTPie/0.2.7dev
|
||||
|
||||
{
|
||||
"age": 29,
|
||||
@ -369,12 +335,12 @@ Forms
|
||||
=====
|
||||
|
||||
Submitting forms is very similar to sending `JSON`_ requests. Often the only
|
||||
difference is in adding the ``--form, -f`` option, which ensures that
|
||||
difference is in adding the ``--form`` / ``-f`` option, which ensures that
|
||||
data fields are serialized as, and ``Content-Type`` is set to,
|
||||
``application/x-www-form-urlencoded; charset=utf-8``.
|
||||
|
||||
It is possible to make form data the implicit content type instead of JSON
|
||||
via the `config`_ file.
|
||||
It is possible to make form data the implicit content type via the `config`_
|
||||
file.
|
||||
|
||||
|
||||
-------------
|
||||
@ -389,6 +355,7 @@ Regular Forms
|
||||
.. code-block:: http
|
||||
|
||||
POST /person/1 HTTP/1.1
|
||||
User-Agent: HTTPie/0.2.7dev
|
||||
Content-Type: application/x-www-form-urlencoded; charset=utf-8
|
||||
|
||||
name=John+Smith&email=john%40example.org
|
||||
@ -467,16 +434,13 @@ come). There are two flags that control authentication:
|
||||
(``-a username``), you'll be prompted for
|
||||
the password before the request is sent.
|
||||
To send a an empty password, pass ``username:``.
|
||||
The ``username:password@hostname`` URL syntax is
|
||||
supported as well (but credentials passed via ``-a``
|
||||
have higher priority).
|
||||
|
||||
``--auth-type`` Specify the auth mechanism. Possible values are
|
||||
``basic`` and ``digest``. The default value is
|
||||
``basic`` so it can often be omitted.
|
||||
=================== ======================================================
|
||||
|
||||
|
||||
Authorization information from ``.netrc`` is respected as well.
|
||||
|
||||
Basic auth:
|
||||
|
||||
@ -501,36 +465,22 @@ With password prompt:
|
||||
$ http -a username example.org
|
||||
|
||||
|
||||
Authorization information from your ``.netrc`` file is honored as well:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ cat .netrc
|
||||
machine httpbin.org
|
||||
login httpie
|
||||
password test
|
||||
$ http httpbin.org/basic-auth/httpie/test
|
||||
HTTP/1.1 200 OK
|
||||
[...]
|
||||
|
||||
|
||||
=======
|
||||
Proxies
|
||||
=======
|
||||
|
||||
You can specify proxies to be used through the ``--proxy`` argument for each
|
||||
protocol (which is included in the value in case of redirects across protocols):
|
||||
You can specify proxies to be used through the ``--proxy`` argument:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http --proxy=http:10.10.1.10:3128 --proxy=https:10.10.1.10:1080 example.org
|
||||
http --proxy=http:10.10.1.10:3128 --https:10.10.1.10:1080 example.org
|
||||
|
||||
|
||||
With Basic authentication:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http --proxy=http:http://user:pass@10.10.1.10:3128 example.org
|
||||
http --proxy=http:http://user:pass@10.10.1.10:3128 example.org
|
||||
|
||||
You can also configure proxies by environment variables ``HTTP_PROXY`` and
|
||||
``HTTPS_PROXY``, and the underlying Requests library will pick them up as well.
|
||||
@ -602,7 +552,7 @@ documentation examples:
|
||||
}
|
||||
|
||||
|
||||
All the other options are just a shortcut for ``--print, -p``.
|
||||
All the other options are just a shortcut for ``--print`` / ``-p``.
|
||||
It accepts a string of characters each of which represents a specific part of
|
||||
the HTTP exchange:
|
||||
|
||||
@ -751,7 +701,6 @@ Also, the following formatting is applied:
|
||||
* HTTP headers are sorted by name.
|
||||
* JSON data is indented, sorted by keys, and unicode escapes are converted
|
||||
to the characters they represent.
|
||||
* XML data is indented for better readability.
|
||||
|
||||
One of these options can be used to control output processing:
|
||||
|
||||
@ -775,7 +724,7 @@ that the response body is binary,
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http example.org/Movie.mov
|
||||
http example.org/Movie.mov
|
||||
|
||||
|
||||
You will nearly instantly see something like this:
|
||||
@ -833,76 +782,6 @@ Force colorizing and formatting, and show both the request and the response in
|
||||
The ``-R`` flag tells ``less`` to interpret color escape sequences included
|
||||
HTTPie`s output.
|
||||
|
||||
You can create a shortcut for invoking HTTPie with colorized and paged output
|
||||
by adding the following to your ``~/.bash_profile``:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
function httpless {
|
||||
# `httpless example.org'
|
||||
http --pretty=all "$@" | less -R;
|
||||
}
|
||||
|
||||
|
||||
=============
|
||||
Download Mode
|
||||
=============
|
||||
|
||||
HTTPie features a download mode in which it acts similarly to ``wget``.
|
||||
|
||||
When enabled using the ``--download, -d`` flag, response headers are printed to
|
||||
the terminal (``stderr``), and a progress bar is shown while the response body
|
||||
is being saved to a file.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http --download https://github.com/jkbr/httpie/tarball/master
|
||||
|
||||
.. code-block:: http
|
||||
|
||||
HTTP/1.1 200 OK
|
||||
Connection: keep-alive
|
||||
Content-Disposition: attachment; filename=jkbr-httpie-0.4.1-33-gfc4f70a.tar.gz
|
||||
Content-Length: 505530
|
||||
Content-Type: application/x-gzip
|
||||
Server: GitHub.com
|
||||
Vary: Accept-Encoding
|
||||
|
||||
Downloading 494.89 kB to "jkbr-httpie-0.4.1-33-gfc4f70a.tar.gz"
|
||||
/ 21.01% 104.00 kB 47.55 kB/s 0:00:08 ETA
|
||||
|
||||
|
||||
If not provided via ``--output, -o``, the output filename will be determined
|
||||
from ``Content-Disposition`` (if available), or from the URL and
|
||||
``Content-Type``. If the guessed filename already exists, HTTPie adds a unique
|
||||
suffix to it.
|
||||
|
||||
You can also redirect the response body to another program while the response
|
||||
headers and progress are still shown in the terminal:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http -d https://github.com/jkbr/httpie/tarball/master | tar zxf -
|
||||
|
||||
|
||||
If ``--output, -o`` is specified, you can resume a partial download using the
|
||||
``--continue, -c`` option. This only works with servers that support
|
||||
``Range`` requests and ``206 Partial Content`` responses. If the server doesn't
|
||||
support that, the whole file will simply be downloaded:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http -dco file.zip example.org/file
|
||||
|
||||
Other notes:
|
||||
|
||||
* The ``--download`` option only changes how the response body is treated.
|
||||
* You can still set custom headers, use sessions, ``--verbose, -v``, etc.
|
||||
* ``--download`` always implies ``--follow`` (redirects are followed).
|
||||
* HTTPie exits with status code ``1`` (error) if the body hasn't been fully
|
||||
downloaded.
|
||||
* ``Accept-Encoding`` cannot be set with ``--download``.
|
||||
|
||||
|
||||
==================
|
||||
Streamed Responses
|
||||
@ -941,28 +820,25 @@ Streamed output by small chunks alá ``tail -f``:
|
||||
$ http --stream -f -a YOUR-TWITTER-NAME https://stream.twitter.com/1/statuses/filter.json track=Apple \
|
||||
| while read tweet; do echo "$tweet" | http POST example.org/tweets ; done
|
||||
|
||||
|
||||
|
||||
========
|
||||
Sessions
|
||||
========
|
||||
|
||||
By default, every request is completely independent of any previous ones.
|
||||
HTTPie also supports persistent sessions, where custom headers (except for the
|
||||
ones starting with ``Content-`` or ``If-``), authorization, and cookies
|
||||
(manually specified or sent by the server) persist between requests
|
||||
to the same host.
|
||||
By default, every request is completely independent of the previous ones.
|
||||
|
||||
--------------
|
||||
Named Sessions
|
||||
--------------
|
||||
HTTPie supports persistent sessions, where custom headers, authorization,
|
||||
and cookies (manually specified or sent by the server) persist between
|
||||
requests. Sessions are named and host-bound.
|
||||
|
||||
Create a new session named ``user1`` for ``example.org``:
|
||||
Create a new session named ``user1``:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http --session=user1 -a user1:password example.org X-Foo:Bar
|
||||
|
||||
Now you can refer to the session by its name, and the previously used
|
||||
authorization and HTTP headers will automatically be set:
|
||||
Now you can refer to the session by its name:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@ -978,32 +854,10 @@ To use a session without updating it from the request/response exchange
|
||||
once it is created, specify the session name via
|
||||
``--session-read-only=SESSION_NAME`` instead.
|
||||
|
||||
Named sessions' data is stored in JSON files in the directory
|
||||
``~/.httpie/sessions/<host>/<name>.json``
|
||||
Sessions are stored as JSON files in ``~/.httpie/sessions/<host>/<name>.json``
|
||||
(``%APPDATA%\httpie\sessions\<host>\<name>.json`` on Windows).
|
||||
|
||||
------------------
|
||||
Anonymous Sessions
|
||||
------------------
|
||||
|
||||
Instead of a name, you can also directly specify a path to a session file. This
|
||||
allows for sessions to be re-used across multiple hosts:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ http --session=/tmp/session.json example.org
|
||||
$ http --session=/tmp/session.json admin.example.org
|
||||
$ http --session=~/.httpie/sessions/another.example.org/test.json example.org
|
||||
$ http --session-read-only=/tmp/session.json example.org
|
||||
|
||||
|
||||
**Warning:** All session data, including credentials, cookie data,
|
||||
and custom headers are stored in plain text.
|
||||
|
||||
Note that session files can also be created and edited manually in a text
|
||||
editor; they are plain JSON.
|
||||
|
||||
See also `Config`_.
|
||||
See also `config`_.
|
||||
|
||||
|
||||
======
|
||||
@ -1014,7 +868,7 @@ HTTPie uses a simple configuration file that contains a JSON object with the
|
||||
following keys:
|
||||
|
||||
========================= =================================================
|
||||
``__meta__`` HTTPie automatically stores some metadata here.
|
||||
``__version__`` HTTPie automatically sets this to its version.
|
||||
Do not change.
|
||||
|
||||
``implicit_content_type`` A ``String`` specifying the implicit content type
|
||||
@ -1024,24 +878,10 @@ following keys:
|
||||
|
||||
``default_options`` An ``Array`` (by default empty) of options
|
||||
that should be applied to every request.
|
||||
|
||||
For instance, you can use this option to change
|
||||
the default style and output options:
|
||||
``"default_options": ["--style=fruity", "--body"]``
|
||||
|
||||
Another useful default option is
|
||||
``"--session=default"`` to make HTTPie always
|
||||
use `sessions`_.
|
||||
|
||||
Default options from config file can be unset
|
||||
for a particular invocation via
|
||||
``--no-OPTION`` arguments passed on the
|
||||
command line (e.g., ``--no-style``
|
||||
or ``--no-session``).
|
||||
========================= =================================================
|
||||
|
||||
The default location of the configuration file is ``~/.httpie/config.json``
|
||||
(or ``%APPDATA%\httpie\config.json`` on Windows).
|
||||
The default location is ``~/.httpie/config.json``
|
||||
(``%APPDATA%\httpie\config.json`` on Windows).
|
||||
|
||||
The config directory location can be changed by setting the
|
||||
``HTTPIE_CONFIG_DIR`` environment variable.
|
||||
@ -1165,11 +1005,6 @@ Please run the existing suite of tests before a pull request is submitted:
|
||||
|
||||
Don't forget to add yourself to `AUTHORS.rst`_.
|
||||
|
||||
=======
|
||||
Logo
|
||||
=======
|
||||
|
||||
See `claudiatd/httpie-artwork`_
|
||||
|
||||
=======
|
||||
Authors
|
||||
@ -1178,6 +1013,7 @@ Authors
|
||||
`Jakub Roztocil`_ (`@jakubroztocil`_) created HTTPie and `these fine people`_
|
||||
have contributed.
|
||||
|
||||
|
||||
=======
|
||||
Licence
|
||||
=======
|
||||
@ -1191,26 +1027,6 @@ Changelog
|
||||
|
||||
*You can click a version name to see a diff with the previous one.*
|
||||
|
||||
* `0.7.0-dev`_
|
||||
* `0.6.0`_
|
||||
* XML data is now formatted.
|
||||
* ``--session`` and ``--session-read-only`` now also accept paths to
|
||||
session files (eg. ``http --session=/tmp/session.json example.org``).
|
||||
* `0.5.1`_ (2013-05-13)
|
||||
* ``Content-*`` and ``If-*`` request headers are not stored in sessions
|
||||
anymore as they are request-specific.
|
||||
* `0.5.0`_ (2013-04-27)
|
||||
* Added a `download mode`_ via ``--download``.
|
||||
* Bugfixes.
|
||||
* `0.4.1`_ (2013-02-26)
|
||||
* Fixed ``setup.py``.
|
||||
* `0.4.0`_ (2013-02-22)
|
||||
* Python 3.3 compatibility.
|
||||
* Requests >= v1.0.4 compatibility.
|
||||
* Added support for credentials in URL.
|
||||
* Added ``--no-option`` for every ``--option`` to be config-friendly.
|
||||
* Mutually exclusive arguments can be specified multiple times. The
|
||||
last value is used.
|
||||
* `0.3.0`_ (2012-09-21)
|
||||
* Allow output redirection on Windows.
|
||||
* Added configuration file.
|
||||
@ -1226,7 +1042,7 @@ Changelog
|
||||
``--ugly`` has bee removed in favor of ``--pretty=none``.
|
||||
* `0.2.7`_ (2012-08-07)
|
||||
* Compatibility with Requests 0.13.6.
|
||||
* Streamed terminal output. ``--stream, -S`` can be used to enable
|
||||
* Streamed terminal output. ``--stream`` / ``-S`` can be used to enable
|
||||
streaming also with ``--pretty`` and to ensure a more frequent output
|
||||
flushing.
|
||||
* Support for efficient large file downloads.
|
||||
@ -1296,7 +1112,6 @@ Changelog
|
||||
.. _Jakub Roztocil: http://roztocil.name
|
||||
.. _@jakubroztocil: https://twitter.com/jakubroztocil
|
||||
.. _existing issues: https://github.com/jkbr/httpie/issues?state=open
|
||||
.. _claudiatd/httpie-artwork: https://github.com/claudiatd/httpie-artwork
|
||||
.. _0.1.6: https://github.com/jkbr/httpie/compare/0.1.4...0.1.6
|
||||
.. _0.2.0: https://github.com/jkbr/httpie/compare/0.1.6...0.2.0
|
||||
.. _0.2.1: https://github.com/jkbr/httpie/compare/0.2.0...0.2.1
|
||||
@ -1305,11 +1120,6 @@ Changelog
|
||||
.. _0.2.6: https://github.com/jkbr/httpie/compare/0.2.5...0.2.6
|
||||
.. _0.2.7: https://github.com/jkbr/httpie/compare/0.2.5...0.2.7
|
||||
.. _0.3.0: https://github.com/jkbr/httpie/compare/0.2.7...0.3.0
|
||||
.. _0.4.0: https://github.com/jkbr/httpie/compare/0.3.0...0.4.0
|
||||
.. _0.4.1: https://github.com/jkbr/httpie/compare/0.4.0...0.4.1
|
||||
.. _0.5.0: https://github.com/jkbr/httpie/compare/0.4.1...0.5.0
|
||||
.. _0.5.1: https://github.com/jkbr/httpie/compare/0.5.0...0.5.1
|
||||
.. _0.6.0: https://github.com/jkbr/httpie/compare/0.5.1...0.6.0
|
||||
.. _0.7.0-dev: https://github.com/jkbr/httpie/compare/0.6.0...master
|
||||
.. _stable version: https://github.com/jkbr/httpie/tree/0.3.0#readme
|
||||
.. _AUTHORS.rst: https://github.com/jkbr/httpie/blob/master/AUTHORS.rst
|
||||
.. _LICENSE: https://github.com/jkbr/httpie/blob/master/LICENSE
|
||||
|
||||
@ -1,14 +1,13 @@
|
||||
"""
|
||||
HTTPie - a CLI, cURL-like tool for humans.
|
||||
HTTPie - cURL for humans.
|
||||
|
||||
"""
|
||||
__author__ = 'Jakub Roztocil'
|
||||
__version__ = '0.6.0'
|
||||
__version__ = '0.3.0'
|
||||
__licence__ = 'BSD'
|
||||
|
||||
|
||||
class ExitStatus:
|
||||
"""Exit status code constants."""
|
||||
class exit:
|
||||
OK = 0
|
||||
ERROR = 1
|
||||
ERROR_TIMEOUT = 2
|
||||
|
||||
208
httpie/cli.py
208
httpie/cli.py
@ -1,10 +1,14 @@
|
||||
"""CLI arguments definition.
|
||||
|
||||
NOTE: the CLI interface may change before reaching v1.0.
|
||||
TODO: make the options config friendly, i.e., no mutually exclusive groups to
|
||||
allow options overwriting.
|
||||
|
||||
"""
|
||||
from argparse import FileType, OPTIONAL, ZERO_OR_MORE, SUPPRESS
|
||||
|
||||
from requests.compat import is_windows
|
||||
|
||||
from . import __doc__
|
||||
from . import __version__
|
||||
from .sessions import DEFAULT_SESSIONS_DIR
|
||||
@ -13,7 +17,7 @@ from .input import (Parser, AuthCredentialsArgType, KeyValueArgType,
|
||||
SEP_PROXY, SEP_CREDENTIALS, SEP_GROUP_ITEMS,
|
||||
OUT_REQ_HEAD, OUT_REQ_BODY, OUT_RESP_HEAD,
|
||||
OUT_RESP_BODY, OUTPUT_OPTIONS,
|
||||
PRETTY_MAP, PRETTY_STDOUT_TTY_ONLY, SessionNameValidator)
|
||||
PRETTY_MAP, PRETTY_STDOUT_TTY_ONLY)
|
||||
|
||||
|
||||
def _(text):
|
||||
@ -23,13 +27,14 @@ def _(text):
|
||||
|
||||
parser = Parser(
|
||||
description='%s <http://httpie.org>' % __doc__.strip(),
|
||||
epilog='For every --option there is a --no-option'
|
||||
' that reverts the option to its default value.\n\n'
|
||||
'Suggestions and bug reports are greatly appreciated:\n'
|
||||
'https://github.com/jkbr/httpie/issues'
|
||||
epilog=_('''
|
||||
Suggestions and bug reports are greatly appreciated:
|
||||
https://github.com/jkbr/httpie/issues
|
||||
''')
|
||||
)
|
||||
|
||||
|
||||
|
||||
###############################################################################
|
||||
# Positional arguments.
|
||||
###############################################################################
|
||||
@ -38,12 +43,11 @@ positional = parser.add_argument_group(
|
||||
title='Positional arguments',
|
||||
description=_('''
|
||||
These arguments come after any flags and in the
|
||||
order they are listed here. Only URL is required.
|
||||
''')
|
||||
order they are listed here. Only URL is required.'''
|
||||
)
|
||||
)
|
||||
positional.add_argument(
|
||||
'method',
|
||||
metavar='METHOD',
|
||||
'method', metavar='METHOD',
|
||||
nargs=OPTIONAL,
|
||||
default=None,
|
||||
help=_('''
|
||||
@ -55,16 +59,14 @@ positional.add_argument(
|
||||
''')
|
||||
)
|
||||
positional.add_argument(
|
||||
'url',
|
||||
metavar='URL',
|
||||
'url', metavar='URL',
|
||||
help=_('''
|
||||
The protocol defaults to http:// if the
|
||||
URL does not include one.
|
||||
''')
|
||||
)
|
||||
positional.add_argument(
|
||||
'items',
|
||||
metavar='REQUEST ITEM',
|
||||
'items', metavar='REQUEST ITEM',
|
||||
nargs=ZERO_OR_MORE,
|
||||
type=KeyValueArgType(*SEP_GROUP_ITEMS),
|
||||
help=_('''
|
||||
@ -87,11 +89,10 @@ positional.add_argument(
|
||||
content_type = parser.add_argument_group(
|
||||
title='Predefined content types',
|
||||
description=None
|
||||
)
|
||||
).add_mutually_exclusive_group(required=False)
|
||||
|
||||
content_type.add_argument(
|
||||
'--json', '-j',
|
||||
action='store_true',
|
||||
'--json', '-j', action='store_true',
|
||||
help=_('''
|
||||
(default) Data items from the command
|
||||
line are serialized as a JSON object.
|
||||
@ -100,8 +101,7 @@ content_type.add_argument(
|
||||
''')
|
||||
)
|
||||
content_type.add_argument(
|
||||
'--form', '-f',
|
||||
action='store_true',
|
||||
'--form', '-f', action='store_true',
|
||||
help=_('''
|
||||
Data items from the command line are serialized as form fields.
|
||||
The Content-Type is set to application/x-www-form-urlencoded
|
||||
@ -119,9 +119,20 @@ content_type.add_argument(
|
||||
output_processing = parser.add_argument_group(title='Output processing')
|
||||
|
||||
output_processing.add_argument(
|
||||
'--pretty',
|
||||
dest='prettify',
|
||||
default=PRETTY_STDOUT_TTY_ONLY,
|
||||
'--output', '-o', type=FileType('w+b'),
|
||||
metavar='FILE',
|
||||
help= SUPPRESS if not is_windows else _(
|
||||
'''
|
||||
Save output to FILE.
|
||||
This option is a replacement for piping output to FILE,
|
||||
which would on Windows result in corrupted data
|
||||
being saved.
|
||||
|
||||
'''
|
||||
)
|
||||
)
|
||||
output_processing.add_argument(
|
||||
'--pretty', dest='prettify', default=PRETTY_STDOUT_TTY_ONLY,
|
||||
choices=sorted(PRETTY_MAP.keys()),
|
||||
help=_('''
|
||||
Controls output processing. The value can be "none" to not prettify
|
||||
@ -131,10 +142,7 @@ output_processing.add_argument(
|
||||
''')
|
||||
)
|
||||
output_processing.add_argument(
|
||||
'--style', '-s',
|
||||
dest='style',
|
||||
metavar='STYLE',
|
||||
default=DEFAULT_STYLE,
|
||||
'--style', '-s', dest='style', default=DEFAULT_STYLE, metavar='STYLE',
|
||||
choices=AVAILABLE_STYLES,
|
||||
help=_('''
|
||||
Output coloring style. One of %s. Defaults to "%s".
|
||||
@ -145,14 +153,14 @@ output_processing.add_argument(
|
||||
)
|
||||
|
||||
|
||||
|
||||
###############################################################################
|
||||
# Output options
|
||||
###############################################################################
|
||||
output_options = parser.add_argument_group(title='Output options')
|
||||
|
||||
output_options.add_argument(
|
||||
'--print', '-p',
|
||||
dest='output_options',
|
||||
output_print = output_options.add_mutually_exclusive_group(required=False)
|
||||
output_print.add_argument('--print', '-p', dest='output_options',
|
||||
metavar='WHAT',
|
||||
help=_('''
|
||||
String specifying what the output should contain:
|
||||
@ -164,46 +172,39 @@ output_options.add_argument(
|
||||
headers and body is printed), if standard output is not redirected.
|
||||
If the output is piped to another program or to a file,
|
||||
then only the body is printed by default.
|
||||
'''.format(request_headers=OUT_REQ_HEAD,
|
||||
request_body=OUT_REQ_BODY,
|
||||
response_headers=OUT_RESP_HEAD,
|
||||
response_body=OUT_RESP_BODY,))
|
||||
'''.format(
|
||||
request_headers=OUT_REQ_HEAD,
|
||||
request_body=OUT_REQ_BODY,
|
||||
response_headers=OUT_RESP_HEAD,
|
||||
response_body=OUT_RESP_BODY,
|
||||
))
|
||||
)
|
||||
output_options.add_argument(
|
||||
'--verbose', '-v',
|
||||
dest='output_options',
|
||||
action='store_const',
|
||||
const=''.join(OUTPUT_OPTIONS),
|
||||
output_print.add_argument(
|
||||
'--verbose', '-v', dest='output_options',
|
||||
action='store_const', const=''.join(OUTPUT_OPTIONS),
|
||||
help=_('''
|
||||
Print the whole request as well as the response.
|
||||
Shortcut for --print={0}.
|
||||
'''.format(''.join(OUTPUT_OPTIONS)))
|
||||
)
|
||||
output_options.add_argument(
|
||||
'--headers', '-h',
|
||||
dest='output_options',
|
||||
action='store_const',
|
||||
const=OUT_RESP_HEAD,
|
||||
output_print.add_argument(
|
||||
'--headers', '-h', dest='output_options',
|
||||
action='store_const', const=OUT_RESP_HEAD,
|
||||
help=_('''
|
||||
Print only the response headers.
|
||||
Shortcut for --print={0}.
|
||||
'''.format(OUT_RESP_HEAD))
|
||||
)
|
||||
output_options.add_argument(
|
||||
'--body', '-b',
|
||||
dest='output_options',
|
||||
action='store_const',
|
||||
const=OUT_RESP_BODY,
|
||||
output_print.add_argument(
|
||||
'--body', '-b', dest='output_options',
|
||||
action='store_const', const=OUT_RESP_BODY,
|
||||
help=_('''
|
||||
Print only the response body.
|
||||
Shortcut for --print={0}.
|
||||
'''.format(OUT_RESP_BODY))
|
||||
)
|
||||
|
||||
output_options.add_argument(
|
||||
'--stream', '-S',
|
||||
action='store_true',
|
||||
default=False,
|
||||
output_options.add_argument('--stream', '-S', action='store_true', default=False,
|
||||
help=_('''
|
||||
Always stream the output by line, i.e., behave like `tail -f'.
|
||||
|
||||
@ -216,72 +217,27 @@ output_options.add_argument(
|
||||
It is useful also without --pretty: It ensures that the output is flushed
|
||||
more often and in smaller chunks.
|
||||
|
||||
''')
|
||||
)
|
||||
output_processing.add_argument(
|
||||
'--output', '-o',
|
||||
type=FileType('a+b'),
|
||||
dest='output_file',
|
||||
metavar='FILE',
|
||||
help=_(
|
||||
'''
|
||||
Save output to FILE. If --download is set, then only the response
|
||||
body is saved to the file. Other parts of the HTTP exchange are
|
||||
printed to stderr.
|
||||
|
||||
'''
|
||||
)
|
||||
)
|
||||
|
||||
output_options.add_argument(
|
||||
'--download', '-d',
|
||||
action='store_true',
|
||||
default=False,
|
||||
help=_('''
|
||||
Do not print the response body to stdout. Rather, download it and store it
|
||||
in a file. The filename is guessed unless specified with --output
|
||||
[filename]. This action is similar to the default behaviour of wget.
|
||||
|
||||
''')
|
||||
)
|
||||
|
||||
output_options.add_argument(
|
||||
'--continue', '-c',
|
||||
dest='download_resume',
|
||||
action='store_true',
|
||||
default=False,
|
||||
help=_('''
|
||||
Resume an interrupted download.
|
||||
The --output option needs to be specified as well.
|
||||
''')
|
||||
)
|
||||
'''
|
||||
))
|
||||
|
||||
|
||||
###############################################################################
|
||||
# Sessions
|
||||
###############################################################################
|
||||
|
||||
sessions = parser.add_argument_group(title='Sessions')\
|
||||
.add_mutually_exclusive_group(required=False)
|
||||
|
||||
session_name_validator = SessionNameValidator(
|
||||
'Session name contains invalid characters.')
|
||||
|
||||
sessions.add_argument(
|
||||
'--session',
|
||||
metavar='SESSION_NAME_OR_PATH',
|
||||
type=session_name_validator,
|
||||
'--session', metavar='SESSION_NAME',
|
||||
help=_('''
|
||||
Create, or reuse and update a session.
|
||||
Within a session, custom headers, auth credential, as well as any
|
||||
Withing a session, custom headers, auth credential, as well as any
|
||||
cookies sent by the server persist between requests.
|
||||
Session files are stored in %s/<HOST>/<SESSION_NAME>.json.
|
||||
''' % DEFAULT_SESSIONS_DIR)
|
||||
)
|
||||
sessions.add_argument(
|
||||
'--session-read-only',
|
||||
metavar='SESSION_NAME_OR_PATH',
|
||||
type=session_name_validator,
|
||||
'--session-read-only', metavar='SESSION_NAME',
|
||||
help=_('''
|
||||
Create or read a session without updating it form the
|
||||
request/response exchange.
|
||||
@ -292,12 +248,10 @@ sessions.add_argument(
|
||||
###############################################################################
|
||||
# Authentication
|
||||
###############################################################################
|
||||
|
||||
# ``requests.request`` keyword arguments.
|
||||
auth = parser.add_argument_group(title='Authentication')
|
||||
auth.add_argument(
|
||||
'--auth', '-a',
|
||||
metavar='USER[:PASS]',
|
||||
'--auth', '-a', metavar='USER[:PASS]',
|
||||
type=AuthCredentialsArgType(SEP_CREDENTIALS),
|
||||
help=_('''
|
||||
If only the username is provided (-a username),
|
||||
@ -306,9 +260,7 @@ auth.add_argument(
|
||||
)
|
||||
|
||||
auth.add_argument(
|
||||
'--auth-type',
|
||||
choices=['basic', 'digest'],
|
||||
default='basic',
|
||||
'--auth-type', choices=['basic', 'digest'], default='basic',
|
||||
help=_('''
|
||||
The authentication mechanism to be used.
|
||||
Defaults to "basic".
|
||||
@ -316,17 +268,14 @@ auth.add_argument(
|
||||
)
|
||||
|
||||
|
||||
###############################################################################
|
||||
|
||||
# Network
|
||||
###############################################################################
|
||||
#############################################
|
||||
|
||||
network = parser.add_argument_group(title='Network')
|
||||
|
||||
network.add_argument(
|
||||
'--proxy',
|
||||
default=[],
|
||||
action='append',
|
||||
metavar='PROTOCOL:HOST',
|
||||
'--proxy', default=[], action='append', metavar='PROTOCOL:HOST',
|
||||
type=KeyValueArgType(SEP_PROXY),
|
||||
help=_('''
|
||||
String mapping protocol to the URL of the proxy
|
||||
@ -335,17 +284,14 @@ network.add_argument(
|
||||
''')
|
||||
)
|
||||
network.add_argument(
|
||||
'--follow',
|
||||
default=False,
|
||||
action='store_true',
|
||||
'--follow', default=False, action='store_true',
|
||||
help=_('''
|
||||
Set this flag if full redirects are allowed
|
||||
(e.g. re-POST-ing of data at new ``Location``)
|
||||
''')
|
||||
)
|
||||
network.add_argument(
|
||||
'--verify',
|
||||
default='yes',
|
||||
'--verify', default='yes',
|
||||
help=_('''
|
||||
Set to "no" to skip checking the host\'s SSL certificate.
|
||||
You can also pass the path to a CA_BUNDLE
|
||||
@ -356,19 +302,14 @@ network.add_argument(
|
||||
)
|
||||
|
||||
network.add_argument(
|
||||
'--timeout',
|
||||
type=float,
|
||||
default=30,
|
||||
metavar='SECONDS',
|
||||
'--timeout', type=float, default=30, metavar='SECONDS',
|
||||
help=_('''
|
||||
The connection timeout of the request in seconds.
|
||||
The default value is 30 seconds.
|
||||
''')
|
||||
)
|
||||
network.add_argument(
|
||||
'--check-status',
|
||||
default=False,
|
||||
action='store_true',
|
||||
'--check-status', default=False, action='store_true',
|
||||
help=_('''
|
||||
By default, HTTPie exits with 0 when no network or other fatal
|
||||
errors occur.
|
||||
@ -395,25 +336,16 @@ troubleshooting = parser.add_argument_group(title='Troubleshooting')
|
||||
|
||||
troubleshooting.add_argument(
|
||||
'--help',
|
||||
action='help',
|
||||
default=SUPPRESS,
|
||||
action='help', default=SUPPRESS,
|
||||
help='Show this help message and exit'
|
||||
)
|
||||
troubleshooting.add_argument('--version', action='version', version=__version__)
|
||||
troubleshooting.add_argument(
|
||||
'--version',
|
||||
action='version',
|
||||
version=__version__
|
||||
)
|
||||
troubleshooting.add_argument(
|
||||
'--traceback',
|
||||
action='store_true',
|
||||
default=False,
|
||||
'--traceback', action='store_true', default=False,
|
||||
help='Prints exception traceback should one occur.'
|
||||
)
|
||||
troubleshooting.add_argument(
|
||||
'--debug',
|
||||
action='store_true',
|
||||
default=False,
|
||||
'--debug', action='store_true', default=False,
|
||||
help=_('''
|
||||
Prints exception traceback should one occur, and also other
|
||||
information that is useful for debugging HTTPie itself and
|
||||
|
||||
@ -4,6 +4,7 @@ from pprint import pformat
|
||||
|
||||
import requests
|
||||
import requests.auth
|
||||
from requests.defaults import defaults
|
||||
|
||||
from . import sessions
|
||||
from . import __version__
|
||||
@ -20,52 +21,41 @@ def get_response(args, config_dir):
|
||||
requests_kwargs = get_requests_kwargs(args)
|
||||
|
||||
if args.debug:
|
||||
sys.stderr.write('\n>>> requests.request(%s)\n\n'
|
||||
% pformat(requests_kwargs))
|
||||
sys.stderr.write(
|
||||
'\n>>> requests.request(%s)\n\n' % pformat(requests_kwargs))
|
||||
|
||||
if not args.session and not args.session_read_only:
|
||||
response = requests.request(**requests_kwargs)
|
||||
return requests.request(**requests_kwargs)
|
||||
else:
|
||||
response = sessions.get_response(
|
||||
return sessions.get_response(
|
||||
config_dir=config_dir,
|
||||
session_name=args.session or args.session_read_only,
|
||||
requests_kwargs=requests_kwargs,
|
||||
name=args.session or args.session_read_only,
|
||||
request_kwargs=requests_kwargs,
|
||||
read_only=bool(args.session_read_only),
|
||||
)
|
||||
|
||||
return response
|
||||
|
||||
|
||||
def get_requests_kwargs(args):
|
||||
"""Translate our `args` into `requests.request` keyword arguments."""
|
||||
|
||||
implicit_headers = {
|
||||
'User-Agent': DEFAULT_UA
|
||||
}
|
||||
base_headers = defaults['base_headers'].copy()
|
||||
base_headers['User-Agent'] = DEFAULT_UA
|
||||
|
||||
auto_json = args.data and not args.form
|
||||
# FIXME: Accept is set to JSON with `http url @./file.txt`.
|
||||
if args.json or auto_json:
|
||||
implicit_headers['Accept'] = 'application/json'
|
||||
if args.json or (auto_json and args.data):
|
||||
implicit_headers['Content-Type'] = JSON
|
||||
base_headers['Accept'] = 'application/json'
|
||||
if args.data:
|
||||
base_headers['Content-Type'] = JSON
|
||||
|
||||
if isinstance(args.data, dict):
|
||||
if args.data:
|
||||
args.data = json.dumps(args.data)
|
||||
else:
|
||||
# We need to set data to an empty string to prevent requests
|
||||
# from assigning an empty list to `response.request.data`.
|
||||
args.data = ''
|
||||
# If not empty, serialize the data `dict` parsed from arguments.
|
||||
# Otherwise set it to `None` avoid sending "{}".
|
||||
args.data = json.dumps(args.data) if args.data else None
|
||||
|
||||
elif args.form and not args.files:
|
||||
# If sending files, `requests` will set
|
||||
# the `Content-Type` for us.
|
||||
implicit_headers['Content-Type'] = FORM
|
||||
|
||||
for name, value in implicit_headers.items():
|
||||
if name not in args.headers:
|
||||
args.headers[name] = value
|
||||
base_headers['Content-Type'] = FORM
|
||||
|
||||
credentials = None
|
||||
if args.auth:
|
||||
@ -75,7 +65,7 @@ def get_requests_kwargs(args):
|
||||
}[args.auth_type](args.auth.key, args.auth.value)
|
||||
|
||||
kwargs = {
|
||||
'stream': True,
|
||||
'prefetch': False,
|
||||
'method': args.method.lower(),
|
||||
'url': args.url,
|
||||
'headers': args.headers,
|
||||
@ -90,6 +80,9 @@ def get_requests_kwargs(args):
|
||||
'files': args.files,
|
||||
'allow_redirects': args.follow,
|
||||
'params': args.params,
|
||||
'config': {
|
||||
'base_headers': base_headers
|
||||
}
|
||||
}
|
||||
|
||||
return kwargs
|
||||
|
||||
@ -1,18 +0,0 @@
|
||||
"""
|
||||
Python 2/3 compatibility.
|
||||
|
||||
"""
|
||||
#noinspection PyUnresolvedReferences
|
||||
from requests.compat import (
|
||||
is_windows,
|
||||
bytes,
|
||||
str,
|
||||
is_py3,
|
||||
is_py26,
|
||||
)
|
||||
|
||||
try:
|
||||
from urllib.parse import urlsplit
|
||||
except ImportError:
|
||||
from urlparse import urlsplit
|
||||
|
||||
@ -3,7 +3,7 @@ import json
|
||||
import errno
|
||||
|
||||
from . import __version__
|
||||
from .compat import is_windows
|
||||
from requests.compat import is_windows
|
||||
|
||||
|
||||
DEFAULT_CONFIG_DIR = os.environ.get(
|
||||
@ -16,9 +16,8 @@ DEFAULT_CONFIG_DIR = os.environ.get(
|
||||
class BaseConfigDict(dict):
|
||||
|
||||
name = None
|
||||
helpurl = None
|
||||
about = None
|
||||
directory = DEFAULT_CONFIG_DIR
|
||||
help = None
|
||||
directory=DEFAULT_CONFIG_DIR
|
||||
|
||||
def __init__(self, directory=None, *args, **kwargs):
|
||||
super(BaseConfigDict, self).__init__(*args, **kwargs)
|
||||
@ -28,24 +27,18 @@ class BaseConfigDict(dict):
|
||||
def __getattr__(self, item):
|
||||
return self[item]
|
||||
|
||||
def _get_path(self):
|
||||
"""Return the config file path without side-effects."""
|
||||
return os.path.join(self.directory, self.name + '.json')
|
||||
|
||||
@property
|
||||
def path(self):
|
||||
"""Return the config file path creating basedir, if needed."""
|
||||
path = self._get_path()
|
||||
try:
|
||||
os.makedirs(os.path.dirname(path), mode=0o700)
|
||||
os.makedirs(self.directory, mode=0o700)
|
||||
except OSError as e:
|
||||
if e.errno != errno.EEXIST:
|
||||
raise
|
||||
return path
|
||||
return os.path.join(self.directory, self.name + '.json')
|
||||
|
||||
@property
|
||||
def is_new(self):
|
||||
return not os.path.exists(self._get_path())
|
||||
return not os.path.exists(self.path)
|
||||
|
||||
def load(self):
|
||||
try:
|
||||
@ -63,15 +56,7 @@ class BaseConfigDict(dict):
|
||||
raise
|
||||
|
||||
def save(self):
|
||||
self['__meta__'] = {
|
||||
'httpie': __version__
|
||||
}
|
||||
if self.helpurl:
|
||||
self['__meta__']['help'] = self.helpurl
|
||||
|
||||
if self.about:
|
||||
self['__meta__']['about'] = self.about
|
||||
|
||||
self['__version__'] = __version__
|
||||
with open(self.path, 'w') as f:
|
||||
json.dump(self, f, indent=4, sort_keys=True, ensure_ascii=True)
|
||||
f.write('\n')
|
||||
@ -87,8 +72,6 @@ class BaseConfigDict(dict):
|
||||
class Config(BaseConfigDict):
|
||||
|
||||
name = 'config'
|
||||
helpurl = 'https://github.com/jkbr/httpie#config'
|
||||
about = 'HTTPie configuration file'
|
||||
|
||||
DEFAULTS = {
|
||||
'implicit_content_type': 'json',
|
||||
|
||||
105
httpie/core.py
105
httpie/core.py
@ -14,32 +14,31 @@ import sys
|
||||
import errno
|
||||
|
||||
import requests
|
||||
from requests.compat import str, is_py3
|
||||
from httpie import __version__ as httpie_version
|
||||
from requests import __version__ as requests_version
|
||||
from pygments import __version__ as pygments_version
|
||||
|
||||
from .cli import parser
|
||||
from .compat import str, is_py3
|
||||
from .client import get_response
|
||||
from .downloads import Download
|
||||
from .models import Environment
|
||||
from .output import build_output_stream, write, write_with_colors_win_py3
|
||||
from . import ExitStatus
|
||||
from .output import output_stream, write, write_with_colors_win_p3k
|
||||
from . import exit
|
||||
|
||||
|
||||
def get_exit_status(http_status, follow=False):
|
||||
"""Translate HTTP status code to exit status code."""
|
||||
if 300 <= http_status <= 399 and not follow:
|
||||
def get_exist_status(code, follow=False):
|
||||
"""Translate HTTP status code to exit status."""
|
||||
if 300 <= code <= 399 and not follow:
|
||||
# Redirect
|
||||
return ExitStatus.ERROR_HTTP_3XX
|
||||
elif 400 <= http_status <= 499:
|
||||
return exit.ERROR_HTTP_3XX
|
||||
elif 400 <= code <= 499:
|
||||
# Client Error
|
||||
return ExitStatus.ERROR_HTTP_4XX
|
||||
elif 500 <= http_status <= 599:
|
||||
return exit.ERROR_HTTP_4XX
|
||||
elif 500 <= code <= 599:
|
||||
# Server Error
|
||||
return ExitStatus.ERROR_HTTP_5XX
|
||||
return exit.ERROR_HTTP_5XX
|
||||
else:
|
||||
return ExitStatus.OK
|
||||
return exit.OK
|
||||
|
||||
|
||||
def print_debug_info(env):
|
||||
@ -55,110 +54,70 @@ def print_debug_info(env):
|
||||
def main(args=sys.argv[1:], env=Environment()):
|
||||
"""Run the main program and write the output to ``env.stdout``.
|
||||
|
||||
Return exit status code.
|
||||
Return exit status.
|
||||
|
||||
"""
|
||||
if env.config.default_options:
|
||||
args = env.config.default_options + args
|
||||
|
||||
def error(msg, *args, **kwargs):
|
||||
def error(msg, *args):
|
||||
msg = msg % args
|
||||
level = kwargs.get('level', 'error')
|
||||
env.stderr.write('\nhttp: %s: %s\n' % (level, msg))
|
||||
env.stderr.write('\nhttp: error: %s\n' % msg)
|
||||
|
||||
debug = '--debug' in args
|
||||
traceback = debug or '--traceback' in args
|
||||
exit_status = ExitStatus.OK
|
||||
status = exit.OK
|
||||
|
||||
if debug:
|
||||
print_debug_info(env)
|
||||
if args == ['--debug']:
|
||||
return exit_status
|
||||
|
||||
download = None
|
||||
sys.exit(exit.OK)
|
||||
|
||||
try:
|
||||
args = parser.parse_args(args=args, env=env)
|
||||
|
||||
if args.download:
|
||||
args.follow = True # --download implies --follow.
|
||||
download = Download(
|
||||
output_file=args.output_file,
|
||||
progress_file=env.stderr,
|
||||
resume=args.download_resume
|
||||
)
|
||||
download.pre_request(args.headers)
|
||||
|
||||
response = get_response(args, config_dir=env.config.directory)
|
||||
|
||||
if args.check_status or download:
|
||||
if args.check_status:
|
||||
status = get_exist_status(response.status_code,
|
||||
args.follow)
|
||||
if status and not env.stdout_isatty:
|
||||
error('%s %s', response.raw.status, response.raw.reason)
|
||||
|
||||
exit_status = get_exit_status(
|
||||
http_status=response.status_code,
|
||||
follow=args.follow
|
||||
)
|
||||
|
||||
if not env.stdout_isatty and exit_status != ExitStatus.OK:
|
||||
error('HTTP %s %s',
|
||||
response.raw.status,
|
||||
response.raw.reason,
|
||||
level='warning')
|
||||
stream = output_stream(args, env, response.request, response)
|
||||
|
||||
write_kwargs = {
|
||||
'stream': build_output_stream(
|
||||
args, env, response.request, response),
|
||||
|
||||
# This will in fact be `stderr` with `--download`
|
||||
'stream': stream,
|
||||
'outfile': env.stdout,
|
||||
|
||||
'flush': env.stdout_isatty or args.stream
|
||||
}
|
||||
|
||||
try:
|
||||
|
||||
if env.is_windows and is_py3 and 'colors' in args.prettify:
|
||||
write_with_colors_win_py3(**write_kwargs)
|
||||
write_with_colors_win_p3k(**write_kwargs)
|
||||
else:
|
||||
write(**write_kwargs)
|
||||
|
||||
if download and exit_status == ExitStatus.OK:
|
||||
# Response body download.
|
||||
download_stream, download_to = download.start(response)
|
||||
write(
|
||||
stream=download_stream,
|
||||
outfile=download_to,
|
||||
flush=False,
|
||||
)
|
||||
download.finish()
|
||||
if download.interrupted:
|
||||
exit_status = ExitStatus.ERROR
|
||||
|
||||
except IOError as e:
|
||||
if not traceback and e.errno == errno.EPIPE:
|
||||
# Ignore broken pipes unless --traceback.
|
||||
env.stderr.write('\n')
|
||||
else:
|
||||
raise
|
||||
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
if traceback:
|
||||
raise
|
||||
env.stderr.write('\n')
|
||||
exit_status = ExitStatus.ERROR
|
||||
|
||||
status = exit.ERROR
|
||||
except requests.Timeout:
|
||||
exit_status = ExitStatus.ERROR_TIMEOUT
|
||||
status = exit.ERROR_TIMEOUT
|
||||
error('Request timed out (%ss).', args.timeout)
|
||||
|
||||
except Exception as e:
|
||||
# TODO: Better distinction between expected and unexpected errors.
|
||||
# Network errors vs. bugs, etc.
|
||||
# TODO: distinguish between expected and unexpected errors.
|
||||
# network errors vs. bugs, etc.
|
||||
if traceback:
|
||||
raise
|
||||
error('%s: %s', type(e).__name__, str(e))
|
||||
exit_status = ExitStatus.ERROR
|
||||
status = exit.ERROR
|
||||
|
||||
finally:
|
||||
if download and not download.finished:
|
||||
download.failed()
|
||||
|
||||
return exit_status
|
||||
return status
|
||||
|
||||
@ -1,423 +0,0 @@
|
||||
# coding=utf-8
|
||||
"""
|
||||
Download mode implementation.
|
||||
|
||||
"""
|
||||
from __future__ import division
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import mimetypes
|
||||
import threading
|
||||
from time import sleep, time
|
||||
|
||||
from .output import RawStream
|
||||
from .models import HTTPResponse
|
||||
from .utils import humanize_bytes
|
||||
from .compat import urlsplit
|
||||
|
||||
|
||||
PARTIAL_CONTENT = 206
|
||||
|
||||
|
||||
CLEAR_LINE = '\r\033[K'
|
||||
PROGRESS = (
|
||||
'{percentage: 6.2f} %'
|
||||
' {downloaded: >10}'
|
||||
' {speed: >10}/s'
|
||||
' {eta: >8} ETA'
|
||||
)
|
||||
PROGRESS_NO_CONTENT_LENGTH = '{downloaded: >10} {speed: >10}/s'
|
||||
SUMMARY = 'Done. {downloaded} in {time:0.5f}s ({speed}/s)\n'
|
||||
SPINNER = '|/-\\'
|
||||
|
||||
|
||||
class ContentRangeError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
def parse_content_range(content_range, resumed_from):
|
||||
"""
|
||||
Parse and validate Content-Range header.
|
||||
|
||||
<http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html>
|
||||
|
||||
:param content_range: the value of a Content-Range response header
|
||||
eg. "bytes 21010-47021/47022"
|
||||
:param resumed_from: first byte pos. from the Range request header
|
||||
:return: total size of the response body when fully downloaded.
|
||||
|
||||
"""
|
||||
if content_range is None:
|
||||
raise ContentRangeError('Missing Content-Range')
|
||||
|
||||
pattern = (
|
||||
'^bytes (?P<first_byte_pos>\d+)-(?P<last_byte_pos>\d+)'
|
||||
'/(\*|(?P<instance_length>\d+))$'
|
||||
)
|
||||
match = re.match(pattern, content_range)
|
||||
|
||||
if not match:
|
||||
raise ContentRangeError(
|
||||
'Invalid Content-Range format %r' % content_range)
|
||||
|
||||
content_range_dict = match.groupdict()
|
||||
first_byte_pos = int(content_range_dict['first_byte_pos'])
|
||||
last_byte_pos = int(content_range_dict['last_byte_pos'])
|
||||
instance_length = (
|
||||
int(content_range_dict['instance_length'])
|
||||
if content_range_dict['instance_length']
|
||||
else None
|
||||
)
|
||||
|
||||
# "A byte-content-range-spec with a byte-range-resp-spec whose
|
||||
# last- byte-pos value is less than its first-byte-pos value,
|
||||
# or whose instance-length value is less than or equal to its
|
||||
# last-byte-pos value, is invalid. The recipient of an invalid
|
||||
# byte-content-range- spec MUST ignore it and any content
|
||||
# transferred along with it."
|
||||
if (first_byte_pos >= last_byte_pos
|
||||
or (instance_length is not None
|
||||
and instance_length <= last_byte_pos)):
|
||||
raise ContentRangeError(
|
||||
'Invalid Content-Range returned: %r' % content_range)
|
||||
|
||||
if (first_byte_pos != resumed_from
|
||||
or (instance_length is not None
|
||||
and last_byte_pos + 1 != instance_length)):
|
||||
# Not what we asked for.
|
||||
raise ContentRangeError(
|
||||
'Unexpected Content-Range returned (%r)'
|
||||
' for the requested Range ("bytes=%d-")'
|
||||
% (content_range, resumed_from)
|
||||
)
|
||||
|
||||
return last_byte_pos + 1
|
||||
|
||||
|
||||
def filename_from_content_disposition(content_disposition):
|
||||
"""
|
||||
Extract and validate filename from a Content-Disposition header.
|
||||
|
||||
:param content_disposition: Content-Disposition value
|
||||
:return: the filename if present and valid, otherwise `None`
|
||||
|
||||
"""
|
||||
# attachment; filename=jkbr-httpie-0.4.1-20-g40bd8f6.tar.gz
|
||||
match = re.search('filename=(\S+)', content_disposition)
|
||||
if match and match.group(1):
|
||||
fn = match.group(1).strip('."')
|
||||
if re.match('^[a-zA-Z0-9._-]+$', fn):
|
||||
return fn
|
||||
|
||||
|
||||
def filename_from_url(url, content_type):
|
||||
fn = urlsplit(url).path.rstrip('/')
|
||||
fn = os.path.basename(fn) if fn else 'index'
|
||||
if '.' not in fn and content_type:
|
||||
content_type = content_type.split(';')[0]
|
||||
if content_type == 'text/plain':
|
||||
# mimetypes returns '.ksh'
|
||||
ext = '.txt'
|
||||
else:
|
||||
ext = mimetypes.guess_extension(content_type)
|
||||
|
||||
if ext == '.htm': # Python 3
|
||||
ext = '.html'
|
||||
|
||||
if ext:
|
||||
fn += ext
|
||||
|
||||
return fn
|
||||
|
||||
|
||||
def get_unique_filename(fn, exists=os.path.exists):
|
||||
attempt = 0
|
||||
while True:
|
||||
suffix = '-' + str(attempt) if attempt > 0 else ''
|
||||
if not exists(fn + suffix):
|
||||
return fn + suffix
|
||||
attempt += 1
|
||||
|
||||
|
||||
class Download(object):
|
||||
|
||||
def __init__(self, output_file=None,
|
||||
resume=False, progress_file=sys.stderr):
|
||||
"""
|
||||
:param resume: Should the download resume if partial download
|
||||
already exists.
|
||||
:type resume: bool
|
||||
|
||||
:param output_file: The file to store response body in. If not
|
||||
provided, it will be guessed from the response.
|
||||
:type output_file: file
|
||||
|
||||
:param progress_file: Where to report download progress.
|
||||
:type progress_file: file
|
||||
|
||||
"""
|
||||
self._output_file = output_file
|
||||
self._resume = resume
|
||||
self._resumed_from = 0
|
||||
self.finished = False
|
||||
|
||||
self._status = Status()
|
||||
self._progress_reporter = ProgressReporterThread(
|
||||
status=self._status,
|
||||
output=progress_file
|
||||
)
|
||||
|
||||
def pre_request(self, request_headers):
|
||||
"""Called just before the HTTP request is sent.
|
||||
|
||||
Might alter `request_headers`.
|
||||
|
||||
:type request_headers: dict
|
||||
|
||||
"""
|
||||
# Disable content encoding so that we can resume, etc.
|
||||
request_headers['Accept-Encoding'] = None
|
||||
if self._resume:
|
||||
bytes_have = os.path.getsize(self._output_file.name)
|
||||
if bytes_have:
|
||||
# Set ``Range`` header to resume the download
|
||||
# TODO: Use "If-Range: mtime" to make sure it's fresh?
|
||||
request_headers['Range'] = 'bytes=%d-' % bytes_have
|
||||
self._resumed_from = bytes_have
|
||||
|
||||
def start(self, response):
|
||||
"""
|
||||
Initiate and return a stream for `response` body with progress
|
||||
callback attached. Can be called only once.
|
||||
|
||||
:param response: Initiated response object with headers already fetched
|
||||
:type response: requests.models.Response
|
||||
|
||||
:return: RawStream, output_file
|
||||
|
||||
"""
|
||||
assert not self._status.time_started
|
||||
|
||||
try:
|
||||
total_size = int(response.headers['Content-Length'])
|
||||
except (KeyError, ValueError, TypeError):
|
||||
total_size = None
|
||||
|
||||
if self._output_file:
|
||||
if self._resume and response.status_code == PARTIAL_CONTENT:
|
||||
total_size = parse_content_range(
|
||||
response.headers.get('Content-Range'),
|
||||
self._resumed_from
|
||||
)
|
||||
|
||||
else:
|
||||
self._resumed_from = 0
|
||||
try:
|
||||
self._output_file.seek(0)
|
||||
self._output_file.truncate()
|
||||
except IOError:
|
||||
pass # stdout
|
||||
else:
|
||||
# TODO: Should the filename be taken from response.history[0].url?
|
||||
# Output file not specified. Pick a name that doesn't exist yet.
|
||||
fn = None
|
||||
if 'Content-Disposition' in response.headers:
|
||||
fn = filename_from_content_disposition(
|
||||
response.headers['Content-Disposition'])
|
||||
if not fn:
|
||||
fn = filename_from_url(
|
||||
url=response.url,
|
||||
content_type=response.headers.get('Content-Type'),
|
||||
)
|
||||
self._output_file = open(get_unique_filename(fn), mode='a+b')
|
||||
|
||||
self._status.started(
|
||||
resumed_from=self._resumed_from,
|
||||
total_size=total_size
|
||||
)
|
||||
|
||||
stream = RawStream(
|
||||
msg=HTTPResponse(response),
|
||||
with_headers=False,
|
||||
with_body=True,
|
||||
on_body_chunk_downloaded=self._chunk_downloaded,
|
||||
chunk_size=1024 * 8
|
||||
)
|
||||
|
||||
self._progress_reporter.output.write(
|
||||
'Downloading %sto "%s"\n' % (
|
||||
(humanize_bytes(total_size) + ' '
|
||||
if total_size is not None
|
||||
else ''),
|
||||
self._output_file.name
|
||||
)
|
||||
)
|
||||
self._progress_reporter.start()
|
||||
|
||||
return stream, self._output_file
|
||||
|
||||
def finish(self):
|
||||
assert not self.finished
|
||||
self.finished = True
|
||||
self._status.finished()
|
||||
|
||||
def failed(self):
|
||||
self._progress_reporter.stop()
|
||||
|
||||
@property
|
||||
def interrupted(self):
|
||||
return (
|
||||
self.finished
|
||||
and self._status.total_size
|
||||
and self._status.total_size != self._status.downloaded
|
||||
)
|
||||
|
||||
def _chunk_downloaded(self, chunk):
|
||||
"""
|
||||
A download progress callback.
|
||||
|
||||
:param chunk: A chunk of response body data that has just
|
||||
been downloaded and written to the output.
|
||||
:type chunk: bytes
|
||||
|
||||
"""
|
||||
self._status.chunk_downloaded(len(chunk))
|
||||
|
||||
|
||||
class Status(object):
|
||||
"""Holds details about the downland status."""
|
||||
|
||||
def __init__(self):
|
||||
self.downloaded = 0
|
||||
self.total_size = None
|
||||
self.resumed_from = 0
|
||||
self.time_started = None
|
||||
self.time_finished = None
|
||||
|
||||
def started(self, resumed_from=0, total_size=None):
|
||||
assert self.time_started is None
|
||||
if total_size is not None:
|
||||
self.total_size = total_size
|
||||
self.downloaded = self.resumed_from = resumed_from
|
||||
self.time_started = time()
|
||||
|
||||
def chunk_downloaded(self, size):
|
||||
assert self.time_finished is None
|
||||
self.downloaded += size
|
||||
|
||||
@property
|
||||
def has_finished(self):
|
||||
return self.time_finished is not None
|
||||
|
||||
def finished(self):
|
||||
assert self.time_started is not None
|
||||
assert self.time_finished is None
|
||||
self.time_finished = time()
|
||||
|
||||
|
||||
class ProgressReporterThread(threading.Thread):
|
||||
"""
|
||||
Reports download progress based on its status.
|
||||
|
||||
Uses threading to periodically update the status (speed, ETA, etc.).
|
||||
|
||||
"""
|
||||
def __init__(self, status, output, tick=.1, update_interval=1):
|
||||
"""
|
||||
|
||||
:type status: Status
|
||||
:type output: file
|
||||
"""
|
||||
super(ProgressReporterThread, self).__init__()
|
||||
self.status = status
|
||||
self.output = output
|
||||
self._tick = tick
|
||||
self._update_interval = update_interval
|
||||
self._spinner_pos = 0
|
||||
self._status_line = ''
|
||||
self._prev_bytes = 0
|
||||
self._prev_time = time()
|
||||
self._should_stop = threading.Event()
|
||||
|
||||
def stop(self):
|
||||
"""Stop reporting on next tick."""
|
||||
self._should_stop.set()
|
||||
|
||||
def run(self):
|
||||
while not self._should_stop.is_set():
|
||||
if self.status.has_finished:
|
||||
self.sum_up()
|
||||
break
|
||||
|
||||
self.report_speed()
|
||||
sleep(self._tick)
|
||||
|
||||
def report_speed(self):
|
||||
|
||||
now = time()
|
||||
|
||||
if now - self._prev_time >= self._update_interval:
|
||||
downloaded = self.status.downloaded
|
||||
try:
|
||||
speed = ((downloaded - self._prev_bytes)
|
||||
/ (now - self._prev_time))
|
||||
except ZeroDivisionError:
|
||||
speed = 0
|
||||
|
||||
if not self.status.total_size:
|
||||
self._status_line = PROGRESS_NO_CONTENT_LENGTH.format(
|
||||
downloaded=humanize_bytes(downloaded),
|
||||
speed=humanize_bytes(speed),
|
||||
)
|
||||
else:
|
||||
try:
|
||||
percentage = downloaded / self.status.total_size * 100
|
||||
except ZeroDivisionError:
|
||||
percentage = 0
|
||||
|
||||
if not speed:
|
||||
eta = '-:--:--'
|
||||
else:
|
||||
s = int((self.status.total_size - downloaded) / speed)
|
||||
h, s = divmod(s, 60 * 60)
|
||||
m, s = divmod(s, 60)
|
||||
eta = '{0}:{1:0>2}:{2:0>2}'.format(h, m, s)
|
||||
|
||||
self._status_line = PROGRESS.format(
|
||||
percentage=percentage,
|
||||
downloaded=humanize_bytes(downloaded),
|
||||
speed=humanize_bytes(speed),
|
||||
eta=eta,
|
||||
)
|
||||
|
||||
self._prev_time = now
|
||||
self._prev_bytes = downloaded
|
||||
|
||||
self.output.write(
|
||||
CLEAR_LINE
|
||||
+ ' '
|
||||
+ SPINNER[self._spinner_pos]
|
||||
+ ' '
|
||||
+ self._status_line
|
||||
)
|
||||
self.output.flush()
|
||||
|
||||
self._spinner_pos = (self._spinner_pos + 1
|
||||
if self._spinner_pos + 1 != len(SPINNER)
|
||||
else 0)
|
||||
|
||||
def sum_up(self):
|
||||
actually_downloaded = (self.status.downloaded
|
||||
- self.status.resumed_from)
|
||||
time_taken = self.status.time_finished - self.status.time_started
|
||||
|
||||
self.output.write(CLEAR_LINE)
|
||||
self.output.write(SUMMARY.format(
|
||||
downloaded=humanize_bytes(actually_downloaded),
|
||||
total=(self.status.total_size
|
||||
and humanize_bytes(self.status.total_size)),
|
||||
speed=humanize_bytes(actually_downloaded / time_taken),
|
||||
time=time_taken,
|
||||
))
|
||||
self.output.flush()
|
||||
284
httpie/input.py
284
httpie/input.py
@ -8,19 +8,15 @@ import json
|
||||
import mimetypes
|
||||
import getpass
|
||||
from io import BytesIO
|
||||
from argparse import ArgumentParser, ArgumentTypeError, ArgumentError
|
||||
from argparse import ArgumentParser, ArgumentTypeError
|
||||
|
||||
try:
|
||||
from collections import OrderedDict
|
||||
except ImportError:
|
||||
OrderedDict = dict
|
||||
|
||||
# TODO: Use MultiDict for headers once added to `requests`.
|
||||
# https://github.com/jkbr/httpie/issues/130
|
||||
from requests.structures import CaseInsensitiveDict
|
||||
|
||||
from .compat import urlsplit, str
|
||||
from .sessions import VALID_SESSION_NAME_PATTERN
|
||||
from requests.compat import str, urlparse
|
||||
|
||||
|
||||
HTTP_POST = 'POST'
|
||||
@ -99,32 +95,37 @@ class Parser(ArgumentParser):
|
||||
def parse_args(self, env, args=None, namespace=None):
|
||||
|
||||
self.env = env
|
||||
self.args, no_options = super(Parser, self)\
|
||||
.parse_known_args(args, namespace)
|
||||
|
||||
if self.args.debug:
|
||||
self.args.traceback = True
|
||||
args = super(Parser, self).parse_args(args, namespace)
|
||||
|
||||
if not args.json and env.config.implicit_content_type == 'form':
|
||||
args.form = True
|
||||
|
||||
if args.debug:
|
||||
args.traceback = True
|
||||
|
||||
if args.output:
|
||||
env.stdout = args.output
|
||||
env.stdout_isatty = False
|
||||
|
||||
self._process_output_options(args, env)
|
||||
self._process_pretty_options(args, env)
|
||||
self._guess_method(args, env)
|
||||
self._parse_items(args)
|
||||
|
||||
# Arguments processing and environment setup.
|
||||
self._apply_no_options(no_options)
|
||||
self._apply_config()
|
||||
self._validate_download_options()
|
||||
self._setup_standard_streams()
|
||||
self._process_output_options()
|
||||
self._process_pretty_options()
|
||||
self._guess_method()
|
||||
self._parse_items()
|
||||
if not env.stdin_isatty:
|
||||
self._body_from_file(self.env.stdin)
|
||||
if not (self.args.url.startswith((HTTP, HTTPS))):
|
||||
# Default to 'https://' if invoked as `https args`.
|
||||
scheme = HTTPS if self.env.progname == 'https' else HTTP
|
||||
self.args.url = scheme + self.args.url
|
||||
self._process_auth()
|
||||
self._body_from_file(args, env.stdin)
|
||||
|
||||
return self.args
|
||||
if not (args.url.startswith(HTTP) or args.url.startswith(HTTPS)):
|
||||
scheme = HTTPS if env.progname == 'https' else HTTP
|
||||
args.url = scheme + args.url
|
||||
|
||||
if args.auth and not args.auth.has_password():
|
||||
# Stdin already read (if not a tty) so it's save to prompt.
|
||||
args.auth.prompt_password(urlparse(args.url).netloc)
|
||||
|
||||
return args
|
||||
|
||||
# noinspection PyShadowingBuiltins
|
||||
def _print_message(self, message, file=None):
|
||||
# Sneak in our stderr/stdout.
|
||||
file = {
|
||||
@ -135,226 +136,113 @@ class Parser(ArgumentParser):
|
||||
|
||||
super(Parser, self)._print_message(message, file)
|
||||
|
||||
def _setup_standard_streams(self):
|
||||
"""
|
||||
Modify `env.stdout` and `env.stdout_isatty` based on args, if needed.
|
||||
|
||||
"""
|
||||
if not self.env.stdout_isatty and self.args.output_file:
|
||||
self.error('Cannot use --output, -o with redirected output.')
|
||||
|
||||
# FIXME: Come up with a cleaner solution.
|
||||
if self.args.download:
|
||||
|
||||
if not self.env.stdout_isatty:
|
||||
# Use stdout as tge download output file.
|
||||
self.args.output_file = self.env.stdout
|
||||
|
||||
# With `--download`, we write everything that would normally go to
|
||||
# `stdout` to `stderr` instead. Let's replace the stream so that
|
||||
# we don't have to use many `if`s throughout the codebase.
|
||||
# The response body will be treated separately.
|
||||
self.env.stdout = self.env.stderr
|
||||
self.env.stdout_isatty = self.env.stderr_isatty
|
||||
|
||||
elif self.args.output_file:
|
||||
# When not `--download`ing, then `--output` simply replaces
|
||||
# `stdout`. The file is opened for appending, which isn't what
|
||||
# we want in this case.
|
||||
self.args.output_file.seek(0)
|
||||
self.args.output_file.truncate()
|
||||
|
||||
self.env.stdout = self.args.output_file
|
||||
self.env.stdout_isatty = False
|
||||
|
||||
def _apply_config(self):
|
||||
if (not self.args.json
|
||||
and self.env.config.implicit_content_type == 'form'):
|
||||
self.args.form = True
|
||||
|
||||
def _process_auth(self):
|
||||
"""
|
||||
If only a username provided via --auth, then ask for a password.
|
||||
Or, take credentials from the URL, if provided.
|
||||
|
||||
"""
|
||||
url = urlsplit(self.args.url)
|
||||
|
||||
if self.args.auth:
|
||||
if not self.args.auth.has_password():
|
||||
# Stdin already read (if not a tty) so it's save to prompt.
|
||||
self.args.auth.prompt_password(url.netloc)
|
||||
|
||||
elif url.username is not None:
|
||||
# Handle http://username:password@hostname/
|
||||
username, password = url.username, url.password
|
||||
self.args.auth = AuthCredentials(
|
||||
key=username,
|
||||
value=password,
|
||||
sep=SEP_CREDENTIALS,
|
||||
orig=SEP_CREDENTIALS.join([username, password])
|
||||
)
|
||||
|
||||
def _apply_no_options(self, no_options):
|
||||
"""For every `--no-OPTION` in `no_options`, set `args.OPTION` to
|
||||
its default value. This allows for un-setting of options, e.g.,
|
||||
specified in config.
|
||||
|
||||
"""
|
||||
invalid = []
|
||||
|
||||
for option in no_options:
|
||||
if not option.startswith('--no-'):
|
||||
invalid.append(option)
|
||||
continue
|
||||
|
||||
# --no-option => --option
|
||||
inverted = '--' + option[5:]
|
||||
for action in self._actions:
|
||||
if inverted in action.option_strings:
|
||||
setattr(self.args, action.dest, action.default)
|
||||
break
|
||||
else:
|
||||
invalid.append(option)
|
||||
|
||||
if invalid:
|
||||
msg = 'unrecognized arguments: %s'
|
||||
self.error(msg % ' '.join(invalid))
|
||||
|
||||
def _body_from_file(self, fd):
|
||||
def _body_from_file(self, args, fd):
|
||||
"""There can only be one source of request data.
|
||||
|
||||
Bytes are always read.
|
||||
|
||||
"""
|
||||
if self.args.data:
|
||||
if args.data:
|
||||
self.error('Request body (from stdin or a file) and request '
|
||||
'data (key=value) cannot be mixed.')
|
||||
self.args.data = getattr(fd, 'buffer', fd).read()
|
||||
args.data = getattr(fd, 'buffer', fd).read()
|
||||
|
||||
def _guess_method(self):
|
||||
def _guess_method(self, args, env):
|
||||
"""Set `args.method` if not specified to either POST or GET
|
||||
based on whether the request has data or not.
|
||||
|
||||
"""
|
||||
if self.args.method is None:
|
||||
if args.method is None:
|
||||
# Invoked as `http URL'.
|
||||
assert not self.args.items
|
||||
if not self.env.stdin_isatty:
|
||||
self.args.method = HTTP_POST
|
||||
assert not args.items
|
||||
if not env.stdin_isatty:
|
||||
args.method = HTTP_POST
|
||||
else:
|
||||
self.args.method = HTTP_GET
|
||||
args.method = HTTP_GET
|
||||
|
||||
# FIXME: False positive, e.g., "localhost" matches but is a valid URL.
|
||||
elif not re.match('^[a-zA-Z]+$', self.args.method):
|
||||
elif not re.match('^[a-zA-Z]+$', args.method):
|
||||
# Invoked as `http URL item+'. The URL is now in `args.method`
|
||||
# and the first ITEM is now incorrectly in `args.url`.
|
||||
try:
|
||||
# Parse the URL as an ITEM and store it as the first ITEM arg.
|
||||
self.args.items.insert(
|
||||
0,
|
||||
KeyValueArgType(*SEP_GROUP_ITEMS).__call__(self.args.url)
|
||||
)
|
||||
args.items.insert(
|
||||
0, KeyValueArgType(*SEP_GROUP_ITEMS).__call__(args.url))
|
||||
|
||||
except ArgumentTypeError as e:
|
||||
if self.args.traceback:
|
||||
if args.traceback:
|
||||
raise
|
||||
self.error(e.message)
|
||||
|
||||
else:
|
||||
# Set the URL correctly
|
||||
self.args.url = self.args.method
|
||||
args.url = args.method
|
||||
# Infer the method
|
||||
has_data = not self.env.stdin_isatty or any(
|
||||
item.sep in SEP_GROUP_DATA_ITEMS
|
||||
for item in self.args.items
|
||||
)
|
||||
self.args.method = HTTP_POST if has_data else HTTP_GET
|
||||
has_data = not env.stdin_isatty or any(
|
||||
item.sep in SEP_GROUP_DATA_ITEMS for item in args.items)
|
||||
args.method = HTTP_POST if has_data else HTTP_GET
|
||||
|
||||
def _parse_items(self):
|
||||
"""Parse `args.items` into `args.headers`, `args.data`, `args.params`,
|
||||
and `args.files`.
|
||||
def _parse_items(self, args):
|
||||
"""Parse `args.items` into `args.headers`, `args.data`,
|
||||
`args.`, and `args.files`.
|
||||
|
||||
"""
|
||||
self.args.headers = CaseInsensitiveDict()
|
||||
self.args.data = ParamDict() if self.args.form else OrderedDict()
|
||||
self.args.files = OrderedDict()
|
||||
self.args.params = ParamDict()
|
||||
args.headers = CaseInsensitiveDict()
|
||||
args.data = ParamDict() if args.form else OrderedDict()
|
||||
args.files = OrderedDict()
|
||||
args.params = ParamDict()
|
||||
|
||||
try:
|
||||
parse_items(items=self.args.items,
|
||||
headers=self.args.headers,
|
||||
data=self.args.data,
|
||||
files=self.args.files,
|
||||
params=self.args.params)
|
||||
parse_items(items=args.items,
|
||||
headers=args.headers,
|
||||
data=args.data,
|
||||
files=args.files,
|
||||
params=args.params)
|
||||
except ParseError as e:
|
||||
if self.args.traceback:
|
||||
if args.traceback:
|
||||
raise
|
||||
self.error(e.message)
|
||||
|
||||
if self.args.files and not self.args.form:
|
||||
if args.files and not args.form:
|
||||
# `http url @/path/to/file`
|
||||
file_fields = list(self.args.files.keys())
|
||||
file_fields = list(args.files.keys())
|
||||
if file_fields != ['']:
|
||||
self.error(
|
||||
'Invalid file fields (perhaps you meant --form?): %s'
|
||||
% ','.join(file_fields))
|
||||
|
||||
fn, fd = self.args.files['']
|
||||
self.args.files = {}
|
||||
|
||||
self._body_from_file(fd)
|
||||
|
||||
if 'Content-Type' not in self.args.headers:
|
||||
fn, fd = args.files['']
|
||||
args.files = {}
|
||||
self._body_from_file(args, fd)
|
||||
if 'Content-Type' not in args.headers:
|
||||
mime, encoding = mimetypes.guess_type(fn, strict=False)
|
||||
if mime:
|
||||
content_type = mime
|
||||
if encoding:
|
||||
content_type = '%s; charset=%s' % (mime, encoding)
|
||||
self.args.headers['Content-Type'] = content_type
|
||||
args.headers['Content-Type'] = content_type
|
||||
|
||||
def _process_output_options(self):
|
||||
"""Apply defaults to output options, or validate the provided ones.
|
||||
def _process_output_options(self, args, env):
|
||||
"""Apply defaults to output options or validate the provided ones.
|
||||
|
||||
The default output options are stdout-type-sensitive.
|
||||
|
||||
"""
|
||||
if not self.args.output_options:
|
||||
self.args.output_options = (
|
||||
OUTPUT_OPTIONS_DEFAULT
|
||||
if self.env.stdout_isatty
|
||||
else OUTPUT_OPTIONS_DEFAULT_STDOUT_REDIRECTED
|
||||
)
|
||||
if not args.output_options:
|
||||
args.output_options = (OUTPUT_OPTIONS_DEFAULT if env.stdout_isatty
|
||||
else OUTPUT_OPTIONS_DEFAULT_STDOUT_REDIRECTED)
|
||||
|
||||
unknown_output_options = set(self.args.output_options) - OUTPUT_OPTIONS
|
||||
if unknown_output_options:
|
||||
self.error(
|
||||
'Unknown output options: %s' % ','.join(unknown_output_options)
|
||||
)
|
||||
unknown = set(args.output_options) - OUTPUT_OPTIONS
|
||||
if unknown:
|
||||
self.error('Unknown output options: %s' % ','.join(unknown))
|
||||
|
||||
if self.args.download and OUT_RESP_BODY in self.args.output_options:
|
||||
# Response body is always downloaded with --download and it goes
|
||||
# through a different routine, so we remove it.
|
||||
self.args.output_options = str(
|
||||
set(self.args.output_options) - set(OUT_RESP_BODY))
|
||||
|
||||
def _process_pretty_options(self):
|
||||
if self.args.prettify == PRETTY_STDOUT_TTY_ONLY:
|
||||
self.args.prettify = PRETTY_MAP[
|
||||
'all' if self.env.stdout_isatty else 'none']
|
||||
elif self.args.prettify and self.env.is_windows:
|
||||
def _process_pretty_options(self, args, env):
|
||||
if args.prettify == PRETTY_STDOUT_TTY_ONLY:
|
||||
args.prettify = PRETTY_MAP['all' if env.stdout_isatty else 'none']
|
||||
elif args.prettify and env.is_windows:
|
||||
self.error('Only terminal output can be colorized on Windows.')
|
||||
else:
|
||||
# noinspection PyTypeChecker
|
||||
self.args.prettify = PRETTY_MAP[self.args.prettify]
|
||||
|
||||
def _validate_download_options(self):
|
||||
if not self.args.download:
|
||||
if self.args.download_resume:
|
||||
self.error('--continue only works with --download')
|
||||
if self.args.download_resume and not (
|
||||
self.args.download and self.args.output_file):
|
||||
self.error('--continue requires --output to be specified')
|
||||
args.prettify = PRETTY_MAP[args.prettify]
|
||||
|
||||
|
||||
class ParseError(Exception):
|
||||
@ -374,19 +262,6 @@ class KeyValue(object):
|
||||
return self.__dict__ == other.__dict__
|
||||
|
||||
|
||||
class SessionNameValidator(object):
|
||||
|
||||
def __init__(self, error_message):
|
||||
self.error_message = error_message
|
||||
|
||||
def __call__(self, value):
|
||||
# Session name can be a path or just a name.
|
||||
if (os.path.sep not in value
|
||||
and not VALID_SESSION_NAME_PATTERN.search(value)):
|
||||
raise ArgumentError(None, self.error_message)
|
||||
return value
|
||||
|
||||
|
||||
class KeyValueArgType(object):
|
||||
"""A key-value pair argument type used with `argparse`.
|
||||
|
||||
@ -528,6 +403,9 @@ class ParamDict(OrderedDict):
|
||||
data and URL params.
|
||||
|
||||
"""
|
||||
# NOTE: Won't work when used for form data with multiple values
|
||||
# for a field and a file field is present:
|
||||
# https://github.com/kennethreitz/requests/issues/737
|
||||
if key not in self:
|
||||
super(ParamDict, self).__setitem__(key, value)
|
||||
else:
|
||||
|
||||
30
httpie/manage.py
Normal file
30
httpie/manage.py
Normal file
@ -0,0 +1,30 @@
|
||||
"""
|
||||
Provides the `httpie' management command.
|
||||
|
||||
Note that the main `http' command points to `httpie.__main__.main()`.
|
||||
|
||||
"""
|
||||
import argparse
|
||||
|
||||
from . import sessions
|
||||
from . import __version__
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description='The HTTPie management command.',
|
||||
version=__version__
|
||||
)
|
||||
subparsers = parser.add_subparsers()
|
||||
|
||||
|
||||
# Only sessions as of now.
|
||||
sessions.add_commands(subparsers)
|
||||
|
||||
|
||||
def main():
|
||||
args = parser.parse_args()
|
||||
args.command(args)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@ -1,8 +1,8 @@
|
||||
import os
|
||||
import sys
|
||||
from requests.compat import urlparse, is_windows, bytes, str
|
||||
|
||||
from .config import DEFAULT_CONFIG_DIR, Config
|
||||
from .compat import urlsplit, is_windows, bytes, str
|
||||
|
||||
|
||||
class Environment(object):
|
||||
@ -13,32 +13,29 @@ class Environment(object):
|
||||
|
||||
"""
|
||||
|
||||
#noinspection PyUnresolvedReferences
|
||||
is_windows = is_windows
|
||||
|
||||
progname = os.path.basename(sys.argv[0])
|
||||
if progname not in ['http', 'https']:
|
||||
progname = 'http'
|
||||
|
||||
stdin_isatty = sys.stdin.isatty()
|
||||
stdin = sys.stdin
|
||||
stdout_isatty = sys.stdout.isatty()
|
||||
|
||||
config_dir = DEFAULT_CONFIG_DIR
|
||||
|
||||
# Can be set to 0 to disable colors completely.
|
||||
colors = 256 if '256color' in os.environ.get('TERM', '') else 88
|
||||
|
||||
stdin = sys.stdin
|
||||
stdin_isatty = sys.stdin.isatty()
|
||||
|
||||
stdout_isatty = sys.stdout.isatty()
|
||||
stderr_isatty = sys.stderr.isatty()
|
||||
if is_windows:
|
||||
# noinspection PyUnresolvedReferences
|
||||
if stdout_isatty and is_windows:
|
||||
from colorama.initialise import wrap_stream
|
||||
stdout = wrap_stream(sys.stdout, convert=None,
|
||||
strip=None, autoreset=True, wrap=True)
|
||||
stderr = wrap_stream(sys.stderr, convert=None,
|
||||
strip=None, autoreset=True, wrap=True)
|
||||
else:
|
||||
stdout = sys.stdout
|
||||
stderr = sys.stderr
|
||||
stderr = sys.stderr
|
||||
|
||||
# Can be set to 0 to disable colors completely.
|
||||
colors = 256 if '256color' in os.environ.get('TERM', '') else 88
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
assert all(hasattr(type(self), attr)
|
||||
@ -88,13 +85,7 @@ class HTTPMessage(object):
|
||||
@property
|
||||
def content_type(self):
|
||||
"""Return the message content type."""
|
||||
ct = self._orig.headers.get(
|
||||
b'Content-Type',
|
||||
self._orig.headers.get(
|
||||
'Content-Type',
|
||||
''
|
||||
)
|
||||
)
|
||||
ct = self._orig.headers.get('Content-Type', '')
|
||||
if isinstance(ct, bytes):
|
||||
ct = ct.decode()
|
||||
return ct
|
||||
@ -152,18 +143,33 @@ class HTTPRequest(HTTPMessage):
|
||||
|
||||
@property
|
||||
def headers(self):
|
||||
url = urlsplit(self._orig.url)
|
||||
"""Return Request-Line"""
|
||||
url = urlparse(self._orig.url)
|
||||
|
||||
# Querystring
|
||||
qs = ''
|
||||
if url.query or self._orig.params:
|
||||
qs = '?'
|
||||
if url.query:
|
||||
qs += url.query
|
||||
# Requests doesn't make params part of ``request.url``.
|
||||
if self._orig.params:
|
||||
if url.query:
|
||||
qs += '&'
|
||||
#noinspection PyUnresolvedReferences
|
||||
qs += type(self._orig)._encode_params(self._orig.params)
|
||||
|
||||
# Request-Line
|
||||
request_line = '{method} {path}{query} HTTP/1.1'.format(
|
||||
method=self._orig.method,
|
||||
path=url.path or '/',
|
||||
query='?' + url.query if url.query else ''
|
||||
query=qs
|
||||
)
|
||||
|
||||
headers = dict(self._orig.headers)
|
||||
|
||||
if 'Host' not in headers:
|
||||
headers['Host'] = url.netloc
|
||||
headers['Host'] = urlparse(self._orig.url).netloc
|
||||
|
||||
headers = ['%s: %s' % (name, value)
|
||||
for name, value in headers.items()]
|
||||
@ -178,8 +184,26 @@ class HTTPRequest(HTTPMessage):
|
||||
|
||||
@property
|
||||
def body(self):
|
||||
body = self._orig.body
|
||||
if isinstance(body, str):
|
||||
# Happens with JSON/form request data parsed from the command line.
|
||||
body = body.encode('utf8')
|
||||
return body or b''
|
||||
"""Reconstruct and return the original request body bytes."""
|
||||
if self._orig.files:
|
||||
# TODO: would be nice if we didn't need to encode the files again
|
||||
# FIXME: Also the boundary header doesn't match the one used.
|
||||
for fn, fd in self._orig.files.values():
|
||||
# Rewind the files as they have already been read before.
|
||||
fd.seek(0)
|
||||
body, _ = self._orig._encode_files(self._orig.files)
|
||||
else:
|
||||
try:
|
||||
body = self._orig.data
|
||||
except AttributeError:
|
||||
# requests < 0.12.1
|
||||
body = self._orig._enc_data
|
||||
|
||||
if isinstance(body, dict):
|
||||
#noinspection PyUnresolvedReferences
|
||||
body = type(self._orig)._encode_params(body)
|
||||
|
||||
if isinstance(body, str):
|
||||
body = body.encode('utf8')
|
||||
|
||||
return body
|
||||
|
||||
@ -2,7 +2,6 @@
|
||||
|
||||
"""
|
||||
import json
|
||||
import xml.dom.minidom
|
||||
from functools import partial
|
||||
from itertools import chain
|
||||
|
||||
@ -13,17 +12,14 @@ from pygments.lexers import get_lexer_for_mimetype, get_lexer_by_name
|
||||
from pygments.formatters.terminal import TerminalFormatter
|
||||
from pygments.formatters.terminal256 import Terminal256Formatter
|
||||
from pygments.util import ClassNotFound
|
||||
from requests.compat import is_windows
|
||||
|
||||
from .compat import is_windows
|
||||
from .solarized import Solarized256Style
|
||||
from .models import HTTPRequest, HTTPResponse, Environment
|
||||
from .input import (OUT_REQ_BODY, OUT_REQ_HEAD,
|
||||
OUT_RESP_HEAD, OUT_RESP_BODY)
|
||||
|
||||
|
||||
# The default number of spaces to indent when pretty printing
|
||||
DEFAULT_INDENT = 4
|
||||
|
||||
# Colors on Windows via colorama don't look that
|
||||
# great and fruity seems to give the best result there.
|
||||
AVAILABLE_STYLES = set(STYLE_MAP.keys())
|
||||
@ -65,7 +61,7 @@ def write(stream, outfile, flush):
|
||||
outfile.flush()
|
||||
|
||||
|
||||
def write_with_colors_win_py3(stream, outfile, flush):
|
||||
def write_with_colors_win_p3k(stream, outfile, flush):
|
||||
"""Like `write`, but colorized chunks are written as text
|
||||
directly to `outfile` to ensure it gets processed by colorama.
|
||||
Applies only to Windows with Python 3 and colorized terminal output.
|
||||
@ -82,21 +78,23 @@ def write_with_colors_win_py3(stream, outfile, flush):
|
||||
outfile.flush()
|
||||
|
||||
|
||||
def build_output_stream(args, env, request, response):
|
||||
def output_stream(args, env, request, response):
|
||||
"""Build and return a chain of iterators over the `request`-`response`
|
||||
exchange each of which yields `bytes` chunks.
|
||||
|
||||
"""
|
||||
|
||||
Stream = make_stream(env, args)
|
||||
|
||||
req_h = OUT_REQ_HEAD in args.output_options
|
||||
req_b = OUT_REQ_BODY in args.output_options
|
||||
resp_h = OUT_RESP_HEAD in args.output_options
|
||||
resp_b = OUT_RESP_BODY in args.output_options
|
||||
|
||||
req = req_h or req_b
|
||||
resp = resp_h or resp_b
|
||||
|
||||
output = []
|
||||
Stream = get_stream_type(env, args)
|
||||
|
||||
if req:
|
||||
output.append(Stream(
|
||||
@ -122,7 +120,7 @@ def build_output_stream(args, env, request, response):
|
||||
return chain(*output)
|
||||
|
||||
|
||||
def get_stream_type(env, args):
|
||||
def make_stream(env, args):
|
||||
"""Pick the right stream type based on `env` and `args`.
|
||||
Wrap it in a partial with the type-specific args so that
|
||||
we don't need to think what stream we are dealing with.
|
||||
@ -149,42 +147,37 @@ def get_stream_type(env, args):
|
||||
|
||||
|
||||
class BaseStream(object):
|
||||
"""Base HTTP message output stream class."""
|
||||
"""Base HTTP message stream class."""
|
||||
|
||||
def __init__(self, msg, with_headers=True, with_body=True,
|
||||
on_body_chunk_downloaded=None):
|
||||
def __init__(self, msg, with_headers=True, with_body=True):
|
||||
"""
|
||||
:param msg: a :class:`models.HTTPMessage` subclass
|
||||
:param with_headers: if `True`, headers will be included
|
||||
:param with_body: if `True`, body will be included
|
||||
|
||||
"""
|
||||
assert with_headers or with_body
|
||||
self.msg = msg
|
||||
self.with_headers = with_headers
|
||||
self.with_body = with_body
|
||||
self.on_body_chunk_downloaded = on_body_chunk_downloaded
|
||||
|
||||
def _get_headers(self):
|
||||
def _headers(self):
|
||||
"""Return the headers' bytes."""
|
||||
return self.msg.headers.encode('ascii')
|
||||
|
||||
def _iter_body(self):
|
||||
def _body(self):
|
||||
"""Return an iterator over the message body."""
|
||||
raise NotImplementedError()
|
||||
|
||||
def __iter__(self):
|
||||
"""Return an iterator over `self.msg`."""
|
||||
if self.with_headers:
|
||||
yield self._get_headers()
|
||||
yield self._headers()
|
||||
yield b'\r\n\r\n'
|
||||
|
||||
if self.with_body:
|
||||
try:
|
||||
for chunk in self._iter_body():
|
||||
for chunk in self._body():
|
||||
yield chunk
|
||||
if self.on_body_chunk_downloaded:
|
||||
self.on_body_chunk_downloaded(chunk)
|
||||
except BinarySuppressedError as e:
|
||||
if self.with_headers:
|
||||
yield b'\n'
|
||||
@ -195,13 +188,13 @@ class RawStream(BaseStream):
|
||||
"""The message is streamed in chunks with no processing."""
|
||||
|
||||
CHUNK_SIZE = 1024 * 100
|
||||
CHUNK_SIZE_BY_LINE = 1
|
||||
CHUNK_SIZE_BY_LINE = 1024 * 5
|
||||
|
||||
def __init__(self, chunk_size=CHUNK_SIZE, **kwargs):
|
||||
super(RawStream, self).__init__(**kwargs)
|
||||
self.chunk_size = chunk_size
|
||||
|
||||
def _iter_body(self):
|
||||
def _body(self):
|
||||
return self.msg.iter_body(self.chunk_size)
|
||||
|
||||
|
||||
@ -213,7 +206,7 @@ class EncodedStream(BaseStream):
|
||||
is suppressed. The body is always streamed by line.
|
||||
|
||||
"""
|
||||
CHUNK_SIZE = 1
|
||||
CHUNK_SIZE = 1024 * 5
|
||||
|
||||
def __init__(self, env=Environment(), **kwargs):
|
||||
|
||||
@ -229,7 +222,7 @@ class EncodedStream(BaseStream):
|
||||
# Default to utf8 when unsure.
|
||||
self.output_encoding = output_encoding or 'utf8'
|
||||
|
||||
def _iter_body(self):
|
||||
def _body(self):
|
||||
|
||||
for line, lf in self.msg.iter_lines(self.CHUNK_SIZE):
|
||||
|
||||
@ -249,17 +242,17 @@ class PrettyStream(EncodedStream):
|
||||
|
||||
"""
|
||||
|
||||
CHUNK_SIZE = 1
|
||||
CHUNK_SIZE = 1024 * 5
|
||||
|
||||
def __init__(self, processor, **kwargs):
|
||||
super(PrettyStream, self).__init__(**kwargs)
|
||||
self.processor = processor
|
||||
|
||||
def _get_headers(self):
|
||||
def _headers(self):
|
||||
return self.processor.process_headers(
|
||||
self.msg.headers).encode(self.output_encoding)
|
||||
|
||||
def _iter_body(self):
|
||||
def _body(self):
|
||||
for line, lf in self.msg.iter_lines(self.CHUNK_SIZE):
|
||||
if b'\0' in line:
|
||||
raise BinarySuppressedError()
|
||||
@ -268,9 +261,8 @@ class PrettyStream(EncodedStream):
|
||||
def _process_body(self, chunk):
|
||||
return (self.processor
|
||||
.process_body(
|
||||
content=chunk.decode(self.msg.encoding, 'replace'),
|
||||
content_type=self.msg.content_type,
|
||||
encoding=self.msg.encoding)
|
||||
chunk.decode(self.msg.encoding, 'replace'),
|
||||
self.msg.content_type)
|
||||
.encode(self.output_encoding, 'replace'))
|
||||
|
||||
|
||||
@ -284,8 +276,9 @@ class BufferedPrettyStream(PrettyStream):
|
||||
|
||||
CHUNK_SIZE = 1024 * 10
|
||||
|
||||
def _iter_body(self):
|
||||
def _body(self):
|
||||
|
||||
#noinspection PyArgumentList
|
||||
# Read the whole body before prettifying it,
|
||||
# but bail out immediately if the body is binary.
|
||||
body = bytearray()
|
||||
@ -372,13 +365,12 @@ class BaseProcessor(object):
|
||||
"""
|
||||
return headers
|
||||
|
||||
def process_body(self, content, content_type, subtype, encoding):
|
||||
def process_body(self, content, content_type, subtype):
|
||||
"""Return processed `content`.
|
||||
|
||||
:param content: The body content as text
|
||||
:param content_type: Full content type, e.g., 'application/atom+xml'.
|
||||
:param subtype: E.g. 'xml'.
|
||||
:param encoding: The original content encoding.
|
||||
|
||||
"""
|
||||
return content
|
||||
@ -387,7 +379,7 @@ class BaseProcessor(object):
|
||||
class JSONProcessor(BaseProcessor):
|
||||
"""JSON body processor."""
|
||||
|
||||
def process_body(self, content, content_type, subtype, encoding):
|
||||
def process_body(self, content, content_type, subtype):
|
||||
if subtype == 'json':
|
||||
try:
|
||||
# Indent the JSON data, sort keys by name, and
|
||||
@ -395,29 +387,13 @@ class JSONProcessor(BaseProcessor):
|
||||
content = json.dumps(json.loads(content),
|
||||
sort_keys=True,
|
||||
ensure_ascii=False,
|
||||
indent=DEFAULT_INDENT)
|
||||
indent=4)
|
||||
except ValueError:
|
||||
# Invalid JSON but we don't care.
|
||||
pass
|
||||
return content
|
||||
|
||||
|
||||
class XMLProcessor(BaseProcessor):
|
||||
"""XML body processor."""
|
||||
# TODO: tests
|
||||
|
||||
def process_body(self, content, content_type, subtype, encoding):
|
||||
if subtype == 'xml':
|
||||
try:
|
||||
# Pretty print the XML
|
||||
doc = xml.dom.minidom.parseString(content.encode(encoding))
|
||||
content = doc.toprettyxml(indent=' ' * DEFAULT_INDENT)
|
||||
except xml.parsers.expat.ExpatError:
|
||||
# Ignore invalid XML errors (skips attempting to pretty print)
|
||||
pass
|
||||
return content
|
||||
|
||||
|
||||
class PygmentsProcessor(BaseProcessor):
|
||||
"""A processor that applies syntax-highlighting using Pygments
|
||||
to the headers, and to the body as well if its content type is recognized.
|
||||
@ -449,7 +425,7 @@ class PygmentsProcessor(BaseProcessor):
|
||||
return pygments.highlight(
|
||||
headers, HTTPLexer(), self.formatter).strip()
|
||||
|
||||
def process_body(self, content, content_type, subtype, encoding):
|
||||
def process_body(self, content, content_type, subtype):
|
||||
try:
|
||||
lexer = self.lexers_by_type.get(content_type)
|
||||
if not lexer:
|
||||
@ -482,8 +458,7 @@ class OutputProcessor(object):
|
||||
installed_processors = {
|
||||
'format': [
|
||||
HeadersProcessor,
|
||||
JSONProcessor,
|
||||
XMLProcessor
|
||||
JSONProcessor
|
||||
],
|
||||
'colors': [
|
||||
PygmentsProcessor
|
||||
@ -509,18 +484,13 @@ class OutputProcessor(object):
|
||||
headers = processor.process_headers(headers)
|
||||
return headers
|
||||
|
||||
def process_body(self, content, content_type, encoding):
|
||||
def process_body(self, content, content_type):
|
||||
# e.g., 'application/atom+xml'
|
||||
content_type = content_type.split(';')[0]
|
||||
# e.g., 'xml'
|
||||
subtype = content_type.split('/')[-1].split('+')[-1]
|
||||
|
||||
for processor in self.processors:
|
||||
content = processor.process_body(
|
||||
content,
|
||||
content_type,
|
||||
subtype,
|
||||
encoding
|
||||
)
|
||||
content = processor.process_body(content, content_type, subtype)
|
||||
|
||||
return content
|
||||
|
||||
@ -1,116 +1,116 @@
|
||||
"""Persistent, JSON-serialized sessions.
|
||||
|
||||
"""
|
||||
import re
|
||||
import os
|
||||
import sys
|
||||
import glob
|
||||
import errno
|
||||
import codecs
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
import requests
|
||||
from requests.compat import urlparse
|
||||
from requests.cookies import RequestsCookieJar, create_cookie
|
||||
from requests.auth import HTTPBasicAuth, HTTPDigestAuth
|
||||
from argparse import OPTIONAL
|
||||
|
||||
from .compat import urlsplit
|
||||
from .config import BaseConfigDict, DEFAULT_CONFIG_DIR
|
||||
from .output import PygmentsProcessor
|
||||
|
||||
|
||||
SESSIONS_DIR_NAME = 'sessions'
|
||||
DEFAULT_SESSIONS_DIR = os.path.join(DEFAULT_CONFIG_DIR, SESSIONS_DIR_NAME)
|
||||
VALID_SESSION_NAME_PATTERN = re.compile('^[a-zA-Z0-9_.-]+$')
|
||||
# Request headers starting with these prefixes won't be stored in sessions.
|
||||
# They are specific to each request.
|
||||
# http://en.wikipedia.org/wiki/List_of_HTTP_header_fields#Requests
|
||||
SESSION_IGNORED_HEADER_PREFIXES = ['Content-', 'If-']
|
||||
|
||||
|
||||
def get_response(session_name, requests_kwargs, config_dir, read_only=False):
|
||||
def get_response(name, request_kwargs, config_dir, read_only=False):
|
||||
"""Like `client.get_response`, but applies permanent
|
||||
aspects of the session to the request.
|
||||
|
||||
"""
|
||||
if os.path.sep in session_name:
|
||||
path = os.path.expanduser(session_name)
|
||||
else:
|
||||
hostname = (
|
||||
requests_kwargs['headers'].get('Host', None)
|
||||
or urlsplit(requests_kwargs['url']).netloc.split('@')[-1]
|
||||
)
|
||||
sessions_dir = os.path.join(config_dir, SESSIONS_DIR_NAME)
|
||||
host = Host(
|
||||
root_dir=sessions_dir,
|
||||
name=request_kwargs['headers'].get('Host', None)
|
||||
or urlparse(request_kwargs['url']).netloc.split('@')[-1]
|
||||
)
|
||||
|
||||
assert re.match('^[a-zA-Z0-9_.:-]+$', hostname)
|
||||
|
||||
# host:port => host_port
|
||||
hostname = hostname.replace(':', '_')
|
||||
path = os.path.join(config_dir,
|
||||
SESSIONS_DIR_NAME,
|
||||
hostname,
|
||||
session_name + '.json')
|
||||
|
||||
session = Session(path)
|
||||
session = Session(host, name)
|
||||
session.load()
|
||||
|
||||
request_headers = requests_kwargs.get('headers', {})
|
||||
requests_kwargs['headers'] = dict(session.headers, **request_headers)
|
||||
session.update_headers(request_headers)
|
||||
# Update session headers with the request headers.
|
||||
session['headers'].update(request_kwargs.get('headers', {}))
|
||||
# Use the merged headers for the request
|
||||
request_kwargs['headers'] = session['headers']
|
||||
|
||||
auth = requests_kwargs.get('auth', None)
|
||||
auth = request_kwargs.get('auth', None)
|
||||
if auth:
|
||||
session.auth = auth
|
||||
elif session.auth:
|
||||
requests_kwargs['auth'] = session.auth
|
||||
|
||||
requests_session = requests.Session()
|
||||
requests_session.cookies = session.cookies
|
||||
request_kwargs['auth'] = session.auth
|
||||
|
||||
rsession = requests.Session(cookies=session.cookies)
|
||||
try:
|
||||
response = requests_session.request(**requests_kwargs)
|
||||
response = rsession.request(**request_kwargs)
|
||||
except Exception:
|
||||
raise
|
||||
else:
|
||||
# Existing sessions with `read_only=True` don't get updated.
|
||||
if session.is_new or not read_only:
|
||||
session.cookies = requests_session.cookies
|
||||
session.cookies = rsession.cookies
|
||||
session.save()
|
||||
return response
|
||||
|
||||
|
||||
class Session(BaseConfigDict):
|
||||
helpurl = 'https://github.com/jkbr/httpie#sessions'
|
||||
about = 'HTTPie session file'
|
||||
class Host(object):
|
||||
"""A host is a per-host directory on the disk containing sessions files."""
|
||||
|
||||
def __init__(self, path, *args, **kwargs):
|
||||
super(Session, self).__init__(*args, **kwargs)
|
||||
self._path = path
|
||||
self['headers'] = {}
|
||||
self['cookies'] = {}
|
||||
self['auth'] = {
|
||||
'type': None,
|
||||
'username': None,
|
||||
'password': None
|
||||
}
|
||||
def __init__(self, name, root_dir=DEFAULT_CONFIG_DIR):
|
||||
self.name = name
|
||||
self.root_dir = root_dir
|
||||
|
||||
def _get_path(self):
|
||||
return self._path
|
||||
def __iter__(self):
|
||||
"""Return a iterator yielding `(session_name, session_path)`."""
|
||||
for fn in sorted(glob.glob1(self.path, '*.json')):
|
||||
yield os.path.splitext(fn)[0], os.path.join(self.path, fn)
|
||||
|
||||
def update_headers(self, request_headers):
|
||||
"""
|
||||
Update the session headers with the request ones while ignoring
|
||||
certain name prefixes.
|
||||
|
||||
:type request_headers: dict
|
||||
|
||||
"""
|
||||
for name, value in request_headers.items():
|
||||
|
||||
if name == 'User-Agent' and value.startswith('HTTPie/'):
|
||||
continue
|
||||
|
||||
for prefix in SESSION_IGNORED_HEADER_PREFIXES:
|
||||
if name.lower().startswith(prefix.lower()):
|
||||
break
|
||||
else:
|
||||
self['headers'][name] = value
|
||||
def delete(self):
|
||||
shutil.rmtree(self.path)
|
||||
|
||||
@property
|
||||
def headers(self):
|
||||
return self['headers']
|
||||
def path(self):
|
||||
# Name will include ':' if a port is specified, which is invalid
|
||||
# on windows. DNS does not allow '_' in a domain, or for it to end
|
||||
# in a number (I think?)
|
||||
path = os.path.join(self.root_dir, self.name.replace(':', '_'))
|
||||
try:
|
||||
os.makedirs(path, mode=0o700)
|
||||
except OSError as e:
|
||||
if e.errno != errno.EEXIST:
|
||||
raise
|
||||
return path
|
||||
|
||||
@classmethod
|
||||
def all(cls):
|
||||
"""Return a generator yielding a host at a time."""
|
||||
for name in sorted(glob.glob1(DEFAULT_SESSIONS_DIR, '*')):
|
||||
if os.path.isdir(os.path.join(DEFAULT_SESSIONS_DIR, name)):
|
||||
yield Host(name)
|
||||
|
||||
|
||||
class Session(BaseConfigDict):
|
||||
""""""
|
||||
|
||||
def __init__(self, host, name, *args, **kwargs):
|
||||
super(Session, self).__init__(*args, **kwargs)
|
||||
self.host = host
|
||||
self.name = name
|
||||
self['headers'] = {}
|
||||
self['cookies'] = {}
|
||||
|
||||
@property
|
||||
def directory(self):
|
||||
return self.host.path
|
||||
|
||||
@property
|
||||
def cookies(self):
|
||||
@ -123,23 +123,26 @@ class Session(BaseConfigDict):
|
||||
|
||||
@cookies.setter
|
||||
def cookies(self, jar):
|
||||
"""
|
||||
:type jar: CookieJar
|
||||
"""
|
||||
# http://docs.python.org/2/library/cookielib.html#cookie-objects
|
||||
stored_attrs = ['value', 'path', 'secure', 'expires']
|
||||
excluded = [
|
||||
'_rest', 'name', 'port_specified',
|
||||
'domain_specified', 'domain_initial_dot',
|
||||
'path_specified', 'comment', 'comment_url'
|
||||
]
|
||||
self['cookies'] = {}
|
||||
for cookie in jar:
|
||||
self['cookies'][cookie.name] = dict(
|
||||
(attname, getattr(cookie, attname))
|
||||
for attname in stored_attrs
|
||||
)
|
||||
for host in jar._cookies.values():
|
||||
for path in host.values():
|
||||
for name, cookie in path.items():
|
||||
cookie_dict = {}
|
||||
for k, v in cookie.__dict__.items():
|
||||
if k not in excluded:
|
||||
cookie_dict[k] = v
|
||||
self['cookies'][name] = cookie_dict
|
||||
|
||||
@property
|
||||
def auth(self):
|
||||
auth = self.get('auth', None)
|
||||
if not auth or not auth['type']:
|
||||
return
|
||||
if not auth:
|
||||
return None
|
||||
Auth = {'basic': HTTPBasicAuth,
|
||||
'digest': HTTPDigestAuth}[auth['type']]
|
||||
return Auth(auth['username'], auth['password'])
|
||||
@ -152,3 +155,79 @@ class Session(BaseConfigDict):
|
||||
'username': cred.username,
|
||||
'password': cred.password,
|
||||
}
|
||||
|
||||
|
||||
# The commands are disabled for now.
|
||||
# TODO: write tests for the commands.
|
||||
|
||||
def list_command(args):
|
||||
if args.host:
|
||||
for name, path in Host(args.host):
|
||||
print(name + ' [' + path + ']')
|
||||
else:
|
||||
for host in Host.all():
|
||||
print(host.name)
|
||||
for name, path in host:
|
||||
print(' ' + name + ' [' + path + ']')
|
||||
|
||||
|
||||
def show_command(args):
|
||||
path = Session(Host(args.host), args.name).path
|
||||
if not os.path.exists(path):
|
||||
sys.stderr.write('Session "%s" does not exist [%s].\n'
|
||||
% (args.name, path))
|
||||
sys.exit(1)
|
||||
|
||||
with codecs.open(path, encoding='utf8') as f:
|
||||
print(path + ':\n')
|
||||
proc = PygmentsProcessor()
|
||||
print(proc.process_body(f.read(), 'application/json', 'json'))
|
||||
print('')
|
||||
|
||||
|
||||
def delete_command(args):
|
||||
host = Host(args.host)
|
||||
if not args.name:
|
||||
host.delete()
|
||||
else:
|
||||
Session(host, args.name).delete()
|
||||
|
||||
|
||||
def edit_command(args):
|
||||
editor = os.environ.get('EDITOR', None)
|
||||
if not editor:
|
||||
sys.stderr.write(
|
||||
'You need to configure the environment variable EDITOR.\n')
|
||||
sys.exit(1)
|
||||
command = editor.split()
|
||||
command.append(Session(Host(args.host), args.name).path)
|
||||
subprocess.call(command)
|
||||
|
||||
|
||||
def add_commands(subparsers):
|
||||
|
||||
# List
|
||||
list_ = subparsers.add_parser('session-list', help='list sessions')
|
||||
list_.set_defaults(command=list_command)
|
||||
list_.add_argument('host', nargs=OPTIONAL)
|
||||
|
||||
# Show
|
||||
show = subparsers.add_parser('session-show', help='show a session')
|
||||
show.set_defaults(command=show_command)
|
||||
show.add_argument('host')
|
||||
show.add_argument('name')
|
||||
|
||||
# Edit
|
||||
edit = subparsers.add_parser(
|
||||
'session-edit', help='edit a session in $EDITOR')
|
||||
edit.set_defaults(command=edit_command)
|
||||
edit.add_argument('host')
|
||||
edit.add_argument('name')
|
||||
|
||||
# Delete
|
||||
delete = subparsers.add_parser('session-delete', help='delete a session')
|
||||
delete.set_defaults(command=delete_command)
|
||||
delete.add_argument('host')
|
||||
delete.add_argument('name', nargs=OPTIONAL,
|
||||
help='The name of the session to be deleted.'
|
||||
' If not specified, all host sessions are deleted.')
|
||||
|
||||
@ -1,46 +0,0 @@
|
||||
from __future__ import division
|
||||
|
||||
|
||||
def humanize_bytes(n, precision=2):
|
||||
# Author: Doug Latornell
|
||||
# Licence: MIT
|
||||
# URL: http://code.activestate.com/recipes/577081/
|
||||
"""Return a humanized string representation of a number of bytes.
|
||||
|
||||
Assumes `from __future__ import division`.
|
||||
|
||||
>>> humanize_bytes(1)
|
||||
'1 byte'
|
||||
>>> humanize_bytes(1024)
|
||||
'1.0 kB'
|
||||
>>> humanize_bytes(1024 * 123)
|
||||
'123.0 kB'
|
||||
>>> humanize_bytes(1024 * 12342)
|
||||
'12.1 MB'
|
||||
>>> humanize_bytes(1024 * 12342, 2)
|
||||
'12.05 MB'
|
||||
>>> humanize_bytes(1024 * 1234, 2)
|
||||
'1.21 MB'
|
||||
>>> humanize_bytes(1024 * 1234 * 1111, 2)
|
||||
'1.31 GB'
|
||||
>>> humanize_bytes(1024 * 1234 * 1111, 1)
|
||||
'1.3 GB'
|
||||
|
||||
"""
|
||||
abbrevs = [
|
||||
(1 << 50, 'PB'),
|
||||
(1 << 40, 'TB'),
|
||||
(1 << 30, 'GB'),
|
||||
(1 << 20, 'MB'),
|
||||
(1 << 10, 'kB'),
|
||||
(1, 'B')
|
||||
]
|
||||
|
||||
if n == 1:
|
||||
return '1 B'
|
||||
|
||||
for factor, suffix in abbrevs:
|
||||
if n >= factor:
|
||||
break
|
||||
|
||||
return '%.*f %s' % (precision, n / factor, suffix)
|
||||
@ -1,3 +0,0 @@
|
||||
tox
|
||||
git+git://github.com/kennethreitz/httpbin.git@7c96875e87a448f08fb1981e85eb79e77d592d98
|
||||
docutils
|
||||
5
setup.py
5
setup.py
@ -12,7 +12,8 @@ if sys.argv[-1] == 'test':
|
||||
|
||||
|
||||
requirements = [
|
||||
'requests>=1.2.3',
|
||||
# Debian has only requests==0.10.1 and httpie.deb depends on that.
|
||||
'requests>=0.10.1',
|
||||
'Pygments>=1.5'
|
||||
]
|
||||
if sys.version_info[:2] in ((2, 6), (3, 1)):
|
||||
@ -46,6 +47,8 @@ setup(
|
||||
entry_points={
|
||||
'console_scripts': [
|
||||
'http = httpie.__main__:main',
|
||||
# Not ready yet.
|
||||
# 'httpie = httpie.manage:main',
|
||||
],
|
||||
},
|
||||
install_requires=requirements,
|
||||
|
||||
430
tests/tests.py
430
tests/tests.py
@ -19,6 +19,7 @@ To make it run faster and offline you can::
|
||||
HTTPBIN_URL=http://localhost:5000 tox
|
||||
|
||||
"""
|
||||
from functools import partial
|
||||
import subprocess
|
||||
import os
|
||||
import sys
|
||||
@ -27,20 +28,16 @@ import argparse
|
||||
import tempfile
|
||||
import unittest
|
||||
import shutil
|
||||
import time
|
||||
from requests.structures import CaseInsensitiveDict
|
||||
|
||||
from requests.compat import urlparse
|
||||
try:
|
||||
from urllib.request import urlopen
|
||||
except ImportError:
|
||||
# noinspection PyUnresolvedReferences
|
||||
from urllib2 import urlopen
|
||||
try:
|
||||
from unittest import skipIf, skip
|
||||
except ImportError:
|
||||
skip = lambda msg: lambda self: None
|
||||
|
||||
# noinspection PyUnusedLocal
|
||||
def skipIf(cond, reason):
|
||||
def decorator(test_method):
|
||||
if cond:
|
||||
@ -49,6 +46,7 @@ except ImportError:
|
||||
return decorator
|
||||
|
||||
from requests import __version__ as requests_version
|
||||
from requests.compat import is_windows, is_py26, bytes, str
|
||||
|
||||
|
||||
#################################################################
|
||||
@ -59,21 +57,12 @@ from requests import __version__ as requests_version
|
||||
TESTS_ROOT = os.path.abspath(os.path.dirname(__file__))
|
||||
sys.path.insert(0, os.path.realpath(os.path.join(TESTS_ROOT, '..')))
|
||||
|
||||
from httpie import ExitStatus
|
||||
from httpie import exit
|
||||
from httpie import input
|
||||
from httpie.models import Environment
|
||||
from httpie.core import main
|
||||
from httpie.output import BINARY_SUPPRESSED_NOTICE
|
||||
from httpie.input import ParseError
|
||||
from httpie.compat import is_windows, is_py26, bytes, str
|
||||
from httpie.downloads import (
|
||||
parse_content_range,
|
||||
filename_from_content_disposition,
|
||||
filename_from_url,
|
||||
get_unique_filename,
|
||||
ContentRangeError,
|
||||
Download,
|
||||
)
|
||||
|
||||
|
||||
CRLF = '\r\n'
|
||||
@ -143,7 +132,6 @@ class TestEnvironment(Environment):
|
||||
if self.delete_config_dir:
|
||||
self._shutil.rmtree(self.config_dir)
|
||||
|
||||
|
||||
def has_docutils():
|
||||
try:
|
||||
#noinspection PyUnresolvedReferences
|
||||
@ -152,7 +140,6 @@ def has_docutils():
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
|
||||
def get_readme_errors():
|
||||
p = subprocess.Popen([
|
||||
'rst2pseudoxml.py',
|
||||
@ -167,8 +154,6 @@ def get_readme_errors():
|
||||
|
||||
class BytesResponse(bytes):
|
||||
stderr = json = exit_status = None
|
||||
|
||||
|
||||
class StrResponse(str):
|
||||
stderr = json = exit_status = None
|
||||
|
||||
@ -192,25 +177,20 @@ def http(*args, **kwargs):
|
||||
if not env:
|
||||
env = kwargs['env'] = TestEnvironment()
|
||||
|
||||
stdout = env.stdout
|
||||
stderr = env.stderr
|
||||
try:
|
||||
|
||||
try:
|
||||
exit_status = main(args=['--traceback'] + list(args), **kwargs)
|
||||
if '--download' in args:
|
||||
# Let the progress reporter thread finish.
|
||||
time.sleep(.5)
|
||||
except Exception:
|
||||
sys.stderr.write(stderr.read())
|
||||
sys.stderr.write(env.stderr.read())
|
||||
raise
|
||||
except SystemExit:
|
||||
exit_status = ExitStatus.ERROR
|
||||
exit_status = exit.ERROR
|
||||
|
||||
stdout.seek(0)
|
||||
stderr.seek(0)
|
||||
env.stdout.seek(0)
|
||||
env.stderr.seek(0)
|
||||
|
||||
output = stdout.read()
|
||||
output = env.stdout.read()
|
||||
try:
|
||||
r = StrResponse(output.decode('utf8'))
|
||||
except UnicodeDecodeError:
|
||||
@ -232,20 +212,18 @@ def http(*args, **kwargs):
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
r.stderr = stderr.read()
|
||||
r.stderr = env.stderr.read()
|
||||
r.exit_status = exit_status
|
||||
|
||||
return r
|
||||
|
||||
finally:
|
||||
stdout.close()
|
||||
stderr.close()
|
||||
env.stdout.close()
|
||||
env.stderr.close()
|
||||
|
||||
|
||||
class BaseTestCase(unittest.TestCase):
|
||||
|
||||
maxDiff = 100000
|
||||
|
||||
if is_py26:
|
||||
def assertIn(self, member, container, msg=None):
|
||||
self.assertTrue(member in container, msg)
|
||||
@ -257,15 +235,11 @@ class BaseTestCase(unittest.TestCase):
|
||||
self.assertEqual(set(d1.keys()), set(d2.keys()), msg)
|
||||
self.assertEqual(sorted(d1.values()), sorted(d2.values()), msg)
|
||||
|
||||
def assertIsNone(self, obj, msg=None):
|
||||
self.assertEqual(obj, None, msg=msg)
|
||||
|
||||
|
||||
#################################################################
|
||||
# High-level tests using httpbin.
|
||||
#################################################################
|
||||
|
||||
|
||||
class HTTPieTest(BaseTestCase):
|
||||
|
||||
def test_GET(self):
|
||||
@ -289,7 +263,7 @@ class HTTPieTest(BaseTestCase):
|
||||
'foo=bar'
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
self.assertIn(r'\"foo\": \"bar\"', r)
|
||||
self.assertIn('"foo": "bar"', r)
|
||||
|
||||
def test_POST_JSON_data(self):
|
||||
r = http(
|
||||
@ -298,7 +272,7 @@ class HTTPieTest(BaseTestCase):
|
||||
'foo=bar'
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
self.assertIn(r'\"foo\": \"bar\"', r)
|
||||
self.assertIn('"foo": "bar"', r)
|
||||
|
||||
def test_POST_form(self):
|
||||
r = http(
|
||||
@ -457,9 +431,7 @@ class AutoContentTypeAndAcceptHeadersTest(BaseTestCase):
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
self.assertEqual(r.json['headers']['Accept'], 'application/json')
|
||||
# Make sure Content-Type gets set even with no data.
|
||||
# https://github.com/jkbr/httpie/issues/137
|
||||
self.assertIn('application/json', r.json['headers']['Content-Type'])
|
||||
self.assertFalse(r.json['headers'].get('Content-Type'))
|
||||
|
||||
def test_GET_explicit_JSON_explicit_headers(self):
|
||||
r = http(
|
||||
@ -542,7 +514,7 @@ class ImplicitHTTPMethodTest(BaseTestCase):
|
||||
'hello=world'
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
self.assertIn(r'\"hello\": \"world\"', r)
|
||||
self.assertIn('"hello": "world"', r)
|
||||
|
||||
def test_implicit_POST_form(self):
|
||||
r = http(
|
||||
@ -681,8 +653,8 @@ class VerboseFlagTest(BaseTestCase):
|
||||
'baz=bar'
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
self.assertIn('"baz": "bar"', r) # request
|
||||
self.assertIn(r'\"baz\": \"bar\"', r) # response
|
||||
#noinspection PyUnresolvedReferences
|
||||
self.assertEqual(r.count('"baz": "bar"'), 2)
|
||||
|
||||
|
||||
class MultipartFormDataFileUploadTest(BaseTestCase):
|
||||
@ -805,8 +777,6 @@ class RequestBodyFromFilePathTest(BaseTestCase):
|
||||
|
||||
"""
|
||||
def test_request_body_from_file_by_path(self):
|
||||
# FIXME: *sometimes* fails on py33, the content-type is form.
|
||||
# https://github.com/jkbr/httpie/issues/140
|
||||
r = http(
|
||||
'POST',
|
||||
httpbin('/post'),
|
||||
@ -817,8 +787,6 @@ class RequestBodyFromFilePathTest(BaseTestCase):
|
||||
self.assertIn('"Content-Type": "text/plain"', r)
|
||||
|
||||
def test_request_body_from_file_by_path_with_explicit_content_type(self):
|
||||
# FIXME: *sometimes* fails on py33, the content-type is form.
|
||||
# https://github.com/jkbr/httpie/issues/140
|
||||
r = http(
|
||||
'POST',
|
||||
httpbin('/post'),
|
||||
@ -872,8 +840,8 @@ class AuthTest(BaseTestCase):
|
||||
httpbin('/digest-auth/auth/user/password')
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
self.assertIn(r'"authenticated": true', r)
|
||||
self.assertIn(r'"user": "user"', r)
|
||||
self.assertIn('"authenticated": true', r)
|
||||
self.assertIn('"user": "user"', r)
|
||||
|
||||
def test_password_prompt(self):
|
||||
|
||||
@ -890,31 +858,6 @@ class AuthTest(BaseTestCase):
|
||||
self.assertIn('"authenticated": true', r)
|
||||
self.assertIn('"user": "user"', r)
|
||||
|
||||
def test_credentials_in_url(self):
|
||||
url = httpbin('/basic-auth/user/password')
|
||||
url = 'http://user:password@' + url.split('http://', 1)[1]
|
||||
r = http(
|
||||
'GET',
|
||||
url
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
self.assertIn('"authenticated": true', r)
|
||||
self.assertIn('"user": "user"', r)
|
||||
|
||||
def test_credentials_in_url_auth_flag_has_priority(self):
|
||||
"""When credentials are passed in URL and via -a at the same time,
|
||||
then the ones from -a are used."""
|
||||
url = httpbin('/basic-auth/user/password')
|
||||
url = 'http://user:wrong_password@' + url.split('http://', 1)[1]
|
||||
r = http(
|
||||
'--auth=user:password',
|
||||
'GET',
|
||||
url
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
self.assertIn('"authenticated": true', r)
|
||||
self.assertIn('"user": "user"', r)
|
||||
|
||||
|
||||
class ExitStatusTest(BaseTestCase):
|
||||
|
||||
@ -924,7 +867,7 @@ class ExitStatusTest(BaseTestCase):
|
||||
httpbin('/status/200')
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
self.assertEqual(r.exit_status, ExitStatus.OK)
|
||||
self.assertEqual(r.exit_status, exit.OK)
|
||||
|
||||
def test_error_response_exits_0_without_check_status(self):
|
||||
r = http(
|
||||
@ -932,7 +875,7 @@ class ExitStatusTest(BaseTestCase):
|
||||
httpbin('/status/500')
|
||||
)
|
||||
self.assertIn('HTTP/1.1 500', r)
|
||||
self.assertEqual(r.exit_status, ExitStatus.OK)
|
||||
self.assertEqual(r.exit_status, exit.OK)
|
||||
self.assertTrue(not r.stderr)
|
||||
|
||||
def test_timeout_exit_status(self):
|
||||
@ -941,7 +884,7 @@ class ExitStatusTest(BaseTestCase):
|
||||
'GET',
|
||||
httpbin('/delay/1')
|
||||
)
|
||||
self.assertEqual(r.exit_status, ExitStatus.ERROR_TIMEOUT)
|
||||
self.assertEqual(r.exit_status, exit.ERROR_TIMEOUT)
|
||||
|
||||
def test_3xx_check_status_exits_3_and_stderr_when_stdout_redirected(self):
|
||||
r = http(
|
||||
@ -952,7 +895,7 @@ class ExitStatusTest(BaseTestCase):
|
||||
env=TestEnvironment(stdout_isatty=False,)
|
||||
)
|
||||
self.assertIn('HTTP/1.1 301', r)
|
||||
self.assertEqual(r.exit_status, ExitStatus.ERROR_HTTP_3XX)
|
||||
self.assertEqual(r.exit_status, exit.ERROR_HTTP_3XX)
|
||||
self.assertIn('301 moved permanently', r.stderr.lower())
|
||||
|
||||
@skipIf(requests_version == '0.13.6',
|
||||
@ -966,7 +909,7 @@ class ExitStatusTest(BaseTestCase):
|
||||
)
|
||||
# The redirect will be followed so 200 is expected.
|
||||
self.assertIn('HTTP/1.1 200 OK', r)
|
||||
self.assertEqual(r.exit_status, ExitStatus.OK)
|
||||
self.assertEqual(r.exit_status, exit.OK)
|
||||
|
||||
def test_4xx_check_status_exits_4(self):
|
||||
r = http(
|
||||
@ -975,7 +918,7 @@ class ExitStatusTest(BaseTestCase):
|
||||
httpbin('/status/401')
|
||||
)
|
||||
self.assertIn('HTTP/1.1 401', r)
|
||||
self.assertEqual(r.exit_status, ExitStatus.ERROR_HTTP_4XX)
|
||||
self.assertEqual(r.exit_status, exit.ERROR_HTTP_4XX)
|
||||
# Also stderr should be empty since stdout isn't redirected.
|
||||
self.assertTrue(not r.stderr)
|
||||
|
||||
@ -986,7 +929,7 @@ class ExitStatusTest(BaseTestCase):
|
||||
httpbin('/status/500')
|
||||
)
|
||||
self.assertIn('HTTP/1.1 500', r)
|
||||
self.assertEqual(r.exit_status, ExitStatus.ERROR_HTTP_5XX)
|
||||
self.assertEqual(r.exit_status, exit.ERROR_HTTP_5XX)
|
||||
|
||||
|
||||
class WindowsOnlyTests(BaseTestCase):
|
||||
@ -1039,8 +982,7 @@ class StreamTest(BaseTestCase):
|
||||
#self.assertIn(OK_COLOR, r)
|
||||
|
||||
def test_encoded_stream(self):
|
||||
"""Test that --stream works with non-prettified
|
||||
redirected terminal output."""
|
||||
"""Test that --stream works with non-prettified redirected terminal output."""
|
||||
with open(BIN_FILE_PATH, 'rb') as f:
|
||||
r = http(
|
||||
'--pretty=none',
|
||||
@ -1058,8 +1000,7 @@ class StreamTest(BaseTestCase):
|
||||
#self.assertIn(OK, r)
|
||||
|
||||
def test_redirected_stream(self):
|
||||
"""Test that --stream works with non-prettified
|
||||
redirected terminal output."""
|
||||
"""Test that --stream works with non-prettified redirected terminal output."""
|
||||
with open(BIN_FILE_PATH, 'rb') as f:
|
||||
r = http(
|
||||
'--pretty=none',
|
||||
@ -1083,6 +1024,7 @@ class LineEndingsTest(BaseTestCase):
|
||||
as the headers/body separator."""
|
||||
|
||||
def _validate_crlf(self, msg):
|
||||
#noinspection PyUnresolvedReferences
|
||||
lines = iter(msg.splitlines(True))
|
||||
for header in lines:
|
||||
if header == CRLF:
|
||||
@ -1117,7 +1059,7 @@ class LineEndingsTest(BaseTestCase):
|
||||
'GET',
|
||||
httpbin('/get')
|
||||
)
|
||||
self.assertEqual(r.exit_status, 0)
|
||||
self.assertEqual(r.exit_status,0)
|
||||
self._validate_crlf(r)
|
||||
|
||||
def test_CRLF_ugly_request(self):
|
||||
@ -1170,8 +1112,6 @@ class ItemParsingTest(BaseTestCase):
|
||||
# files
|
||||
self.key_value_type('bar\\@baz@%s' % FILE_PATH_ARG)
|
||||
])
|
||||
# `requests.structures.CaseInsensitiveDict` => `dict`
|
||||
headers = dict(headers._store.values())
|
||||
self.assertDictEqual(headers, {
|
||||
'foo:bar': 'baz',
|
||||
'jack@jill': 'hill',
|
||||
@ -1201,8 +1141,6 @@ class ItemParsingTest(BaseTestCase):
|
||||
self.key_value_type('test-file@%s' % FILE_PATH_ARG),
|
||||
self.key_value_type('query==value'),
|
||||
])
|
||||
# `requests.structures.CaseInsensitiveDict` => `dict`
|
||||
headers = dict(headers._store.values())
|
||||
self.assertDictEqual(headers, {
|
||||
'header': 'value',
|
||||
'eh': ''
|
||||
@ -1226,108 +1164,78 @@ class ArgumentParserTestCase(unittest.TestCase):
|
||||
self.parser = input.Parser()
|
||||
|
||||
def test_guess_when_method_set_and_valid(self):
|
||||
self.parser.args = argparse.Namespace()
|
||||
self.parser.args.method = 'GET'
|
||||
self.parser.args.url = 'http://example.com/'
|
||||
self.parser.args.items = []
|
||||
args = argparse.Namespace()
|
||||
args.method = 'GET'
|
||||
args.url = 'http://example.com/'
|
||||
args.items = []
|
||||
|
||||
self.parser.env = TestEnvironment()
|
||||
self.parser._guess_method(args, TestEnvironment())
|
||||
|
||||
self.parser._guess_method()
|
||||
|
||||
self.assertEqual(self.parser.args.method, 'GET')
|
||||
self.assertEqual(self.parser.args.url, 'http://example.com/')
|
||||
self.assertEqual(self.parser.args.items, [])
|
||||
self.assertEqual(args.method, 'GET')
|
||||
self.assertEqual(args.url, 'http://example.com/')
|
||||
self.assertEqual(args.items, [])
|
||||
|
||||
def test_guess_when_method_not_set(self):
|
||||
args = argparse.Namespace()
|
||||
args.method = None
|
||||
args.url = 'http://example.com/'
|
||||
args.items = []
|
||||
|
||||
self.parser.args = argparse.Namespace()
|
||||
self.parser.args.method = None
|
||||
self.parser.args.url = 'http://example.com/'
|
||||
self.parser.args.items = []
|
||||
self.parser.env = TestEnvironment()
|
||||
self.parser._guess_method(args, TestEnvironment())
|
||||
|
||||
self.parser._guess_method()
|
||||
|
||||
self.assertEqual(self.parser.args.method, 'GET')
|
||||
self.assertEqual(self.parser.args.url, 'http://example.com/')
|
||||
self.assertEqual(self.parser.args.items, [])
|
||||
self.assertEqual(args.method, 'GET')
|
||||
self.assertEqual(args.url, 'http://example.com/')
|
||||
self.assertEqual(args.items, [])
|
||||
|
||||
def test_guess_when_method_set_but_invalid_and_data_field(self):
|
||||
self.parser.args = argparse.Namespace()
|
||||
self.parser.args.method = 'http://example.com/'
|
||||
self.parser.args.url = 'data=field'
|
||||
self.parser.args.items = []
|
||||
self.parser.env = TestEnvironment()
|
||||
self.parser._guess_method()
|
||||
args = argparse.Namespace()
|
||||
args.method = 'http://example.com/'
|
||||
args.url = 'data=field'
|
||||
args.items = []
|
||||
|
||||
self.assertEqual(self.parser.args.method, 'POST')
|
||||
self.assertEqual(self.parser.args.url, 'http://example.com/')
|
||||
self.parser._guess_method(args, TestEnvironment())
|
||||
|
||||
self.assertEqual(args.method, 'POST')
|
||||
self.assertEqual(args.url, 'http://example.com/')
|
||||
self.assertEqual(
|
||||
self.parser.args.items,
|
||||
args.items,
|
||||
[input.KeyValue(
|
||||
key='data', value='field', sep='=', orig='data=field')])
|
||||
|
||||
def test_guess_when_method_set_but_invalid_and_header_field(self):
|
||||
self.parser.args = argparse.Namespace()
|
||||
self.parser.args.method = 'http://example.com/'
|
||||
self.parser.args.url = 'test:header'
|
||||
self.parser.args.items = []
|
||||
args = argparse.Namespace()
|
||||
args.method = 'http://example.com/'
|
||||
args.url = 'test:header'
|
||||
args.items = []
|
||||
|
||||
self.parser.env = TestEnvironment()
|
||||
self.parser._guess_method(args, TestEnvironment())
|
||||
|
||||
self.parser._guess_method()
|
||||
|
||||
self.assertEqual(self.parser.args.method, 'GET')
|
||||
self.assertEqual(self.parser.args.url, 'http://example.com/')
|
||||
self.assertEqual(args.method, 'GET')
|
||||
self.assertEqual(args.url, 'http://example.com/')
|
||||
self.assertEqual(
|
||||
self.parser.args.items,
|
||||
args.items,
|
||||
[input.KeyValue(
|
||||
key='test', value='header', sep=':', orig='test:header')])
|
||||
|
||||
def test_guess_when_method_set_but_invalid_and_item_exists(self):
|
||||
self.parser.args = argparse.Namespace()
|
||||
self.parser.args.method = 'http://example.com/'
|
||||
self.parser.args.url = 'new_item=a'
|
||||
self.parser.args.items = [
|
||||
args = argparse.Namespace()
|
||||
args.method = 'http://example.com/'
|
||||
args.url = 'new_item=a'
|
||||
args.items = [
|
||||
input.KeyValue(
|
||||
key='old_item', value='b', sep='=', orig='old_item=b')
|
||||
]
|
||||
|
||||
self.parser.env = TestEnvironment()
|
||||
self.parser._guess_method(args, TestEnvironment())
|
||||
|
||||
self.parser._guess_method()
|
||||
|
||||
self.assertEqual(self.parser.args.items, [
|
||||
self.assertEqual(args.items, [
|
||||
input.KeyValue(
|
||||
key='new_item', value='a', sep='=', orig='new_item=a'),
|
||||
input.KeyValue(
|
||||
key='old_item', value='b', sep='=', orig='old_item=b'),
|
||||
input.KeyValue(key
|
||||
='old_item', value='b', sep='=', orig='old_item=b'),
|
||||
])
|
||||
|
||||
|
||||
class TestNoOptions(BaseTestCase):
|
||||
|
||||
def test_valid_no_options(self):
|
||||
r = http(
|
||||
'--verbose',
|
||||
'--no-verbose',
|
||||
'GET',
|
||||
httpbin('/get')
|
||||
)
|
||||
self.assertNotIn('GET /get HTTP/1.1', r)
|
||||
|
||||
def test_invalid_no_options(self):
|
||||
r = http(
|
||||
'--no-war',
|
||||
'GET',
|
||||
httpbin('/get')
|
||||
)
|
||||
self.assertEqual(r.exit_status, 1)
|
||||
self.assertIn('unrecognized arguments: --no-war', r.stderr)
|
||||
self.assertNotIn('GET /get HTTP/1.1', r)
|
||||
|
||||
|
||||
class READMETest(BaseTestCase):
|
||||
|
||||
@skipIf(not has_docutils(), 'docutils not installed')
|
||||
@ -1374,26 +1282,6 @@ class SessionTest(BaseTestCase):
|
||||
self.assertEqual(r.json['headers']['Cookie'], 'hello=world')
|
||||
self.assertIn('Basic ', r.json['headers']['Authorization'])
|
||||
|
||||
def test_session_ignored_header_prefixes(self):
|
||||
r = http(
|
||||
'--session=test',
|
||||
'GET',
|
||||
httpbin('/get'),
|
||||
'Content-Type: text/plain',
|
||||
'If-Unmodified-Since: Sat, 29 Oct 1994 19:43:31 GMT',
|
||||
env=self.env
|
||||
)
|
||||
self.assertIn(OK, r)
|
||||
|
||||
r2 = http(
|
||||
'--session=test',
|
||||
'GET',
|
||||
httpbin('/get')
|
||||
)
|
||||
self.assertIn(OK, r2)
|
||||
self.assertNotIn('Content-Type', r2.json['headers'])
|
||||
self.assertNotIn('If-Unmodified-Since', r2.json['headers'])
|
||||
|
||||
def test_session_update(self):
|
||||
# Get a response to a request from the original session.
|
||||
r1 = http(
|
||||
@ -1462,182 +1350,10 @@ class SessionTest(BaseTestCase):
|
||||
)
|
||||
self.assertIn(OK, r3)
|
||||
|
||||
# Origin can differ on Travis.
|
||||
del r1.json['origin'], r3.json['origin']
|
||||
|
||||
# Should be the same as before r2.
|
||||
self.assertDictEqual(r1.json, r3.json)
|
||||
|
||||
def test_session_by_path(self):
|
||||
session_path = os.path.join(self.config_dir, 'session-by-path.json')
|
||||
|
||||
r1 = http(
|
||||
'--session=' + session_path,
|
||||
'GET',
|
||||
httpbin('/get'),
|
||||
'Foo:Bar',
|
||||
env=self.env
|
||||
)
|
||||
self.assertIn(OK, r1)
|
||||
|
||||
r2 = http(
|
||||
'--session=' + session_path,
|
||||
'GET',
|
||||
httpbin('/get'),
|
||||
env=self.env
|
||||
)
|
||||
self.assertIn(OK, r2)
|
||||
|
||||
self.assertEqual(r2.json['headers']['Foo'], 'Bar')
|
||||
|
||||
|
||||
class DownloadUtilsTest(BaseTestCase):
|
||||
|
||||
def test_Content_Range_parsing(self):
|
||||
|
||||
parse = parse_content_range
|
||||
|
||||
self.assertEqual(parse('bytes 100-199/200', 100), 200)
|
||||
self.assertEqual(parse('bytes 100-199/*', 100), 200)
|
||||
|
||||
# missing
|
||||
self.assertRaises(ContentRangeError, parse, None, 100)
|
||||
|
||||
# syntax error
|
||||
self.assertRaises(ContentRangeError, parse, 'beers 100-199/*', 100)
|
||||
|
||||
# unexpected range
|
||||
self.assertRaises(ContentRangeError, parse, 'bytes 100-199/*', 99)
|
||||
|
||||
# invalid instance-length
|
||||
self.assertRaises(ContentRangeError, parse, 'bytes 100-199/199', 100)
|
||||
|
||||
# invalid byte-range-resp-spec
|
||||
self.assertRaises(ContentRangeError, parse, 'bytes 100-99/199', 100)
|
||||
|
||||
# invalid byte-range-resp-spec
|
||||
self.assertRaises(ContentRangeError, parse, 'bytes 100-100/*', 100)
|
||||
|
||||
def test_Content_Disposition_parsing(self):
|
||||
parse = filename_from_content_disposition
|
||||
self.assertEqual(
|
||||
parse('attachment; filename=hello-WORLD_123.txt'),
|
||||
'hello-WORLD_123.txt'
|
||||
)
|
||||
self.assertEqual(
|
||||
parse('attachment; filename=".hello-WORLD_123.txt"'),
|
||||
'hello-WORLD_123.txt'
|
||||
)
|
||||
self.assertIsNone(parse('attachment; filename='))
|
||||
self.assertIsNone(parse('attachment; filename=/etc/hosts'))
|
||||
self.assertIsNone(parse('attachment; filename=hello@world'))
|
||||
|
||||
def test_filename_from_url(self):
|
||||
self.assertEqual(filename_from_url(
|
||||
url='http://example.org/foo',
|
||||
content_type='text/plain'
|
||||
), 'foo.txt')
|
||||
self.assertEqual(filename_from_url(
|
||||
url='http://example.org/foo',
|
||||
content_type='text/html; charset=utf8'
|
||||
), 'foo.html')
|
||||
self.assertEqual(filename_from_url(
|
||||
url='http://example.org/foo',
|
||||
content_type=None
|
||||
), 'foo')
|
||||
self.assertEqual(filename_from_url(
|
||||
url='http://example.org/foo',
|
||||
content_type='x-foo/bar'
|
||||
), 'foo')
|
||||
|
||||
def test_unique_filename(self):
|
||||
|
||||
def make_exists(unique_on_attempt=0):
|
||||
# noinspection PyUnresolvedReferences,PyUnusedLocal
|
||||
def exists(filename):
|
||||
if exists.attempt == unique_on_attempt:
|
||||
return False
|
||||
exists.attempt += 1
|
||||
return True
|
||||
exists.attempt = 0
|
||||
return exists
|
||||
|
||||
self.assertEqual(
|
||||
get_unique_filename('foo.bar', exists=make_exists()),
|
||||
'foo.bar'
|
||||
)
|
||||
self.assertEqual(
|
||||
get_unique_filename('foo.bar', exists=make_exists(1)),
|
||||
'foo.bar-1'
|
||||
)
|
||||
self.assertEqual(
|
||||
get_unique_filename('foo.bar', exists=make_exists(10)),
|
||||
'foo.bar-10'
|
||||
)
|
||||
|
||||
|
||||
class Response(object):
|
||||
|
||||
# noinspection PyDefaultArgument
|
||||
def __init__(self, url, headers={}, status_code=200):
|
||||
self.url = url
|
||||
self.headers = CaseInsensitiveDict(headers)
|
||||
self.status_code = status_code
|
||||
|
||||
|
||||
# noinspection PyTypeChecker
|
||||
class DownloadTest(BaseTestCase):
|
||||
# TODO: more tests
|
||||
|
||||
def test_actual_download(self):
|
||||
url = httpbin('/robots.txt')
|
||||
body = urlopen(url).read().decode()
|
||||
r = http(
|
||||
'--download',
|
||||
url,
|
||||
env=TestEnvironment(
|
||||
stdin_isatty=True,
|
||||
stdout_isatty=False
|
||||
)
|
||||
)
|
||||
self.assertIn('Downloading', r.stderr)
|
||||
self.assertIn('[K', r.stderr)
|
||||
self.assertIn('Done', r.stderr)
|
||||
self.assertEqual(body, r)
|
||||
|
||||
def test_download_with_Content_Length(self):
|
||||
download = Download(output_file=open(os.devnull, 'w'))
|
||||
download.start(Response(
|
||||
url=httpbin('/'),
|
||||
headers={'Content-Length': 10}
|
||||
))
|
||||
time.sleep(1.1)
|
||||
download._chunk_downloaded(b'12345')
|
||||
time.sleep(1.1)
|
||||
download._chunk_downloaded(b'12345')
|
||||
download.finish()
|
||||
self.assertFalse(download.interrupted)
|
||||
|
||||
def test_download_no_Content_Length(self):
|
||||
download = Download(output_file=open(os.devnull, 'w'))
|
||||
download.start(Response(url=httpbin('/')))
|
||||
time.sleep(1.1)
|
||||
download._chunk_downloaded(b'12345')
|
||||
download.finish()
|
||||
self.assertFalse(download.interrupted)
|
||||
|
||||
def test_download_interrupted(self):
|
||||
download = Download(
|
||||
output_file=open(os.devnull, 'w')
|
||||
)
|
||||
download.start(Response(
|
||||
url=httpbin('/'),
|
||||
headers={'Content-Length': 5}
|
||||
))
|
||||
download._chunk_downloaded(b'1234')
|
||||
download.finish()
|
||||
self.assertTrue(download.interrupted)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
#noinspection PyCallingNonCallable
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user