Commit Graph

675 Commits

Author SHA1 Message Date
Mickaël Schoentgen
3869c3ce99 Remove unused import 2021-10-07 14:41:03 +02:00
Jakub Roztocil
ef4fa20ceb Tweak 2021-10-06 19:31:43 +02:00
Jakub Roztocil
7e0bed4e54 Add more types and docs for plugins API II. 2021-10-06 19:28:21 +02:00
Jakub Roztocil
e1627803fe Add more types and docs for plugins API 2021-10-06 19:24:10 +02:00
Mickaël Schoentgen
4f1c9441c5
Fix encoding error with non-prettified encoded responses (#1168)
* Fix encoding error with non-prettified encoded responses

Removed `--format-option response.as` an promote `--response-as`: using
the format option would be misleading as it is now also used by non-prettified
responses.

* Encoding refactoring

* split --response-as into --response-mime and --response-charset
* add support for Content-Type charset for requests printed to terminal
* add support charset detection for requests printed to terminal without a Content-Type charset
* etc.

* `test_unicode.py` → `test_encoding.py`

* Drop sequence length check

* Clean-up tests

* [skip ci] Tweaks

* Use the compatible release clause for `charset_normalizer` requirement

Cf. https://www.python.org/dev/peps/pep-0440/#version-specifiers

* Clean-up

* Partially revert d52a4833e4

* Changelog

* Tweak tests

* [skip ci] Better test name

* Cleanup tests and add request body charset detection

* More test suite cleanups

* Cleanup

* Fix code style in test

* Improve detect_encoding() docstring

* Uniformize pytest.mark.parametrize() calls

* [skip ci] Comment out TODOs (will be tackled in a specific PR)

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-10-06 17:27:07 +02:00
Jakub Roztocil
4ef31ecf71
Update utils.py 2021-10-02 16:43:29 +02:00
Mickaël Schoentgen
71adcd97d0
Improve handling of prettified responses without correct content-type encoding (#1110)
* Improve handling of responses without correct content-type charset

* [skip ci] Minor tweaks in tests

* [skip ci] Add documentation

Co-authored-by: claudiatd <claudiatd@gmail.com>

* Improve unknown encoding test

[skip ci]

* Review mime and options retrieval

* Add full content-type example in help output

* Simplify decoder

* [skip ci] s/charset/encoding/

* Tweaks

* [skip ci] Fix type annotation

* [skip ci] s/charset/encoding/

* Tweaks

* Fix type annoation

* Improvement

* Introduce `codec.encode()`

* [skip ci] Tweak changelog

Co-authored-by: claudiatd <claudiatd@gmail.com>
2021-09-29 20:22:19 +02:00
Mickaël Schoentgen
b50f9aa7e7 Use PYthon 3 documentation
[skip ci]
2021-09-28 12:54:16 +02:00
Mickaël Schoentgen
fe96b2af20 Use httpie.io/docs everywhere
[skip ci]
2021-09-28 12:53:53 +02:00
Mickaël Schoentgen
727b8a2c05
Sort available style choices (#1166) 2021-09-27 16:55:10 +02:00
Mickaël Schoentgen
9c89c703ae
Allow to overwrite the response Content-Type from options (#1134)
* Allow to override the response `Content-Type` from options

* Apply suggestions from code review

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Rename the option from `--response.content-type` to `--response-as`

* Update CHANGELOG.md

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-09-27 13:58:19 +02:00
Mickaël Schoentgen
474093acdf
Include plugin info in --debug output (#1165)
* Include plugin info in `--debug` output

* Adapt issue number

* Fix docs
2021-09-23 17:15:14 +02:00
Mickaël Schoentgen
d7ed45bbcd
Fix duplicate keys preservation of JSON data (#1163)
* Fix duplicate keys preservation of JSON data

* Update issue number

* Fix type annotations

* Changes after review

* Rewording
2021-09-21 19:07:59 +02:00
Mickaël Schoentgen
e6c5cd3e4b
Improve JSON output when there is leading data before the actual JSON body (#1130)
In some special cases, to prevent against Cross Site Script Inclusion (XSSI)
attacks, the JSON response body starts with a magic prefix line that must be
stripped before feeding the rest of the response body to the JSON parser.
Such prefix is now simply ignored from the parser but still printed in the
terminal.

* Fix Windows tests
2021-09-21 11:15:43 +02:00
Mickaël Schoentgen
273134123a
Bump the version to 2.6.0.dev0 (#1162)
[skip ci]
2021-09-21 10:40:09 +02:00
Mickaël Schoentgen
4eaa4d67c5
v2.5.0 (#1140)
[skip ci]
2021-09-06 20:23:14 +02:00
Mickaël Schoentgen
a62391e789
Tiny clean-up in program() (#1135) 2021-09-02 16:47:01 +02:00
Mickaël Schoentgen
fdd486415a Fix XML formatter tests 2021-09-01 10:28:03 +02:00
Jakub Roztocil
6c501d23c3 Change default XML indent to 2 spaces 2021-08-31 22:52:16 +02:00
Mickaël Schoentgen
d10e108b5f
Added support for XML formatting (#1129)
As a side effect, XHTML responses will be pretty-printed too.
2021-08-31 22:49:53 +02:00
Mickaël Schoentgen
d7caeaf372
Fix handling of session files with Cookie: followed by other headers (#1127)
* Fix the handling of cookies from session files

* Apply suggestions from code review

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Fix test docstring formatting

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-08-16 14:50:46 +02:00
Mickaël Schoentgen
54c8612452
Remove unused code in BasicConfig (#1123) 2021-08-06 18:04:08 +02:00
Mickaël Schoentgen
4ff22defe4
Rework __main__.py to follow best practices (#1124)
It also simplifies how the `main()` function could be tested.
2021-08-06 16:57:19 +02:00
Mickaël Schoentgen
6bd6648545
Simplify get_content_type() (#1125)
We were using the potential `encoding` returned by `mimetypes.guess_type()`
to expand the `Content-Type` header.
According to the RFC-7231 [1] the `Content-Type` should contain a charset
and nothing more. But as stated in the `mimetypes.guess_type()` doc [2],
the `encoding` would be the name of the program used to encode (e.g. compress
or gzip) the payload. The `encoding` is suitable for use as a `Content-Encoding`
header. See [3] for potential `encoding`s, none is a IANA registered one [4], and
so a valid charset to be used by the `Content-Type` header.

[1] https://httpwg.org/specs/rfc7231.html#header.content-type
[2] https://docs.python.org/3/library/mimetypes.html#guess_type
[3] 938e84b4fa/Lib/mimetypes.py (L416-L422)
[4] https://www.iana.org/assignments/character-sets/character-sets.xhtml
2021-08-06 12:35:38 +02:00
Jakub Roztocil
6633b5ae9b Use UTF8 constant in FORM_CONTENT_TYPE as well 2021-08-05 21:00:17 +02:00
Mickaël Schoentgen
c6cbc7dfa5
Uniformize UTF-8 naming (#1115)
* Uniformize UTF-8 naming

Replace `utf8` -> `utf-8` everywhere.
It should have no impact, `utf8` is an alias of `utf-8` [1].

[1] ee03bad25e/Lib/encodings/aliases.py (L534)

* Always specify the encoding

Let's be explicit over implicit. And prevent future warnings from PEP-597 [1].

[1] https://www.python.org/dev/peps/pep-0597/#using-the-default-encoding-is-a-common-mistake

* Update `UTF8` constant (`utf-8` -> `utf_8`)

* Remove default argument from `str.encode()` and `bytes.decode()`

* Clean-up
2021-08-05 20:58:43 +02:00
Mickaël Schoentgen
11399dde76
Refine abstract methods and properties (#1118) 2021-08-05 20:57:23 +02:00
Mickaël Schoentgen
de13423839
--download: Use time.monotonic() and rework code to prevent ZeroDivisionError specific handling (#1113) 2021-07-29 16:05:56 +02:00
Mickaël Schoentgen
04d05a8abd
Minor clean-up (#1112)
* Remove Python 2 clean-up misses

* Remove unused `Environment.devnull` setter

* Simplifies `get_filename_max_length()`
2021-07-26 23:56:38 +02:00
Anton Emelyanov
aee77a23af
Simplify spinner_pos calculation a little (#1111) 2021-07-20 18:24:49 +02:00
Ilya Sukhanov
147a066dbe
Add internal support for file-like object responses to improve adapter plugin support (#1094)
* Support `requests.response.raw` being a file-like object

Previously HTTPie relied on `requests.models.Response.raw` being
`urllib3.HTTPResponse`. The `requests` documentation specifies that
(requests.models.Response.raw)[https://docs.python-requests.org/en/master/api/#requests.Response.raw]
is a file-like object but allows for other types for internal use.

This change introduces graceful handling for scenarios when
`requests.models.Response.raw` is not `urllib3.HTTPResponse`. In such a scenario
HTTPie now falls back to extracting metadata from `requests.models.Response`
directly instead of direct access from protected protected members such as
`response.raw._original_response`. A side effect in this fallback procedure is
that we can no longer determine HTTP protocol version and report it as `1.1`.

This change is necessary to make it possible to implement `TransportPlugins`
without having to also needing to emulate internal behavior of `urlib3` and
`http.client`.

* Load cookies from `response.headers` instead of `response.raw._original_response.msg._headers`

`response.cookies` was not utilized as it not possible to construct original
payload from `http.cookiejar.Cookie`. Data is stored in lossy format. For example
`Cookie.secure` defaults to `False` so we cannot distinguish if `Cookie.secure` was
set to `False` or was not set at all. Same problem applies to other fields also.

* Simpler HTTP envelope data extraction

* Test cookie extraction and make cookie presentment backwards compatible

Co-authored-by: Mickaël Schoentgen <contact@tiger-222.fr>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-07-06 21:00:06 +02:00
Mickaël Schoentgen
5717fb1ad5
Normalize the version (#1101)
To fix that warning:

    setuptools/dist.py:473: UserWarning: Normalizing '2.5.0-dev' to '2.5.0.dev0'
2021-07-01 10:34:51 +02:00
nixbytes
c8d70e8c0b
Simplify return statements in client.py (#1096)
Co-authored-by: Mickaël Schoentgen <contact@tiger-222.fr>
2021-06-28 09:05:24 +02:00
Mickaël Schoentgen
2d55c01c7e
Fix printing redirected prepared request in verbose mode (#1088) 2021-06-15 13:39:46 +02:00
Mickaël Schoentgen
8d35a12d27
Fix several issues found with flake8 (#1081) 2021-06-01 14:46:58 +02:00
Mickaël Schoentgen
8374a9ed83
Review OSError exceptions handling (#1080)
- Replace obsolete `IOError` (Python 2) with `OSError`,
  cf https://docs.python.org/3/library/exceptions.html#OSError.
- Improve `OSError` catches at different places, simplifying
  the code.
2021-05-31 10:10:41 +02:00
Mickaël Schoentgen
a61f9e1114
Minor clean-up (#1078)
- Remove default arguments to `open()`.
- Make use of `pytest` mechanisms for temporary folders.
2021-05-29 12:06:06 +02:00
Mickaël Schoentgen
611b278b63
Fix --style colors list help indentation (#1077) 2021-05-28 12:45:40 +02:00
Miro Hrončok
06ef27c576
Remove an useless shebang form non-executable file (#1073)
Shebangs have no function in non-executable files.
This file does not need to be directly executed.
2021-05-27 19:17:04 +02:00
Mickaël Schoentgen
464b5b4c1d
Polish Python 2 removal (#1070) 2021-05-27 13:05:41 +02:00
Mickaël Schoentgen
0ff0874fa3
Modernize the code base with f-strings (#1068) 2021-05-25 20:49:07 +02:00
Mickaël Schoentgen
0001297f41
Add --raw to allow specifying the raw request body as an alternative to stdin (#1062)
* Add --raw to allow specifying the raw request body without extra processing

As an alternative to `stdin`.

Co-authored-by: Elena Lape <elapinskaite@gmail.com>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Update README.rst

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Update README.rst

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Fix default HTTP method on empty data

Co-authored-by: Elena Lape <elapinskaite@gmail.com>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-05-24 14:29:54 +02:00
Mickaël Schoentgen
a3a08a9a22
Use relative imports (#1057)
* Use relative imports in test

* Use relative imports

* Add myself to contributors :)
2021-05-05 14:13:39 +02:00
Jan Verbeek
611bcdaab1
Fail gracefully if multiple request data files are supplied (#1042) 2021-04-15 09:35:50 +02:00
Jan Verbeek
84c7327057
Correctly handle single-byte Content-Range (#1032)
HTTPie previously failed if it continued a download with a single byte left.
2021-02-14 13:30:58 +01:00
Jakub Roztocil
e944dbd7fa 2.5.0-dev 2021-02-06 13:34:04 +01:00
Jakub Roztocil
bb36897054 2.4.0 2021-02-06 11:17:24 +01:00
Jakub Roztocil
5414d1853e Refactoring
#1029
2021-02-06 10:58:36 +01:00
Denis Belavin
3c07a25326
Add support for max-age=0 cookie expiry (#1029)
Close #998
2021-02-06 10:50:34 +01:00
Jakub Roztocil
cf78a12e46
Show --check-status warning with --quiet as well. (#1026)
Fixes #1028
2021-01-31 00:58:56 +01:00
Jakub Roztocil
0f1e098cc4
Fix incorrect separators and introduce assert_output_matches() (close #1027) 2021-01-30 22:14:57 +01:00
Gian Ortiz
db685d58b5
Decode headers using utf-8 only if they are not str (#1020) 2021-01-13 21:45:56 +01:00
Jakub Roztocil
2afdc958c6 Update URLs 2020-12-23 22:07:27 +01:00
Shubhendra Singh Chauhan
9883a46575
Cleanup (#993) 2020-12-21 12:03:25 +01:00
Jakub Roztocil
42c4a7596b 2.4.0-dev 2020-10-25 21:36:24 +01:00
Jakub Roztocil
1573058811 v2.3.0 2020-10-25 21:12:38 +01:00
Jakub Roztocil
2f8d330b57 Fix --offline --chunked, add more tests 2020-09-28 16:40:16 +02:00
Jakub Roztocil
32d8b481e9 Fix --offline --multipart, add more tests 2020-09-28 16:22:34 +02:00
Jakub Roztocil
5a5b42340f PEP8 2020-09-28 12:50:45 +02:00
Jakub Roztocil
6925d930da Add support for streamed uploads, --chunked, finish --multipart, etc.
Close #201
Close #753
Close #684
Close #903
Related: #452
2020-09-28 12:16:57 +02:00
Jakub Roztocil
b7754f92ce Merge branch 'master' into feature/uploads2020
# Conflicts:
#	httpie/cli/argparser.py
#	httpie/uploads.py
2020-09-25 14:46:19 +02:00
Jakub Roztocil
e4e40e5b06 Request content type 2020-09-25 14:44:22 +02:00
Jakub Roztocil
d12af4a569 WIP 2020-09-25 13:44:28 +02:00
Jakub Roztocil
16ef08a159 Gracefully ignore cookie expiry dates in invalid format
Close #963
2020-09-20 09:21:10 +02:00
Jakub Roztocil
1813cf6156 Add --multipart and --boundary 2020-08-19 10:22:50 +02:00
Jakub Roztocil
ce185bd0fa
Update __init__.py 2020-08-17 13:16:57 +02:00
Jakub Roztocil
5a908aa411 pep8 2020-08-15 17:51:43 +02:00
Jakub Roztocil
6cd934d1b8 Add support for multipart upload streaming
Close #684, #201
2020-08-15 17:50:00 +02:00
Jakub Roztocil
1aa1366f99 Finish --quiet 2020-08-15 15:26:29 +02:00
Nicolas Beltran
c90d039a0b fixed issues related to downloading and using quiet at the same time 2020-08-15 15:26:29 +02:00
Nicolas Beltran
ae22d4e754 Additional Aesthetic changes 2020-08-15 15:26:29 +02:00
Nicolas Beltran
69e1067a2c Aesthetic changes 2020-08-15 15:26:29 +02:00
Nicolas Beltran
d546081340 Solved issue pertaining to downloads and added additional testing functionality for devnull 2020-08-15 15:26:29 +02:00
Nicolas Beltran
6421c145d9 Added changes suggested in the PR review 2020-08-15 15:26:29 +02:00
Nicolas Beltran
4bd2e622a5 Added tests for --quiet flag 2020-08-15 15:26:29 +02:00
Nicolas Beltran
a4a1e8d43b Added a quiet functionality 2020-08-15 15:26:28 +02:00
Jakub Roztocil
0d480139e4 Split session JSON serialization and writing to file
To avoid writing invalid JSON in case of presence of unserializable data due to an internal bug.
2020-07-07 13:26:47 +02:00
Jakub Roztocil
14677bd25d Cleanup inline to-dos
I.
2020-06-25 11:36:09 +02:00
Katherine Bancroft
9500ce136a
Combine cookies from original request and session file
Close #932
Co-authored-by: kbanc <katherine.bancoft@gmail.com>
Co-authored-by: Gabriel Cruz <gabs.oficial98@gmail.com>
2020-06-18 23:17:33 +02:00
Jakub Roztocil
93d07cfe57 v2.3.0-dev 2020-06-18 22:25:07 +02:00
Jakub Roztocil
5945845420 v2.2.0 2020-06-18 22:20:12 +02:00
Jakub Roztocil
d35864e79d Cleanup 2020-06-16 13:01:48 +02:00
Jakub Roztocil
8a106781be Add --sorted
Also add --no-(sorted|unsorted) to allow the documented resetting to default via --no-<option>.
2020-06-16 12:54:50 +02:00
Jakub Roztocil
23dd80563f Cleanup 2020-06-16 12:25:46 +02:00
Jakub Roztocil
2bab69d9fb Fix default value 2020-06-16 12:24:03 +02:00
Jakub Roztocil
826489950d Added --unsorted
It acts as a shortcut for --format-options=json.sort_keys:false,headers.sort:false

#128
2020-06-16 12:20:13 +02:00
Jakub Roztocil
b86598886e Added netrc support for auth plugins.
Enabled for --auth-type=basic and digest, 3rd parties may opt in.

This closes #718, closes #719, closes #852, and also closes #934
2020-06-16 11:05:00 +02:00
Jakub Roztocil
37200eb055 Cleanup 2020-06-15 23:02:16 +02:00
Eyitayo Ogunbiyi
9c68d7dd87
Remove expired cookies (#929)
* added a test for expiring cookies

* updated tests

* set up util for extracting expired cookies from response header

* Revert "updated tests"

This reverts commit a4eb5c4498.

* Revert "Revert "updated tests""

This reverts commit d242e21bce.

* added more functionality to get-expired-cookies

* add 'clear expired cookies' from session.json files

* refactored get_expired_cookies

* fixed formatting issues

* ensured key exists in cookie_header dict

* fixed linting errors

* removed unused import

* Added tests for get_expired_cookies util

* Added additional test for get_expired_cookies

* added remove_expired_cookies method directly to sessions class

* extracted logic to clear cookies to sessions.py

* refactored utils

* added tests to check expired cookies being removed from session obj

* added type annotations for methods

* Refactored test_sessions

* Seperated out expiry related tests into own class

* Refactored get_expired_cookies in utils

* Refactored remove cookie methods

* fixed linting errors

* fixed indentation and also pluralized test class name

* removed inheritance from SessionTestbase class

* Moved related test to TestExpiredCookies class

Co-authored-by: kbanc <katherine.bancoft@gmail.com>
2020-06-15 22:28:04 +02:00
Jakub Roztocil
7ee519ef46 Update CHANGELOG 2020-06-08 18:02:04 +02:00
Carlo Sciolla
c4627cc882
Custom file upload MIME type (#927)
* Support curl-like syntax for custom MIME type for files

In order to specify a custom MIME type for file uploads, a syntax
similar to that used by cURL is used so that

http -F test_file@/path/to/file.bin;type=application/zip https://...

forwards the user-provided file type if provided, otherwise falling
back to the usual guesswork out of the file extension.
2020-06-08 17:59:41 +02:00
Jakub Roztocil
caeef2fb7c Use : instead of = in `--format-options 2020-05-28 14:24:15 +02:00
Jakub Roztocil
aae596d472 Improve --format-options error messages 2020-05-27 16:19:32 +02:00
Jakub Roztocil
cb51faec51 pep8 2020-05-27 16:12:31 +02:00
Jakub Roztocil
c2a0cef76e Add --format-options to allow disabling sorting, etc.
#128
2020-05-27 16:01:17 +02:00
Jakub Roztocil
ca02e51420 Improve plugin API docs 2020-05-26 10:07:53 +02:00
Jakub Roztocil
cd085cbc0d Refactor built-in plugin registry to avoid circular imports
Fix #925
2020-05-26 10:07:34 +02:00
Jakub Roztocil
27d57ce773 Cleanup 2020-05-23 20:30:25 +02:00
Jakub Roztocil
4c4efff56a Pass cert_reqs to context 2020-05-23 20:19:16 +02:00
Jakub Roztocil
a53505f26e Fix SSL context 2020-05-23 15:01:33 +02:00