Commit Graph

411 Commits

Author SHA1 Message Date
4f1c9441c5 Fix encoding error with non-prettified encoded responses (#1168)
* Fix encoding error with non-prettified encoded responses

Removed `--format-option response.as` an promote `--response-as`: using
the format option would be misleading as it is now also used by non-prettified
responses.

* Encoding refactoring

* split --response-as into --response-mime and --response-charset
* add support for Content-Type charset for requests printed to terminal
* add support charset detection for requests printed to terminal without a Content-Type charset
* etc.

* `test_unicode.py` → `test_encoding.py`

* Drop sequence length check

* Clean-up tests

* [skip ci] Tweaks

* Use the compatible release clause for `charset_normalizer` requirement

Cf. https://www.python.org/dev/peps/pep-0440/#version-specifiers

* Clean-up

* Partially revert d52a4833e4

* Changelog

* Tweak tests

* [skip ci] Better test name

* Cleanup tests and add request body charset detection

* More test suite cleanups

* Cleanup

* Fix code style in test

* Improve detect_encoding() docstring

* Uniformize pytest.mark.parametrize() calls

* [skip ci] Comment out TODOs (will be tackled in a specific PR)

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-10-06 17:27:07 +02:00
71adcd97d0 Improve handling of prettified responses without correct content-type encoding (#1110)
* Improve handling of responses without correct content-type charset

* [skip ci] Minor tweaks in tests

* [skip ci] Add documentation

Co-authored-by: claudiatd <claudiatd@gmail.com>

* Improve unknown encoding test

[skip ci]

* Review mime and options retrieval

* Add full content-type example in help output

* Simplify decoder

* [skip ci] s/charset/encoding/

* Tweaks

* [skip ci] Fix type annotation

* [skip ci] s/charset/encoding/

* Tweaks

* Fix type annoation

* Improvement

* Introduce `codec.encode()`

* [skip ci] Tweak changelog

Co-authored-by: claudiatd <claudiatd@gmail.com>
2021-09-29 20:22:19 +02:00
9c89c703ae Allow to overwrite the response Content-Type from options (#1134)
* Allow to override the response `Content-Type` from options

* Apply suggestions from code review

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Rename the option from `--response.content-type` to `--response-as`

* Update CHANGELOG.md

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-09-27 13:58:19 +02:00
8f8851f1db Remove trailing comma in test 2021-09-24 10:37:59 +02:00
507514b795 Add workflow to test with pyOpenSSL active (#1164)
* Add workflow to test with pyOpenSSL active

Original patch by @gmelodie.

* Fix tests on Windows with Python 3.6
2021-09-23 10:37:23 +02:00
d7ed45bbcd Fix duplicate keys preservation of JSON data (#1163)
* Fix duplicate keys preservation of JSON data

* Update issue number

* Fix type annotations

* Changes after review

* Rewording
2021-09-21 19:07:59 +02:00
e6c5cd3e4b Improve JSON output when there is leading data before the actual JSON body (#1130)
In some special cases, to prevent against Cross Site Script Inclusion (XSSI)
attacks, the JSON response body starts with a magic prefix line that must be
stripped before feeding the rest of the response body to the JSON parser.
Such prefix is now simply ignored from the parser but still printed in the
terminal.

* Fix Windows tests
2021-09-21 11:15:43 +02:00
7c9f415107 Add a workflow to check documentations (#1151)
* Add a workflow to check documentations

* Fix markdown issues

* Install Ruby 2.7

* Finally, handle and fix GitHub templates

* Minor improvement in the feature request template

* Verbose mode to be sure all files are checked
2021-09-09 15:52:24 +02:00
6039bd8582 Switch from reStructuredText to Markdown and add docs/ (#1139)
* Convert most of the documentation from the frontend `README.rst` to `docs/REAME.md`

Also converted all reStructuredText files to Markdown.

* Tell `mdformat` to use LF for end on lines

* `--check` is not needed in the help message

* Skip tests on GitHub Windows.

Those tests pass on a real Windows machine.
Let's revisit those failure later, if needed.

* Move `mdoformat` requirement from `test` to `dev` extra

To fix Fedora CI.
2021-09-06 17:36:13 +02:00
71008bbedb Move example URL in a global variable for XML tests 2021-09-01 16:56:23 +02:00
85110643e7 Move example URL in a global variable for tests 2021-09-01 16:52:10 +02:00
fdd486415a Fix XML formatter tests 2021-09-01 10:28:03 +02:00
d10e108b5f Added support for XML formatting (#1129)
As a side effect, XHTML responses will be pretty-printed too.
2021-08-31 22:49:53 +02:00
d7caeaf372 Fix handling of session files with Cookie: followed by other headers (#1127)
* Fix the handling of cookies from session files

* Apply suggestions from code review

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Fix test docstring formatting

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-08-16 14:50:46 +02:00
4ff22defe4 Rework __main__.py to follow best practices (#1124)
It also simplifies how the `main()` function could be tested.
2021-08-06 16:57:19 +02:00
c6cbc7dfa5 Uniformize UTF-8 naming (#1115)
* Uniformize UTF-8 naming

Replace `utf8` -> `utf-8` everywhere.
It should have no impact, `utf8` is an alias of `utf-8` [1].

[1] ee03bad25e/Lib/encodings/aliases.py (L534)

* Always specify the encoding

Let's be explicit over implicit. And prevent future warnings from PEP-597 [1].

[1] https://www.python.org/dev/peps/pep-0597/#using-the-default-encoding-is-a-common-mistake

* Update `UTF8` constant (`utf-8` -> `utf_8`)

* Remove default argument from `str.encode()` and `bytes.decode()`

* Clean-up
2021-08-05 20:58:43 +02:00
a66af2497a Add converter plugin streaming tests (#1117)
Also fixed minor glitches here and there and re-enabled a unicode test.
2021-08-05 14:37:08 +02:00
04d05a8abd Minor clean-up (#1112)
* Remove Python 2 clean-up misses

* Remove unused `Environment.devnull` setter

* Simplifies `get_filename_max_length()`
2021-07-26 23:56:38 +02:00
2f8d7f77bd Add more download tests (#1114) 2021-07-26 20:27:36 +02:00
147a066dbe Add internal support for file-like object responses to improve adapter plugin support (#1094)
* Support `requests.response.raw` being a file-like object

Previously HTTPie relied on `requests.models.Response.raw` being
`urllib3.HTTPResponse`. The `requests` documentation specifies that
(requests.models.Response.raw)[https://docs.python-requests.org/en/master/api/#requests.Response.raw]
is a file-like object but allows for other types for internal use.

This change introduces graceful handling for scenarios when
`requests.models.Response.raw` is not `urllib3.HTTPResponse`. In such a scenario
HTTPie now falls back to extracting metadata from `requests.models.Response`
directly instead of direct access from protected protected members such as
`response.raw._original_response`. A side effect in this fallback procedure is
that we can no longer determine HTTP protocol version and report it as `1.1`.

This change is necessary to make it possible to implement `TransportPlugins`
without having to also needing to emulate internal behavior of `urlib3` and
`http.client`.

* Load cookies from `response.headers` instead of `response.raw._original_response.msg._headers`

`response.cookies` was not utilized as it not possible to construct original
payload from `http.cookiejar.Cookie`. Data is stored in lossy format. For example
`Cookie.secure` defaults to `False` so we cannot distinguish if `Cookie.secure` was
set to `False` or was not set at all. Same problem applies to other fields also.

* Simpler HTTP envelope data extraction

* Test cookie extraction and make cookie presentment backwards compatible

Co-authored-by: Mickaël Schoentgen <contact@tiger-222.fr>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-07-06 21:00:06 +02:00
ae6f57dc76 Skip tests that randomly fail on Windows in CI 2021-06-26 14:03:31 +02:00
7ceb313ccf Add --follow test for HTTP 308 2021-06-15 17:31:56 +02:00
3228e74df5 Add --follow --verbose test for HTTP 308 2021-06-15 17:24:26 +02:00
07a0359316 Final touches for #1088 (#1091)
* Make sure there’s no trailing \n in test files for easier output inspection

* Refactor output matching test utils

* More robust `test_http_307_allow_redirect_post_verbose()`

* Changelog

* Mention HTTP 307 Temporary Redirect re-post behaviour in README
2021-06-15 14:48:44 +02:00
2d55c01c7e Fix printing redirected prepared request in verbose mode (#1088) 2021-06-15 13:39:46 +02:00
9792513c68 Add a reproduction test case for #1082 (#1085)
The fix may actually be slightly more complex than I expected.
We would need, at first sight, to loose the prepared requests + responses streaming.

A potential solution will come in a near future.
2021-06-09 09:43:06 +02:00
350f973f70 Switch from pycodestyle to flake8 for code style checks (#1083) 2021-06-02 11:06:46 +02:00
8d35a12d27 Fix several issues found with flake8 (#1081) 2021-06-01 14:46:58 +02:00
8374a9ed83 Review OSError exceptions handling (#1080)
- Replace obsolete `IOError` (Python 2) with `OSError`,
  cf https://docs.python.org/3/library/exceptions.html#OSError.
- Improve `OSError` catches at different places, simplifying
  the code.
2021-05-31 10:10:41 +02:00
a61f9e1114 Minor clean-up (#1078)
- Remove default arguments to `open()`.
- Make use of `pytest` mechanisms for temporary folders.
2021-05-29 12:06:06 +02:00
3865fabf09 Adapt doctest of tests.utils.http to work on Python 3.10 as well (#1075)
https://docs.python.org/3.10/whatsnew/3.10.html#enum

Python 3.10 changed the repr of enum members, and the doctest of tests.utils.http failed.
Exact reprs are unfortunately not considered stable API between Python releases:

    =================================== FAILURES ===================================
    __________________________ [doctest] tests.utils.http __________________________
    209
    210     Example:
    211
    212     $ http --auth=user:password GET pie.dev/basic-auth/user/password
    213
    214         >>> httpbin = getfixture('httpbin')
    215         >>> r = http('-a', 'user:pw', httpbin.url + '/basic-auth/user/pw')
    216         >>> type(r) == StrCLIResponse
    217         True
    218         >>> r.exit_status
    Expected:
        <ExitStatus.SUCCESS: 0>
    Got:
        ExitStatus.SUCCESS

A simple replacement of the expected output however breaks the doctest on Python 3.9.

This is the best solution I could think of
that keeps the docstring readable and doctest working in Pythons both old and new.
2021-05-27 19:54:33 +02:00
355befcbfc Skip http://pie.dev tests when offline (#1072) 2021-05-27 19:30:36 +02:00
464b5b4c1d Polish Python 2 removal (#1070) 2021-05-27 13:05:41 +02:00
264d45cdf5 Modernize the code base with f-strings in tests (#1069)
Simple concatenations were kept for readability purpose.
2021-05-26 14:09:38 +02:00
0001297f41 Add --raw to allow specifying the raw request body as an alternative to stdin (#1062)
* Add --raw to allow specifying the raw request body without extra processing

As an alternative to `stdin`.

Co-authored-by: Elena Lape <elapinskaite@gmail.com>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Update README.rst

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Update README.rst

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Fix default HTTP method on empty data

Co-authored-by: Elena Lape <elapinskaite@gmail.com>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-05-24 14:29:54 +02:00
a3a08a9a22 Use relative imports (#1057)
* Use relative imports in test

* Use relative imports

* Add myself to contributors :)
2021-05-05 14:13:39 +02:00
1274d869f6 Replace usage of mock with unittest.mock (#1054)
Since Python 3, the mock dependency is no more required as
it is already part of the unittest module.
2021-04-30 15:08:27 +02:00
611bcdaab1 Fail gracefully if multiple request data files are supplied (#1042) 2021-04-15 09:35:50 +02:00
84c7327057 Correctly handle single-byte Content-Range (#1032)
HTTPie previously failed if it continued a download with a single byte left.
2021-02-14 13:30:58 +01:00
61dbadb730 Tests 2021-02-06 13:21:21 +01:00
1ac8f69651 Add more output matching tests 2021-02-06 10:52:30 +01:00
3c07a25326 Add support for max-age=0 cookie expiry (#1029)
Close #998
2021-02-06 10:50:34 +01:00
cf78a12e46 Show --check-status warning with --quiet as well. (#1026)
Fixes #1028
2021-01-31 00:58:56 +01:00
0f1e098cc4 Fix incorrect separators and introduce assert_output_matches() (close #1027) 2021-01-30 22:14:57 +01:00
db685d58b5 Decode headers using utf-8 only if they are not str (#1020) 2021-01-13 21:45:56 +01:00
6922a0c912 Switch from httpbin.org to pie.dev 2020-12-24 21:34:30 +01:00
2afdc958c6 Update URLs 2020-12-23 22:07:27 +01:00
57b1baf1d1 Add a test to reproduce #1006 2020-12-22 22:56:45 +01:00
9883a46575 Cleanup (#993) 2020-12-21 12:03:25 +01:00
1573058811 v2.3.0 2020-10-25 21:12:38 +01:00