Commit Graph

362 Commits

Author SHA1 Message Date
dkreeft
6befaf9067
Added the ability to silence warnings via double -q or --quiet (#1175)
* change behavior of '--quiet' to silence errors and warnings when passed twice together with '--check-status'

* Apply suggestions from code review

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* remove header, trailing comma, rename constant and variable

* fix flags for tests

* [skip ci] Update ticket number

Co-authored-by: Dave <d.kreeft@outlook.com>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
Co-authored-by: Mickaël Schoentgen <contact@tiger-222.fr>
2021-10-08 14:18:11 +02:00
Mickaël Schoentgen
4f1c9441c5
Fix encoding error with non-prettified encoded responses (#1168)
* Fix encoding error with non-prettified encoded responses

Removed `--format-option response.as` an promote `--response-as`: using
the format option would be misleading as it is now also used by non-prettified
responses.

* Encoding refactoring

* split --response-as into --response-mime and --response-charset
* add support for Content-Type charset for requests printed to terminal
* add support charset detection for requests printed to terminal without a Content-Type charset
* etc.

* `test_unicode.py` → `test_encoding.py`

* Drop sequence length check

* Clean-up tests

* [skip ci] Tweaks

* Use the compatible release clause for `charset_normalizer` requirement

Cf. https://www.python.org/dev/peps/pep-0440/#version-specifiers

* Clean-up

* Partially revert d52a4833e4

* Changelog

* Tweak tests

* [skip ci] Better test name

* Cleanup tests and add request body charset detection

* More test suite cleanups

* Cleanup

* Fix code style in test

* Improve detect_encoding() docstring

* Uniformize pytest.mark.parametrize() calls

* [skip ci] Comment out TODOs (will be tackled in a specific PR)

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-10-06 17:27:07 +02:00
Mickaël Schoentgen
71adcd97d0
Improve handling of prettified responses without correct content-type encoding (#1110)
* Improve handling of responses without correct content-type charset

* [skip ci] Minor tweaks in tests

* [skip ci] Add documentation

Co-authored-by: claudiatd <claudiatd@gmail.com>

* Improve unknown encoding test

[skip ci]

* Review mime and options retrieval

* Add full content-type example in help output

* Simplify decoder

* [skip ci] s/charset/encoding/

* Tweaks

* [skip ci] Fix type annotation

* [skip ci] s/charset/encoding/

* Tweaks

* Fix type annoation

* Improvement

* Introduce `codec.encode()`

* [skip ci] Tweak changelog

Co-authored-by: claudiatd <claudiatd@gmail.com>
2021-09-29 20:22:19 +02:00
Mickaël Schoentgen
9c89c703ae
Allow to overwrite the response Content-Type from options (#1134)
* Allow to override the response `Content-Type` from options

* Apply suggestions from code review

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Rename the option from `--response.content-type` to `--response-as`

* Update CHANGELOG.md

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-09-27 13:58:19 +02:00
Mickaël Schoentgen
8f8851f1db Remove trailing comma in test 2021-09-24 10:37:59 +02:00
Mickaël Schoentgen
507514b795
Add workflow to test with pyOpenSSL active (#1164)
* Add workflow to test with pyOpenSSL active

Original patch by @gmelodie.

* Fix tests on Windows with Python 3.6
2021-09-23 10:37:23 +02:00
Mickaël Schoentgen
d7ed45bbcd
Fix duplicate keys preservation of JSON data (#1163)
* Fix duplicate keys preservation of JSON data

* Update issue number

* Fix type annotations

* Changes after review

* Rewording
2021-09-21 19:07:59 +02:00
Mickaël Schoentgen
e6c5cd3e4b
Improve JSON output when there is leading data before the actual JSON body (#1130)
In some special cases, to prevent against Cross Site Script Inclusion (XSSI)
attacks, the JSON response body starts with a magic prefix line that must be
stripped before feeding the rest of the response body to the JSON parser.
Such prefix is now simply ignored from the parser but still printed in the
terminal.

* Fix Windows tests
2021-09-21 11:15:43 +02:00
Mickaël Schoentgen
7c9f415107
Add a workflow to check documentations (#1151)
* Add a workflow to check documentations

* Fix markdown issues

* Install Ruby 2.7

* Finally, handle and fix GitHub templates

* Minor improvement in the feature request template

* Verbose mode to be sure all files are checked
2021-09-09 15:52:24 +02:00
Mickaël Schoentgen
6039bd8582
Switch from reStructuredText to Markdown and add docs/ (#1139)
* Convert most of the documentation from the frontend `README.rst` to `docs/REAME.md`

Also converted all reStructuredText files to Markdown.

* Tell `mdformat` to use LF for end on lines

* `--check` is not needed in the help message

* Skip tests on GitHub Windows.

Those tests pass on a real Windows machine.
Let's revisit those failure later, if needed.

* Move `mdoformat` requirement from `test` to `dev` extra

To fix Fedora CI.
2021-09-06 17:36:13 +02:00
Mickaël Schoentgen
71008bbedb Move example URL in a global variable for XML tests 2021-09-01 16:56:23 +02:00
Mickaël Schoentgen
85110643e7 Move example URL in a global variable for tests 2021-09-01 16:52:10 +02:00
Mickaël Schoentgen
fdd486415a Fix XML formatter tests 2021-09-01 10:28:03 +02:00
Mickaël Schoentgen
d10e108b5f
Added support for XML formatting (#1129)
As a side effect, XHTML responses will be pretty-printed too.
2021-08-31 22:49:53 +02:00
Mickaël Schoentgen
d7caeaf372
Fix handling of session files with Cookie: followed by other headers (#1127)
* Fix the handling of cookies from session files

* Apply suggestions from code review

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Fix test docstring formatting

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-08-16 14:50:46 +02:00
Mickaël Schoentgen
4ff22defe4
Rework __main__.py to follow best practices (#1124)
It also simplifies how the `main()` function could be tested.
2021-08-06 16:57:19 +02:00
Mickaël Schoentgen
c6cbc7dfa5
Uniformize UTF-8 naming (#1115)
* Uniformize UTF-8 naming

Replace `utf8` -> `utf-8` everywhere.
It should have no impact, `utf8` is an alias of `utf-8` [1].

[1] ee03bad25e/Lib/encodings/aliases.py (L534)

* Always specify the encoding

Let's be explicit over implicit. And prevent future warnings from PEP-597 [1].

[1] https://www.python.org/dev/peps/pep-0597/#using-the-default-encoding-is-a-common-mistake

* Update `UTF8` constant (`utf-8` -> `utf_8`)

* Remove default argument from `str.encode()` and `bytes.decode()`

* Clean-up
2021-08-05 20:58:43 +02:00
Mickaël Schoentgen
a66af2497a
Add converter plugin streaming tests (#1117)
Also fixed minor glitches here and there and re-enabled a unicode test.
2021-08-05 14:37:08 +02:00
Mickaël Schoentgen
04d05a8abd
Minor clean-up (#1112)
* Remove Python 2 clean-up misses

* Remove unused `Environment.devnull` setter

* Simplifies `get_filename_max_length()`
2021-07-26 23:56:38 +02:00
Mickaël Schoentgen
2f8d7f77bd
Add more download tests (#1114) 2021-07-26 20:27:36 +02:00
Ilya Sukhanov
147a066dbe
Add internal support for file-like object responses to improve adapter plugin support (#1094)
* Support `requests.response.raw` being a file-like object

Previously HTTPie relied on `requests.models.Response.raw` being
`urllib3.HTTPResponse`. The `requests` documentation specifies that
(requests.models.Response.raw)[https://docs.python-requests.org/en/master/api/#requests.Response.raw]
is a file-like object but allows for other types for internal use.

This change introduces graceful handling for scenarios when
`requests.models.Response.raw` is not `urllib3.HTTPResponse`. In such a scenario
HTTPie now falls back to extracting metadata from `requests.models.Response`
directly instead of direct access from protected protected members such as
`response.raw._original_response`. A side effect in this fallback procedure is
that we can no longer determine HTTP protocol version and report it as `1.1`.

This change is necessary to make it possible to implement `TransportPlugins`
without having to also needing to emulate internal behavior of `urlib3` and
`http.client`.

* Load cookies from `response.headers` instead of `response.raw._original_response.msg._headers`

`response.cookies` was not utilized as it not possible to construct original
payload from `http.cookiejar.Cookie`. Data is stored in lossy format. For example
`Cookie.secure` defaults to `False` so we cannot distinguish if `Cookie.secure` was
set to `False` or was not set at all. Same problem applies to other fields also.

* Simpler HTTP envelope data extraction

* Test cookie extraction and make cookie presentment backwards compatible

Co-authored-by: Mickaël Schoentgen <contact@tiger-222.fr>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-07-06 21:00:06 +02:00
Jakub Roztocil
ae6f57dc76 Skip tests that randomly fail on Windows in CI 2021-06-26 14:03:31 +02:00
Jakub Roztocil
7ceb313ccf Add --follow test for HTTP 308 2021-06-15 17:31:56 +02:00
Jakub Roztocil
3228e74df5 Add --follow --verbose test for HTTP 308 2021-06-15 17:24:26 +02:00
Jakub Roztocil
07a0359316
Final touches for #1088 (#1091)
* Make sure there’s no trailing \n in test files for easier output inspection

* Refactor output matching test utils

* More robust `test_http_307_allow_redirect_post_verbose()`

* Changelog

* Mention HTTP 307 Temporary Redirect re-post behaviour in README
2021-06-15 14:48:44 +02:00
Mickaël Schoentgen
2d55c01c7e
Fix printing redirected prepared request in verbose mode (#1088) 2021-06-15 13:39:46 +02:00
Mickaël Schoentgen
9792513c68
Add a reproduction test case for #1082 (#1085)
The fix may actually be slightly more complex than I expected.
We would need, at first sight, to loose the prepared requests + responses streaming.

A potential solution will come in a near future.
2021-06-09 09:43:06 +02:00
Mickaël Schoentgen
350f973f70
Switch from pycodestyle to flake8 for code style checks (#1083) 2021-06-02 11:06:46 +02:00
Mickaël Schoentgen
8d35a12d27
Fix several issues found with flake8 (#1081) 2021-06-01 14:46:58 +02:00
Mickaël Schoentgen
8374a9ed83
Review OSError exceptions handling (#1080)
- Replace obsolete `IOError` (Python 2) with `OSError`,
  cf https://docs.python.org/3/library/exceptions.html#OSError.
- Improve `OSError` catches at different places, simplifying
  the code.
2021-05-31 10:10:41 +02:00
Mickaël Schoentgen
a61f9e1114
Minor clean-up (#1078)
- Remove default arguments to `open()`.
- Make use of `pytest` mechanisms for temporary folders.
2021-05-29 12:06:06 +02:00
Miro Hrončok
3865fabf09
Adapt doctest of tests.utils.http to work on Python 3.10 as well (#1075)
https://docs.python.org/3.10/whatsnew/3.10.html#enum

Python 3.10 changed the repr of enum members, and the doctest of tests.utils.http failed.
Exact reprs are unfortunately not considered stable API between Python releases:

    =================================== FAILURES ===================================
    __________________________ [doctest] tests.utils.http __________________________
    209
    210     Example:
    211
    212     $ http --auth=user:password GET pie.dev/basic-auth/user/password
    213
    214         >>> httpbin = getfixture('httpbin')
    215         >>> r = http('-a', 'user:pw', httpbin.url + '/basic-auth/user/pw')
    216         >>> type(r) == StrCLIResponse
    217         True
    218         >>> r.exit_status
    Expected:
        <ExitStatus.SUCCESS: 0>
    Got:
        ExitStatus.SUCCESS

A simple replacement of the expected output however breaks the doctest on Python 3.9.

This is the best solution I could think of
that keeps the docstring readable and doctest working in Pythons both old and new.
2021-05-27 19:54:33 +02:00
Miro Hrončok
355befcbfc
Skip http://pie.dev tests when offline (#1072) 2021-05-27 19:30:36 +02:00
Mickaël Schoentgen
464b5b4c1d
Polish Python 2 removal (#1070) 2021-05-27 13:05:41 +02:00
Mickaël Schoentgen
264d45cdf5
Modernize the code base with f-strings in tests (#1069)
Simple concatenations were kept for readability purpose.
2021-05-26 14:09:38 +02:00
Mickaël Schoentgen
0001297f41
Add --raw to allow specifying the raw request body as an alternative to stdin (#1062)
* Add --raw to allow specifying the raw request body without extra processing

As an alternative to `stdin`.

Co-authored-by: Elena Lape <elapinskaite@gmail.com>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Update README.rst

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Update README.rst

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>

* Fix default HTTP method on empty data

Co-authored-by: Elena Lape <elapinskaite@gmail.com>
Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
2021-05-24 14:29:54 +02:00
Mickaël Schoentgen
a3a08a9a22
Use relative imports (#1057)
* Use relative imports in test

* Use relative imports

* Add myself to contributors :)
2021-05-05 14:13:39 +02:00
Mickaël Schoentgen
1274d869f6
Replace usage of mock with unittest.mock (#1054)
Since Python 3, the mock dependency is no more required as
it is already part of the unittest module.
2021-04-30 15:08:27 +02:00
Jan Verbeek
611bcdaab1
Fail gracefully if multiple request data files are supplied (#1042) 2021-04-15 09:35:50 +02:00
Jan Verbeek
84c7327057
Correctly handle single-byte Content-Range (#1032)
HTTPie previously failed if it continued a download with a single byte left.
2021-02-14 13:30:58 +01:00
Jakub Roztocil
61dbadb730 Tests 2021-02-06 13:21:21 +01:00
Jakub Roztocil
1ac8f69651 Add more output matching tests 2021-02-06 10:52:30 +01:00
Denis Belavin
3c07a25326
Add support for max-age=0 cookie expiry (#1029)
Close #998
2021-02-06 10:50:34 +01:00
Jakub Roztocil
cf78a12e46
Show --check-status warning with --quiet as well. (#1026)
Fixes #1028
2021-01-31 00:58:56 +01:00
Jakub Roztocil
0f1e098cc4
Fix incorrect separators and introduce assert_output_matches() (close #1027) 2021-01-30 22:14:57 +01:00
Gian Ortiz
db685d58b5
Decode headers using utf-8 only if they are not str (#1020) 2021-01-13 21:45:56 +01:00
Jakub Roztocil
6922a0c912 Switch from httpbin.org to pie.dev 2020-12-24 21:34:30 +01:00
Jakub Roztocil
2afdc958c6 Update URLs 2020-12-23 22:07:27 +01:00
Jakub Roztocil
57b1baf1d1 Add a test to reproduce #1006 2020-12-22 22:56:45 +01:00
Shubhendra Singh Chauhan
9883a46575
Cleanup (#993) 2020-12-21 12:03:25 +01:00