Implement basic metrics layout & total elapsed time (#1250)

* Initial metadata processing

* Dynamic coloring and other stuff

* Use -vv / --meta

* More testing

* Cleanup

* Tweek message

Co-authored-by: Jakub Roztocil <jakub@roztocil.co>
This commit is contained in:
Batuhan Taskaya 2021-12-23 23:13:25 +03:00 committed by GitHub
parent e0e03f3237
commit f3b500119c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
23 changed files with 334 additions and 94 deletions

View File

@ -14,6 +14,7 @@ This project adheres to [Semantic Versioning](https://semver.org/).
- Added support for _receiving_ multiple HTTP headers lines with the same name. ([#1207](https://github.com/httpie/httpie/issues/1207)) - Added support for _receiving_ multiple HTTP headers lines with the same name. ([#1207](https://github.com/httpie/httpie/issues/1207))
- Added support for basic JSON types on `--form`/`--multipart` when using JSON only operators (`:=`/`:=@`). ([#1212](https://github.com/httpie/httpie/issues/1212)) - Added support for basic JSON types on `--form`/`--multipart` when using JSON only operators (`:=`/`:=@`). ([#1212](https://github.com/httpie/httpie/issues/1212))
- Added support for automatically enabling `--stream` when `Content-Type` is `text/event-stream`. ([#376](https://github.com/httpie/httpie/issues/376)) - Added support for automatically enabling `--stream` when `Content-Type` is `text/event-stream`. ([#376](https://github.com/httpie/httpie/issues/376))
- Added support for displaying the total elapsed time throguh `--meta`/`-vv` or `--print=m`. ([#243](https://github.com/httpie/httpie/issues/243))
- Added new `pie-dark`/`pie-light` (and `pie`) styles that match with [HTTPie for Web and Desktop](https://httpie.io/product). ([#1237](https://github.com/httpie/httpie/issues/1237)) - Added new `pie-dark`/`pie-light` (and `pie`) styles that match with [HTTPie for Web and Desktop](https://httpie.io/product). ([#1237](https://github.com/httpie/httpie/issues/1237))
- Added support for better error handling on DNS failures. ([#1248](https://github.com/httpie/httpie/issues/1248)) - Added support for better error handling on DNS failures. ([#1248](https://github.com/httpie/httpie/issues/1248))
- Broken plugins will no longer crash the whole application. ([#1204](https://github.com/httpie/httpie/issues/1204)) - Broken plugins will no longer crash the whole application. ([#1204](https://github.com/httpie/httpie/issues/1204))

View File

@ -1497,13 +1497,15 @@ By default, HTTPie only outputs the final response and the whole response
message is printed (headers as well as the body). You can control what should message is printed (headers as well as the body). You can control what should
be printed via several options: be printed via several options:
| Option | What is printed | | Option | What is printed |
| --------------: | -------------------------------------------------------------------------------------------------- | | -------------------------: | -------------------------------------------------------------------------------------------------- |
| `--headers, -h` | Only the response headers are printed | | `--headers, -h` | Only the response headers are printed |
| `--body, -b` | Only the response body is printed | | `--body, -b` | Only the response body is printed |
| `--verbose, -v` | Print the whole HTTP exchange (request and response). This option also enables `--all` (see below) | | `--meta, -m` | Only the response metadata is printed (various metrics like total elapsed time) |
| `--print, -p` | Selects parts of the HTTP exchange | | `--verbose, -v` | Print the whole HTTP exchange (request and response). This option also enables `--all` (see below) |
| `--quiet, -q` | Don't print anything to `stdout` and `stderr` | | `--verbose --verbose, -vv` | Just like `-v`, but also include the response metadata. |
| `--print, -p` | Selects parts of the HTTP exchange |
| `--quiet, -q` | Don't print anything to `stdout` and `stderr` |
### What parts of the HTTP exchange should be printed ### What parts of the HTTP exchange should be printed
@ -1516,6 +1518,7 @@ It accepts a string of characters each of which represents a specific part of th
| `B` | request body | | `B` | request body |
| `h` | response headers | | `h` | response headers |
| `b` | response body | | `b` | response body |
| `m` | response meta |
Print request and response headers: Print request and response headers:
@ -1552,6 +1555,15 @@ Server: gunicorn/0.13.4
} }
``` ```
#### Verbosity Level: 2
If you run HTTPie with `-vv` or `--verbose --verbose`, then it would also display the response metadata.
```bash
# Just like the above, but with additional columns like the total elapsed time
$ http -vv pie.dev/get
```
### Quiet output ### Quiet output
`--quiet` redirects all output that would otherwise go to `stdout` and `stderr` to `/dev/null` (except for errors and warnings). `--quiet` redirects all output that would otherwise go to `stdout` and `stderr` to `/dev/null` (except for errors and warnings).

View File

@ -15,7 +15,7 @@ from .argtypes import (
parse_format_options, parse_format_options,
) )
from .constants import ( from .constants import (
HTTP_GET, HTTP_POST, OUTPUT_OPTIONS, OUTPUT_OPTIONS_DEFAULT, HTTP_GET, HTTP_POST, BASE_OUTPUT_OPTIONS, OUTPUT_OPTIONS, OUTPUT_OPTIONS_DEFAULT,
OUTPUT_OPTIONS_DEFAULT_OFFLINE, OUTPUT_OPTIONS_DEFAULT_STDOUT_REDIRECTED, OUTPUT_OPTIONS_DEFAULT_OFFLINE, OUTPUT_OPTIONS_DEFAULT_STDOUT_REDIRECTED,
OUT_RESP_BODY, PRETTY_MAP, PRETTY_STDOUT_TTY_ONLY, RequestType, OUT_RESP_BODY, PRETTY_MAP, PRETTY_STDOUT_TTY_ONLY, RequestType,
SEPARATOR_CREDENTIALS, SEPARATOR_CREDENTIALS,
@ -456,8 +456,10 @@ class HTTPieArgumentParser(BaseHTTPieArgumentParser):
self.args.all = True self.args.all = True
if self.args.output_options is None: if self.args.output_options is None:
if self.args.verbose: if self.args.verbose >= 2:
self.args.output_options = ''.join(OUTPUT_OPTIONS) self.args.output_options = ''.join(OUTPUT_OPTIONS)
elif self.args.verbose == 1:
self.args.output_options = ''.join(BASE_OUTPUT_OPTIONS)
elif self.args.offline: elif self.args.offline:
self.args.output_options = OUTPUT_OPTIONS_DEFAULT_OFFLINE self.args.output_options = OUTPUT_OPTIONS_DEFAULT_OFFLINE
elif not self.env.stdout_isatty: elif not self.env.stdout_isatty:

View File

@ -73,12 +73,18 @@ OUT_REQ_HEAD = 'H'
OUT_REQ_BODY = 'B' OUT_REQ_BODY = 'B'
OUT_RESP_HEAD = 'h' OUT_RESP_HEAD = 'h'
OUT_RESP_BODY = 'b' OUT_RESP_BODY = 'b'
OUT_RESP_META = 'm'
OUTPUT_OPTIONS = frozenset({ BASE_OUTPUT_OPTIONS = frozenset({
OUT_REQ_HEAD, OUT_REQ_HEAD,
OUT_REQ_BODY, OUT_REQ_BODY,
OUT_RESP_HEAD, OUT_RESP_HEAD,
OUT_RESP_BODY OUT_RESP_BODY,
})
OUTPUT_OPTIONS = frozenset({
*BASE_OUTPUT_OPTIONS,
OUT_RESP_META,
}) })
# Pretty # Pretty

View File

@ -14,7 +14,7 @@ from .argtypes import (
from .constants import ( from .constants import (
DEFAULT_FORMAT_OPTIONS, OUTPUT_OPTIONS, DEFAULT_FORMAT_OPTIONS, OUTPUT_OPTIONS,
OUTPUT_OPTIONS_DEFAULT, OUT_REQ_BODY, OUT_REQ_HEAD, OUTPUT_OPTIONS_DEFAULT, OUT_REQ_BODY, OUT_REQ_HEAD,
OUT_RESP_BODY, OUT_RESP_HEAD, PRETTY_MAP, PRETTY_STDOUT_TTY_ONLY, OUT_RESP_BODY, OUT_RESP_HEAD, OUT_RESP_META, PRETTY_MAP, PRETTY_STDOUT_TTY_ONLY,
RequestType, SEPARATOR_GROUP_ALL_ITEMS, SEPARATOR_PROXY, RequestType, SEPARATOR_GROUP_ALL_ITEMS, SEPARATOR_PROXY,
SORTED_FORMAT_OPTIONS_STRING, SORTED_FORMAT_OPTIONS_STRING,
UNSORTED_FORMAT_OPTIONS_STRING, UNSORTED_FORMAT_OPTIONS_STRING,
@ -401,6 +401,16 @@ output_options.add_argument(
''' '''
) )
output_options.add_argument(
'--meta', '-m',
dest='output_options',
action='store_const',
const=OUT_RESP_META,
help=f'''
Print only the response metadata. Shortcut for --print={OUT_RESP_META}.
'''
)
output_options.add_argument( output_options.add_argument(
'--body', '-b', '--body', '-b',
dest='output_options', dest='output_options',
@ -415,7 +425,8 @@ output_options.add_argument(
output_options.add_argument( output_options.add_argument(
'--verbose', '-v', '--verbose', '-v',
dest='verbose', dest='verbose',
action='store_true', action='count',
default=0,
help=f''' help=f'''
Verbose output. Print the whole request as well as the response. Also print Verbose output. Print the whole request as well as the response. Also print
any intermediary requests/responses (such as redirects). any intermediary requests/responses (such as redirects).

View File

@ -3,21 +3,20 @@ import os
import platform import platform
import sys import sys
import socket import socket
from typing import List, Optional, Tuple, Union, Callable from typing import List, Optional, Union, Callable
import requests import requests
from pygments import __version__ as pygments_version from pygments import __version__ as pygments_version
from requests import __version__ as requests_version from requests import __version__ as requests_version
from . import __version__ as httpie_version from . import __version__ as httpie_version
from .cli.constants import OUT_REQ_BODY, OUT_REQ_HEAD, OUT_RESP_BODY, OUT_RESP_HEAD from .cli.constants import OUT_REQ_BODY
from .client import collect_messages from .client import collect_messages
from .context import Environment from .context import Environment
from .downloads import Downloader from .downloads import Downloader
from .models import ( from .models import (
RequestsMessage,
RequestsMessageKind, RequestsMessageKind,
infer_requests_message_kind OutputOptions,
) )
from .output.writer import write_message, write_stream, MESSAGE_SEPARATOR_BYTES from .output.writer import write_message, write_stream, MESSAGE_SEPARATOR_BYTES
from .plugins.registry import plugin_manager from .plugins.registry import plugin_manager
@ -112,9 +111,9 @@ def raw_main(
original_exc = unwrap_context(exc) original_exc = unwrap_context(exc)
if isinstance(original_exc, socket.gaierror): if isinstance(original_exc, socket.gaierror):
if original_exc.errno == socket.EAI_AGAIN: if original_exc.errno == socket.EAI_AGAIN:
annotation = '\nCouldn\'t connect to a DNS server. Perhaps check your connection and try again.' annotation = '\nCouldnt connect to a DNS server. Please check your connection and try again.'
elif original_exc.errno == socket.EAI_NONAME: elif original_exc.errno == socket.EAI_NONAME:
annotation = '\nCouldn\'t resolve the given hostname. Perhaps check it and try again.' annotation = '\nCouldnt resolve the given hostname. Please check the URL and try again.'
propagated_exc = original_exc propagated_exc = original_exc
else: else:
propagated_exc = exc propagated_exc = exc
@ -153,22 +152,6 @@ def main(
) )
def get_output_options(
args: argparse.Namespace,
message: RequestsMessage
) -> Tuple[bool, bool]:
return {
RequestsMessageKind.REQUEST: (
OUT_REQ_HEAD in args.output_options,
OUT_REQ_BODY in args.output_options,
),
RequestsMessageKind.RESPONSE: (
OUT_RESP_HEAD in args.output_options,
OUT_RESP_BODY in args.output_options,
),
}[infer_requests_message_kind(message)]
def program(args: argparse.Namespace, env: Environment) -> ExitStatus: def program(args: argparse.Namespace, env: Environment) -> ExitStatus:
""" """
The main program without error handling. The main program without error handling.
@ -197,7 +180,8 @@ def program(args: argparse.Namespace, env: Environment) -> ExitStatus:
msg.is_body_upload_chunk = True msg.is_body_upload_chunk = True
msg.body = chunk msg.body = chunk
msg.headers = initial_request.headers msg.headers = initial_request.headers
write_message(requests_message=msg, env=env, args=args, with_body=True, with_headers=False) msg_output_options = OutputOptions.from_message(msg, body=True, headers=False)
write_message(requests_message=msg, env=env, args=args, output_options=msg_output_options)
try: try:
if args.download: if args.download:
@ -211,17 +195,17 @@ def program(args: argparse.Namespace, env: Environment) -> ExitStatus:
# Process messages as theyre generated # Process messages as theyre generated
for message in messages: for message in messages:
is_request = isinstance(message, requests.PreparedRequest) output_options = OutputOptions.from_message(message, args.output_options)
with_headers, with_body = get_output_options(args=args, message=message)
do_write_body = with_body do_write_body = output_options.body
if prev_with_body and (with_headers or with_body) and (force_separator or not env.stdout_isatty): if prev_with_body and output_options.any() and (force_separator or not env.stdout_isatty):
# Separate after a previous message with body, if needed. See test_tokens.py. # Separate after a previous message with body, if needed. See test_tokens.py.
separate() separate()
force_separator = False force_separator = False
if is_request: if output_options.kind is RequestsMessageKind.REQUEST:
if not initial_request: if not initial_request:
initial_request = message initial_request = message
if with_body: if output_options.body:
is_streamed_upload = not isinstance(message.body, (str, bytes)) is_streamed_upload = not isinstance(message.body, (str, bytes))
do_write_body = not is_streamed_upload do_write_body = not is_streamed_upload
force_separator = is_streamed_upload and env.stdout_isatty force_separator = is_streamed_upload and env.stdout_isatty
@ -231,9 +215,10 @@ def program(args: argparse.Namespace, env: Environment) -> ExitStatus:
exit_status = http_status_to_exit_status(http_status=message.status_code, follow=args.follow) exit_status = http_status_to_exit_status(http_status=message.status_code, follow=args.follow)
if exit_status != ExitStatus.SUCCESS and (not env.stdout_isatty or args.quiet == 1): if exit_status != ExitStatus.SUCCESS and (not env.stdout_isatty or args.quiet == 1):
env.log_error(f'HTTP {message.raw.status} {message.raw.reason}', level='warning') env.log_error(f'HTTP {message.raw.status} {message.raw.reason}', level='warning')
write_message(requests_message=message, env=env, args=args, with_headers=with_headers, write_message(requests_message=message, env=env, args=args, output_options=output_options._replace(
with_body=do_write_body) body=do_write_body
prev_with_body = with_body ))
prev_with_body = output_options.body
# Cleanup # Cleanup
if force_separator: if force_separator:

View File

@ -14,7 +14,7 @@ from urllib.parse import urlsplit
import requests import requests
from .models import HTTPResponse from .models import HTTPResponse, OutputOptions
from .output.streams import RawStream from .output.streams import RawStream
from .utils import humanize_bytes from .utils import humanize_bytes
@ -266,10 +266,10 @@ class Downloader:
total_size=total_size total_size=total_size
) )
output_options = OutputOptions.from_message(final_response, headers=False, body=True)
stream = RawStream( stream = RawStream(
msg=HTTPResponse(final_response), msg=HTTPResponse(final_response),
with_headers=False, output_options=output_options,
with_body=True,
on_body_chunk_downloaded=self.chunk_downloaded, on_body_chunk_downloaded=self.chunk_downloaded,
) )

View File

@ -1,11 +1,18 @@
import requests import requests
from enum import Enum, auto from enum import Enum, auto
from typing import Iterable, Union from typing import Iterable, Union, NamedTuple
from urllib.parse import urlsplit from urllib.parse import urlsplit
from .utils import split_cookies, parse_content_type_header from .cli.constants import (
OUT_REQ_BODY,
OUT_REQ_HEAD,
OUT_RESP_BODY,
OUT_RESP_HEAD,
OUT_RESP_META
)
from .compat import cached_property from .compat import cached_property
from .utils import split_cookies, parse_content_type_header
class HTTPMessage: class HTTPMessage:
@ -27,6 +34,11 @@ class HTTPMessage:
"""Return a `str` with the message's headers.""" """Return a `str` with the message's headers."""
raise NotImplementedError raise NotImplementedError
@property
def metadata(self) -> str:
"""Return metadata about the current message."""
raise NotImplementedError
@cached_property @cached_property
def encoding(self) -> str: def encoding(self) -> str:
ct, params = parse_content_type_header(self.content_type) ct, params = parse_content_type_header(self.content_type)
@ -81,6 +93,15 @@ class HTTPResponse(HTTPMessage):
) )
return '\r\n'.join(headers) return '\r\n'.join(headers)
@property
def metadata(self) -> str:
data = {}
data['Elapsed time'] = str(self._orig.elapsed.total_seconds()) + 's'
return '\n'.join(
f'{key}: {value}'
for key, value in data.items()
)
class HTTPRequest(HTTPMessage): class HTTPRequest(HTTPMessage):
"""A :class:`requests.models.Request` wrapper.""" """A :class:`requests.models.Request` wrapper."""
@ -138,3 +159,50 @@ def infer_requests_message_kind(message: RequestsMessage) -> RequestsMessageKind
return RequestsMessageKind.RESPONSE return RequestsMessageKind.RESPONSE
else: else:
raise TypeError(f"Unexpected message type: {type(message).__name__}") raise TypeError(f"Unexpected message type: {type(message).__name__}")
OPTION_TO_PARAM = {
RequestsMessageKind.REQUEST: {
'headers': OUT_REQ_HEAD,
'body': OUT_REQ_BODY,
},
RequestsMessageKind.RESPONSE: {
'headers': OUT_RESP_HEAD,
'body': OUT_RESP_BODY,
'meta': OUT_RESP_META
}
}
class OutputOptions(NamedTuple):
kind: RequestsMessageKind
headers: bool
body: bool
meta: bool = False
def any(self):
return (
self.headers
or self.body
or self.meta
)
@classmethod
def from_message(
cls,
message: RequestsMessage,
raw_args: str = '',
**kwargs
):
kind = infer_requests_message_kind(message)
options = {
option: param in raw_args
for option, param in OPTION_TO_PARAM[kind].items()
}
options.update(kwargs)
return cls(
kind=kind,
**options
)

View File

@ -16,6 +16,7 @@ from pygments.lexers.text import HttpLexer as PygmentsHttpLexer
from pygments.util import ClassNotFound from pygments.util import ClassNotFound
from ..lexers.json import EnhancedJsonLexer from ..lexers.json import EnhancedJsonLexer
from ..lexers.metadata import MetadataLexer
from ..ui.palette import SHADE_NAMES, get_color from ..ui.palette import SHADE_NAMES, get_color
from ...compat import is_windows from ...compat import is_windows
from ...context import Environment from ...context import Environment
@ -50,6 +51,7 @@ class ColorFormatter(FormatterPlugin):
""" """
group_name = 'colors' group_name = 'colors'
metadata_lexer = MetadataLexer()
def __init__( def __init__(
self, self,
@ -68,9 +70,8 @@ class ColorFormatter(FormatterPlugin):
has_256_colors = env.colors == 256 has_256_colors = env.colors == 256
if use_auto_style or not has_256_colors: if use_auto_style or not has_256_colors:
http_lexer = PygmentsHttpLexer() http_lexer = PygmentsHttpLexer()
formatter = TerminalFormatter() body_formatter = header_formatter = TerminalFormatter()
body_formatter = formatter precise = False
header_formatter = formatter
else: else:
from ..lexers.http import SimplifiedHTTPLexer from ..lexers.http import SimplifiedHTTPLexer
header_formatter, body_formatter, precise = self.get_formatters(color_scheme) header_formatter, body_formatter, precise = self.get_formatters(color_scheme)
@ -80,6 +81,7 @@ class ColorFormatter(FormatterPlugin):
self.header_formatter = header_formatter self.header_formatter = header_formatter
self.body_formatter = body_formatter self.body_formatter = body_formatter
self.http_lexer = http_lexer self.http_lexer = http_lexer
self.metadata_lexer = MetadataLexer(precise=precise)
def format_headers(self, headers: str) -> str: def format_headers(self, headers: str) -> str:
return pygments.highlight( return pygments.highlight(
@ -98,6 +100,13 @@ class ColorFormatter(FormatterPlugin):
) )
return body return body
def format_metadata(self, metadata: str) -> str:
return pygments.highlight(
code=metadata,
lexer=self.metadata_lexer,
formatter=self.header_formatter,
).strip()
def get_lexer_for_body( def get_lexer_for_body(
self, mime: str, self, mime: str,
body: str body: str
@ -288,6 +297,13 @@ PIE_HEADER_STYLE = {
pygments.token.Number.HTTP.REDIRECT: 'bold yellow', pygments.token.Number.HTTP.REDIRECT: 'bold yellow',
pygments.token.Number.HTTP.CLIENT_ERR: 'bold orange', pygments.token.Number.HTTP.CLIENT_ERR: 'bold orange',
pygments.token.Number.HTTP.SERVER_ERR: 'bold red', pygments.token.Number.HTTP.SERVER_ERR: 'bold red',
# Metadata
pygments.token.Name.Decorator: 'grey',
pygments.token.Number.SPEED.FAST: 'bold green',
pygments.token.Number.SPEED.AVG: 'bold yellow',
pygments.token.Number.SPEED.SLOW: 'bold orange',
pygments.token.Number.SPEED.VERY_SLOW: 'bold red',
} }
PIE_BODY_STYLE = { PIE_BODY_STYLE = {

View File

@ -0,0 +1,12 @@
def precise(lexer, precise_token, parent_token):
# Due to a pygments bug*, custom tokens will look bad
# on outside styles. Until it is fixed on upstream, we'll
# convey whether the client is using pie style or not
# through precise option and return more precise tokens
# depending on it's value.
#
# [0]: https://github.com/pygments/pygments/issues/1986
if precise_token is None or not lexer.options.get("precise"):
return parent_token
else:
return precise_token

View File

@ -1,6 +1,6 @@
import re import re
import pygments import pygments
from httpie.output.lexers.common import precise
RE_STATUS_LINE = re.compile(r'(\d{3})( +)(.+)') RE_STATUS_LINE = re.compile(r'(\d{3})( +)(.+)')
@ -22,20 +22,6 @@ RESPONSE_TYPES = {
} }
def precise(lexer, precise_token, parent_token):
# Due to a pygments bug*, custom tokens will look bad
# on outside styles. Until it is fixed on upstream, we'll
# convey whether the client is using pie style or not
# through precise option and return more precise tokens
# depending on it's value.
#
# [0]: https://github.com/pygments/pygments/issues/1986
if precise_token is None or not lexer.options.get("precise"):
return parent_token
else:
return precise_token
def http_response_type(lexer, match, ctx): def http_response_type(lexer, match, ctx):
status_match = RE_STATUS_LINE.match(match.group()) status_match = RE_STATUS_LINE.match(match.group())
if status_match is None: if status_match is None:

View File

@ -0,0 +1,57 @@
import pygments
from httpie.output.lexers.common import precise
SPEED_TOKENS = {
0.45: pygments.token.Number.SPEED.FAST,
1.00: pygments.token.Number.SPEED.AVG,
2.50: pygments.token.Number.SPEED.SLOW,
}
def speed_based_token(lexer, match, ctx):
try:
value = float(match.group())
except ValueError:
return pygments.token.Number
for limit, token in SPEED_TOKENS.items():
if value <= limit:
break
else:
token = pygments.token.Number.SPEED.VERY_SLOW
response_type = precise(
lexer,
token,
pygments.token.Number
)
yield match.start(), response_type, match.group()
class MetadataLexer(pygments.lexer.RegexLexer):
"""Simple HTTPie metadata lexer."""
tokens = {
'root': [
(
r'(Elapsed time)( *)(:)( *)(\d+\.\d+)(s)', pygments.lexer.bygroups(
pygments.token.Name.Decorator, # Name
pygments.token.Text,
pygments.token.Operator, # Colon
pygments.token.Text,
speed_based_token,
pygments.token.Name.Builtin # Value
)
),
# Generic item
(
r'(.*?)( *)(:)( *)(.+)', pygments.lexer.bygroups(
pygments.token.Name.Decorator, # Name
pygments.token.Text,
pygments.token.Operator, # Colon
pygments.token.Text,
pygments.token.Text # Value
)
),
]
}

View File

@ -51,3 +51,8 @@ class Formatting:
for p in self.enabled_plugins: for p in self.enabled_plugins:
content = p.format_body(content, mime) content = p.format_body(content, mime)
return content return content
def format_metadata(self, metadata: str) -> str:
for p in self.enabled_plugins:
metadata = p.format_metadata(metadata)
return metadata

View File

@ -5,7 +5,7 @@ from typing import Callable, Iterable, Optional, Union
from .processing import Conversion, Formatting from .processing import Conversion, Formatting
from ..context import Environment from ..context import Environment
from ..encoding import smart_decode, smart_encode, UTF8 from ..encoding import smart_decode, smart_encode, UTF8
from ..models import HTTPMessage from ..models import HTTPMessage, OutputOptions
from ..utils import parse_content_type_header from ..utils import parse_content_type_header
@ -33,47 +33,58 @@ class BaseStream(metaclass=ABCMeta):
def __init__( def __init__(
self, self,
msg: HTTPMessage, msg: HTTPMessage,
with_headers=True, output_options: OutputOptions,
with_body=True,
on_body_chunk_downloaded: Callable[[bytes], None] = None on_body_chunk_downloaded: Callable[[bytes], None] = None
): ):
""" """
:param msg: a :class:`models.HTTPMessage` subclass :param msg: a :class:`models.HTTPMessage` subclass
:param with_headers: if `True`, headers will be included :param output_options: a :class:`OutputOptions` instance to represent
:param with_body: if `True`, body will be included which parts of the message is printed.
""" """
assert with_headers or with_body assert output_options.any()
self.msg = msg self.msg = msg
self.with_headers = with_headers self.output_options = output_options
self.with_body = with_body
self.on_body_chunk_downloaded = on_body_chunk_downloaded self.on_body_chunk_downloaded = on_body_chunk_downloaded
def get_headers(self) -> bytes: def get_headers(self) -> bytes:
"""Return the headers' bytes.""" """Return the headers' bytes."""
return self.msg.headers.encode() return self.msg.headers.encode()
def get_metadata(self) -> bytes:
"""Return the message metadata."""
return self.msg.metadata.encode()
@abstractmethod @abstractmethod
def iter_body(self) -> Iterable[bytes]: def iter_body(self) -> Iterable[bytes]:
"""Return an iterator over the message body.""" """Return an iterator over the message body."""
def __iter__(self) -> Iterable[bytes]: def __iter__(self) -> Iterable[bytes]:
"""Return an iterator over `self.msg`.""" """Return an iterator over `self.msg`."""
if self.with_headers: if self.output_options.headers:
yield self.get_headers() yield self.get_headers()
yield b'\r\n\r\n' yield b'\r\n\r\n'
if self.with_body: if self.output_options.body:
try: try:
for chunk in self.iter_body(): for chunk in self.iter_body():
yield chunk yield chunk
if self.on_body_chunk_downloaded: if self.on_body_chunk_downloaded:
self.on_body_chunk_downloaded(chunk) self.on_body_chunk_downloaded(chunk)
except DataSuppressedError as e: except DataSuppressedError as e:
if self.with_headers: if self.output_options.headers:
yield b'\n' yield b'\n'
yield e.message yield e.message
if self.output_options.meta:
mixed = self.output_options.headers or self.output_options.body
if mixed:
yield b'\n\n'
yield self.get_metadata()
if not mixed:
yield b'\n'
class RawStream(BaseStream): class RawStream(BaseStream):
"""The message is streamed in chunks with no processing.""" """The message is streamed in chunks with no processing."""
@ -181,6 +192,10 @@ class PrettyStream(EncodedStream):
return self.formatting.format_headers( return self.formatting.format_headers(
self.msg.headers).encode(self.output_encoding) self.msg.headers).encode(self.output_encoding)
def get_metadata(self) -> bytes:
return self.formatting.format_metadata(
self.msg.metadata).encode(self.output_encoding)
def iter_body(self) -> Iterable[bytes]: def iter_body(self) -> Iterable[bytes]:
first_chunk = True first_chunk = True
iter_lines = self.msg.iter_lines(self.CHUNK_SIZE) iter_lines = self.msg.iter_lines(self.CHUNK_SIZE)

View File

@ -10,7 +10,7 @@ from ..models import (
HTTPMessage, HTTPMessage,
RequestsMessage, RequestsMessage,
RequestsMessageKind, RequestsMessageKind,
infer_requests_message_kind OutputOptions
) )
from .processing import Conversion, Formatting from .processing import Conversion, Formatting
from .streams import ( from .streams import (
@ -26,18 +26,16 @@ def write_message(
requests_message: RequestsMessage, requests_message: RequestsMessage,
env: Environment, env: Environment,
args: argparse.Namespace, args: argparse.Namespace,
with_headers=False, output_options: OutputOptions,
with_body=False,
): ):
if not (with_body or with_headers): if not output_options.any():
return return
write_stream_kwargs = { write_stream_kwargs = {
'stream': build_output_stream_for_message( 'stream': build_output_stream_for_message(
args=args, args=args,
env=env, env=env,
requests_message=requests_message, requests_message=requests_message,
with_body=with_body, output_options=output_options,
with_headers=with_headers,
), ),
# NOTE: `env.stdout` will in fact be `stderr` with `--download` # NOTE: `env.stdout` will in fact be `stderr` with `--download`
'outfile': env.stdout, 'outfile': env.stdout,
@ -100,13 +98,12 @@ def build_output_stream_for_message(
args: argparse.Namespace, args: argparse.Namespace,
env: Environment, env: Environment,
requests_message: RequestsMessage, requests_message: RequestsMessage,
with_headers: bool, output_options: OutputOptions,
with_body: bool,
): ):
message_type = { message_type = {
RequestsMessageKind.REQUEST: HTTPRequest, RequestsMessageKind.REQUEST: HTTPRequest,
RequestsMessageKind.RESPONSE: HTTPResponse, RequestsMessageKind.RESPONSE: HTTPResponse,
}[infer_requests_message_kind(requests_message)] }[output_options.kind]
stream_class, stream_kwargs = get_stream_type_and_kwargs( stream_class, stream_kwargs = get_stream_type_and_kwargs(
env=env, env=env,
args=args, args=args,
@ -115,11 +112,10 @@ def build_output_stream_for_message(
) )
yield from stream_class( yield from stream_class(
msg=message_type(requests_message), msg=message_type(requests_message),
with_headers=with_headers, output_options=output_options,
with_body=with_body,
**stream_kwargs, **stream_kwargs,
) )
if (env.stdout_isatty and with_body if (env.stdout_isatty and output_options.body
and not getattr(requests_message, 'is_body_upload_chunk', False)): and not getattr(requests_message, 'is_body_upload_chunk', False)):
# Ensure a blank line after the response body. # Ensure a blank line after the response body.
# For terminal output only. # For terminal output only.

View File

@ -155,3 +155,11 @@ class FormatterPlugin(BasePlugin):
""" """
return content return content
def format_metadata(self, metadata: str) -> str:
"""Return processed `metadata`.
:param metadata: The metadata as text.
"""
return metadata

View File

@ -1,6 +1,7 @@
import os import os
import tempfile import tempfile
import time import time
import requests
from unittest import mock from unittest import mock
from urllib.request import urlopen from urllib.request import urlopen
@ -14,7 +15,7 @@ from httpie.downloads import (
from .utils import http, MockEnvironment from .utils import http, MockEnvironment
class Response: class Response(requests.Response):
# noinspection PyDefaultArgument # noinspection PyDefaultArgument
def __init__(self, url, headers={}, status_code=200): def __init__(self, url, headers={}, status_code=200):
self.url = url self.url = url

7
tests/test_meta.py Normal file
View File

@ -0,0 +1,7 @@
from .utils import http
def test_meta_elapsed_time(httpbin, monkeypatch):
r = http('--meta', httpbin + '/get')
for line in r.splitlines():
assert 'Elapsed time' in r

View File

@ -17,7 +17,7 @@ from httpie.cli.argtypes import (
) )
from httpie.cli.definition import parser from httpie.cli.definition import parser
from httpie.encoding import UTF8 from httpie.encoding import UTF8
from httpie.output.formatters.colors import get_lexer from httpie.output.formatters.colors import PIE_STYLES, get_lexer
from httpie.status import ExitStatus from httpie.status import ExitStatus
from .fixtures import XML_DATA_RAW, XML_DATA_FORMATTED from .fixtures import XML_DATA_RAW, XML_DATA_FORMATTED
from .utils import COLOR, CRLF, HTTP_OK, MockEnvironment, http, DUMMY_URL from .utils import COLOR, CRLF, HTTP_OK, MockEnvironment, http, DUMMY_URL
@ -227,6 +227,13 @@ def test_ensure_contents_colored(httpbin, endpoint):
assert COLOR in r assert COLOR in r
@pytest.mark.parametrize('style', PIE_STYLES.keys())
def test_ensure_meta_is_colored(httpbin, style):
env = MockEnvironment(colors=256)
r = http('--meta', '--style', style, 'GET', httpbin + '/get', env=env)
assert COLOR in r
class TestPrettyOptions: class TestPrettyOptions:
"""Test the --pretty handling.""" """Test the --pretty handling."""

View File

@ -101,3 +101,18 @@ def test_verbose_chunked(httpbin_with_chunked_support):
def test_request_headers_response_body(httpbin): def test_request_headers_response_body(httpbin):
r = http('--print=Hb', httpbin + '/get') r = http('--print=Hb', httpbin + '/get')
assert_output_matches(r, ExpectSequence.TERMINAL_REQUEST) assert_output_matches(r, ExpectSequence.TERMINAL_REQUEST)
def test_request_single_verbose(httpbin):
r = http('-v', httpbin + '/post', 'hello=world')
assert_output_matches(r, ExpectSequence.TERMINAL_EXCHANGE)
def test_request_double_verbose(httpbin):
r = http('-vv', httpbin + '/post', 'hello=world')
assert_output_matches(r, ExpectSequence.TERMINAL_EXCHANGE_META)
def test_request_meta(httpbin):
r = http('--meta', httpbin + '/get')
assert_output_matches(r, [Expect.RESPONSE_META])

View File

@ -7,6 +7,7 @@ from ...utils import CRLF
SEPARATOR_RE = re.compile(f'^{MESSAGE_SEPARATOR}') SEPARATOR_RE = re.compile(f'^{MESSAGE_SEPARATOR}')
KEY_VALUE_RE = re.compile(r'[\n]*((.*?):(.+)[\n]?)+[\n]*')
def make_headers_re(message_type: Expect): def make_headers_re(message_type: Expect):
@ -43,6 +44,7 @@ BODY_ENDINGS = [
TOKEN_REGEX_MAP = { TOKEN_REGEX_MAP = {
Expect.REQUEST_HEADERS: make_headers_re(Expect.REQUEST_HEADERS), Expect.REQUEST_HEADERS: make_headers_re(Expect.REQUEST_HEADERS),
Expect.RESPONSE_HEADERS: make_headers_re(Expect.RESPONSE_HEADERS), Expect.RESPONSE_HEADERS: make_headers_re(Expect.RESPONSE_HEADERS),
Expect.RESPONSE_META: KEY_VALUE_RE,
Expect.SEPARATOR: SEPARATOR_RE, Expect.SEPARATOR: SEPARATOR_RE,
} }

View File

@ -107,6 +107,29 @@ def test_assert_output_matches_headers_with_body_and_separator():
) )
def test_assert_output_matches_response_meta():
assert_output_matches(
(
'Key: Value\n'
'Elapsed Time: 3.3s'
),
[Expect.RESPONSE_META]
)
def test_assert_output_matches_whole_response():
assert_output_matches(
(
f'HTTP/1.1{CRLF}'
f'AAA:BBB{CRLF}'
f'{CRLF}'
f'CCC{MESSAGE_SEPARATOR}'
'Elapsed Time: 3.3s'
),
[Expect.RESPONSE_HEADERS, Expect.BODY, Expect.RESPONSE_META]
)
def test_assert_output_matches_multiple_messages(): def test_assert_output_matches_multiple_messages():
assert_output_matches( assert_output_matches(
( (

View File

@ -8,6 +8,7 @@ class Expect(Enum):
""" """
REQUEST_HEADERS = auto() REQUEST_HEADERS = auto()
RESPONSE_HEADERS = auto() RESPONSE_HEADERS = auto()
RESPONSE_META = auto()
BODY = auto() BODY = auto()
SEPARATOR = auto() SEPARATOR = auto()
@ -45,6 +46,10 @@ class ExpectSequence:
*TERMINAL_REQUEST, *TERMINAL_REQUEST,
*TERMINAL_RESPONSE, *TERMINAL_RESPONSE,
] ]
TERMINAL_EXCHANGE_META = [
*TERMINAL_EXCHANGE,
Expect.RESPONSE_META
]
TERMINAL_BODY = [ TERMINAL_BODY = [
RAW_BODY, RAW_BODY,
Expect.SEPARATOR Expect.SEPARATOR