nushell/crates/nu-source
Yehuda Katz d07789677f
Clean up lexer (#2956)
* Document the lexer and lightly improve its names

The bulk of this pull request adds a substantial amount of new inline
documentation for the lexer. Along the way, I made a few minor changes
to the names in the lexer, most of which were internal.

The main change that affects other files is renaming `group` to `block`,
since the function is actually parsing a block (a list of groups).

* Further clean up the lexer

- Consolidate the logic of the various token builders into a single type
- Improve and clean up the event-driven BlockParser
- Clean up comment parsing. Comments now contain their original leading
  whitespace as well as trailing whitespace, and know how to move some
  leading whitespace back into the body based on how the lexer decides
  to dedent the comments. This preserves the original whitespace
  information while still making it straight-forward to eliminate leading
  whitespace in help comments.

* Update meta.rs

* WIP

* fix clippy

* remove unwraps

* remove unwraps

Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
Co-authored-by: Jonathan Turner <jonathan.d.turner@gmail.com>
2021-02-04 20:20:21 +13:00
..
src Clean up lexer (#2956) 2021-02-04 20:20:21 +13:00
Cargo.toml Bump to 0.26.0 (#2974) 2021-01-26 23:07:08 +13:00
README.md First pass at updating all documentation formatting and cleaning up output of examples (#2031) 2020-06-24 06:21:47 +12:00

nu-source

Overview

The nu-source crate contains types and traits used for keeping track of metadata about values being processed. Nu uses Tags to keep track of where a value came from, an AnchorLocation, as well as positional information about the value, a Span. An AnchorLocation can be a Url, File, or Source text that a value was parsed from. The source Text is special in that it is a type similar to a String that comes with the ability to be cheaply cloned. A Span keeps track of a value's start and end positions. These types make up the metadata for a value and are wrapped up together in a Tagged struct, which holds everything needed to track and locate a value.

Nu's metadata system can be seen when reporting errors. In the following example Nu is able to report to the user where the typo of a column originated from.

1 | ls | get typ
  |          ^^^ did you mean 'type'?

In addition to metadata tracking, nu-source also contains types and traits related to debugging, tracing, and formatting the metadata and values it processes.

Other Resources