Yehuda Katz d07789677f Clean up lexer (#2956) ... * Document the lexer and lightly improve its names The bulk of this pull request adds a substantial amount of new inline documentation for the lexer. Along the way, I made a few minor changes to the names in the lexer, most of which were internal. The main change that affects other files is renaming `group` to `block`, since the function is actually parsing a block (a list of groups). * Further clean up the lexer - Consolidate the logic of the various token builders into a single type - Improve and clean up the event-driven BlockParser - Clean up comment parsing. Comments now contain their original leading whitespace as well as trailing whitespace, and know how to move some leading whitespace back into the body based on how the lexer decides to dedent the comments. This preserves the original whitespace information while still making it straight-forward to eliminate leading whitespace in help comments. * Update meta.rs * WIP * fix clippy * remove unwraps * remove unwraps Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com> Co-authored-by: Jonathan Turner <jonathan.d.turner@gmail.com>		2021-02-04 20:20:21 +13:00
..
src	Clean up lexer (#2956)	2021-02-04 20:20:21 +13:00
Cargo.toml	Clean up lexer (#2956)	2021-02-04 20:20:21 +13:00