Finish the job of moving shapes into the stream

This commit should finish the `coloring_in_tokens` feature, which moves
the shape accumulator into the token stream. This allows rollbacks of
the token stream to also roll back any shapes that were added.

This commit also adds a much nicer syntax highlighter trace, which shows
all of the paths the highlighter took to arrive at a particular coloring
output. This change is fairly substantial, but really improves the
understandability of the flow. I intend to update the normal parser with
a similar tracing view.

In general, this change also fleshes out the concept of "atomic" token
stream operations.

A good next step would be to try to make the parser more
error-correcting, using the coloring infrastructure. A follow-up step
would involve merging the parser and highlighter shapes themselves.
This commit is contained in:
Yehuda Katz
2019-10-21 08:18:43 -07:00
parent 82b24d9beb
commit 6a7c00eaef
22 changed files with 888 additions and 361 deletions

View File

@ -3,9 +3,6 @@ use log::LevelFilter;
use std::error::Error;
fn main() -> Result<(), Box<dyn Error>> {
#[cfg(feature1)]
println!("feature1 is enabled");
let matches = App::new("nushell")
.version(clap::crate_version!())
.arg(