Add top-level crate documentation/READMEs (#12907)

# Description
Add `README.md` files to each crate in our workspace (-plugins) and also
include it in the `lib.rs` documentation for <docs.rs> (if there is no
existing `lib.rs` crate documentation)

In all new README I added the defensive comment that the crates are not
considered stable for public consumption. If necessary we can adjust
this if we deem a crate useful for plugin authors.

crates/nu-cli/README.md

@@ -0,0 +1,7 @@
+This crate implements the core functionality of the interactive Nushell REPL and interfaces with `reedline`.
+Currently implements the syntax highlighting and completions logic.
+Furthermore includes a few commands that are specific to `reedline`
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-cli/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod commands;
 mod completions;
 mod config_files;

crates/nu-cmd-base/README.md

@@ -0,0 +1,5 @@
+Utilities used by the different `nu-command`/`nu-cmd-*` crates, should not contain any full `Command` implementations.
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-cmd-base/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 pub mod formats;
 pub mod hook;
 pub mod input_handler;

crates/nu-cmd-extra/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod example_test;
 pub mod extra;
 pub use extra::*;

crates/nu-cmd-lang/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod core_commands;
 mod default_context;
 pub mod example_support;

crates/nu-color-config/README.md

@@ -0,0 +1,5 @@
+Logic to resolve colors for syntax highlighting and output formatting
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-color-config/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod color_config;
 mod matching_brackets_style;
 mod nu_style;

crates/nu-command/README.md

@@ -0,0 +1,7 @@
+This crate contains the majority of our commands
+
+We allow ourselves to move some of the commands in `nu-command` to `nu-cmd-*` crates as needed.
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-command/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod bytes;
 mod charting;
 mod conversions;

crates/nu-engine/README.md

@@ -0,0 +1,9 @@
+This crate primarily drives the evaluation of expressions.
+
+(Some overlap with nu-protocol)
+
+- Provides `CallExt`
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-engine/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod call_ext;
 mod closure_eval;
 pub mod column;

crates/nu-explore/README.md

@@ -0,0 +1,5 @@
+Implementation of the interactive `explore` command pager.
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-explore/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod commands;
 mod default_context;
 mod explore;

crates/nu-json/README.md

@@ -24,7 +24,7 @@ nu-json = "0.76"
 ## From the Commandline
 
 Add with:
-```
+```sh
  cargo add serde
  cargo add nu-json
 ```
@@ -43,7 +43,7 @@ fn main() {
 
     let sample_text=r#"
     {
-        # specify rate in requests/second
+        ## specify rate in requests/second
         rate: 1000
         array:
         [

crates/nu-json/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 pub use self::de::{
     from_iter, from_reader, from_slice, from_str, Deserializer, StreamDeserializer,
 };

crates/nu-lsp/README.md

@@ -0,0 +1,7 @@
+Implementation of the Nushell language server.
+
+See [the Language Server Protocol specification](https://microsoft.github.io/language-server-protocol/)
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-lsp/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 use lsp_server::{Connection, IoThreads, Message, Response, ResponseError};
 use lsp_types::{
     request::{Completion, GotoDefinition, HoverRequest, Request},

crates/nu-parser/README.md

@@ -4,7 +4,7 @@ Nushell's parser is a type-directed parser, meaning that the parser will use typ
 
 Nushell's base language is whitespace-separated tokens with the command (Nushell's term for a function) name in the head position:
 
-```
+```nushell
 head1 arg1 arg2 | head2
 ```
 
@@ -12,7 +12,7 @@ head1 arg1 arg2 | head2
 
 The first job of the parser is to a lexical analysis to find where the tokens start and end in the input. This turns the above into:
 
-```
+```text
 <item: "head1">, <item: "arg1">, <item: "arg2">, <pipe>, <item: "head2">
 ```
 
@@ -24,7 +24,7 @@ As Nushell is a language of pipelines, pipes form a key role in both separating
 
 The above tokens are converted the following during the lite parse phase:
 
-```
+```text
 Pipeline:
   Command #1:
     <item: "head1">, <item: "arg1">, <item: "arg2">
@@ -45,7 +45,7 @@ Each command has a shape assigned to each of the arguments it reads in. These sh
 
 For example, if the command is written as:
 
-```sql
+```text
 where $x > 10
 ```
 
@@ -53,7 +53,7 @@ When the parsing happens, the parser will look up the `where` command and find i
 
 In the above example, if the Signature of `where` said that it took three String values, the result would be:
 
-```
+```text
 CallInfo:
   Name: `where`
   Args:
@@ -64,7 +64,7 @@ CallInfo:
 
 Or, the Signature could state that it takes in three positional arguments: a Variable, an Operator, and a Number, which would give:
 
-```
+```text
 CallInfo:
   Name: `where`
   Args:
@@ -77,7 +77,7 @@ Note that in this case, each would be checked at compile time to confirm that th
 
 Finally, some Shapes can consume more than one token. In the above, if the `where` command stated it took in a single required argument, and that the Shape of this argument was a MathExpression, then the parser would treat the remaining tokens as part of the math expression.
 
-```
+```text
 CallInfo:
   Name: `where`
   Args:

crates/nu-parser/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod deparse;
 mod exportable;
 mod flatten;

crates/nu-path/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod assert_path_eq;
 mod components;
 pub mod dots;

crates/nu-protocol/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod alias;
 pub mod ast;
 pub mod config;

crates/nu-protocol/src/pipeline/byte_stream.rs

@@ -159,14 +159,14 @@ impl From<ByteStreamType> for Type {
 ///   Try not to use this method if possible. Rather, please use [`reader`](ByteStream::reader)
 ///   (or [`lines`](ByteStream::lines) if it matches the situation).
 ///
-/// Additionally, there are few methods to collect a [`Bytestream`] into memory:
+/// Additionally, there are few methods to collect a [`ByteStream`] into memory:
 /// - [`into_bytes`](ByteStream::into_bytes): collects all bytes into a [`Vec<u8>`].
 /// - [`into_string`](ByteStream::into_string): collects all bytes into a [`String`], erroring if utf-8 decoding failed.
 /// - [`into_value`](ByteStream::into_value): collects all bytes into a value typed appropriately
 ///   for the [type](.type_()) of this stream. If the type is [`Unknown`](ByteStreamType::Unknown),
 ///   it will produce a string value if the data is valid UTF-8, or a binary value otherwise.
 ///
-/// There are also a few other methods to consume all the data of a [`Bytestream`]:
+/// There are also a few other methods to consume all the data of a [`ByteStream`]:
 /// - [`drain`](ByteStream::drain): consumes all bytes and outputs nothing.
 /// - [`write_to`](ByteStream::write_to): writes all bytes to the given [`Write`] destination.
 /// - [`print`](ByteStream::print): a convenience wrapper around [`write_to`](ByteStream::write_to).

crates/nu-std/README.md

@@ -7,7 +7,7 @@ The standard library is a pure-`nushell` collection of custom commands which
 provide interactive utilities and building blocks for users writing casual scripts or complex applications.
 
 To see what's here:
-```
+```text
 > use std
 > scope commands | select name usage | where name =~ "std "
 #┬───────────name────────────┬──────────────────────usage──────────────────────

crates/nu-std/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 use log::trace;
 use nu_engine::eval_block;
 use nu_parser::parse;

crates/nu-system/README.md

@@ -0,0 +1,7 @@
+Operating system specific bindings used by Nushell.
+
+Currently primarily wrappers around processes and ways to gather process info from the system
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-system/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod foreground;
 
 #[cfg(target_os = "freebsd")]

crates/nu-table/README.md

@@ -0,0 +1,7 @@
+The layout logic for Nushell's table viewer.
+
+See also the separate `table` command implementation
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-table/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod table;
 mod table_theme;
 mod types;

crates/nu-term-grid/README.md

@@ -0,0 +1,5 @@
+Implementation of the layout engine for the `grid` command.
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-term-grid/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 pub mod grid;
 
 pub use grid::Grid;

crates/nu-test-support/README.md

@@ -0,0 +1,7 @@
+This crate provides utilities for testing of Nushell
+
+Plugin authors should instead refer to `nu-plugin-test-support`
+
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-test-support/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 pub mod commands;
 pub mod fs;
 pub mod locale_override;

crates/nu-utils/README.md

@@ -0,0 +1,6 @@
+Collection of small utilities that are shared across Nushell crates.
+
+This crate should compile early in the crate graph and thus not depend on major dependencies or core-nushell crates itself.
+## Internal Nushell crate
+
+This crate implements components of Nushell and is not designed to support plugin authors or other users directly.

crates/nu-utils/src/lib.rs

@@ -1,3 +1,4 @@
+#![doc = include_str!("../README.md")]
 mod casing;
 mod deansi;
 pub mod emoji;

crates/nuon/README.md

@@ -4,7 +4,7 @@ The NUON format is a superset of JSON designed to fit the feel of Nushell.
 Some of its extra features are
 - trailing commas are allowed
 - commas are optional in lists
-- quotes are not required around keys or any _bare_ string that do not contain spaces
+- quotes are not required around keys or any _bare_ string that do not contain spaces or special characters
 - comments are allowed, though not preserved when using [`from_nuon`]
 
 ## Example