Module: support defining const and use const variables inside of function (#9773)

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Relative: #8248 

After this pr, user can define const variable inside a module.

![image](https://github.com/nushell/nushell/assets/22256154/e3e03e56-c4b5-4144-a944-d1b20bec1cbd)

And user can export const variables, the following screenshot shows how
it works (it follows
https://github.com/nushell/nushell/issues/8248#issuecomment-1637442612):

![image](https://github.com/nushell/nushell/assets/22256154/b2c14760-3f27-41cc-af77-af70a4367f2a)

## About the change
1. To make module support const, we need to change `parse_module_block`
to support `const` keyword.
2. To suport export `const`, we need to make module tracking variables,
so we add `variables` attribute to `Module`
3. During eval, the const variable may not exists in `stack`, because we
don't eval `const` when we define a module, so we need to find variables
which are already registered in `engine_state`

## One more thing to note about the const value.
Consider the following code
```
module foo { const b = 3; export def bar [] { $b } }
use foo bar
const b = 4;
bar
```
The result will be 3 (which is defined in module) rather than 4. I think
it's expected behavior.

It's something like [dynamic
binding](https://www.gnu.org/software/emacs/manual/html_node/elisp/Dynamic-Binding-Tips.html)
vs [lexical
binding](https://www.gnu.org/software/emacs/manual/html_node/elisp/Lexical-Binding.html)
in lisp like language, and lexical binding should be right behavior
which generates more predicable result, and it doesn't introduce really
subtle bugs in nushell code.

What if user want dynamic-binding?(For example: the example code returns
`4`)
There is no way to do this, user should consider passing the value as
argument to custom command rather than const.

## TODO
- [X] adding tests for the feature.
- [X] support export const out of module to use.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

crates/nu-protocol/src/ast/import_pattern.rs

@@ -1,6 +1,6 @@
 use serde::{Deserialize, Serialize};
 
-use crate::{span, ModuleId, Span};
+use crate::{span, ModuleId, Span, VarId};
 use std::collections::HashSet;
 
 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -24,6 +24,7 @@ pub struct ImportPattern {
     // communicate to eval which decls/aliases were hidden during `parse_hide()` so it does not
     // interpret these as env var names:
     pub hidden: HashSet<Vec<u8>>,
+    pub module_name_var_id: Option<VarId>,
 }
 
 impl ImportPattern {
@@ -36,6 +37,7 @@ impl ImportPattern {
             },
             members: vec![],
             hidden: HashSet::new(),
+            module_name_var_id: None,
         }
     }
 
@@ -62,6 +64,7 @@ impl ImportPattern {
             head: self.head,
             members: self.members,
             hidden,
+            module_name_var_id: self.module_name_var_id,
         }
     }
 }

crates/nu-protocol/src/engine/engine_state.rs

@@ -1161,6 +1161,17 @@ impl<'a> StateWorkingSet<'a> {
         }
     }
 
+    pub fn use_variables(&mut self, variables: Vec<(Vec<u8>, VarId)>) {
+        let overlay_frame = self.last_overlay_mut();
+
+        for (mut name, var_id) in variables {
+            if !name.starts_with(b"$") {
+                name.insert(0, b'$');
+            }
+            overlay_frame.insert_variable(name, var_id);
+        }
+    }
+
     pub fn add_predecl(&mut self, decl: Box<dyn Command>) -> Option<DeclId> {
         let name = decl.name().as_bytes().to_vec();
 
@@ -1530,11 +1541,15 @@ impl<'a> StateWorkingSet<'a> {
     }
 
     pub fn find_variable(&self, name: &[u8]) -> Option<VarId> {
+        let mut name = name.to_vec();
+        if !name.starts_with(b"$") {
+            name.insert(0, b'$');
+        }
         let mut removed_overlays = vec![];
 
         for scope_frame in self.delta.scope.iter().rev() {
             for overlay_frame in scope_frame.active_overlays(&mut removed_overlays).rev() {
-                if let Some(var_id) = overlay_frame.vars.get(name) {
+                if let Some(var_id) = overlay_frame.vars.get(&name) {
                     return Some(*var_id);
                 }
             }
@@ -1545,7 +1560,7 @@ impl<'a> StateWorkingSet<'a> {
             .active_overlays(&removed_overlays)
             .rev()
         {
-            if let Some(var_id) = overlay_frame.vars.get(name) {
+            if let Some(var_id) = overlay_frame.vars.get(&name) {
                 return Some(*var_id);
             }
         }

crates/nu-protocol/src/engine/overlay.rs

@@ -206,6 +206,10 @@ impl OverlayFrame {
         self.modules.insert(name, module_id)
     }
 
+    pub fn insert_variable(&mut self, name: Vec<u8>, variable_id: VarId) -> Option<VarId> {
+        self.vars.insert(name, variable_id)
+    }
+
     pub fn get_decl(&self, name: &[u8]) -> Option<DeclId> {
         self.decls.get(name).cloned()
     }

crates/nu-protocol/src/exportable.rs

@@ -1,6 +1,7 @@
-use crate::{DeclId, ModuleId};
+use crate::{DeclId, ModuleId, VarId};
 
 pub enum Exportable {
     Decl { name: Vec<u8>, id: DeclId },
     Module { name: Vec<u8>, id: ModuleId },
+    VarDecl { name: Vec<u8>, id: VarId },
 }

crates/nu-protocol/src/module.rs

@@ -1,5 +1,6 @@
 use crate::{
     ast::ImportPatternMember, engine::StateWorkingSet, BlockId, DeclId, ModuleId, ParseError, Span,
+    VarId,
 };
 
 use indexmap::IndexMap;
@@ -7,11 +8,20 @@ use indexmap::IndexMap;
 pub struct ResolvedImportPattern {
     pub decls: Vec<(Vec<u8>, DeclId)>,
     pub modules: Vec<(Vec<u8>, ModuleId)>,
+    pub variables: Vec<(Vec<u8>, VarId)>,
 }
 
 impl ResolvedImportPattern {
-    pub fn new(decls: Vec<(Vec<u8>, DeclId)>, modules: Vec<(Vec<u8>, ModuleId)>) -> Self {
-        ResolvedImportPattern { decls, modules }
+    pub fn new(
+        decls: Vec<(Vec<u8>, DeclId)>,
+        modules: Vec<(Vec<u8>, ModuleId)>,
+        variables: Vec<(Vec<u8>, VarId)>,
+    ) -> Self {
+        ResolvedImportPattern {
+            decls,
+            modules,
+            variables,
+        }
     }
 }
 
@@ -21,6 +31,7 @@ pub struct Module {
     pub name: Vec<u8>,
     pub decls: IndexMap<Vec<u8>, DeclId>,
     pub submodules: IndexMap<Vec<u8>, ModuleId>,
+    pub variables: IndexMap<Vec<u8>, VarId>,
     pub env_block: Option<BlockId>, // `export-env { ... }` block
     pub main: Option<DeclId>,       // `export def main`
     pub span: Option<Span>,
@@ -32,6 +43,7 @@ impl Module {
             name,
             decls: IndexMap::new(),
             submodules: IndexMap::new(),
+            variables: IndexMap::new(),
             env_block: None,
             main: None,
             span: None,
@@ -43,6 +55,7 @@ impl Module {
             name,
             decls: IndexMap::new(),
             submodules: IndexMap::new(),
+            variables: IndexMap::new(),
             env_block: None,
             main: None,
             span: Some(span),
@@ -61,6 +74,10 @@ impl Module {
         self.submodules.insert(name, module_id)
     }
 
+    pub fn add_variable(&mut self, name: Vec<u8>, var_id: VarId) -> Option<VarId> {
+        self.variables.insert(name, var_id)
+    }
+
     pub fn add_env_block(&mut self, block_id: BlockId) {
         self.env_block = Some(block_id);
     }
@@ -87,7 +104,8 @@ impl Module {
             (head, rest)
         } else {
             // Import pattern was just name without any members
-            let mut results = vec![];
+            let mut decls = vec![];
+            let mut vars = vec![];
             let mut errors = vec![];
 
             for (_, id) in &self.submodules {
@@ -101,14 +119,19 @@ impl Module {
                     new_name.push(b' ');
                     new_name.extend(sub_name);
 
-                    results.push((new_name, sub_decl_id));
+                    decls.push((new_name, sub_decl_id));
+                }
+
+                for (sub_name, sub_var_id) in sub_results.variables {
+                    vars.push((sub_name, sub_var_id));
                 }
             }
 
-            results.extend(self.decls_with_head(&final_name));
+            decls.extend(self.decls_with_head(&final_name));
+            vars.extend(self.vars());
 
             return (
-                ResolvedImportPattern::new(results, vec![(final_name, self_id)]),
+                ResolvedImportPattern::new(decls, vec![(final_name, self_id)], vars),
                 errors,
             );
         };
@@ -118,18 +141,27 @@ impl Module {
                 if name == b"main" {
                     if let Some(main_decl_id) = self.main {
                         (
-                            ResolvedImportPattern::new(vec![(final_name, main_decl_id)], vec![]),
+                            ResolvedImportPattern::new(
+                                vec![(final_name, main_decl_id)],
+                                vec![],
+                                vec![],
+                            ),
                             vec![],
                         )
                     } else {
                         (
-                            ResolvedImportPattern::new(vec![], vec![]),
+                            ResolvedImportPattern::new(vec![], vec![], vec![]),
                             vec![ParseError::ExportNotFound(*span)],
                         )
                     }
                 } else if let Some(decl_id) = self.decls.get(name) {
                     (
-                        ResolvedImportPattern::new(vec![(name.clone(), *decl_id)], vec![]),
+                        ResolvedImportPattern::new(vec![(name.clone(), *decl_id)], vec![], vec![]),
+                        vec![],
+                    )
+                } else if let Some(var_id) = self.variables.get(name) {
+                    (
+                        ResolvedImportPattern::new(vec![], vec![], vec![(name.clone(), *var_id)]),
                         vec![],
                     )
                 } else if let Some(submodule_id) = self.submodules.get(name) {
@@ -137,7 +169,7 @@ impl Module {
                     submodule.resolve_import_pattern(working_set, *submodule_id, rest, None)
                 } else {
                     (
-                        ResolvedImportPattern::new(vec![], vec![]),
+                        ResolvedImportPattern::new(vec![], vec![], vec![]),
                         vec![ParseError::ExportNotFound(*span)],
                     )
                 }
@@ -145,6 +177,7 @@ impl Module {
             ImportPatternMember::Glob { .. } => {
                 let mut decls = vec![];
                 let mut submodules = vec![];
+                let mut variables = vec![];
                 let mut errors = vec![];
 
                 for (_, id) in &self.submodules {
@@ -152,18 +185,25 @@ impl Module {
                     let (sub_results, sub_errors) =
                         submodule.resolve_import_pattern(working_set, *id, &[], None);
                     decls.extend(sub_results.decls);
+
                     submodules.extend(sub_results.modules);
+                    variables.extend(sub_results.variables);
                     errors.extend(sub_errors);
                 }
 
                 decls.extend(self.decls());
+                variables.extend(self.variables.clone());
                 submodules.extend(self.submodules());
 
-                (ResolvedImportPattern::new(decls, submodules), errors)
+                (
+                    ResolvedImportPattern::new(decls, submodules, variables),
+                    errors,
+                )
             }
             ImportPatternMember::List { names } => {
                 let mut decls = vec![];
                 let mut submodules = vec![];
+                let mut variables = vec![];
                 let mut errors = vec![];
 
                 for (name, span) in names {
@@ -175,6 +215,8 @@ impl Module {
                         }
                     } else if let Some(decl_id) = self.decls.get(name) {
                         decls.push((name.clone(), *decl_id));
+                    } else if let Some(var_id) = self.variables.get(name) {
+                        variables.push((name.clone(), *var_id));
                     } else if let Some(submodule_id) = self.submodules.get(name) {
                         submodules.push((name.clone(), *submodule_id));
                     } else {
@@ -182,7 +224,10 @@ impl Module {
                     }
                 }
 
-                (ResolvedImportPattern::new(decls, submodules), errors)
+                (
+                    ResolvedImportPattern::new(decls, submodules, variables),
+                    errors,
+                )
             }
         }
     }
@@ -217,6 +262,13 @@ impl Module {
         result
     }
 
+    pub fn vars(&self) -> Vec<(Vec<u8>, VarId)> {
+        self.variables
+            .iter()
+            .map(|(name, id)| (name.to_vec(), *id))
+            .collect()
+    }
+
     pub fn decl_names_with_head(&self, head: &[u8]) -> Vec<Vec<u8>> {
         let mut result: Vec<Vec<u8>> = self
             .decls