nushell/crates/nu-std/std/dirs.nu
Douglas 00709fc5bd
Improves startup time when using std-lib (#13842)
Updated summary for commit
[612e0e2](612e0e2160)
- While folks are welcome to read through the entire comments, the core
information is summarized here.

# Description

This PR drastically improves startup times of Nushell by only parsing a
single submodule of the Standard Library that provides the `banner` and
`pwd` commands. All other Standard Library commands and submodules are
parsed when imported by the user. This cuts startup times by more than
60%.

At the moment, we have stopped adding to `std-lib` because every
addition adds a small amount to the Nushell startup time.
With this change, we should once again be able to allow new
functionality to be added to the Standard Library without it impacting
`nu` startup times.

# User-Facing Changes

* Nushell now starts about 60% faster
* Breaking change: The `dirs` (Shells) aliases will return a warning
message that it will not be auto-loaded in the following release, along
with instructions on how to restore it (and disable the message)
* The `use std <submodule> *` syntax is available for convenience, but
should be avoided in scripts as it parses the entire `std` module and
all other submodules and places it in scope. The correct syntax to
*just* load a submodule is `use std/<submodule> *` (asterisk optional).
The slash is important. This will be documented.
* `use std *` can be used for convenience to load all of the library but
still incurs the full loading-time.
* `std/dirs`: Semi-breaking change. The `dirs` command replaces the
`show` command. This is more in line with the directory-stack
functionality found in other shells. Existing users will not be impacted
by this as the alias (`shells`) remains the same.

* Breaking-change: Technically a breaking change, but probably only
impacts maintainers of `std`. The virtual path for the standard library
has changed. It could previously be imported using its virtual path (and
technically, this would have been the correct way to do it):

  ```nu
  use NU_STDLIB_VIRTUAL_DIR/std
  ```

  The path is now simply `std/`:

  ```nu
  use std
  ```

  All submodules have moved accordingly.
  

# Timings

Comparisons below were made:

* In a temporary, clean config directory using `$env.XDG_CONFIG_HOME =
(mktemp -d)`.
* `nu` was run with a release build
* `nu` was run one time to generate the default `config.nu` (etc.) files
- Otherwise timings would include the user-prompt
* The shell was exited and then restarted several times to get timing
samples

(Note: Old timings based on 0.97 rather than 0.98, but in the range of
being accurate)

| Scenario | `$nu.startup-time` |
| --- | --- |
| 0.97.2
([aaaab8e](aaaab8e070))
Without this PR | 23ms - 24ms |
| This PR with deprecated commands | 9ms - <11ms |
| This PR after deprecated commands are removed in following release |
8ms - <10ms |
| Final PR (remove deprecated), using `--no-std-lib` | 6.1ms to 6.4ms |
| Final PR (remove deprecated), using `--no-config-file` | 3.1ms - 3.6ms
|
| Final PR (remove deprecated), using `--no-config-file --no-std-lib` |
1ms - 1.5ms |

*These last two timings point to the opportunity for further
optimization (see comment in thread below (will link once I write it).*

# Implementation details for future maintenance

* `use std banner` is a ridiculously deceptive call. That call parses
and imports *all* of `std` into scope. Simply replacing it with `use
std/core *` is essentially what saves ~14-15ms. This *only* imports the
submodule with the `banner` and `pwd` commands.

* From the code-comments, the reason that `NU_STDLIB_VIRTUAL_DIR` was
used as a prefix was so that there wouldn't be an issue if a user had a
`./std/mod.nu` in the current directory. This does **not** appear to be
an issue. After removing the prefix, I tested with both a relative
module as well as one in the `$env.NU_LIB_DIRS` path, and in all cases
the *internal* `std` still took precedence.

* By removing the prefix, users can now `use std` (and variants) without
requiring that it already be parsed and in scope.

* In the next release, we'll stop autoloading the `dirs` (shells)
functionality. While this only costs an additional 1-1.5ms, I think it's
better moved to the `config.nu` where the user can optionally remove it.
The main reason is its use of aliases (which have also caused issues) -
The `n`, `p`, and `g` short-commands are valuable real-estate, and users
may want to map these to something else.
  
For this release, there's an `deprecated_dirs` module that is still
autoloaded. As with the top-level commands, use of these will give a
deprecation warning with instructions on how to handle going forward.

To help with this, moved the aliases to their own submodule inside the
`dirs` module.

* Also sneaks in a small change where the top-level `dirs` command is
now the replacement for `dirs show`

* Fixed a double-import of `assert` in `dirs.nu`
* The `show_banner` step is replaced with simply `banner` rather than
re-importing it.

* A `virtual_path` may now be referenced with either a forward-slash or
a backward-slash on Windows. This allows `use std/<submodule>` to work
on all platforms.

# Performance side-notes:

* Future parsing and/or IR improvements should improve performance even
further.
* While the existing load time penalty of `std-lib` was not noticeable
on many systems, Nushell runs on a wide-variety of hardware and OS
platforms. Slower platforms will naturally see a bigger jump in
performance here. For users starting multiple Nushell sessions
frequently (e.g., `tmux`, Zellij, `screen`, et. al.) it is recommended
to keep total startup time (including user configuration) under ~250ms.

# Tests + Formatting

* All tests are green

* Updated tests:
- Removed the test that confirmed that `std` was loaded (since we
don't).
- Removed the `shells` test since it is not autoloaded. Main `dirs.nu`
functionality is tested through `stdlib-test`.
- Many tests assumed that the library was fully loaded, because it was
(even though we didn't intend for it to be). Fixed those tests.
- Tests now import only the necessary submodules (e.g., `use
std/assert`, rather than `use std assert`)
- Some tests *thought* they were loading `std/log`, but were doing so
improperly. This was masked by the now-fixed "load-everything-into-scope
bug". Local CI would pass due the `$env.NU_LOG_<...>` variables being
inherited from the calling process, but would fail in the "clean" GitHub
CI environment. These tests have also been fixed.

 * Added additional tests for the changes

# After Submitting

Will update the Standard Library doc page
2024-10-03 06:28:22 -05:00

148 lines
4.4 KiB
Plaintext

# Maintain a list of working directories and navigate them
# The directory stack.
#
# Exception: the entry for the current directory contains an
# irrelevant value. Instead, the source of truth for the working
# directory is $env.PWD. It has to be this way because cd doesn't
# know about this module.
#
# Example: the following state represents a user-facing directory
# stack of [/a, /var/tmp, /c], and we are currently in /var/tmp .
#
# PWD = /var/tmp
# DIRS_POSITION = 1
# DIRS_LIST = [/a, /b, /c]
#
# This situation could arise if we started with [/a, /b, /c], then
# we changed directories from /b to /var/tmp.
export-env {
$env.DIRS_POSITION = 0
$env.DIRS_LIST = [($env.PWD | path expand)]
}
# Add one or more directories to the list.
# The first directory listed becomes the new
# active directory.
export def --env add [
...paths: string # directory or directories to add to working list
] {
mut abspaths = []
for p in $paths {
let exp = ($p | path expand)
if ($exp | path type) != 'dir' {
let span = (metadata $p).span
error make {msg: "not a directory", label: {text: "not a directory", span: $span } }
}
$abspaths = ($abspaths | append $exp)
}
$env.DIRS_LIST = ($env.DIRS_LIST | insert ($env.DIRS_POSITION + 1) $abspaths | flatten)
_fetch 1
}
# Make the next directory on the list the active directory.
# If the currenta ctive directory is the last in the list,
# then cycle to the top of the list.
export def --env next [
N:int = 1 # number of positions to move.
] {
_fetch $N
}
# Make the previous directory on the list the active directory.
# If the current active directory is the first in the list,
# then cycle to the end of the list.
export def --env prev [
N:int = 1 # number of positions to move.
] {
_fetch (-1 * $N)
}
# Drop the current directory from the list.
# The previous directory in the list becomes
# the new active directory.
#
# If there is only one directory in the list,
# then this command has no effect.
export def --env drop [] {
if ($env.DIRS_LIST | length) > 1 {
$env.DIRS_LIST = ($env.DIRS_LIST | reject $env.DIRS_POSITION)
if ($env.DIRS_POSITION >= ($env.DIRS_LIST | length)) {$env.DIRS_POSITION = 0}
}
# step to previous slot
_fetch -1 --forget_current --always_cd
}
# Display current working directories
export def --env main [] {
mut out = []
for $p in ($env.DIRS_LIST | enumerate) {
let is_act_slot = $p.index == $env.DIRS_POSITION
$out = ($out | append [
[active, path];
[($is_act_slot),
(if $is_act_slot {$env.PWD} else {$p.item}) # show current PWD in lieu of active slot
]
])
}
$out
}
# Jump to directory by index
export def --env goto [dir_idx?: int] {
if $dir_idx == null {
return (main)
}
if $dir_idx < 0 or $dir_idx >= ($env.DIRS_LIST | length) {
let span = (metadata $dir_idx | get span)
error make {
msg: $"(ansi red_bold)invalid_dirs_index(ansi reset)"
label: {
text: $"`idx` should be between 0 and (($env.DIRS_LIST | length) - 1)"
span: $span
}
}
}
_fetch ($dir_idx - $env.DIRS_POSITION)
}
# fetch item helper
def --env _fetch [
offset: int, # signed change to position
--forget_current # true to skip saving PWD
--always_cd # true to always cd
] {
if not ($forget_current) {
# first record current working dir in current slot of ring, to track what CD may have done.
$env.DIRS_LIST = ($env.DIRS_LIST | upsert $env.DIRS_POSITION $env.PWD)
}
# figure out which entry to move to
# nushell 'mod' operator is really 'remainder', can return negative values.
# see: https://stackoverflow.com/questions/13683563/whats-the-difference-between-mod-and-remainder
let len = ($env.DIRS_LIST | length)
mut pos = ($env.DIRS_POSITION + $offset) mod $len
if ($pos < 0) { $pos += $len}
# if using a different position in ring, CD there.
if ($always_cd or $pos != $env.DIRS_POSITION) {
$env.DIRS_POSITION = $pos
cd ($env.DIRS_LIST | get $pos )
}
}
export module shells-aliases {
export alias shells = main
export alias enter = add
export alias dexit = drop
export alias p = prev
export alias n = next
export alias g = goto
}