REFACTOR: clean the root of the repo (#9231)

# Description
i've almost always wanted to clean up the root of the repo, so here is
my take at it, with some important advice given by @fdncred 😌

- `README.release.txt` is now gone and directly inline in the
`release-pkg` script used in the `release` *workflow*
- `build.rs` has been moved to `scripts/` and its path has been changed
in
[`Cargo.toml`](https://github.com/amtoine/nushell/blob/refactor/clean-root/Cargo.toml#L3)
according to the [*Build Scripts*
section](https://doc.rust-lang.org/cargo/reference/build-scripts.html#build-scripts)
of *The Cargo Book*
- i've merged `images/` into `assets/` and fix the only mention to the
GIF in the README
- i've moved the `docs/README.md` inside the main `README.md` as a new
[*Configuration*
section](https://github.com/amtoine/nushell/tree/refactor/clean-root#configuration)
- the very deprecated `pkg_mgrs/` has been removed
- all the `.nu`, `.sh`, `.ps1` and `.cmd` scripts have been moved to
`scripts/`

### things i've left as-is
- all the other `.md` documents
- the configuration files
- all the Rust and core stuff
- `docker/`
- `toolkit.nu`
- the `wix/` diretory which appears to be important for `winget`

# User-Facing Changes
scripts that used to rely on the paths to some of the scripts should now
call the scripts inside `scripts/` => i think this for the greater good,
it was not pretty nor scalable to have a bunch of scripts in the root of
our main `nushell` 😱

*i even think we might want to move these scripts outside the main
`nushell` repo*
maybe to `nu_scripts` or some other tool 👍 

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
-  `toolkit test`
-  `toolkit test stdlib`

# After Submitting
```
$nothing
```
This commit is contained in:
Antoine Stevan
2023-05-20 14:57:51 +02:00
committed by GitHub
parent 4b9f479e5c
commit 8eece32a8d
20 changed files with 180 additions and 131 deletions

27
scripts/README.md Normal file
View File

@ -0,0 +1,27 @@
## run the scripts
> **Note**
> the following table must be read as follows:
> - an `x` means *it works*
> - a `?` means *no data available*
>
> `.nu` scripts must be run as `nu .../foo.nu`
> `.sh` scripts must be run as `./.../foo.sh`
> `.ps1` scripts must be run as `powershell .../foo.ps1`
>
> let's say a script is called `foo`
> - an `x` in the *`./scripts`* column means *`foo` can be run from `./scripts`*
> - an `x` in the *root* column means *`foo` can be run from the root of `nushell`*
> - an `x` in the *anywhere* column means *`foo` can be run from anywhere!*
| script | `./scripts/` | root | anywhere |
| ----------------------- | ------------ | ---- | -------- |
| `build-all-maclin.sh` | x | x | x |
| `build-all-windows.cmd` | ? | x | ? |
| `build-all.nu` | x | x | x |
| `coverage-local.nu` | x | x | x |
| `coverage-local.sh` | x | x | x |
| `install-all.ps1` | ? | x | ? |
| `install-all.sh` | x | x | x |
| `register-plugins.nu` | x | x | x |
| `uninstall-all.sh` | x | x | x |

35
scripts/build-all-maclin.sh Executable file
View File

@ -0,0 +1,35 @@
#!/usr/bin/env bash
set -euo pipefail
DIR=$(readlink -f $(dirname "${BASH_SOURCE[0]}"))
REPO_ROOT=$(dirname $DIR)
echo "---------------------------------------------------------------"
echo "Building nushell (nu) with dataframes and all the plugins"
echo "---------------------------------------------------------------"
echo ""
NU_PLUGINS=(
'nu_plugin_example'
'nu_plugin_gstat'
'nu_plugin_inc'
'nu_plugin_query'
'nu_plugin_custom_values'
)
echo "Building nushell"
(
cd $REPO_ROOT
cargo build --features=dataframe
)
for plugin in "${NU_PLUGINS[@]}"
do
echo "Building $plugin..."
echo "-----------------------------"
(
cd "$REPO_ROOT/crates/$plugin"
cargo build
)
done

View File

@ -0,0 +1,30 @@
@echo off
echo -------------------------------------------------------------------
echo Building nushell (nu.exe) with dataframes and all the plugins
echo -------------------------------------------------------------------
echo.
echo Building nushell.exe
cargo build --features=dataframe
echo.
call :build crates\nu_plugin_example nu_plugin_example.exe
call :build ..\..\crates\nu_plugin_gstat nu_plugin_gstat.exe
call :build ..\..\crates\nu_plugin_inc nu_plugin_inc.exe
call :build ..\..\crates\nu_plugin_query nu_plugin_query.exe
call :build ..\..\crates\nu_plugin_custom_values nu_plugin_custom_values.exe
cd ..\..
exit /b 0
:build
setlocal
set "location=%~1"
set "target=%~2"
cd "%location%"
echo Building %target%
cargo build
echo.
endlocal
exit /b 0

36
scripts/build-all.nu Normal file
View File

@ -0,0 +1,36 @@
print '-------------------------------------------------------------------'
print 'Building nushell (nu) with dataframes and all the plugins'
print '-------------------------------------------------------------------'
let repo_root = ($env.CURRENT_FILE | path dirname -n 2)
def build-nushell [] {
print $'(char nl)Building nushell'
print '----------------------------'
cd $repo_root
cargo build --features=dataframe
}
def build-plugin [] {
let plugin = $in
print $'(char nl)Building ($plugin)'
print '----------------------------'
cd $'($repo_root)/crates/($plugin)'
cargo build
}
let plugins = [
nu_plugin_inc,
nu_plugin_gstat,
nu_plugin_query,
nu_plugin_example,
nu_plugin_custom_values,
nu_plugin_formats,
]
for plugin in $plugins {
$plugin | build-plugin
}

13
scripts/build.rs Normal file
View File

@ -0,0 +1,13 @@
#[cfg(windows)]
fn main() {
let mut res = winresource::WindowsResource::new();
res.set("ProductName", "Nushell");
res.set("FileDescription", "Nushell");
res.set("LegalCopyright", "Copyright (C) 2022");
res.set_icon("assets/nu_logo.ico");
res.compile()
.expect("Failed to run the Windows resource compiler (rc.exe)");
}
#[cfg(not(windows))]
fn main() {}

60
scripts/coverage-local.nu Executable file
View File

@ -0,0 +1,60 @@
#!/usr/bin/env nu
def compute-coverage [] {
cd ($env.CURRENT_FILE | path dirname -n 2)
print "Setting up environment variables for coverage"
# Enable LLVM coverage tracking through environment variables
# show env outputs .ini/.toml style description of the variables
# In order to use from toml, we need to make sure our string literals are single quoted
# This is especially important when running on Windows since "C:\blah" is treated as an escape
cargo llvm-cov show-env | str replace (char dq) (char sq) -a | from toml | load-env
print "Cleaning up coverage data"
cargo llvm-cov clean --workspace
print "Building with workspace and profile=ci"
# Apparently we need to explicitly build the necessary parts
# using the `--profile=ci` is basically `debug` build with unnecessary symbols stripped
# leads to smaller binaries and potential savings when compiling and running
cargo build --workspace --profile=ci
print "Running tests with --workspace and profile=ci"
cargo test --workspace --profile=ci
# You need to provide the used profile to find the raw data
print "Generating coverage report as lcov.info"
cargo llvm-cov report --lcov --output-path lcov.info --profile=ci
}
let start = (date now)
# Script to generate coverage locally
#
# Output: `lcov.info` file
#
# Relies on `cargo-llvm-cov`. Install via `cargo install cargo-llvm-cov`
# https://github.com/taiki-e/cargo-llvm-cov
# You probably have to run `cargo llvm-cov clean` once manually,
# as you have to confirm to install additional tooling for your rustup toolchain.
# Else the script might stall waiting for your `y<ENTER>`
# Some of the internal tests rely on the exact cargo profile
# (This is somewhat criminal itself)
# but we have to signal to the tests that we use the `ci` `--profile`
let-env NUSHELL_CARGO_TARGET = "ci"
# Manual gathering of coverage to catch invocation of the `nu` binary.
# This is relevant for tests using the `nu!` macro from `nu-test-support`
# see: https://github.com/taiki-e/cargo-llvm-cov#get-coverage-of-external-tests
compute-coverage
let end = (date now)
print $"Coverage generation took ($end - $start)."
# To display the coverage in your editor see:
#
# - https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters
# - https://github.com/umaumax/vim-lcov
# - https://github.com/andythigpen/nvim-coverage (probably needs some additional config)

44
scripts/coverage-local.sh Executable file
View File

@ -0,0 +1,44 @@
#!/usr/bin/env bash
DIR=$(readlink -f $(dirname "${BASH_SOURCE[0]}"))
REPO_ROOT=$(dirname $DIR)
# Script to generate coverage locally
#
# Output: `lcov.info` file
#
# Relies on `cargo-llvm-cov`. Install via `cargo install cargo-llvm-cov`
# https://github.com/taiki-e/cargo-llvm-cov
# You probably have to run `cargo llvm-cov clean` once manually,
# as you have to confirm to install additional tooling for your rustup toolchain.
# Else the script might stall waiting for your `y<ENTER>`
# Some of the internal tests rely on the exact cargo profile
# (This is somewhat criminal itself)
# but we have to signal to the tests that we use the `ci` `--profile`
export NUSHELL_CARGO_TARGET=ci
# Manual gathering of coverage to catch invocation of the `nu` binary.
# This is relevant for tests using the `nu!` macro from `nu-test-support`
# see: https://github.com/taiki-e/cargo-llvm-cov#get-coverage-of-external-tests
(
cd $REPO_ROOT
# Enable LLVM coverage tracking through environment variables
source <(cargo llvm-cov show-env --export-prefix)
cargo llvm-cov clean --workspace
# Apparently we need to explicitly build the necessary parts
# using the `--profile=ci` is basically `debug` build with unnecessary symbols stripped
# leads to smaller binaries and potential savings when compiling and running
cargo build --workspace --profile=ci
cargo test --workspace --profile=ci
# You need to provide the used profile to find the raw data
cargo llvm-cov report --lcov --output-path lcov.info --profile=ci
)
# To display the coverage in your editor see:
#
# - https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters
# - https://github.com/umaumax/vim-lcov
# - https://github.com/andythigpen/nvim-coverage (probably needs some additional config)

31
scripts/install-all.ps1 Normal file
View File

@ -0,0 +1,31 @@
# Usage: Just run `powershell install-all.ps1` in nushell root directory
Write-Output "-----------------------------------------------------------------"
Write-Output "Installing nushell (nu) with dataframes and all the plugins"
Write-Output "-----------------------------------------------------------------"
Write-Output ""
Write-Output "Install nushell from local..."
Write-Output "----------------------------------------------"
cargo install --force --path . --features=dataframe
$NU_PLUGINS = @(
'nu_plugin_example',
'nu_plugin_gstat',
'nu_plugin_inc',
'nu_plugin_query',
'nu_plugin_custom_values',
'nu_plugin_formats'
)
foreach ( $plugin in $NU_PLUGINS) {
Write-Output ''
Write-Output "----------------------------------------------"
Write-Output "Install plugin $plugin from local..."
Write-Output "----------------------------------------------"
Set-Location crates/$plugin
cargo install --force --path .
Set-Location ../../
}

33
scripts/install-all.sh Executable file
View File

@ -0,0 +1,33 @@
#!/usr/bin/env bash
set -euo pipefail
DIR=$(readlink -f $(dirname "${BASH_SOURCE[0]}"))
REPO_ROOT=$(dirname $DIR)
echo "-----------------------------------------------------------------"
echo "Installing nushell (nu) with dataframes and all the plugins"
echo "-----------------------------------------------------------------"
echo ""
echo "Install nushell from local..."
echo "----------------------------------------------"
cargo install --force --path "$REPO_ROOT" --features=dataframe
NU_PLUGINS=(
'nu_plugin_inc'
'nu_plugin_gstat'
'nu_plugin_query'
'nu_plugin_example'
'nu_plugin_custom_values'
'nu_plugin_formats'
)
for plugin in "${NU_PLUGINS[@]}"
do
echo ''
echo "----------------------------------------------"
echo "Install plugin $plugin from local..."
echo "----------------------------------------------"
cargo install --force --path "$REPO_ROOT/crates/$plugin"
done

View File

@ -0,0 +1,51 @@
# are we on windows or not?
def windows? [] {
$nu.os-info.name == windows
}
# filter out files that end in .d
def keep-plugin-executables [] {
if (windows?) { where name ends-with '.exe' } else { where name !~ '\.d' }
}
# get list of all plugin files from their installed directory
let plugins = (ls ((which nu).path.0 | path dirname) | where name =~ nu_plugin | keep-plugin-executables)
for plugin in $plugins {
print -n $"registering ($plugin.name), "
nu -c $"register '($plugin.name)'"
print "success!"
}
# print helpful message
print "\nplugins registered, please restart nushell"
# Plugin Location
# https://github.com/nushell/nushell/tree/main/crates/nu_plugin_custom_values
# https://github.com/nushell/nushell/tree/main/crates/nu_plugin_example
# https://github.com/nushell/nushell/tree/main/crates/nu_plugin_gstat
# https://github.com/nushell/nushell/tree/main/crates/nu_plugin_inc
# https://github.com/nushell/nushell/tree/main/crates/nu_plugin_python
# https://github.com/nushell/nushell/tree/main/crates/nu_plugin_query
# https://github.com/fdncred/nu_plugin_from_parquet
# https://github.com/fdncred/nu_plugin_from_regex
# https://github.com/fdncred/nu_plugin_pnet
# https://github.com/JosephTLyons/nu_plugin_periodic_table
# https://github.com/Euphrasiologist/nu_plugin_bio
# https://github.com/realcundo/nu_plugin_dcm
# https://github.com/enerdgumen/nu_plugin_dotenv
# https://github.com/bluk/nu_plugin_from_bencode
# Older plugins
# https://github.com/notryanb/nu_plugin_id3
# https://github.com/notryanb/nu_plugin_weather
# https://github.com/tiffany352/nu-plugins/tree/main/from_nbt
# https://github.com/tiffany352/nu-plugins/tree/main/file_exists
# https://github.com/potan/nu_plugin_wifiscan
# https://github.com/autophagy/nu_plugin_from_dhall
# https://github.com/yanganto/nu_plugin_s3
# https://github.com/lukasreuter/nu_plugin_unity
# https://github.com/filaretov/nu_plugin_path_temp
# https://github.com/cdecompilador/nu_plugin_bg
# https://github.com/aJuvan/nu_plugin_kubectl
# https://github.com/hedonihilist/nu_plugin_df

22
scripts/uninstall-all.sh Executable file
View File

@ -0,0 +1,22 @@
#!/usr/bin/env bash
set -euo pipefail
echo ''
echo "----------------------------------------------"
echo "Uninstall nu and all plugins from cargo/bin..."
echo "----------------------------------------------"
NU_PLUGINS=(
'nu_plugin_inc'
'nu_plugin_gstat'
'nu_plugin_query'
'nu_plugin_example'
'nu_plugin_formats'
)
cargo uninstall nu
for plugin in "${NU_PLUGINS[@]}"
do
cargo uninstall "$plugin"
done