Enforce call stack depth limit for all calls (#11729)

# Description
Previously, only direcly-recursive calls were checked for recursion
depth. But most recursive calls in nushell are mutually recursive since
expressions like `for`, `where`, `try` and `do` all execute a separte
block.

```nushell
def f [] {
    do { f }
}
```
Calling `f` would crash nushell with a stack overflow.

I think the only general way to prevent such a stack overflow is to
enforce a maximum call stack depth instead of only disallowing directly
recursive calls.

This commit also moves that logic into `eval_call()` instead of
`eval_block()` because the recursion limit is tracked in the `Stack`,
but not all blocks are evaluated in a new stack. Incrementing the
recursion depth of the caller's stack would permanently increment that
for all future calls.

Fixes #11667

# User-Facing Changes
Any function call can now fail with `recursion_limit_reached` instead of
just directly recursive calls. Mutually-recursive calls no longer crash
nushell.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This commit is contained in:
TrMen
2024-02-07 23:42:24 +01:00
committed by GitHub
parent 366348dea0
commit 4b91ed57dd
5 changed files with 40 additions and 63 deletions

View File

@ -42,6 +42,21 @@ pub fn eval_call(
let mut callee_stack = caller_stack.gather_captures(engine_state, &block.captures);
// Rust does not check recursion limits outside of const evaluation.
// But nu programs run in the same process as the shell.
// To prevent a stack overflow in user code from crashing the shell,
// we limit the recursion depth of function calls.
// Picked 50 arbitrarily, should work on all architectures.
const MAXIMUM_CALL_STACK_DEPTH: u64 = 50;
callee_stack.recursion_count += 1;
if callee_stack.recursion_count > MAXIMUM_CALL_STACK_DEPTH {
callee_stack.recursion_count = 0;
return Err(ShellError::RecursionLimitReached {
recursion_limit: MAXIMUM_CALL_STACK_DEPTH,
span: block.span,
});
}
for (param_idx, (param, required)) in decl
.signature()
.required_positional
@ -635,22 +650,6 @@ pub fn eval_block(
redirect_stdout: bool,
redirect_stderr: bool,
) -> Result<PipelineData, ShellError> {
// if Block contains recursion, make sure we don't recurse too deeply (to avoid stack overflow)
if let Some(recursive) = block.recursive {
// picked 50 arbitrarily, should work on all architectures
const RECURSION_LIMIT: u64 = 50;
if recursive {
if stack.recursion_count >= RECURSION_LIMIT {
stack.recursion_count = 0;
return Err(ShellError::RecursionLimitReached {
recursion_limit: RECURSION_LIMIT,
span: block.span,
});
}
stack.recursion_count += 1;
}
}
let num_pipelines = block.len();
for (pipeline_idx, pipeline) in block.pipelines.iter().enumerate() {