Commit Graph

77 Commits

Author SHA1 Message Date
Leon
220b105efb
Reduced LOC by replacing several instances of Value::Int {}, Value::Float{}, Value::Bool {}, and Value::String {} with Value::int(), Value::float(), Value::boolean() and Value::string() (#7412)
# Description

While perusing Value.rs, I noticed the `Value::int()`, `Value::float()`,
`Value::boolean()` and `Value::string()` constructors, which seem
designed to make it easier to construct various Values, but which aren't
used often at all in the codebase. So, using a few find-replaces
regexes, I increased their usage. This reduces overall LOC because
structures like this:
```
Value::Int {
  val: a,
  span: head
}
```
are changed into
```
Value::int(a, head)
```
and are respected as such by the project's formatter.
There are little readability concerns because the second argument to all
of these is `span`, and it's almost always extremely obvious which is
the span at every callsite.

# User-Facing Changes

None.

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2022-12-09 11:37:51 -05:00
JT
eaec480f42
Improve empty pipelines (#7383)
# Description

This fix changes pipelines to allow them to actually be empty. Mapping
over empty pipelines gives empty pipelines. Empty pipelines immediately
return `None` when iterated.

This removes a some of where `Span::new(0, 0)` was coming from, though
there are other cases where we still use it.

# User-Facing Changes

None

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2022-12-08 07:31:57 +13:00
Daniel Buch Hansen
850ecf648a
Protocol: debug_assert!() Span to reflect a valid slice (#6806)
Also enforce this by #[non_exhaustive] span such that going forward we
cannot, in debug builds (1), construct invalid spans.

The motivation for this stems from #6431 where I've seen crashes due to
invalid slice indexing.

My hope is this will mitigate such senarios

1. https://github.com/nushell/nushell/pull/6431#issuecomment-1278147241

# Description

(description of your pull request here)

# Tests

Make sure you've done the following:

- [ ] Add tests that cover your changes, either in the command examples,
the crate/tests folder, or in the /tests folder.
- [ ] Try to think about corner cases and various ways how your changes
could break. Cover them with tests.
- [ ] If adding tests is not possible, please document in the PR body a
minimal example with steps on how to reproduce so one can verify your
change works.

Make sure you've run and fixed any issues with these commands:

- [x] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [ ] `cargo clippy --workspace --features=extra -- -D warnings -D
clippy::unwrap_used -A clippy::needless_collect` to check that you're
using the standard code style
- [ ] `cargo test --workspace --features=extra` to check that all the
tests pass

# Documentation

- [ ] If your PR touches a user-facing nushell feature then make sure
that there is an entry in the documentation
(https://github.com/nushell/nushell.github.io) for the feature, and
update it if necessary.
2022-12-03 11:44:12 +02:00
Darren Schroeder
b7e5790cd1
fix dfr datetime conversion (#7264)
# Description

Closes #7257

This fixes an issue where dataframes would always try to convert
datetimes with milliseconds. Since the timeunit is passed in, I make use
of it and try to choose the appropriate divisor.


# User-Facing Changes


# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2022-11-30 17:10:28 -06:00
Access
9850fbd77d
chore: chrono_update (#7132)
chrono version update

# Description

upgrade chrono to 0.4.23

# Major Changes

If you're considering making any major change to nushell, before
starting work on it, seek feedback from regular contributors and get
approval for the idea from the core team either on
[Discord](https://discordapp.com/invite/NtAbbGn) or [GitHub
issue](https://github.com/nushell/nushell/issues/new/choose).
Making sure we're all on board with the change saves everybody's time.
Thanks!

# Tests + Formatting

Make sure you've done the following, if applicable:

- Add tests that cover your changes (either in the command examples, the
crate/tests folder, or in the /tests folder)
- Try to think about corner cases and various ways how your changes
could break. Cover those in the tests

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace --features=extra -- -D warnings -D
clippy::unwrap_used -A clippy::needless_collect` to check that you're
using the standard code style
- `cargo test --workspace --features=extra` to check that all tests pass

# After Submitting

* Help us keep the docs up to date: If your PR affects the user
experience of Nushell (adding/removing a command, changing an
input/output type, etc.), make sure the changes are reflected in the
documentation (https://github.com/nushell/nushell.github.io) after the
PR is merged.

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2022-11-27 07:19:02 +13:00
Leon
bdca31cc2d
Rename dataframe describe to summary so that the normal describe isn't overloaded (#7176)
This closes #6770.
2022-11-23 17:58:28 +01:00
JT
13515c5eb0
Limited mutable variables (#7089)
This adds support for (limited) mutable variables. Mutable variables are created with mut much the same way immutable variables are made with let.

Mutable variables allow mutation via the assignment operator (=).

❯ mut x = 100
❯ $x = 200
❯ print $x
200

Mutable variables are limited in that they're only tended to be used in the local code block. Trying to capture a local variable will result in an error:

❯ mut x = 123; {|| $x }
Error: nu::parser::expected_keyword (link)

  × Capture of mutable variable.

The intent of this limitation is to reduce some of the issues with mutable variables in general: namely they make code that's harder to reason about. By reducing the scope that a mutable variable can be used it, we can help create local reasoning about them.

Mutation can occur with fields as well, as in this case:

❯ mut y = {abc: 123}
❯ $y.abc = 456
❯ $y

On a historical note: mutable variables are something that we resisted for quite a long time, leaning as much as we could on the functional style of pipelines and dataflow. That said, we've watched folks struggle to work with reduce as an approximation for patterns that would be trivial to express with local mutation. With that in mind, we're leaning towards the happy path.
2022-11-11 19:51:08 +13:00
WindSoilder
c259ef41bd
update polar to 0.25 (#6988) 2022-11-10 11:07:38 +13:00
WindSoilder
4f7f6a2932
Friendly error message for access beyond end (#6944)
Adds `ShellError::AccessEmptyContent`
2022-10-29 19:47:50 +02:00
Stefan Holderbach
6a7a60429f
Remove unnecessary #[allow(...)] annotations (#6870)
* Remove unnecessary `#[allow]` annots

Reduce the number of lint exceptions that are not necessary with the
current state of the code (or more recent toolchain)

* Remove dead code from `FileStructure` in nu-command

* Replace `allow(unused)` with relevant feature switch

* Deal with `needless_collect` with annotations

* Change hack for needless_collect in `from json`

This change obviates the need for `allow(needless_collect)`
Removes a pessimistic allocation for empty strings, but increases
allocation size to `Value`

Probably not really worth it.

* Revert "Deal with `needless_collect` with annotations"

This reverts commit 05aca98445.

The previous state seems to better from a performance perspective as a
`Vec<String>` is lighter weight than `Vec<Value>`
2022-10-24 20:12:16 +02:00
Leon
24a98f8999
Mildly edited a small handful of help messages (#6868)
* Edited a handful of help messages

* Remove line break as instructed by clippy
2022-10-23 02:02:52 -04:00
Dan Davison
ce809881eb
Rename query dfr -> query df (#6777) 2022-10-16 21:04:48 +02:00
HannahZhuSWE
7d5d53cf85
Add search terms to arg dataframe commands (#6724)
* added search terms for arg prefixed dataframe commands

* remove search terms that already produce results
2022-10-15 12:49:09 -05:00
Zephaniah Ong
804b155035
Add search terms for uppercase (#6720)
* Add search terms for uppercase

* Add search terms

* Add search terms

* Change to parse

* Add search terms for from

* Add search terms for to

* Remove duplicate function

* Remove duplication of search terms

* Remove search term
2022-10-14 05:36:31 -05:00
nibon7
15ebf45f46
Apply clippy fix for rust 1.63.0 (#6576)
* Apply clippy fix to avoid extra allocation

error: `format!(..)` appended to existing `String`
  --> crates/nu-engine/src/documentation.rs:82:9
   |
82 |         long_desc.push_str(&format!("\n{G}Subcommands{RESET}:\n"));
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `-D clippy::format-push-string` implied by `-D warnings`
   = help: consider using `write!` to avoid the extra allocation
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#format_push_string

error: `format!(..)` appended to existing `String`
  --> crates/nu-engine/src/documentation.rs:96:9
   |
96 |         long_desc.push_str(&format!("\n{G}Parameters{RESET}:\n"));
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = help: consider using `write!` to avoid the extra allocation
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#format_push_string

error: `format!(..)` appended to existing `String`
   --> crates/nu-engine/src/documentation.rs:128:9
    |
128 |         long_desc.push_str(&format!("\n{}Examples{}:", G, RESET));
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = help: consider using `write!` to avoid the extra allocation
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#format_push_string

error: `format!(..)` appended to existing `String`
   --> crates/nu-engine/src/documentation.rs:202:5
    |
202 |     long_desc.push_str(&format!("\n{}Flags{}:\n", G, RESET));
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = help: consider using `write!` to avoid the extra allocation
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#format_push_string

error: could not compile `nu-engine` due to 4 previous errors

* Apply clippy fix to avoid deref on an immutable reference

error: deref on an immutable reference
   --> crates/nu-command/src/dataframe/eager/sql_context.rs:188:77
    |
188 |                         SetExpr::Select(select_stmt) => self.execute_select(&*select_stmt)?,
    |                                                                             ^^^^^^^^^^^^^
    |
    = note: `-D clippy::borrow-deref-ref` implied by `-D warnings`
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#borrow_deref_ref
help: if you would like to reborrow, try removing `&*`
    |
188 |                         SetExpr::Select(select_stmt) => self.execute_select(select_stmt)?,
    |                                                                             ~~~~~~~~~~~
help: if you would like to deref, try using `&**`
    |
188 |                         SetExpr::Select(select_stmt) => self.execute_select(&**select_stmt)?,
    |                                                                             ~~~~~~~~~~~~~~

error: deref on an immutable reference
   --> crates/nu-command/src/database/values/dsl/expression.rs:252:15
    |
252 |         match &*expr {
    |               ^^^^^^ help: if you would like to reborrow, try removing `&*`: `expr`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#borrow_deref_ref

error: could not compile `nu-command` due to 2 previous errors
2022-09-17 12:10:32 -05:00
Darren Schroeder
4fdfd3d15e
rename with_sql to query dfr (#6568)
* rename with_sql to query dfr

* add search terms

* update example command
2022-09-16 08:34:58 -05:00
Darren Schroeder
f0ae6ffe12
update sql-parser crate and all the files it touches (#6566)
* update sql-parser crate and all the files it touches

* undo adding extra as a default feature
2022-09-15 18:03:43 -05:00
Darren Schroeder
02e3f49bce
provide a way to use sql to query dataframes (#6537) 2022-09-15 09:22:00 -05:00
Aron Nopanen
d08212409f
Support Arrow IPC file format with dataframes (#6548)
* Add support for Arrow IPC file format

Add support for Arrow IPC file format to dataframes commands. Support
opening of Arrow IPC-format files with extension '.arrow' or '.ipc' in
the open-df command. Add a 'to arrow' command to write a dataframe to
Arrow IPC format.

* Add unit test for open-df on Arrow

* Add -t flag to open-df command

Add a `--type`/`-t` flag to the `open-df` command, to explicitly specify
the type of file being used. Allowed values are the same at the set of
allowed file extensions.
2022-09-12 18:30:20 -05:00
JT
d86350af80
Revert "Make $ on variable names optional (#6434)" (#6446)
This reverts commit 3cb9147f22.
2022-09-06 05:42:47 +12:00
JT
3cb9147f22
Make $ on variable names optional (#6434)
* DRAFT: make var dollar optional

* couple fixes

* fix some tests + cleanup
2022-08-29 14:35:55 +12:00
Fernando Herrera
9d8d305e9d
lazy dataframe reader (#6321)
* lazy dataframe reader

* correct space for polars dependencies
2022-08-14 13:06:31 +01:00
Andrés N. Robalino
eb55fd2383
cmd(df/first): returns the first row by default. (#6312) 2022-08-13 14:08:00 -05:00
Fernando Herrera
ae64c58f59
Polars upgrade 0.23 (#6303)
* more lazy expressions

* upgrade polars and correct functions

* arg-where example

* cargo clippy

* restore modified filter files

* correct string addition with str

* correct string addition with str

* correct message in test
2022-08-12 13:10:36 +01:00
Stefan Holderbach
c2f4969d4f
Clippy fix for Rust 1.63 (#6299)
Take more sensitive lints into account

Somewhat ugly in some cases is the replacement of `.get(0)` with
`.first()`
2022-08-11 11:54:54 -05:00
Darren Schroeder
cdeb8de75d
replace the regex crate with the fancy-regex crate (#6227) 2022-08-04 14:51:02 -05:00
Fernando Herrera
b2ec32fdf0
concat string with lazy expressions (#6093) 2022-07-21 18:05:56 +01:00
Fernando Herrera
604025fe34
append string to series (#6089) 2022-07-21 10:42:12 +01:00
Fernando Herrera
9d0be7d96f
check column type during aggregation (#6058)
* check column type during aggregation

* check first if there is schema
2022-07-16 15:34:12 +01:00
Jakub Žádník
26f31da711
Split merging of parser delta and stack environment (#6005)
* Remove comment

* Split delta and environment merging

* Move table mode to a more logical place

* Cleanup

* Merge environment after reading default_env.nu

* Fmt
2022-07-14 17:09:27 +03:00
Fernando Herrera
c0901ef707
Dataframe with real index (#5892)
* remove extra print

* dataframe with real index

* corrected dataframe tests

* clippy error

* clippy error
2022-06-26 17:32:18 -05:00
Fernando Herrera
d3e84daa49
remove extra print (#5891) 2022-06-26 11:48:30 -05:00
JT
f2989bf704
Move input/output type from Command to Signature (#5880) 2022-06-26 09:23:56 +12:00
Fernando Herrera
6cc8402127
Standardise to commands (#5800)
* standarize to commands

* move from to to into
2022-06-17 07:51:50 -05:00
Fernando Herrera
38f3957edf
update polars (#5791) 2022-06-15 11:45:03 -05:00
Fernando Herrera
de554f8e5f
filesize conversion (#5770) 2022-06-13 14:44:32 -05:00
Fernando Herrera
44979f3051
expression to literal (#5769) 2022-06-13 13:22:46 -05:00
pwygab
caafd26deb
Attempts to add // math operator (#5759)
* attempts to add `div` math operator

* allows `//` to be used too

* fmt:

* clippy issue

* returns appropriate type

* returns appropriate type 2

* fmt

* ensure consistency; rename to `fdiv`

* Update parser.rs
2022-06-13 13:54:47 +03:00
Fernando Herrera
11d7d8ea1e
Remove dfr from dataframe commands (#5760)
* input and output tests

* input and output types for dfr

* expression converter

* remove deprecated command

* correct expressions

* cargo clippy

* identifier for ls

* cargo clippy

* type for head and tail expression

* modify full cell path if block
2022-06-12 14:18:00 -05:00
Fernando Herrera
d5b99ae316
input and output types (#5750)
* input and output types

* added description

* type from stored variable

* string in custom value

* more tests with non custom
2022-06-10 10:59:35 -05:00
Fernando Herrera
997d56a288
Lazy dataframes (#5687)
* change between lazy and eager

* when expressions

* examples for aggregations

* more examples for agg

* examples for dataframes

* checked examples

* cargo fmt
2022-05-31 07:29:55 +01:00
Fernando Herrera
8bd68416e3
Lazy dataframes (#5546)
* lazyframe definition

* expressions and lazy frames

* new alias expression

* more expression commands

* updated to polars main

* more expressions and groupby

* more expressions, fetch and sort-by

* csv reader

* removed open csv

* unique function

* joining functions

* join lazy frames commands with eager commands

* corrected tests

* Update .gitignore

* Update .gitignore

Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
2022-05-16 08:27:43 +01:00
Jakub Žádník
9b99b2f6ac
Overlays (#5375)
* WIP: Start laying overlays

* Rename Overlay->Module; Start adding overlay

* Revamp adding overlay

* Add overlay add tests; Disable debug print

* Fix overlay add; Add overlay remove

* Add overlay remove tests

* Add missing overlay remove file

* Add overlay list command

* (WIP?) Enable overlays for env vars

* Move OverlayFrames to ScopeFrames

* (WIP) Move everything to overlays only

ScopeFrame contains nothing but overlays now

* Fix predecls

* Fix wrong overlay id translation and aliases

* Fix broken env lookup logic

* Remove TODOs

* Add overlay add + remove for environment

* Add a few overlay tests; Fix overlay add name

* Some cleanup; Fix overlay add/remove names

* Clippy

* Fmt

* Remove walls of comments

* List overlays from stack; Add debugging flag

Currently, the engine state ordering is somehow broken.

* Fix (?) overlay list test

* Fix tests on Windows

* Fix activated overlay ordering

* Check for active overlays equality in overlay list

This removes the -p flag: Either both parser and engine will have the
same overlays, or the command will fail.

* Add merging on overlay remove

* Change help message and comment

* Add some remove-merge/discard tests

* (WIP) Track removed overlays properly

* Clippy; Fmt

* Fix getting last overlay; Fix predecls in overlays

* Remove merging; Fix re-add overwriting stuff

Also some error message tweaks.

* Fix overlay error in the engine

* Update variable_completions.rs

* Adds flags and optional arguments to view-source (#5446)

* added flags and optional arguments to view-source

* removed redundant code

* removed redundant code

* fmt

* fix bug in shell_integration (#5450)

* fix bug in shell_integration

* add some comments

* enable cd to work with directory abbreviations (#5452)

* enable cd to work with abbreviations

* add abbreviation example

* fix tests

* make it configurable

* make cd recornize symblic link (#5454)

* implement seq char command to generate single character sequence (#5453)

* add tmp code

* add seq char command

* Add split number flag in `split row` (#5434)

Signed-off-by: Yuheng Su <gipsyh.icu@gmail.com>

* Add two more overlay tests

* Add ModuleId to OverlayFrame

* Fix env conversion accidentally activating overlay

It activated overlay from permanent state prematurely which would
cause `overlay add` to misbehave.

* Remove unused parameter; Add overlay list test

* Remove added traces

* Add overlay commands examples

* Modify TODO

* Fix $nu.scope iteration

* Disallow removing default overlay

* Refactor some parser errors

* Remove last overlay if no argument

* Diversify overlay examples

* Make it possible to update overlay's module

In case the origin module updates, the overlay add loads the new module,
makes it overlay's origin and applies the changes. Before, it was
impossible to update the overlay if the module changed.

Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
Co-authored-by: pwygab <88221256+merelymyself@users.noreply.github.com>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
Co-authored-by: WindSoilder <WindSoilder@outlook.com>
Co-authored-by: Yuheng Su <gipsyh.icu@gmail.com>
2022-05-08 07:39:22 +12:00
panicbit
49cbc30974
Add ends-with operator and fix dataframe operator behavior (#5395)
* add ends-with operator

* escape needles in dataframe operator regex patterns
2022-05-02 20:02:38 +12:00
JT
76079d5183
Move config to be an env var (#5230)
* Move config to be an env var

* fix fmt and tests
2022-04-19 10:28:01 +12:00
Kat Marchán
1314a87cb0
update miette and switch to GenericErrors (#5222) 2022-04-19 00:34:10 +12:00
Reilly Wood
d38a3a8b4e
Fix command descriptions+examples (#5129)
* Fix exit usage

* Move dfr as-date* format examples to extra_usage

* Update command usage and examples

* More docs on `str trim`

Co-authored-by: sholderbach <sholderbach@users.noreply.github.com>
2022-04-08 10:30:49 +02:00
Reilly Wood
b2c52b51b7
Change string contains operators to regex (#5117) 2022-04-07 18:23:14 +12:00
Darren Schroeder
2cb815b7b4
Add starts with operator (#5061)
* add starts_with operator

* added a test
2022-04-01 13:35:46 -05:00
Fernando Herrera
d6669d3f33
Polars update (#4875)
* update to polars 0.20

* add to date parser for series
2022-03-19 11:13:34 +00:00