Add stor family of commands (#11170)

# Description

This PR adds the `stor` family of commands. These commands are meant to
create, open, insert, update, delete, reset data in an in-memory sqlite
database. This is really an experiment to see how creatively we can use
an in-memory database.

```
Usage:
  > stor

Subcommands:
  stor create - Create a table in the in-memory sqlite database
  stor delete - Delete a table or specified rows in the in-memory sqlite database
  stor export - Export the in-memory sqlite database to a sqlite database file
  stor import - Import a sqlite database file into the in-memory sqlite database
  stor insert - Insert information into a specified table in the in-memory sqlite database
  stor open - Opens the in-memory sqlite database
  stor reset - Reset the in-memory database by dropping all tables
  stor update - Update information in a specified table in the in-memory sqlite database

Flags:
  -h, --help - Display the help message for this command

Input/output types:
  ╭─#─┬──input──┬─output─╮
  │ 0 │ nothing │ string │
  ╰───┴─────────┴────────╯
```

### Examples

## stor create
```nushell
❯ stor create --table-name nudb --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}
╭──────┬────────────────╮
│ nudb │ [list 0 items] │
╰──────┴────────────────╯
```
## stor insert
```nushell
❯ stor insert --table-name nudb --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} 
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
```
## stor open
```nushell
❯ stor open | table -e 
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  2 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
```
## stor update
```nushell
❯ stor update --table-name nudb --update-record {str1: toby datetime1: 2021-04-17} --where-clause "bool1 = 1"
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
❯ stor open | table -e
╭──────┬─────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬─str1─┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  2 │ 1.10 │ toby │ 2021-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴──────┴────────────────────────────╯ │
╰──────┴─────────────────────────────────────────────────────────────────╯
```
## insert another row
```nushell
❯ stor insert --table-name nudb --data-record {bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} 
╭──────┬────────────────╮
│ nudb │ [table 2 rows] │
╰──────┴────────────────╯
❯ stor open | table -e
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  2 │ 1.10 │ toby    │ 2021-04-17 00:00:00 +00:00 │ │
│      │ │ 1 │ 2 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
```
## stor delete (specific row(s))
```nushell
❯ stor delete --table-name nudb --where-clause "int1 == 5"
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
```
## insert multiple tables
```nushell
❯ stor create --table-name nudb1 --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}
╭───────┬────────────────╮
│ nudb  │ [table 1 row]  │
│ nudb1 │ [list 0 items] │
╰───────┴────────────────╯
❯ stor insert --table-name nudb1 --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}
╭───────┬───────────────╮
│ nudb  │ [table 1 row] │
│ nudb1 │ [table 1 row] │
╰───────┴───────────────╯
❯ stor create --table-name nudb2 --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}
╭───────┬────────────────╮
│ nudb  │ [table 1 row]  │
│ nudb1 │ [table 1 row]  │
│ nudb2 │ [list 0 items] │
╰───────┴────────────────╯
❯ stor insert --table-name nudb2 --data-record {bool1: true, int1: 2, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}
╭───────┬───────────────╮
│ nudb  │ [table 1 row] │
│ nudb1 │ [table 1 row] │
│ nudb2 │ [table 1 row] │
╰───────┴───────────────╯
```
## stor delete (specific table)
```nushell
❯ stor delete --table-name nudb1
╭───────┬───────────────╮
│ nudb  │ [table 1 row] │
│ nudb2 │ [table 1 row] │
╰───────┴───────────────╯
```
## stor reset (all tables are deleted)
```nushell
❯ stor reset
```
## stor export
```nushell
❯ stor export --file-name nudb.sqlite3
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
❯ open nudb.sqlite3 | table -e
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
❯ open nudb.sqlite3 | schema | table -e
╭────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│        │ ╭──────┬──────────────────────────────────────────────────────────────────────────────────────────────────╮ │
│ tables │ │      │ ╭───────────────┬──────────────────────────────────────────────────────────────────────────────╮ │ │
│        │ │ nudb │ │               │ ╭─#─┬─cid─┬───name────┬─────type─────┬─notnull─┬───────default────────┬─pk─╮ │ │ │
│        │ │      │ │ columns       │ │ 0 │ 0   │ id        │ INTEGER      │ 1       │                      │ 1  │ │ │ │
│        │ │      │ │               │ │ 1 │ 1   │ bool1     │ BOOLEAN      │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 2 │ 2   │ int1      │ INTEGER      │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 3 │ 3   │ float1    │ REAL         │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 4 │ 4   │ str1      │ VARCHAR(255) │ 0       │                      │ 0  │ │ │ │
│        │ │      │ │               │ │ 5 │ 5   │ datetime1 │ DATETIME     │ 0       │ STRFTIME('%Y-%m-%d   │ 0  │ │ │ │
│        │ │      │ │               │ │   │     │           │              │         │ %H:%M:%f', 'NOW')    │    │ │ │ │
│        │ │      │ │               │ ╰─#─┴─cid─┴───name────┴─────type─────┴─notnull─┴───────default────────┴─pk─╯ │ │ │
│        │ │      │ │ constraints   │ [list 0 items]                                                               │ │ │
│        │ │      │ │ foreign_keys  │ [list 0 items]                                                               │ │ │
│        │ │      │ │ indexes       │ [list 0 items]                                                               │ │ │
│        │ │      │ ╰───────────────┴──────────────────────────────────────────────────────────────────────────────╯ │ │
│        │ ╰──────┴──────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```
## Using with `query db`
```nushell
❯ stor open | query db "select * from nudb"
╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮
│ 0 │ 1 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │
╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯
```
## stor import
```nushell
❯ stor open
# note, nothing is returned. there is nothing in memory, atm.
❯ stor import --file-name nudb.sqlite3
╭──────┬───────────────╮
│ nudb │ [table 1 row] │
╰──────┴───────────────╯
❯ stor open | table -e 
╭──────┬────────────────────────────────────────────────────────────────────╮
│      │ ╭─#─┬id─┬bool1┬int1┬float1┬──str1───┬─────────datetime1──────────╮ │
│ nudb │ │ 0 │ 1 │   1 │  5 │ 1.10 │ fdncred │ 2023-04-17 00:00:00 +00:00 │ │
│      │ ╰───┴───┴─────┴────┴──────┴─────────┴────────────────────────────╯ │
╰──────┴────────────────────────────────────────────────────────────────────╯
```

TODO:
- [x] `stor export` - Export a fully formed sqlite db file. 
- [x] `stor import` - Imports a specified sqlite db file.
- [x] Perhaps feature-gate it with the sqlite feature
- [x] Update `query db` to work with the in-memory database
- [x] Remove `open --in-memory`

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This commit is contained in:
Darren Schroeder
2023-11-29 10:02:46 -06:00
committed by GitHub
parent 112306aab5
commit e290fa0e68
17 changed files with 1199 additions and 42 deletions

View File

@ -0,0 +1,152 @@
use crate::database::{SQLiteDatabase, MEMORY_DB};
use nu_engine::CallExt;
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, PipelineData, Record, ShellError, Signature, Span,
SyntaxShape, Type, Value,
};
#[derive(Clone)]
pub struct StorCreate;
impl Command for StorCreate {
fn name(&self) -> &str {
"stor create"
}
fn signature(&self) -> Signature {
Signature::build("stor create")
.input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
.required_named(
"table-name",
SyntaxShape::String,
"name of the table you want to create",
Some('t'),
)
.required_named(
"columns",
SyntaxShape::Record(vec![]),
"a record of column names and datatypes",
Some('c'),
)
.allow_variants_without_examples(true)
.category(Category::Math)
}
fn usage(&self) -> &str {
"Create a table in the in-memory sqlite database"
}
fn search_terms(&self) -> Vec<&str> {
vec!["sqlite", "storing", "table"]
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Create an in-memory sqlite database with specified table name, column names, and column data types",
example: "stor create --table-name nudb --columns {bool1: bool, int1: int, float1: float, str1: str, datetime1: datetime}",
result: None,
}]
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
let span = call.head;
let table_name: Option<String> = call.get_flag(engine_state, stack, "table-name")?;
let columns: Option<Record> = call.get_flag(engine_state, stack, "columns")?;
let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
if table_name.is_none() {
return Err(ShellError::MissingParameter {
param_name: "requires at table name".into(),
span,
});
}
let new_table_name = table_name.unwrap_or("table".into());
if let Ok(conn) = db.open_connection() {
match columns {
Some(record) => {
let mut create_stmt = format!(
"CREATE TABLE {} ( id INTEGER NOT NULL PRIMARY KEY, ",
new_table_name
);
for (column_name, column_datatype) in record {
match column_datatype.as_string()?.as_str() {
"int" => {
create_stmt.push_str(&format!("{} INTEGER, ", column_name));
}
"float" => {
create_stmt.push_str(&format!("{} REAL, ", column_name));
}
"str" => {
create_stmt.push_str(&format!("{} VARCHAR(255), ", column_name));
}
"bool" => {
create_stmt.push_str(&format!("{} BOOLEAN, ", column_name));
}
"datetime" => {
create_stmt.push_str(&format!(
"{} DATETIME DEFAULT(STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')), ",
column_name
));
}
_ => {
return Err(ShellError::UnsupportedInput {
msg: "unsupported column data type".into(),
input: format!("{:?}", column_datatype),
msg_span: column_datatype.span(),
input_span: column_datatype.span(),
});
}
}
}
if create_stmt.ends_with(", ") {
create_stmt.pop();
create_stmt.pop();
}
create_stmt.push_str(" )");
// dbg!(&create_stmt);
conn.execute(&create_stmt, []).map_err(|err| {
ShellError::GenericError(
"Failed to open SQLite connection in memory from create".into(),
err.to_string(),
Some(Span::test_data()),
None,
Vec::new(),
)
})?;
}
None => {
return Err(ShellError::MissingParameter {
param_name: "requires at least one column".into(),
span: call.head,
});
}
};
}
// dbg!(db.clone());
Ok(Value::custom_value(db, span).into_pipeline_data())
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test_examples() {
use crate::test_examples;
test_examples(StorCreate {})
}
}

View File

@ -0,0 +1,141 @@
use crate::database::{SQLiteDatabase, MEMORY_DB};
use nu_engine::CallExt;
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, SyntaxShape,
Type, Value,
};
#[derive(Clone)]
pub struct StorDelete;
impl Command for StorDelete {
fn name(&self) -> &str {
"stor delete"
}
fn signature(&self) -> Signature {
Signature::build("stor delete")
.input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
.required_named(
"table-name",
SyntaxShape::String,
"name of the table you want to insert into",
Some('t'),
)
.named(
"where-clause",
SyntaxShape::String,
"a sql string to use as a where clause without the WHERE keyword",
Some('w'),
)
.allow_variants_without_examples(true)
.category(Category::Math)
}
fn usage(&self) -> &str {
"Delete a table or specified rows in the in-memory sqlite database"
}
fn search_terms(&self) -> Vec<&str> {
vec!["sqlite", "remove", "table", "saving", "drop"]
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Delete a table from the in-memory sqlite database",
example: "stor delete --table-name nudb",
result: None,
},
Example {
description:
"Delete some rows from the in-memory sqlite database with a where clause",
example: "stor delete --table-name nudb --where-clause \"int1 == 5\"",
result: None,
},
]
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
let span = call.head;
// For dropping/deleting an entire table
let table_name_opt: Option<String> = call.get_flag(engine_state, stack, "table-name")?;
// For deleting rows from a table
let where_clause_opt: Option<String> =
call.get_flag(engine_state, stack, "where-clause")?;
if table_name_opt.is_none() && where_clause_opt.is_none() {
return Err(ShellError::MissingParameter {
param_name: "requires at least one of table-name or where-clause".into(),
span,
});
}
if table_name_opt.is_none() && where_clause_opt.is_some() {
return Err(ShellError::MissingParameter {
param_name: "using the where-clause requires the use of a table-name".into(),
span,
});
}
// Open the in-mem database
let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
if let Some(new_table_name) = table_name_opt {
let where_clause = match where_clause_opt {
Some(where_stmt) => where_stmt,
None => String::new(),
};
if let Ok(conn) = db.open_connection() {
let sql_stmt = if where_clause.is_empty() {
// We're deleting an entire table
format!("DROP TABLE {}", new_table_name)
} else {
// We're just deleting some rows
let mut delete_stmt = format!("DELETE FROM {} ", new_table_name);
// Yup, this is a bit janky, but I'm not sure a better way to do this without having
// --and and --or flags as well as supporting ==, !=, <>, is null, is not null, etc.
// and other sql syntax. So, for now, just type a sql where clause as a string.
delete_stmt.push_str(&format!("WHERE {}", where_clause));
delete_stmt
};
// dbg!(&sql_stmt);
conn.execute(&sql_stmt, []).map_err(|err| {
ShellError::GenericError(
"Failed to open SQLite connection in memory from delete".into(),
err.to_string(),
Some(Span::test_data()),
None,
Vec::new(),
)
})?;
}
}
// dbg!(db.clone());
Ok(Value::custom_value(db, span).into_pipeline_data())
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test_examples() {
use crate::test_examples;
test_examples(StorDelete {})
}
}

View File

@ -0,0 +1,98 @@
use crate::database::{SQLiteDatabase, MEMORY_DB};
use nu_engine::CallExt;
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, SyntaxShape,
Type, Value,
};
#[derive(Clone)]
pub struct StorExport;
impl Command for StorExport {
fn name(&self) -> &str {
"stor export"
}
fn signature(&self) -> Signature {
Signature::build("stor export")
.input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
.required_named(
"file-name",
SyntaxShape::String,
"file name to export the sqlite in-memory database to",
Some('f'),
)
.allow_variants_without_examples(true)
.category(Category::Math)
}
fn usage(&self) -> &str {
"Export the in-memory sqlite database to a sqlite database file"
}
fn search_terms(&self) -> Vec<&str> {
vec!["sqlite", "save", "database", "saving", "file"]
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Export the in-memory sqlite database",
example: "stor export --file-name nudb.sqlite",
result: None,
}]
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
let span = call.head;
let file_name_opt: Option<String> = call.get_flag(engine_state, stack, "file-name")?;
let file_name = match file_name_opt {
Some(file_name) => file_name,
None => {
return Err(ShellError::MissingParameter {
param_name: "please supply a file name with the --file-name parameter".into(),
span,
})
}
};
// Open the in-mem database
let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
if let Ok(conn) = db.open_connection() {
// This uses vacuum. I'm not really sure if this is the best way to do this.
// I also added backup in the sqlitedatabase impl. If we have problems, we could switch to that.
db.export_in_memory_database_to_file(&conn, file_name)
.map_err(|err| {
ShellError::GenericError(
"Failed to open SQLite connection in memory from export".into(),
err.to_string(),
Some(Span::test_data()),
None,
Vec::new(),
)
})?;
}
// dbg!(db.clone());
Ok(Value::custom_value(db, span).into_pipeline_data())
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test_examples() {
use crate::test_examples;
test_examples(StorExport {})
}
}

View File

@ -0,0 +1,96 @@
use crate::database::{SQLiteDatabase, MEMORY_DB};
use nu_engine::CallExt;
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, SyntaxShape,
Type, Value,
};
#[derive(Clone)]
pub struct StorImport;
impl Command for StorImport {
fn name(&self) -> &str {
"stor import"
}
fn signature(&self) -> Signature {
Signature::build("stor import")
.input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
.required_named(
"file-name",
SyntaxShape::String,
"file name to export the sqlite in-memory database to",
Some('f'),
)
.allow_variants_without_examples(true)
.category(Category::Math)
}
fn usage(&self) -> &str {
"Import a sqlite database file into the in-memory sqlite database"
}
fn search_terms(&self) -> Vec<&str> {
vec!["sqlite", "open", "database", "restore", "file"]
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Import a sqlite database file into the in-memory sqlite database",
example: "stor import --file-name nudb.sqlite",
result: None,
}]
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
let span = call.head;
let file_name_opt: Option<String> = call.get_flag(engine_state, stack, "file-name")?;
let file_name = match file_name_opt {
Some(file_name) => file_name,
None => {
return Err(ShellError::MissingParameter {
param_name: "please supply a file name with the --file-name parameter".into(),
span,
})
}
};
// Open the in-mem database
let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
if let Ok(mut conn) = db.open_connection() {
db.restore_database_from_file(&mut conn, file_name)
.map_err(|err| {
ShellError::GenericError(
"Failed to open SQLite connection in memory from import".into(),
err.to_string(),
Some(Span::test_data()),
None,
Vec::new(),
)
})?;
}
// dbg!(db.clone());
Ok(Value::custom_value(db, span).into_pipeline_data())
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test_examples() {
use crate::test_examples;
test_examples(StorImport {})
}
}

View File

@ -0,0 +1,155 @@
use crate::database::{SQLiteDatabase, MEMORY_DB};
use nu_engine::CallExt;
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, PipelineData, Record, ShellError, Signature, Span,
SyntaxShape, Type, Value,
};
#[derive(Clone)]
pub struct StorInsert;
impl Command for StorInsert {
fn name(&self) -> &str {
"stor insert"
}
fn signature(&self) -> Signature {
Signature::build("stor insert")
.input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
.required_named(
"table-name",
SyntaxShape::String,
"name of the table you want to insert into",
Some('t'),
)
.required_named(
"data-record",
SyntaxShape::Record(vec![]),
"a record of column names and column values to insert into the specified table",
Some('d'),
)
.allow_variants_without_examples(true)
.category(Category::Math)
}
fn usage(&self) -> &str {
"Insert information into a specified table in the in-memory sqlite database"
}
fn search_terms(&self) -> Vec<&str> {
vec!["sqlite", "storing", "table", "saving"]
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Insert data the in-memory sqlite database using a data-record of column-name and column-value pairs",
example: "stor insert --table-name nudb --data-record {bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}",
result: None,
}]
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
let span = call.head;
let table_name: Option<String> = call.get_flag(engine_state, stack, "table-name")?;
let columns: Option<Record> = call.get_flag(engine_state, stack, "data-record")?;
// let config = engine_state.get_config();
let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
if table_name.is_none() {
return Err(ShellError::MissingParameter {
param_name: "requires at table name".into(),
span,
});
}
let new_table_name = table_name.unwrap_or("table".into());
if let Ok(conn) = db.open_connection() {
match columns {
Some(record) => {
let mut create_stmt = format!("INSERT INTO {} ( ", new_table_name);
let cols = record.columns();
cols.for_each(|col| {
create_stmt.push_str(&format!("{}, ", col));
});
if create_stmt.ends_with(", ") {
create_stmt.pop();
create_stmt.pop();
}
create_stmt.push_str(") VALUES ( ");
let vals = record.values();
vals.for_each(|val| match val {
Value::Int { val, .. } => {
create_stmt.push_str(&format!("{}, ", val));
}
Value::Float { val, .. } => {
create_stmt.push_str(&format!("{}, ", val));
}
Value::String { val, .. } => {
create_stmt.push_str(&format!("'{}', ", val));
}
Value::Date { val, .. } => {
create_stmt.push_str(&format!("'{}', ", val));
}
Value::Bool { val, .. } => {
create_stmt.push_str(&format!("{}, ", val));
}
_ => {
// return Err(ShellError::UnsupportedInput {
// msg: format!("{} is not a valid datepart, expected one of year, month, day, hour, minute, second, millisecond, microsecond, nanosecond", part.item),
// input: "value originates from here".to_string(),
// msg_span: span,
// input_span: val.span(),
// });
}
});
if create_stmt.ends_with(", ") {
create_stmt.pop();
create_stmt.pop();
}
create_stmt.push(')');
// dbg!(&create_stmt);
conn.execute(&create_stmt, []).map_err(|err| {
ShellError::GenericError(
"Failed to open SQLite connection in memory from insert".into(),
err.to_string(),
Some(Span::test_data()),
None,
Vec::new(),
)
})?;
}
None => {
return Err(ShellError::MissingParameter {
param_name: "requires at least one column".into(),
span: call.head,
});
}
};
}
// dbg!(db.clone());
Ok(Value::custom_value(db, span).into_pipeline_data())
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test_examples() {
use crate::test_examples;
test_examples(StorInsert {})
}
}

View File

@ -0,0 +1,35 @@
#[cfg(feature = "sqlite")]
mod create;
#[cfg(feature = "sqlite")]
mod delete;
#[cfg(feature = "sqlite")]
mod export;
#[cfg(feature = "sqlite")]
mod import;
#[cfg(feature = "sqlite")]
mod insert;
#[cfg(feature = "sqlite")]
mod open;
#[cfg(feature = "sqlite")]
mod reset;
mod stor_;
#[cfg(feature = "sqlite")]
mod update;
#[cfg(feature = "sqlite")]
pub use create::StorCreate;
#[cfg(feature = "sqlite")]
pub use delete::StorDelete;
#[cfg(feature = "sqlite")]
pub use export::StorExport;
#[cfg(feature = "sqlite")]
pub use import::StorImport;
#[cfg(feature = "sqlite")]
pub use insert::StorInsert;
#[cfg(feature = "sqlite")]
pub use open::StorOpen;
#[cfg(feature = "sqlite")]
pub use reset::StorReset;
pub use stor_::Stor;
#[cfg(feature = "sqlite")]
pub use update::StorUpdate;

View File

@ -0,0 +1,78 @@
use crate::database::{SQLiteDatabase, MEMORY_DB};
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Type,
};
#[derive(Clone)]
pub struct StorOpen;
impl Command for StorOpen {
fn name(&self) -> &str {
"stor open"
}
fn signature(&self) -> Signature {
Signature::build("stor open")
.input_output_types(vec![(
Type::Nothing,
Type::Custom("sqlite-in-memory".into()),
)])
.allow_variants_without_examples(true)
.category(Category::Math)
}
fn usage(&self) -> &str {
"Opens the in-memory sqlite database"
}
fn search_terms(&self) -> Vec<&str> {
vec!["sqlite", "storing", "access"]
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Open the in-memory sqlite database",
example: "stor open",
result: None,
}]
}
fn run(
&self,
_engine_state: &EngineState,
_stack: &mut Stack,
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
// eprintln!("Initializing nudb");
// eprintln!("Here's some things to try:");
// eprintln!("* stor open | schema | table -e");
// eprintln!("* stor open | query db 'insert into nudb (bool1,int1,float1,str1,datetime1) values (2,200,2.0,'str2','1969-04-17T06:00:00-05:00')'");
// eprintln!("* stor open | query db 'select * from nudb'");
// eprintln!("Now imagine all those examples happening as commands, without sql, in our normal nushell pipelines\n");
// TODO: Think about adding the following functionality
// * stor open --table-name my_table_name
// It returns the output of `select * from my_table_name`
// Just create an empty database with MEMORY_DB and nothing else
let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
// dbg!(db.clone());
Ok(db.into_value(call.head).into_pipeline_data())
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test_examples() {
use crate::test_examples;
test_examples(StorOpen {})
}
}

View File

@ -0,0 +1,77 @@
use crate::database::{SQLiteDatabase, MEMORY_DB};
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, Type, Value,
};
#[derive(Clone)]
pub struct StorReset;
impl Command for StorReset {
fn name(&self) -> &str {
"stor reset"
}
fn signature(&self) -> Signature {
Signature::build("stor reset")
.input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
.allow_variants_without_examples(true)
.category(Category::Math)
}
fn usage(&self) -> &str {
"Reset the in-memory database by dropping all tables"
}
fn search_terms(&self) -> Vec<&str> {
vec!["sqlite", "remove", "table", "saving", "drop"]
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Reset the in-memory sqlite database",
example: "stor reset",
result: None,
}]
}
fn run(
&self,
_engine_state: &EngineState,
_stack: &mut Stack,
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
let span = call.head;
// Open the in-mem database
let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
if let Ok(conn) = db.open_connection() {
db.drop_all_tables(&conn).map_err(|err| {
ShellError::GenericError(
"Failed to open SQLite connection in memory from reset".into(),
err.to_string(),
Some(Span::test_data()),
None,
Vec::new(),
)
})?;
}
// dbg!(db.clone());
Ok(Value::custom_value(db, span).into_pipeline_data())
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test_examples() {
use crate::test_examples;
test_examples(StorReset {})
}
}

View File

@ -0,0 +1,49 @@
use nu_engine::get_full_help;
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, IntoPipelineData, PipelineData, ShellError, Signature, Type, Value,
};
#[derive(Clone)]
pub struct Stor;
impl Command for Stor {
fn name(&self) -> &str {
"stor"
}
fn signature(&self) -> Signature {
Signature::build("stor")
.category(Category::Strings)
.input_output_types(vec![(Type::Nothing, Type::String)])
}
fn usage(&self) -> &str {
"Various commands for working with the in-memory sqlite database."
}
fn extra_usage(&self) -> &str {
"You must use one of the following subcommands. Using this command as-is will only produce this help message."
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
Ok(Value::string(
get_full_help(
&Stor.signature(),
&Stor.examples(),
engine_state,
stack,
self.is_parser_keyword(),
),
call.head,
)
.into_pipeline_data())
}
}

View File

@ -0,0 +1,167 @@
use crate::database::{SQLiteDatabase, MEMORY_DB};
use nu_engine::CallExt;
use nu_protocol::{
ast::Call,
engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, PipelineData, Record, ShellError, Signature, Span,
Spanned, SyntaxShape, Type, Value,
};
#[derive(Clone)]
pub struct StorUpdate;
impl Command for StorUpdate {
fn name(&self) -> &str {
"stor update"
}
fn signature(&self) -> Signature {
Signature::build("stor update")
.input_output_types(vec![(Type::Nothing, Type::Table(vec![]))])
.required_named(
"table-name",
SyntaxShape::String,
"name of the table you want to insert into",
Some('t'),
)
.required_named(
"update-record",
SyntaxShape::Record(vec![]),
"a record of column names and column values to update in the specified table",
Some('u'),
)
.named(
"where-clause",
SyntaxShape::String,
"a sql string to use as a where clause without the WHERE keyword",
Some('w'),
)
.allow_variants_without_examples(true)
.category(Category::Math)
}
fn usage(&self) -> &str {
"Update information in a specified table in the in-memory sqlite database"
}
fn search_terms(&self) -> Vec<&str> {
vec!["sqlite", "storing", "table", "saving", "changing"]
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Update the in-memory sqlite database",
example: "stor update --table-name nudb --update-record {str1: nushell datetime1: 2020-04-17}",
result: None,
},
Example {
description: "Update the in-memory sqlite database with a where clause",
example: "stor update --table-name nudb --update-record {str1: nushell datetime1: 2020-04-17} --where-clause \"bool1 = 1\"",
result: None,
},
]
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
_input: PipelineData,
) -> Result<PipelineData, ShellError> {
let span = call.head;
let table_name: Option<String> = call.get_flag(engine_state, stack, "table-name")?;
let columns: Option<Record> = call.get_flag(engine_state, stack, "update-record")?;
let where_clause_opt: Option<Spanned<String>> =
call.get_flag(engine_state, stack, "where-clause")?;
// Open the in-mem database
let db = Box::new(SQLiteDatabase::new(std::path::Path::new(MEMORY_DB), None));
if table_name.is_none() {
return Err(ShellError::MissingParameter {
param_name: "requires at table name".into(),
span,
});
}
let new_table_name = table_name.unwrap_or("table".into());
if let Ok(conn) = db.open_connection() {
match columns {
Some(record) => {
let mut update_stmt = format!("UPDATE {} ", new_table_name);
update_stmt.push_str("SET ");
let vals = record.iter();
vals.for_each(|(key, val)| match val {
Value::Int { val, .. } => {
update_stmt.push_str(&format!("{} = {}, ", key, val));
}
Value::Float { val, .. } => {
update_stmt.push_str(&format!("{} = {}, ", key, val));
}
Value::String { val, .. } => {
update_stmt.push_str(&format!("{} = '{}', ", key, val));
}
Value::Date { val, .. } => {
update_stmt.push_str(&format!("{} = '{}', ", key, val));
}
Value::Bool { val, .. } => {
update_stmt.push_str(&format!("{} = {}, ", key, val));
}
_ => {
// return Err(ShellError::UnsupportedInput {
// msg: format!("{} is not a valid datepart, expected one of year, month, day, hour, minute, second, millisecond, microsecond, nanosecond", part.item),
// input: "value originates from here".to_string(),
// msg_span: span,
// input_span: val.span(),
// });
}
});
if update_stmt.ends_with(", ") {
update_stmt.pop();
update_stmt.pop();
}
// Yup, this is a bit janky, but I'm not sure a better way to do this without having
// --and and --or flags as well as supporting ==, !=, <>, is null, is not null, etc.
// and other sql syntax. So, for now, just type a sql where clause as a string.
if let Some(where_clause) = where_clause_opt {
update_stmt.push_str(&format!(" WHERE {}", where_clause.item));
}
// dbg!(&update_stmt);
conn.execute(&update_stmt, []).map_err(|err| {
ShellError::GenericError(
"Failed to open SQLite connection in memory from update".into(),
err.to_string(),
Some(Span::test_data()),
None,
Vec::new(),
)
})?;
}
None => {
return Err(ShellError::MissingParameter {
param_name: "requires at least one column".into(),
span: call.head,
});
}
};
}
// dbg!(db.clone());
Ok(Value::custom_value(db, span).into_pipeline_data())
}
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test_examples() {
use crate::test_examples;
test_examples(StorUpdate {})
}
}