mirror of
https://github.com/atuinsh/atuin.git
synced 2025-08-03 13:07:32 +02:00
29f4a93e30
So, to tell a story
1. We introduced the record sync, intended to be the new algorithm to
sync history.
2. On top of this, I added the KV store. This was intended as a simple
test of the record sync, and to see if people wanted that sort of
functionality
3. History remained syncing via the old means, as while it had issues it
worked more-or-less OK. And we are aware of its flaws
4. If KV syncing worked ok, history would be moved across
KV syncing ran ok for 6mo or so, so I started to move across history.
For several weeks, I ran a local fork of Atuin + the server that synced
via records instead.
The record store maintained ordering via a linked list, which was a
mistake. It performed well in testing, but was really difficult to debug
and reason about. So when a few small sync issues occured, they took an
extremely long time to debug.
This PR is huge, which I regret. It involves replacing the "parent"
relationship that records once had (pointing to the previous record)
with a simple index (generally referred to as idx). This also means we
had to change the recordindex, which referenced "tails". Tails were the
last item in the chain.
Now that we use an "array" vs linked list, that logic was also replaced.
And is much simpler :D
Same for the queries that act on this data.
----
This isn't final - we still need to add
1. Proper server/client error handling, which has been lacking for a
while
2. The actual history implementation on top
This exists in a branch, just without deletions. Won't be much to
add that, I just don't want to make this any larger than it already
is
The _only_ caveat here is that we basically lose data synced via the old
record store. This is the KV data from before.
It hasn't been deleted or anything, just no longer hooked up. So it's
totally possible to write a migration script. I just need to do that.
100 lines
3.0 KiB
Rust
100 lines
3.0 KiB
Rust
use std::{io::prelude::*, path::PathBuf};
|
|
|
|
use config::{Config, Environment, File as ConfigFile, FileFormat};
|
|
use eyre::{eyre, Result};
|
|
use fs_err::{create_dir_all, File};
|
|
use serde::{de::DeserializeOwned, Deserialize, Serialize};
|
|
|
|
static EXAMPLE_CONFIG: &str = include_str!("../server.toml");
|
|
|
|
#[derive(Clone, Debug, Deserialize, Serialize)]
|
|
pub struct Metrics {
|
|
pub enable: bool,
|
|
pub host: String,
|
|
pub port: u16,
|
|
}
|
|
|
|
impl Default for Metrics {
|
|
fn default() -> Self {
|
|
Self {
|
|
enable: false,
|
|
host: String::from("127.0.0.1"),
|
|
port: 9001,
|
|
}
|
|
}
|
|
}
|
|
|
|
#[derive(Clone, Debug, Deserialize, Serialize)]
|
|
pub struct Settings<DbSettings> {
|
|
pub host: String,
|
|
pub port: u16,
|
|
pub path: String,
|
|
pub open_registration: bool,
|
|
pub max_history_length: usize,
|
|
pub max_record_size: usize,
|
|
pub page_size: i64,
|
|
pub register_webhook_url: Option<String>,
|
|
pub register_webhook_username: String,
|
|
pub metrics: Metrics,
|
|
|
|
#[serde(flatten)]
|
|
pub db_settings: DbSettings,
|
|
}
|
|
|
|
impl<DbSettings: DeserializeOwned> Settings<DbSettings> {
|
|
pub fn new() -> Result<Self> {
|
|
let mut config_file = if let Ok(p) = std::env::var("ATUIN_CONFIG_DIR") {
|
|
PathBuf::from(p)
|
|
} else {
|
|
let mut config_file = PathBuf::new();
|
|
let config_dir = atuin_common::utils::config_dir();
|
|
config_file.push(config_dir);
|
|
config_file
|
|
};
|
|
|
|
config_file.push("server.toml");
|
|
|
|
// create the config file if it does not exist
|
|
let mut config_builder = Config::builder()
|
|
.set_default("host", "127.0.0.1")?
|
|
.set_default("port", 8888)?
|
|
.set_default("open_registration", false)?
|
|
.set_default("max_history_length", 8192)?
|
|
.set_default("max_record_size", 1024 * 1024 * 1024)? // pretty chonky
|
|
.set_default("path", "")?
|
|
.set_default("register_webhook_username", "")?
|
|
.set_default("page_size", 1100)?
|
|
.set_default("metrics.enable", false)?
|
|
.set_default("metrics.host", "127.0.0.1")?
|
|
.set_default("metrics.port", 9001)?
|
|
.add_source(
|
|
Environment::with_prefix("atuin")
|
|
.prefix_separator("_")
|
|
.separator("__"),
|
|
);
|
|
|
|
config_builder = if config_file.exists() {
|
|
config_builder.add_source(ConfigFile::new(
|
|
config_file.to_str().unwrap(),
|
|
FileFormat::Toml,
|
|
))
|
|
} else {
|
|
create_dir_all(config_file.parent().unwrap())?;
|
|
let mut file = File::create(config_file)?;
|
|
file.write_all(EXAMPLE_CONFIG.as_bytes())?;
|
|
|
|
config_builder
|
|
};
|
|
|
|
let config = config_builder.build()?;
|
|
|
|
config
|
|
.try_deserialize()
|
|
.map_err(|e| eyre!("failed to deserialize: {}", e))
|
|
}
|
|
}
|
|
|
|
pub fn example_config() -> &'static str {
|
|
EXAMPLE_CONFIG
|
|
}
|