Magical shell history
Go to file
Ellie Huxtable 29f4a93e30 feat: rework record sync for improved reliability
So, to tell a story

1. We introduced the record sync, intended to be the new algorithm to
   sync history.
2. On top of this, I added the KV store. This was intended as a simple
   test of the record sync, and to see if people wanted that sort of
   functionality
3. History remained syncing via the old means, as while it had issues it
   worked more-or-less OK. And we are aware of its flaws
4. If KV syncing worked ok, history would be moved across

KV syncing ran ok for 6mo or so, so I started to move across history.
For several weeks, I ran a local fork of Atuin + the server that synced
via records instead.

The record store maintained ordering via a linked list, which was a
mistake. It performed well in testing, but was really difficult to debug
and reason about. So when a few small sync issues occured, they took an
extremely long time to debug.

This PR is huge, which I regret. It involves replacing the "parent"
relationship that records once had (pointing to the previous record)
with a simple index (generally referred to as idx). This also means we
had to change the recordindex, which referenced "tails". Tails were the
last item in the chain.

Now that we use an "array" vs linked list, that logic was also replaced.
And is much simpler :D

Same for the queries that act on this data.

----

This isn't final - we still need to add

1. Proper server/client error handling, which has been lacking for a
   while
2. The actual history implementation on top
    This exists in a branch, just without deletions. Won't be much to
    add that, I just don't want to make this any larger than it already
    is

The _only_ caveat here is that we basically lose data synced via the old
record store. This is the KV data from before.

It hasn't been deleted or anything, just no longer hooked up. So it's
totally possible to write a migration script. I just need to do that.
2024-01-01 18:09:23 +00:00
.github feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
atuin feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
atuin-client feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
atuin-common feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
atuin-server feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
atuin-server-database feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
atuin-server-postgres feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
docs feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
k8s Move all references to the old repo (#1132) 2023-07-30 23:08:00 +01:00
.dockerignore Optimise docker (#34) 2021-04-14 17:40:50 +00:00
.gitignore feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
.mailmap Add history deletion (#791) 2023-03-20 09:26:54 +00:00
.rustfmt.toml ignore JetBrains IDEs, tidy-up imports (#348) 2022-04-28 18:53:59 +01:00
atuin.nix fix(nix): Add Appkit to the package build (#1358) 2023-10-30 08:18:26 +00:00
atuin.plugin.zsh run shellcheck (#97) 2021-05-14 08:31:15 +01:00
Cargo.lock feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
Cargo.toml feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
CHANGELOG.md chore(release): prepare for release v17.1.0 (#1432) 2023-12-10 13:00:59 +00:00
cliff.toml chore: setup git cliff (#1431) 2023-12-10 12:26:28 +00:00
CODE_OF_CONDUCT.md Add code of conduct (#281) 2022-03-17 21:43:54 +00:00
CONTRIBUTING.md Update CONTRIBUTING.md 2023-11-19 20:35:46 +00:00
CONTRIBUTORS chore(release): prepare for release v17.1.0 (#1432) 2023-12-10 13:00:59 +00:00
default.nix nix: add flake-compat (#743) 2023-03-01 19:51:31 +00:00
demo.gif Release v0.7.0 (#103) 2021-05-10 21:28:07 +01:00
deny.toml replace chrono with time (#806) 2023-09-11 09:26:05 +01:00
docker-compose.yml Move all references to the old repo (#1132) 2023-07-30 23:08:00 +01:00
Dockerfile feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
flake.lock Disable server tests in the nix build (#1123) 2023-07-28 08:49:42 +01:00
flake.nix fix(nix): Add Appkit to the package build (#1358) 2023-10-30 08:18:26 +00:00
install.sh feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00
LICENSE Create LICENSE 2021-02-14 16:22:25 +00:00
README.md feat: rework record sync for improved reliability 2024-01-01 18:09:23 +00:00

Atuin logo

magical shell history


English | 简体中文

Atuin replaces your existing shell history with a SQLite database, and records additional context for your commands. Additionally, it provides optional and fully encrypted synchronisation of your history between machines, via an Atuin server.

animated

exit code, duration, time and command shown

As well as the search UI, it can do things like this:

# search for all successful `make` commands, recorded after 3pm yesterday
atuin search --exit 0 --after "yesterday 3pm" make

You may use either the server I host, or host your own! Or just don't use sync at all. As all history sync is encrypted, I couldn't access your data even if I wanted to. And I really don't want to.

Features

  • rebind ctrl-r and up (configurable) to a full screen history search UI
  • store shell history in a sqlite database
  • backup and sync encrypted shell history
  • the same history across terminals, across sessions, and across machines
  • log exit code, cwd, hostname, session, command duration, etc
  • calculate statistics such as "most used command"
  • old history file is not replaced
  • quick-jump to previous items with Alt-<num>
  • switch filter modes via ctrl-r; search history just from the current session, directory, or globally

Documentation

Supported Shells

  • zsh
  • bash
  • fish
  • nushell

Community

Forum

Atuin has a community forum, please ask here for help and support: https://forum.atuin.sh/

Discord

Atuin also has a community Discord, available here

Quickstart

With the default sync server

This will sign you up for the default sync server, hosted by me. Everything is end-to-end encrypted, so your secrets are safe!

Read more below for offline-only usage, or for hosting your own server.

bash <(curl https://raw.githubusercontent.com/atuinsh/atuin/main/install.sh)

atuin register -u <USERNAME> -e <EMAIL>
atuin import auto
atuin sync

Then restart your shell!

Opt-in to activity graph

Alongside the hosted Atuin server, there is also a service which generates activity graphs for your shell history! These are inspired by the GitHub graph.

For example, here is mine:

Activity Graph Example

If you wish to get your own, after signing up for the sync server, run this

curl https://api.atuin.sh/enable -d $(cat ~/.local/share/atuin/session)

The response includes the URL to your graph. Feel free to share and/or embed this URL, the token is not a secret, and simply prevents user enumeration.

Offline only (no sync)

bash <(curl https://raw.githubusercontent.com/atuinsh/atuin/main/install.sh)
            
atuin import auto

By default, Atuin will check for updates. You can disable update checks by modifying config.toml.

Then restart your shell!

Install

The install script will help you through the setup, ensuring your shell is properly configured. It will also use one of the below methods, preferring the system package manager where possible (pacman, homebrew, etc etc).

# do not run this as root, root will be asked for if required
bash <(curl https://raw.githubusercontent.com/atuinsh/atuin/main/install.sh)

And then follow the shell setup

With cargo

It's best to use rustup to get setup with a Rust toolchain, then you can run:

cargo install atuin

And then follow the shell setup

Homebrew

brew install atuin

And then follow the shell setup

MacPorts

Atuin is also available in MacPorts

sudo port install atuin

And then follow the shell setup

Nix

This repository is a flake, and can be installed using nix profile:

nix profile install "github:atuinsh/atuin"

Atuin is also available in nixpkgs:

nix-env -f '<nixpkgs>' -iA atuin

And then follow the shell setup

Pacman

Atuin is available in the Arch Linux [extra] repository:

pacman -S atuin

And then follow the shell setup

Termux

Atuin is available in the Termux package repository:

pkg install atuin

And then follow the shell setup

From source

git clone https://github.com/atuinsh/atuin.git
cd atuin/atuin
cargo install --path .

And then follow the shell setup

Shell plugin

Once the binary is installed, the shell plugin requires installing. If you use the install script, this should all be done for you! After installing, remember to restart your shell.

zsh

echo 'eval "$(atuin init zsh)"' >> ~/.zshrc

Zinit

zinit load atuinsh/atuin

Antigen

antigen bundle atuinsh/atuin@main

bash

ble.sh

Atuin works best in bash when using ble.sh >= 0.4.

With ble.sh (>= 0.4) installed, just add atuin to your .bashrc

echo 'eval "$(atuin init bash)"' >> ~/.bashrc

Please make sure that the above line comes after sourcing ble.sh so atuin knows the presence of ble.sh.

bash-preexec

Bash-preexec can also be used, but you may experience some minor problems with the recorded duration and exit status of some commands.

To use bash-preexec, download and initialize it

curl https://raw.githubusercontent.com/rcaloras/bash-preexec/master/bash-preexec.sh -o ~/.bash-preexec.sh
echo '[[ -f ~/.bash-preexec.sh ]] && source ~/.bash-preexec.sh' >> ~/.bashrc

Then setup Atuin

echo 'eval "$(atuin init bash)"' >> ~/.bashrc

PLEASE NOTE

bash-preexec currently has an issue where it will stop honoring ignorespace. While Atuin will ignore commands prefixed with whitespace, they may still end up in your bash history. Please check your configuration! All other shells do not have this issue.

fish

Add

atuin init fish | source

to your is-interactive block in your ~/.config/fish/config.fish file

Nushell

Run in Nushell:

mkdir ~/.local/share/atuin/
atuin init nu | save ~/.local/share/atuin/init.nu

Add to config.nu:

source ~/.local/share/atuin/init.nu

Contributors

Made with contrib.rocks.