So, to tell a story
1. We introduced the record sync, intended to be the new algorithm to
sync history.
2. On top of this, I added the KV store. This was intended as a simple
test of the record sync, and to see if people wanted that sort of
functionality
3. History remained syncing via the old means, as while it had issues it
worked more-or-less OK. And we are aware of its flaws
4. If KV syncing worked ok, history would be moved across
KV syncing ran ok for 6mo or so, so I started to move across history.
For several weeks, I ran a local fork of Atuin + the server that synced
via records instead.
The record store maintained ordering via a linked list, which was a
mistake. It performed well in testing, but was really difficult to debug
and reason about. So when a few small sync issues occured, they took an
extremely long time to debug.
This PR is huge, which I regret. It involves replacing the "parent"
relationship that records once had (pointing to the previous record)
with a simple index (generally referred to as idx). This also means we
had to change the recordindex, which referenced "tails". Tails were the
last item in the chain.
Now that we use an "array" vs linked list, that logic was also replaced.
And is much simpler :D
Same for the queries that act on this data.
----
This isn't final - we still need to add
1. Proper server/client error handling, which has been lacking for a
while
2. The actual history implementation on top
This exists in a branch, just without deletions. Won't be much to
add that, I just don't want to make this any larger than it already
is
The _only_ caveat here is that we basically lose data synced via the old
record store. This is the KV data from before.
It hasn't been deleted or anything, just no longer hooked up. So it's
totally possible to write a migration script. I just need to do that.
* fix(bash): history should be updated before preexec
* fix(bash): properly execute "--"
With the current implementation, the user command "--" would not be
executed even if it were the intended one. This is because it would
be confused as an option by the "eval" builtin. We can specify "--"
to tell "eval" that the later arguments should be literally treated as
the command.
* fix(bash): correctly restore $? and $_
* fix(bash): fix the use of preexec_ret_value
The exit status of preexec_ret_value is used to suppress the execution
of the corresponding command in the extdebug mode. The current
implementation somehow tries to set $? before the call of stty, which
does not have any effect. Instead, we can manually turn off the
execution of the user command when the condition matches.
* feat(bash): support array PROMPT_COMMAND of Bash >= 5.1
* feat(bash): check version of ble.sh
blehooks are only supported in ble.sh >= 0.4, so we require the ble.sh
version to be larger or equal to 0.4. We also describe the version
requirement in README.md.
* fix(bash): use ble.sh's contrib/integration/bash-preexec
ble.sh provides module "contrib/integration/bash-preexec", which can
be used with the same interface as bash-preexec. This module provides
"preexec_functions" and "precmd_functions" without requiring
bash-preexec.
This module also properly handles it when both ble.sh and bash-preexec
are loaded; the module resolves the conflicts between ble.sh and
bash-preexec, and the module also tries to support bash-preexec in the
detached state of ble.sh.
* fix(bash): use ble.sh's accept-line widget for enter_accept
In ble.sh, one can directly call the widget "accept-line" from a shell
script. The widget "accept-line" is the actual widget that reserves
the command execution in ble.sh, so calling "accept-line" is
equivalent to the normal execution. It includes all the necessary
adjustments and processing including stty and history.
In addition, the command will be executed at the top-level context
instead in a function scope. For example, without ble.sh, running
"declare -A dict=()" through enter_accept will create an associative
array in the function scope unexpectedly. With ble.sh, since the
command is executed at the top-level context, such a problem does not
happen.
When ble.sh is in a detached state, we fall back to the manual
execution of the command. In this case, we cannot assume the
existence of the shell function "__bp_set_ret_value", so we always use
__atuin_set_ret_value.
With a single-line prompt, the last line of the output of the previous
command is overwritten by the prompt on the enter_accept. In this
situation, `tput cuu` receives 0 as the argument, but `tput cuu 0`
emits the control sequence `\e[0A`, which moves the cursor above by
one line unexpectedly. This is because the parameter 0 for CUU means
the default value, 1. In this patch, to avoid moving the cursor when
the prompt offset is 0, we check the offset value before running `tput
cuu`.
* Add TLS to atuin-server
atuin as a project already includes most of the dependencies necessary
for server-side TLS. This allows `atuin server start` to use a TLS
certificate when self-hosting in order to avoid the complication of
wrapping it in a TLS-aware proxy server.
Configuration is handled similar to the metrics server with its own
struct and currently accepts only the private key and certificate file
paths.
Starting a TLS server and a TCP server are divergent because the tests
need to bind to an arbitrary port to avoid collisions across tests. The
API to accomplish this for a TLS server is much more verbose.
* Fix clippy, fmt
* Add TLS section to self-hosting
* feat: add semver checking to client requests
This enforces that the client and the server run the same major version
in order to sync successfully.
We're using the `Atuin-Version` http header to transfer this information
If the user is not on the same MAJOR, then they will see an error like
this
> Atuin version mismatch! In order to successfully sync, the client and the server must run the same *major* version
> Client: 17.1.0
> Server: 18.1.0
> Error: could not sync records due to version mismatch
This change means two things
1. We will now only increment major versions if there is a breaking
change for sync
2. We can now add breaking changes to sync, for any version >17.1.0.
Clients will fail in a meaningful way.
* lint, fmt, etc
* only check for client newer than server
* Add version header to client too
Before this change, when configuring only `common_subcommands` and
not `common_prefix` (so it would take its default value),
atuin produces an error message:
```
Error: could not load client settings Caused by: failed to deserialize: missing field `common_prefix` Location: atuin-client/src/settings.rs:456:26
Error: could not load client settings
Caused by:
failed to deserialize: missing field `common_prefix`
Location:
atuin-client/src/settings.rs:456:26
Error:: command not found
```
With this change, the fields can be specified separately and missing
fields will take their default values.
