* feat: rework record sync for improved reliability
So, to tell a story
1. We introduced the record sync, intended to be the new algorithm to
sync history.
2. On top of this, I added the KV store. This was intended as a simple
test of the record sync, and to see if people wanted that sort of
functionality
3. History remained syncing via the old means, as while it had issues it
worked more-or-less OK. And we are aware of its flaws
4. If KV syncing worked ok, history would be moved across
KV syncing ran ok for 6mo or so, so I started to move across history.
For several weeks, I ran a local fork of Atuin + the server that synced
via records instead.
The record store maintained ordering via a linked list, which was a
mistake. It performed well in testing, but was really difficult to debug
and reason about. So when a few small sync issues occured, they took an
extremely long time to debug.
This PR is huge, which I regret. It involves replacing the "parent"
relationship that records once had (pointing to the previous record)
with a simple index (generally referred to as idx). This also means we
had to change the recordindex, which referenced "tails". Tails were the
last item in the chain.
Now that we use an "array" vs linked list, that logic was also replaced.
And is much simpler :D
Same for the queries that act on this data.
----
This isn't final - we still need to add
1. Proper server/client error handling, which has been lacking for a
while
2. The actual history implementation on top
This exists in a branch, just without deletions. Won't be much to
add that, I just don't want to make this any larger than it already
is
The _only_ caveat here is that we basically lose data synced via the old
record store. This is the KV data from before.
It hasn't been deleted or anything, just no longer hooked up. So it's
totally possible to write a migration script. I just need to do that.
* update .gitignore
* use correct endpoint
* fix for stores with length of 1
* use create/delete enum for history store
* lint, remove unneeded host_id
* remove prints
* add command to import old history
* add enable/disable switch for record sync
* add record sync to auto sync
* satisfy the almighty clippy
* remove file that I did not mean to commit
* feedback
This enabled the Kitty Keyboard Protocol
Read more here: https://sw.kovidgoyal.net/kitty/keyboard-protocol/
No change on unsupported terminals, but means in the future we can be
more creative with keybinding depending on terminal.
Tested on Alacritty and events come through with all modifiers
supported.
Will be useful for #193
Without TLS config, the server fails to load defaults. Until this is
released, add this to your server config
```
[tls]
enable = false
cert_path = ""
pkey_path = ""
```
* fix(import/zsh): zsh use a special format to escape some characters
unescape those correctly rather than throw them away.
zsh side code:
9f57ca4ac8/Src/utils.c (L4889-L4900)
* fix code style
The parameter expansions for the prompt strings, `${PS1@P}`, is only
available in bash >= 4.4. In Bash 4.3 or below w/ bash-preexec, the
current implementation produces error messages. There is no way to
evaluate PS1 with bash < 4.4, so we give up the adjustments for
multiline prompts in bash < 4.4 in this patch.
* fix(bash): preserve the line content on search cancel
In the current implementation for Bash, the line content is lost when
the user cancels the atuin search by pressing ESC, C-g, or Down at the
bottom line. This is because the line content is set to the empty
string returned by atuin on the cancellation of the search.
In the integrations for other shells, zsh and fish, the empty output
is properly handled so that the line content is preserved. This patch
makes the behavior in Bash consistent with that in zsh and fish, i.e.,
we do nothing when the atuin search returns an empty output.
* fix(zsh): ignore confusing line `__atuin_accept__:*` on search cancel
* fix(bash): prefix "__atuin_" to avoid variable conflicts
Because the function "__atuin_history" executes an arbitary user
command for "enter_accept", the local variable names should be
carefully chosen. A local variable can shadow a global variable that
the user wants to use when there is a name conflict. To avoid such a
situation we try to namespace the variables used by atuin by prefixing
"__atuin_".
* fix(bash): work around "shopt -s xpg_echo"
* refactor(bash): simplify the rendering of the prompt
* perf(bash): avoid extra evaluation of PS1
* refactor(bash): count \n by wc
We can simply use "wc -l" to count the number of newline characters.
In the POSIX standard, a line in a text stream is defined as
characters terminated by a newline character, so the unterminated line
is not counted by "wc -l". As a result, "wc -l" actually counts the
number of newline characters.
* refactor(bash): rename localvar `HISTORY => __atuin_command`
This patch renames the local variable `HISTORY` in __atuin_accept_line
to `__atuin_command`. The name of the global variable `HISTORY` set
by `__atuin_history` is kept.
* feat: integrate with zsh-autosuggestions
* Update atuin/src/shell/atuin.zsh
Co-authored-by: Patrick Jackson <patrick@jackson.dev>
* Update atuin/src/shell/atuin.zsh
Co-authored-by: Patrick Jackson <patrick@jackson.dev>
* feedback
---------
Co-authored-by: Patrick Jackson <patrick@jackson.dev>
* fix(bash): history should be updated before preexec
* fix(bash): properly execute "--"
With the current implementation, the user command "--" would not be
executed even if it were the intended one. This is because it would
be confused as an option by the "eval" builtin. We can specify "--"
to tell "eval" that the later arguments should be literally treated as
the command.
* fix(bash): correctly restore $? and $_
* fix(bash): fix the use of preexec_ret_value
The exit status of preexec_ret_value is used to suppress the execution
of the corresponding command in the extdebug mode. The current
implementation somehow tries to set $? before the call of stty, which
does not have any effect. Instead, we can manually turn off the
execution of the user command when the condition matches.
* feat(bash): support array PROMPT_COMMAND of Bash >= 5.1
* feat(bash): check version of ble.sh
blehooks are only supported in ble.sh >= 0.4, so we require the ble.sh
version to be larger or equal to 0.4. We also describe the version
requirement in README.md.
* fix(bash): use ble.sh's contrib/integration/bash-preexec
ble.sh provides module "contrib/integration/bash-preexec", which can
be used with the same interface as bash-preexec. This module provides
"preexec_functions" and "precmd_functions" without requiring
bash-preexec.
This module also properly handles it when both ble.sh and bash-preexec
are loaded; the module resolves the conflicts between ble.sh and
bash-preexec, and the module also tries to support bash-preexec in the
detached state of ble.sh.
* fix(bash): use ble.sh's accept-line widget for enter_accept
In ble.sh, one can directly call the widget "accept-line" from a shell
script. The widget "accept-line" is the actual widget that reserves
the command execution in ble.sh, so calling "accept-line" is
equivalent to the normal execution. It includes all the necessary
adjustments and processing including stty and history.
In addition, the command will be executed at the top-level context
instead in a function scope. For example, without ble.sh, running
"declare -A dict=()" through enter_accept will create an associative
array in the function scope unexpectedly. With ble.sh, since the
command is executed at the top-level context, such a problem does not
happen.
When ble.sh is in a detached state, we fall back to the manual
execution of the command. In this case, we cannot assume the
existence of the shell function "__bp_set_ret_value", so we always use
__atuin_set_ret_value.
With a single-line prompt, the last line of the output of the previous
command is overwritten by the prompt on the enter_accept. In this
situation, `tput cuu` receives 0 as the argument, but `tput cuu 0`
emits the control sequence `\e[0A`, which moves the cursor above by
one line unexpectedly. This is because the parameter 0 for CUU means
the default value, 1. In this patch, to avoid moving the cursor when
the prompt offset is 0, we check the offset value before running `tput
cuu`.
* Add TLS to atuin-server
atuin as a project already includes most of the dependencies necessary
for server-side TLS. This allows `atuin server start` to use a TLS
certificate when self-hosting in order to avoid the complication of
wrapping it in a TLS-aware proxy server.
Configuration is handled similar to the metrics server with its own
struct and currently accepts only the private key and certificate file
paths.
Starting a TLS server and a TCP server are divergent because the tests
need to bind to an arbitrary port to avoid collisions across tests. The
API to accomplish this for a TLS server is much more verbose.
* Fix clippy, fmt
* Add TLS section to self-hosting
* feat: add semver checking to client requests
This enforces that the client and the server run the same major version
in order to sync successfully.
We're using the `Atuin-Version` http header to transfer this information
If the user is not on the same MAJOR, then they will see an error like
this
> Atuin version mismatch! In order to successfully sync, the client and the server must run the same *major* version
> Client: 17.1.0
> Server: 18.1.0
> Error: could not sync records due to version mismatch
This change means two things
1. We will now only increment major versions if there is a breaking
change for sync
2. We can now add breaking changes to sync, for any version >17.1.0.
Clients will fail in a meaningful way.
* lint, fmt, etc
* only check for client newer than server
* Add version header to client too
Before this change, when configuring only `common_subcommands` and
not `common_prefix` (so it would take its default value),
atuin produces an error message:
```
Error: could not load client settings Caused by: failed to deserialize: missing field `common_prefix` Location: atuin-client/src/settings.rs:456:26
Error: could not load client settings
Caused by:
failed to deserialize: missing field `common_prefix`
Location:
atuin-client/src/settings.rs:456:26
Error:: command not found
```
With this change, the fields can be specified separately and missing
fields will take their default values.
