* add importers for xonsh JSON files and SQLite db
* rustfmt xonsh importers
* remove env-dependent tests from xonsh importers
* pass xonsh_data_dir into path resolver instead of looking up in env
* review: run format
* review: fix clippy errors
---------
Co-authored-by: Ellie Huxtable <ellie@elliehuxtable.com>
Instead of lots of small sqlite transaction, do one monster one.
A single tx with 100s of 1000s of rows should be fine on all systems
A very unscientific test shows this to be roughly 10x faster
Low-end devices like RISC-V SBCs are sometimes too slow to initialize SQLite in 0.1s. Option to specify a higher value allows check to pass on such devices with relaxed restrictions.
Replace lots of logging with some progress bars. This looks much nicer
I'd like to move it out of the atuin-client crate and into the atuin
crate. But first, I want to decouple a lot of the record moving, so it
can wait until that's done.
* Allow specifying a timezone in history search/list
* Fix clippy complaints
* Add a bit more comment on supporting named timezones
* Add rudimentary tests
* Ditch local timezone test
* Timezone configuration support
* Set default timezone to `local`
* `--tz` -> `--timezone`
`--tz` is kept as a visible alias
This allows the user to
1. Specify that they want to sync, but ONLY pull new data
2. Specify that they wish to force pull, which will wipe the local store
and download it from the remote
With the other set of changes, this allows the user to perform
sufficient maintenance to recovery from most errors I can think of right
now.
This will
1. Wipe the remote store
2. Upload all of the local store to remote
Imagine the scenario where you end up with some mixed keys locally :(
You confirm this with
```
atuin store verify
```
You then fix it locally with
```
atuin store purge
```
Ensure that your local changes are reflected remotely with
```
atuin store push --force
```
and then (another PR, coming soon), update all other hosts with
```
atuin store pull --force
```
This command will delete all records from the local store that cannot be
decrypted with the current key.
If a verify fails before running this, it should pass _after_ running
it.
Required afterwards:
- A `push --force`, to allow ensuring the remote store equals the local
store (deletions have now occured!)
- A `pull --force`, as once remote has been forced then local needs the
same
Nice to have:
- Provide "old" keys to purge, in case the are not lost. Or maybe rekey.
This ensures that the local store can be decrypted with the current
in-use key.
If it cannot, we can go on to perform maintenance operations and get
back into a happy state.
* feat: add prefers_reduced_motion flag
* use NO_MOTION, and ensure type is bool
* update default config
---------
Co-authored-by: Ellie Huxtable <ellie@elliehuxtable.com>
* feat: add record re-encrypting
* automatically re-encrypt store when logging in with a different key
* fix
* actually save the new key lmao
* add rekey
* save new key
* decode bip key
* "add test for sqlite store re encrypt"
Now that local history is stored encrypted, new_key should not overwrite
an existing one. This may be frustrating, but will remove the risk of
Atuin generating a new key and the user losing their old one.
* add support for getting the total length of a store
* tidy up sync
* auto call init if history is ahead
* fix import order, key regen
* fix import order, key regen
* do not delete key when user deletes account
* message output
* remote init store command; this is now automatic
* should probs make that function return u64 at some point
* Add change-password command & support on server
* Add a test for password change
* review: run format
---------
Co-authored-by: Ellie Huxtable <ellie@elliehuxtable.com>
* feat: make history list format configurable
* Update atuin-client/config.toml
* review: run format
---------
Co-authored-by: Ellie Huxtable <ellie@elliehuxtable.com>
* chore: add feature to allow always disable check update
In the packaging rules of some distributions, the software's self-update check needs to be permanently turned off
This commit will make it easier for these users to
* fix: formatting
---------
Co-authored-by: Ellie Huxtable <ellie@elliehuxtable.com>
An issue with the old sync was that if there was _one_ record encrypted
with a different key, sync would stop. You'd need to delete your account
and start from scratch. This sucked.
This change means we will carry on, and try to encrypt and build with as
much of the history as we are able to decrypt.
This is possible because we can quite happily store data on disk that we
cannot decrypt. The old store couldn't do this.
In future, we might consider a keyring containing multiple keys.
Related: https://forum.atuin.sh/t/search-ignoring-commands/74/5?u=ellie
When a user ran a duplicated command, but in another session, it was
removed by filters. This is because the subquery that was once used did
not have the same filters applied as the main query.
Instead of messing with subqueries, `group by` instead. This aligns with
the search() function
* feat(search): make cursor style configurable
The vim mode of the interactive Atuin search changes the cursor style
on a mode change, but the current implementation has the following
issues.
* The terminal's cursor style set by the Atuin search remains after
Atuin exits. This causes an inconsistency with the shell's setting
for the cursor style.
* Also, the cursor style for each keymap mode is currently hardcoded
in the source code, which is not necessarily consistent with the
user's cursor-style setting in the shell.
* Since the current implementation does not set the cursor style for
the initial keymap mode but only sets the cursor style when the
keymap mode is changed, it also causes inconsistency in the cursor
style and the actual keymap when the shell's keymap and Atuin's
initial keymap mode are different.
This patch solves those issues by introducing an opt-in configuration
variable `keymap_cursor`. By default, the vim mode does not change
the cursor style because there is no way to automatically determine
the cursor style consistent with the shell settings. We enable the
feature only when the user specifies the preferred cursor style in
each mode in their config. Also, the cursor style is set on the
startup of the Atuin search (based on the initial keymap mode) and is
reset on the termination of the Atuin search (based on the shell's
keymap mode that started the Atuin search).
* chore(settings): remove dependency on crossterm
* fix: add acquire timeout to sqlite database connection
This should fix#1503
I wasn't able to trigger enough IO pressure for the SQL connection to be
a problem.
This adds `local_timeout` to the client config. This is a float, and
represents the number of seconds (units in line with the other timeouts,
though those are ints). Users may well want to reduce this if they
regularly have issues, but by default I think 2s is fine and avoids a
non-responsive system in bad situations.
* tests
When printing the history list with either the session or cwd filter
enabled, use to same query method as without either to ensure that the
other options (hide deleted entries etc) are respected.
* feat: add history rebuild
This adds a function that will
1. List all history from the store
2. Segment by create/delete
3. Insert all creates into the database
4. Delete all deleted
This replaces the old history sync.
Presently it's incomplete. There is no incremental rebuild, it can only
do the entire thing at once.
This is ran by `atuin store rebuild history`
* fix tests
* add incremental sync
* add auto sync
Imagine the scenario where a sync fails. The function touched here will
never save the sync time.
That means that external to this function, the sync never happened. The
next command will try and start one immediately.
The happy scenario is that this succeeds. The unhappy scenario is that
this fails. Fails, and isn't saved, so we try again...
Should add proper backoff but this is a good start.
* feat(config): add vim option to config
* feat(ui): simple vim mode
* fix(windows): windows sadly doesn't support the stuff
* feat(ui): blinking
* fix(merge)
* revert: reverts some debugging stuff
* feat(ui): changes the defaut to insert, don't know what should be the default
* feat(ui): implements some vim parity
* doc: adds this to the docs
* docs(keybindings): adds vim mode keybindsings to the list of keybindings
* refactor: rustfmt and remove the docs for pr in own repo
* refactor: use execute!
* Update atuin/src/command/client/search/interactive.rs
---------
Co-authored-by: Ellie Huxtable <ellie@elliehuxtable.com>
* Begin work on command inspector
This is a separate pane in the interactive mode that allows for
exploration and inspecting of specific commands.
I've restructured things a bit. It made logical sense that things
were nested under commands, however the whole point of `atuin` is to
provide commands. Breaking things out like this enables a bit less
crazy nesting as we add more functionality to things like interactive
search. I'd like to add a few more interactive things and it was
starting to feel very cluttered
* Some vague tab things
* functioning inspector with stats
* add interactive delete to inspector
* things
* clippy
* borders
* sus
* revert restructure for another pr
* Revert "sus"
This reverts commit d4bae8cf61.
* feat: rework record sync for improved reliability
So, to tell a story
1. We introduced the record sync, intended to be the new algorithm to
sync history.
2. On top of this, I added the KV store. This was intended as a simple
test of the record sync, and to see if people wanted that sort of
functionality
3. History remained syncing via the old means, as while it had issues it
worked more-or-less OK. And we are aware of its flaws
4. If KV syncing worked ok, history would be moved across
KV syncing ran ok for 6mo or so, so I started to move across history.
For several weeks, I ran a local fork of Atuin + the server that synced
via records instead.
The record store maintained ordering via a linked list, which was a
mistake. It performed well in testing, but was really difficult to debug
and reason about. So when a few small sync issues occured, they took an
extremely long time to debug.
This PR is huge, which I regret. It involves replacing the "parent"
relationship that records once had (pointing to the previous record)
with a simple index (generally referred to as idx). This also means we
had to change the recordindex, which referenced "tails". Tails were the
last item in the chain.
Now that we use an "array" vs linked list, that logic was also replaced.
And is much simpler :D
Same for the queries that act on this data.
----
This isn't final - we still need to add
1. Proper server/client error handling, which has been lacking for a
while
2. The actual history implementation on top
This exists in a branch, just without deletions. Won't be much to
add that, I just don't want to make this any larger than it already
is
The _only_ caveat here is that we basically lose data synced via the old
record store. This is the KV data from before.
It hasn't been deleted or anything, just no longer hooked up. So it's
totally possible to write a migration script. I just need to do that.
* update .gitignore
* use correct endpoint
* fix for stores with length of 1
* use create/delete enum for history store
* lint, remove unneeded host_id
* remove prints
* add command to import old history
* add enable/disable switch for record sync
* add record sync to auto sync
* satisfy the almighty clippy
* remove file that I did not mean to commit
* feedback
* fix(import/zsh): zsh use a special format to escape some characters
unescape those correctly rather than throw them away.
zsh side code:
9f57ca4ac8/Src/utils.c (L4889-L4900)
* fix code style
* feat: add semver checking to client requests
This enforces that the client and the server run the same major version
in order to sync successfully.
We're using the `Atuin-Version` http header to transfer this information
If the user is not on the same MAJOR, then they will see an error like
this
> Atuin version mismatch! In order to successfully sync, the client and the server must run the same *major* version
> Client: 17.1.0
> Server: 18.1.0
> Error: could not sync records due to version mismatch
This change means two things
1. We will now only increment major versions if there is a breaking
change for sync
2. We can now add breaking changes to sync, for any version >17.1.0.
Clients will fail in a meaningful way.
* lint, fmt, etc
* only check for client newer than server
* Add version header to client too
Before this change, when configuring only `common_subcommands` and
not `common_prefix` (so it would take its default value),
atuin produces an error message:
```
Error: could not load client settings Caused by: failed to deserialize: missing field `common_prefix` Location: atuin-client/src/settings.rs:456:26
Error: could not load client settings
Caused by:
failed to deserialize: missing field `common_prefix`
Location:
atuin-client/src/settings.rs:456:26
Error:: command not found
```
With this change, the fields can be specified separately and missing
fields will take their default values.
