David Dworken
352e483db5
Split the flaky testTui/search test into two tests to increase odds of it passing quickly
2023-10-23 18:53:16 -07:00
David Dworken
27eeb7d099
Refactor code to strip out the shell prefix to a separate function
2023-10-23 12:24:43 -07:00
David Dworken
10851528bc
Increase delays for testTui/search since it is the flakiest test
2023-10-23 12:13:40 -07:00
David Dworken
ff79707ccf
Use unqualified test name as test ID since the full package makes the datadog metrics overly verbose
2023-10-22 20:57:21 -07:00
David Dworken
0413e88321
Add flush call to metrics exporter so that metrics are flushed before the github action is killed
2023-10-22 18:10:49 -07:00
David Dworken
13233926a7
Fix test broken by 9433bd6e94
2023-10-22 17:59:21 -07:00
David Dworken
bfce9d4b1c
Add space to fix test that was broken by previous refactoring
2023-10-22 17:21:36 -07:00
David Dworken
d463d99916
Reorder if in test metrics exporter so that it actually skips initializing DD when there is no DD api key
2023-10-22 12:01:51 -07:00
David Dworken
2b1af986d8
Add support for running hishtory redact while offline, even though this may lead to syncing inconsistency bugs
2023-10-22 11:43:56 -07:00
David Dworken
c613e696b4
Update hishtory redact so that it doesn't error out due to it redacting the presaved entry for the current command
2023-10-22 11:34:26 -07:00
David Dworken
fdbe91246e
Add another hook point to check HISHTORY_SIMULATE_NETWORK_ERROR so that it also blocks downloading updates
2023-10-22 11:30:49 -07:00
David Dworken
9433bd6e94
Refactor offline simulation tests so that they reliably set the env variable before ever starting the shell, since we need the env variable to also apply to presaving
2023-10-22 11:28:41 -07:00
David Dworken
956ac736f2
Update metrics exporter output to include whether GLOBAL_STATSD is nil
2023-10-22 11:14:25 -07:00
David Dworken
ff0d2c34d6
Add print line after finishing upload of DD metrics
2023-10-21 17:14:49 -07:00
David Dworken
3d4e48aabd
Add test runtime metric
2023-10-21 16:42:41 -07:00
David Dworken
595ddec235
Add integration to report flaky tests to datadog
2023-10-21 16:28:28 -07:00
David Dworken
df9c6e8786
Swap to using gotestsum for retrying flaky tests
2023-10-21 15:41:32 -07:00
David Dworken
4e0788dccc
Update more test code to use require rather than manual failures
2023-10-21 14:59:12 -07:00
David Dworken
5ec9f85e5e
Fix broken test on ubuntu that was introduced in 08e734a
2023-10-21 13:44:08 -07:00
David Dworken
6bb12241c9
Promote the background updates of the DB from BetaMode to prod
2023-10-21 09:47:20 -07:00
David Dworken
08e734a1fe
Add tests to verify the fix for #117
2023-10-21 09:00:06 -07:00
David Dworken
3ca9de559f
Fix #117 by disown-ing background processes so that they don't stop fish from exiting
2023-10-20 23:23:24 -07:00
David Dworken
9efef65e3a
Update README and make column name matching more relaxed
2023-10-18 19:17:05 -07:00
David Dworken
f1ca76a5a0
Extra delay to make test more reliable
2023-10-15 21:47:01 -07:00
David Dworken
0fec2d5ba9
Ensure that the DB entry is reliably created locally and in the remote for testTui/scroll to make it more reliable
2023-10-15 21:45:20 -07:00
David Dworken
1a04213f11
Add extra delay to make flaky test more reliable
2023-10-15 19:02:57 -07:00
David Dworken
58e92e5760
Refactor by moving methods out of lib.go into more specific packages
2023-10-15 18:30:39 -07:00
David Dworken
539ef74746
Remove TODO that has already been completed
2023-10-15 18:12:28 -07:00
David Dworken
f3727dbeff
Wire through a flag so that we can track when installations come from tests, and delete those from the DB more aggressively
2023-10-15 13:09:48 -07:00
David Dworken
449a4d0000
Add better handling for race condition with deleting presaved entries
2023-10-15 13:00:34 -07:00
David Dworken
4bcc5fb335
Periodically call query endpoint and retrieve data to ensure that the local DB is always reasonably up to date
2023-10-15 12:29:50 -07:00
David Dworken
1c4fe0d379
Add TODO
2023-10-14 17:30:12 -07:00
David Dworken
6fb6498515
Make deletion request processing happen in parallel to speed up query endpoint
2023-10-14 17:21:00 -07:00
David Dworken
fca2b1441f
Always include user and device ID in API request headers, so that they're available in all server-side handlers
2023-10-14 10:52:51 -07:00
David Dworken
66e482de17
Fix broken import test that asserted that fish commands were imported after bash/zsh commands
2023-10-12 21:51:18 -07:00
David Dworken
dbb555e20c
Add progress bar to monitor importing history entries for #93
2023-10-12 19:35:52 -07:00
David Dworken
851283df8d
Add TODO
2023-10-12 19:34:09 -07:00
David Dworken
253ad7f6b6
Bold matches for search queries in TUI for #112 . This was previously available behind the beta-mode flag, but will now be enabled by default
2023-10-12 19:34:09 -07:00
David Dworken
9a947b73cf
Add test for updating that relies on the prod server
2023-10-10 22:14:51 -07:00
David Dworken
dc19fa3a17
Roll-forward "Make history entry start times more precise (down to the nanosecond) to make the recorded runtime more accurate, since currently start times are rounded to the nearest second. Though
...
note that 'date' on MacOS doesn't support %N, so this doesn't apply to MacOS"
This reverts commit 6281ae0601
2023-10-10 07:44:15 -07:00
David Dworken
dfa961072e
Add more precise assertions for update tests
2023-10-10 07:43:37 -07:00
David Dworken
8373c589ea
Improve update tests to run a command post-update with the pre-update shell config
2023-10-10 07:35:23 -07:00
David Dworken
f0dbcd6e3b
Add more thorough tests for updates
2023-10-09 21:41:30 -07:00
David Dworken
45aa218bd3
Move err check so that it only executes if it is reassigned
2023-10-08 20:26:33 -07:00
David Dworken
717cf2c4b0
Add check so that if there are zero leftover entries, we don't try to import them into the DB since this leads to install errors
2023-10-08 20:25:53 -07:00
David Dworken
6281ae0601
Revert "Make history entry start times more precise (down to the nanosecond) to make the recorded runtime more accurate, since currently start times are rounded to the nearest second. Though note that 'date' on MacOS doesn't support %N, so this doesn't apply to MacOS"
...
This reverts commit d404a73c9b
2023-10-07 20:05:21 -07:00
David Dworken
4bb0ebfb2e
Bubble up error when we fail to encrypt history entries
2023-10-07 20:02:02 -07:00
David Dworken
480f8696fb
Also read imports from ~/.zhistory which is sometimes used by zsh
2023-10-07 15:19:51 -07:00
David Dworken
d2788a9f31
Swap interface{} to any
2023-10-07 15:11:49 -07:00
David Dworken
d404a73c9b
Make history entry start times more precise (down to the nanosecond) to make the recorded runtime more accurate, since currently start times are rounded to the nearest second. Though note that 'date' on MacOS doesn't support %N, so this doesn't apply to MacOS
2023-10-07 14:47:54 -07:00
David Dworken
5323c264b3
Further improve history importing performance for #93
2023-10-07 14:41:34 -07:00
David Dworken
8097c31afa
Remove TODO that was fixed in f5264b6eff
2023-10-07 14:36:02 -07:00
David Dworken
69b9b58cd0
Fix test that should have been updated in 3efe624
2023-10-07 11:06:56 -07:00
David Dworken
f5264b6eff
Tweak ImportHistory so that all entries are guaranteed to have sequential timestamps that are monotonically increasing
2023-10-07 11:05:19 -07:00
David Dworken
3efe62401c
Change semicolon to && so that if the cd fails, the command doesn't run. Per feedback on #83
2023-10-06 16:49:50 -07:00
David Dworken
c7d1abdc83
Add additional tests for history importing
2023-10-03 08:00:16 -07:00
David Dworken
d33bd6a046
Improve history import performance so that we can import 100k entries in ~6 seconds (down from ~20 seconds)
2023-10-02 22:27:24 -07:00
David Dworken
f8719e4b9b
Increase number of automatic retries for testTui/delete since it seems to also be somewhat flaky
2023-10-01 18:26:02 -07:00
David Dworken
0b54cebd46
Further increase delay to make tests more reliable
2023-10-01 18:25:19 -07:00
David Dworken
2e793b2c10
Test refactoring to remove a function that can be replaced with testify
2023-10-01 18:21:11 -07:00
David Dworken
33f09ea030
Fix testPresaving/bash by calling SetSid to prevent SIGTTIN signal from killing the test
2023-09-29 21:38:50 -07:00
David Dworken
2c77c3d9c9
Fix bug with bash presaving that caused commands to not get properly recorded due to presaving looking like a duplicate command
2023-09-29 21:03:12 -07:00
David Dworken
0b57034c39
Revert "Revert all commits since v0.223 to enable me to release a patch on top of v0.223"
...
This reverts commit 68ed9f2d5d
2023-09-29 18:21:23 -07:00
David Dworken
68ed9f2d5d
Revert all commits since v0.223 to enable me to release a patch on top of v0.223
2023-09-28 22:15:30 -07:00
David Dworken
98b6eb57d0
Disable broken test on github actions
2023-09-28 22:15:26 -07:00
David Dworken
61a7435fb7
Add additional assertion to presaving test to make it a stronger test, and to help debug a test failure that only reproduces on github actions
2023-09-28 22:15:22 -07:00
David Dworken
b1988d7022
Increase delay since TestTui/search is still flaky
2023-09-28 22:15:12 -07:00
David Dworken
1a84bc2fef
Remove new line
2023-09-28 22:15:09 -07:00
David Dworken
b859b34616
Disable testing of TUI colors on linux
2023-09-28 22:15:03 -07:00
David Dworken
fdec51bd14
Update presaving for bash to use history 1 rather than BASH_COMMAND to ensure we don't log bash functions
2023-09-28 22:14:53 -07:00
David Dworken
5d46c94b13
Improve test for duplicate rows to make it more reliable/simple
2023-09-28 22:14:47 -07:00
David Dworken
2e5f345411
Refactor tmux capturing to take in a struct to avoid having to create so many overloaded functions
2023-09-28 22:14:41 -07:00
David Dworken
06ee1bd6fe
Add comments/clean up TUI highlighting code
2023-09-28 22:14:30 -07:00
David Dworken
83c3664de6
Add test for beta-mode enabling highlighting of matching queries
2023-09-28 22:14:22 -07:00
David Dworken
e04583f62f
Move regexp compilation out of render function so it isn't run once for every single cell
2023-09-28 22:14:15 -07:00
David Dworken
3feb267038
Remove debug log statements
2023-09-28 22:14:11 -07:00
David Dworken
b482a6d6ec
Add very basic TUI test that also tests how the TUI uses escape sequences for colors
2023-09-28 22:14:05 -07:00
David Dworken
79ec2be1e4
Improve testPresaving to cover more potential errors with presaving
2023-09-28 22:14:00 -07:00
David Dworken
89014a8674
Add workaround for bash issues with pre-saving
2023-09-28 22:13:41 -07:00
David Dworken
5ac6f297fb
Fix incorrect shell name in config.fish for pre-saving
2023-09-28 22:13:35 -07:00
David Dworken
9fda54d4c2
Remove testutils.Check(t, err) and replace it with require.NoError which gives a clearer error message and a full stacktrace
2023-09-28 22:13:27 -07:00
David Dworken
beb907d59e
Add bolding of matching search results for #112 , currently behind the beta-mode flag
2023-09-28 22:13:11 -07:00
David Dworken
ec44e94565
Fix NPE that occurs if the initial sqlite query is slow, the user starts typing, and then the latter query is faster and completes first, by forcing an update of the table if it is nil before calling SetRows() on it
2023-09-28 21:52:00 -07:00
David Dworken
6d5a86a8bb
Revert all commits since v0.223 to enable me to release a patch on top of v0.223
2023-09-28 21:49:37 -07:00
David Dworken
d18747e639
Disable broken test on github actions
2023-09-27 21:10:39 -07:00
David Dworken
bed3fd3eb4
Add additional assertion to presaving test to make it a stronger test, and to help debug a test failure that only reproduces on github actions
2023-09-26 22:17:32 -07:00
David Dworken
f84c2d86b5
Increase delay since TestTui/search is still flaky
2023-09-26 20:16:55 -07:00
David Dworken
289fe6e8e1
Remove new line
2023-09-26 20:16:22 -07:00
David Dworken
adc069bd54
Disable testing of TUI colors on linux
2023-09-26 20:15:11 -07:00
David Dworken
d14dc1cd50
Update presaving for bash to use history 1 rather than BASH_COMMAND to ensure we don't log bash functions
2023-09-26 19:41:12 -07:00
David Dworken
753b7241a8
Improve test for duplicate rows to make it more reliable/simple
2023-09-26 19:38:21 -07:00
David Dworken
64db341144
Refactor tmux capturing to take in a struct to avoid having to create so many overloaded functions
2023-09-24 18:28:06 -07:00
David Dworken
6d95b759f6
Add comments/clean up TUI highlighting code
2023-09-24 18:06:32 -07:00
David Dworken
0627891be2
Add test for beta-mode enabling highlighting of matching queries
2023-09-24 17:57:40 -07:00
David Dworken
d2fec54ec0
Move regexp compilation out of render function so it isn't run once for every single cell
2023-09-24 17:19:51 -07:00
David Dworken
5a70a3b776
Remove debug log statements
2023-09-24 17:18:56 -07:00
David Dworken
b3fa5c9a6b
Add very basic TUI test that also tests how the TUI uses escape sequences for colors
2023-09-24 17:03:54 -07:00
David Dworken
354f2872d6
Improve testPresaving to cover more potential errors with presaving
2023-09-24 16:35:40 -07:00
David Dworken
0db27d4217
Add workaround for bash issues with pre-saving
2023-09-24 16:35:00 -07:00
David Dworken
eeacd6fa70
Fix incorrect shell name in config.fish for pre-saving
2023-09-24 16:31:46 -07:00
David Dworken
5121df5a8c
Remove testutils.Check(t, err) and replace it with require.NoError which gives a clearer error message and a full stacktrace
2023-09-24 16:05:01 -07:00
David Dworken
0f375aacfc
Add bolding of matching search results for #112 , currently behind the beta-mode flag
2023-09-24 15:57:39 -07:00
David Dworken
e92c702402
Another exclusion for presaving
2023-09-24 14:19:11 -07:00
David Dworken
cb95d4a396
More test reliability improvements to fix all test failure root causes from 10 sample runs on Github Actions
2023-09-24 06:29:47 -07:00
David Dworken
ce9602d0df
Make testControlR more reliable in cases where control-R doesn't include padding
2023-09-24 06:25:08 -07:00
David Dworken
e3c70beb6b
Fix bug introduced in c00e5d where hishtory import would fail if shell history files didn't exist (e.g. because the user had never used zsh)
2023-09-23 20:37:41 -07:00
David Dworken
c21d443fa6
Make presaving query slightly more specific, and delete res.RowsAffected check since it can cause flaky errors if a command is run twice very quickly
2023-09-23 20:02:05 -07:00
David Dworken
3253883198
Fix syncing consistency bug, but with the caveat that there is currently no easy/practical way to write tests for this
2023-09-23 19:57:41 -07:00
David Dworken
c3782dda18
Add additional error to IsOfflineError
2023-09-23 19:37:25 -07:00
David Dworken
1c2f9cf36c
Run local server for TestAugmentedIsOfflineError and ensure that HISHTORY_TEST is set
2023-09-23 19:35:21 -07:00
David Dworken
16f2751975
Add offline checking for deleting presaved entries so that hishtory works while offline
2023-09-23 19:32:35 -07:00
David Dworken
8443292070
Augment IsOfflineError(err) so that it detects if the hishtory server is down, and will then treat all API errors as offline errors
2023-09-23 16:40:03 -07:00
David Dworken
ab12fa4d47
Fix consistency bug where ClientConfig would get out of date between different parts of the code
2023-09-23 12:40:57 -07:00
David Dworken
8c239a32b9
Rename variable so it doesn't contain TODO
2023-09-23 12:25:58 -07:00
David Dworken
c00e5d357e
Swap hishtory import to use iterators to improve import performance for #93
2023-09-23 12:19:06 -07:00
David Dworken
eb1bc99a22
Add extra delay for testTui_delete test that relies on syncing, which can be slow
2023-09-23 10:58:48 -07:00
David Dworken
68c6bd6654
Improve TestTimestampFormat
2023-09-23 10:56:27 -07:00
David Dworken
e886763144
Make testCustomColumns simpler and more reliable
2023-09-23 05:34:36 -07:00
David Dworken
61c8e26f89
Revert "Make testControlR more reliable by making it more lenient in how it looks for the separator"
...
This reverts commit 5168efb2c8
2023-09-22 21:18:29 -07:00
David Dworken
d613c22e50
Move a bunch of utility functions for tests to a testutils.go file to make client_test.go at least a little shorter
2023-09-22 20:19:12 -07:00
David Dworken
5168efb2c8
Make testControlR more reliable by making it more lenient in how it looks for the separator
2023-09-22 20:03:41 -07:00
David Dworken
51ca2578b7
Follow up to e3d54e: Same thing for querying for pre-saved entries
2023-09-22 19:59:19 -07:00
David Dworken
e3d54e43e8
Make deletion request auto-retry SQLITE_LOCKED errors since deletion requests will now be much more common with pre-saving support
2023-09-22 19:47:10 -07:00
David Dworken
2c9aa099d2
Fix bug in offline sync code that contained off-by-one error leading to missed entries
2023-09-22 19:31:22 -07:00
David Dworken
e089690cbb
no-op formatting change
2023-09-22 19:22:03 -07:00
David Dworken
3f0adbc324
Add support for deleting pre-saved entries on the remote server
2023-09-22 19:00:37 -07:00
David Dworken
1d878195b2
Rework ff98a7907c to use the newly added EntryId column rather than deleting based on the start time
2023-09-22 14:03:41 -07:00
David Dworken
9b847c5e35
Further optimize client-server roundtrips by including deletion and dump requests in submit responses (follow up to 1e43de689f)
2023-09-22 13:49:29 -07:00
David Dworken
a5f11af150
Add initial code to support unique per-entry IDs
...
This code has two caveats for old entries:
1. the ID is being backfiled with a random per-(device,entry) ID. So the ID won't match cross-device.
2. the server-side ID will still be a random ID that is unrelated to the entry ID
2023-09-22 13:16:24 -07:00
David Dworken
2a5a6d65c4
Roll-forward "Add preliminary support for persisting pre-saved history entries remotely"
...
This rolls-forward commit 66916c27cb
2023-09-22 13:13:46 -07:00
David Dworken
dc166ec089
Move fuzz tests to a separate file and add a couple more tests to it
2023-09-22 08:43:58 -07:00
David Dworken
66916c27cb
Revert "Add preliminary support for persisting pre-saved history entries remotely"
...
This reverts commit ff98a7907c
2023-09-21 12:39:20 -07:00
David Dworken
ff98a7907c
Add preliminary support for persisting pre-saved history entries remotely
2023-09-21 12:39:04 -07:00
David Dworken
63dea34007
Add comment to document a somewhat confusing function
2023-09-21 11:35:47 -07:00
David Dworken
1e43de689f
Optimize number of round-trip HTTP connections made by the client by having the submit handler return metadata about whether there are pending dump/deletion requests
...
For now, I'm still keeping the dedicated endpoints for those functionalities, but since most of the time there are no dump/deletion requests this should cut down the number of requests made by the client by 2/3.
2023-09-21 11:35:24 -07:00
David Dworken
b05fb0f818
Fix typo-ed variable name
2023-09-21 11:25:26 -07:00
David Dworken
1c4570586c
Increase max supported line length for #93
2023-09-19 18:59:09 -07:00
David Dworken
69047109ac
Skip presaving the presave command, since it is currently spamming the recorded history entries in bash
2023-09-19 13:35:11 -07:00
David Dworken
216803f325
Bring back logic that was removed in c9da7a10e4 that made it so invalid initial search queries will get replaced with a query for the empty string
2023-09-18 22:21:10 -07:00
David Dworken
c9da7a10e4
Make the initial search query for the TUI async so that the UI opens immediately, so that user's can start typing their query without any delay
2023-09-18 19:35:53 -07:00
David Dworken
4f592f4aef
Automatically retry search queries when the DB is locked to increase reliability (especially when the backend is down, which leads to more queries, and thus more instances of the DB being locked)
2023-09-18 19:22:26 -07:00
David Dworken
f589a0b1ba
Increment the test_retry_count metric for failures too to ensure that it captures tests that fail all retries
2023-09-17 23:07:10 -07:00
David Dworken
134061085b
Fix bug in searching logic that caused gorm to template in NULL for non-null values
2023-09-17 20:24:48 -07:00
David Dworken
e6c76eb222
Make WHERE query for deleting presaved entries stricter and re-add in the error check that was removed in 713ee96a96
2023-09-17 14:38:48 -07:00
David Dworken
4c912cfaed
Improve sh config file tweak function to make it stricter with presaving
2023-09-17 14:35:56 -07:00
David Dworken
1d7ba41289
Avoid performance hit caused by pre-saving by presaving in the background
2023-09-17 14:11:44 -07:00
David Dworken
713ee96a96
Delete error checking branch that isn't necessary because the filter condition above is much more precise now
2023-09-17 12:06:19 -07:00
David Dworken
4b99e4d0e8
Fix misleading error message that contained the wrong shell name due to a copy/paste mistake
2023-09-14 23:09:15 -07:00
David Dworken
48e33d5034
Make the query input box in the TUI match the terminal dimensions
2023-09-14 21:14:16 -07:00
David Dworken
30ba0bd35c
Presaving clean up improvements: Skip presaving empty commands and make timestamp format clearer
2023-09-14 21:04:23 -07:00
