Commit Graph

101 Commits

Author SHA1 Message Date
David Dworken
33bc582d8b
Run integration tests in parallel to speed up testing (#175)
* Remove a few direct DB insertions to prepare for parallel tests

* Revert "Remove a few direct DB insertions to prepare for parallel tests"

This reverts commit f8a3552ad8.

* Add rudimentary experiment of splitting tests into two chunks to make them faster

* Add missing tag

* Remove code that enforces that all goldens are used, since it is incompatible with how tests are currently split into chunks

* Lay out the framework for checking goldens being used across all test runs

* Fix missing brace

* Revert "Remove code that enforces that all goldens are used, since it is incompatible with how tests are currently split into chunks"

This reverts commit 06cc3eedbc.

* Add initial work towards checking that all goldens are used

* Delete incorrect and unreferenced matrix

* Upgrade actions/upload-artifact to see if that makes the download in the next job work

* Alternatively, try downloading the artifact by name

* Update golden checker to read all the golden artifacts

* Swap to using glob to enumerate all golden files, rather than hardcoding them

* Remove debugging commands

* Remove goldens that are actually used

* Remove another golden that is actually used

* Add more comprehensive support for test sharding

* Fix references to test shards and increase shard count

* Shard the fuzz test

* Add debug prints

* Mark additional tests for sharding

* Fix logic error that broke test sharding

* Remove debug print

* Fix incorrect logic with skipping the fuzz test

* Move sharding functions to testutils and add some comments

* Upgrade all setup-go actions to enable caching of deps

* Remove goldens that don't exist

* Remove new line

* Reduce delay

* Correct stage name

* Remove incorrect skip code from the first version of sharding

* Remove unused import

* Reduce number of test shards to match GitHub's limit of 5 concurrent macos jobs

* Use cask for installing homebrew to speed up github actions

* More cleanup for unused goldens
2024-08-11 12:29:36 -07:00
David Dworken
1c963bed8a
Update go action to enable caching of dependencies 2024-08-11 12:29:36 -07:00
David Dworken
a6333ea995
Skip DD integration for m1 mac since it seems to fail for mysterious beta-related reasons 2024-08-11 12:29:36 -07:00
David Dworken
212c24c30c
Add tests running on macos-14 with m1 (#169)
* Add tests running on macos-14 with m1

* Run macos setup steps on macos-14 with m1
2024-02-04 10:55:11 -08:00
David Dworken
d8eb079e5f
Add .zip file extension for go tests artifact 2023-12-28 19:32:09 -08:00
David Dworken
aa4afcfa29
Add OS to testlog output to separate macos and linux test logs 2023-12-21 18:27:46 -08:00
David Dworken
2b327f965a
Update to latest version of gotestsum from main rather than from my fork, now that my patch has been merged 2023-12-20 19:57:02 -08:00
David Dworken
81081073a3
Add test log as an exported file from github actions 2023-12-20 15:02:16 -08:00
David Dworken
f88e1d5b3b
Add dollar sign to fix github actions templating for outputs 2023-12-03 16:58:13 -08:00
David Dworken
bdaf0f8a65
Make TUI always take exactly the full width of the terminal through more precise calculations of the overhead used for the table (#136)
* Make TUI always take exactly the full width of the terminal through more precise calculations of the overhead used for the table

* Update TUI goldens for more accurate TUI sizing

* More golden updates for wider table sizing

* Include failed test goldens as artifacts for easier test fixing on github actions
2023-12-03 13:15:38 -08:00
David Dworken
1a450c0817 Add basic sanity test to confirm that our interactions with the OpenAI API are correct 2023-11-12 02:57:23 -08:00
David Dworken
982f7b5f02
Add extra delay to see if that allows DD metrics to go through 2023-10-22 19:37:09 -07:00
David Dworken
238177897e
Swap to cloning gotestsum over https:// rather than SSH to fix auth 2023-10-22 17:09:13 -07:00
David Dworken
772ab6cc87
Update makefile to use my forked version of gotestsum that contains https://github.com/gotestyourself/gotestsum/pull/377 2023-10-22 17:06:36 -07:00
David Dworken
6e1589c1d8
Update github actions to export the test results json for both OSs, and to do it even if the test are an overall failure 2023-10-22 11:12:11 -07:00
David Dworken
8b9e3dfb10
Export testrun.json as a file so that we can also download it for manual inspection/processing 2023-10-22 00:21:47 -07:00
David Dworken
df9c6e8786
Swap to using gotestsum for retrying flaky tests 2023-10-21 15:41:32 -07:00
David Dworken
63da1d9cd4
Run tests for releases too 2023-10-10 22:20:07 -07:00
David Dworken
7ad1e2fb03
Upgrade to go 1.21 which offers fully reproducible builds (on top of SLSA's attestations) 2023-10-01 18:00:58 -07:00
David Dworken
b478eadeae
Merge pull request #106 from lsmoura/sergio/isolated-server
Create isolated server struct that encapsulates all server logic
2023-09-13 21:31:51 -07:00
Sergio Moura
b93a365055 use actions/checkout@v4 for go-test.yml 2023-09-13 10:35:18 -04:00
David Dworken
4c40a8a177
Skip datadog setup for release commits since we also skip tests for release commits 2023-09-12 18:57:21 -07:00
David Dworken
392ebfb0fd
Fix yaml syntax error introduced in 27d5c59263 2023-09-08 19:41:42 -07:00
David Dworken
27d5c59263
Skip datadog integratio on macos if colima fails to start, since colima appears to be flaky due to a code signing error 2023-09-08 19:27:16 -07:00
David Dworken
cc10689d09
Skip running DD on non-master branches so that PRs can run successfully 2023-09-08 09:59:55 -07:00
David Dworken
fe75d67ede
Link colima's docker socket to the default location 2023-09-07 22:02:08 -07:00
David Dworken
c67b6b8c82
Install docker on macos github action so the datadog integration works 2023-09-07 22:00:47 -07:00
David Dworken
daa4fa2a00
Fix indentation error in go-test.yml 2023-09-07 18:32:17 -07:00
David Dworken
a13549851c
Add basic datadog integration to monitor test failures in github actions 2023-09-07 18:30:47 -07:00
David Dworken
22abfbc733
Add overall test retries for all tests to make github action tests even more reliable 2023-09-04 19:30:38 -07:00
David Dworken
25eae144d8
Revert all tmpfs related commits, since running in a tmpfs doesn't seem to meaningfully improve test reliability 2023-09-04 18:03:46 -07:00
David Dworken
069705ef41
Move setting of TZ variable into Makefile so it is always set even if doing local development in a different timezone 2023-09-02 18:28:37 -07:00
David Dworken
a2617e970a
Fix tmpfs creation for macos 2023-09-02 18:25:09 -07:00
David Dworken
9de10560a6
Re-enable tmate debugging 2023-09-02 18:17:16 -07:00
David Dworken
9dafa40872
Remove || true to force macos setup to complete successfully 2023-09-02 18:16:53 -07:00
David Dworken
4c2e49885f
Fix tmpfs mount command for linux 2023-09-02 18:11:29 -07:00
David Dworken
46b2d5a26a
Enable tmate for debugging 2023-09-02 18:06:46 -07:00
David Dworken
f0bd50dac6
Fix string quotes in github actions template 2023-09-02 18:04:31 -07:00
David Dworken
d9893397b7
Split test setup into separate stages for each OS 2023-09-02 18:03:26 -07:00
David Dworken
d51dba8375
Split tests and setup into different stages 2023-09-02 17:58:36 -07:00
David Dworken
0ee8c1c796
Add sudo for ram disk creation 2023-09-02 17:35:22 -07:00
David Dworken
96446eb62c
Configure a ram disk for ~/.hishtory/ to make sqlite run faster 2023-09-02 17:33:02 -07:00
David Dworken
4e93727dd0
Disable tmate session for Github Actions 2023-08-27 18:23:21 -07:00
David Dworken
477c30230a
Enable tmate debugging 2023-02-13 20:36:12 -08:00
David Dworken
c29fa90207
Disable tmate 2022-12-18 22:03:02 -08:00
David Dworken
d606b5c2ba
Uprade tmux on ubuntu to see if that fixes the test failures on actions 2022-12-13 17:15:11 -08:00
David Dworken
5c3f023260
enable tmate for debugging 2022-12-12 20:32:50 -08:00
David Dworken
0cd7d2e4a4
Remove tmate 2022-11-16 20:43:29 -08:00
David Dworken
5b66943b9a
Update isAction golden after last commit + remove shell:bash that was only needed for windows 2022-11-12 08:07:49 -08:00
David Dworken
6e7822ebae
Attempt to fix github action only test failures 2022-11-12 06:18:05 -08:00